profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/joe-elliott/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

grafana/tempo 1627

Grafana Tempo is a high volume, minimal dependency distributed tracing backend.

joe-elliott/cert-exporter 183

A Prometheus exporter that publishes cert expirations on disk and in Kubernetes secrets

grafana/kubernetes-diff-logger 77

Logs updates to Kubernetes Objects for storing and querying with Loki

grafana/xk6-distributed-tracing 25

A k6 extension for distributed tracing.

joe-elliott/go-rev-proxy 13

Learning project with code samples for using prometheus, redis and jaeger in go

joe-elliott/kubernetes-grafana-controller 6

A Kubernetes controller used to sync CRDs with Grafana

joe-elliott/go-minikube-vscode-dev 1

Development environment for vscode and minikube

push eventjoe-elliott/tempo

Joe Elliott

commit sha b68d7d73ad884abdc03e9e7aa3e1f502c774b83b

[Tempo CLI] Additional Features (#972) * blockslurp test Signed-off-by: Joe Elliott <number101010@gmail.com> * blockslurp Signed-off-by: Joe Elliott <number101010@gmail.com> * pod Signed-off-by: Joe Elliott <number101010@gmail.com> * added slurp buffer Signed-off-by: Joe Elliott <number101010@gmail.com> * prefetch iter Signed-off-by: Joe Elliott <number101010@gmail.com> * queueimpl7 Signed-off-by: Joe Elliott <number101010@gmail.com> * ballast Signed-off-by: Joe Elliott <number101010@gmail.com> * megaslurp Signed-off-by: Joe Elliott <number101010@gmail.com> * remove megaslurp Signed-off-by: Joe Elliott <number101010@gmail.com> * dumbness Signed-off-by: Joe Elliott <number101010@gmail.com> * done with this garbage Signed-off-by: Joe Elliott <number101010@gmail.com> * mega deslurp, iteratorchannel Signed-off-by: Joe Elliott <number101010@gmail.com> * added parse Signed-off-by: Joe Elliott <number101010@gmail.com> * increasing chunk sizes Signed-off-by: Joe Elliott <number101010@gmail.com> * increase base chunk Signed-off-by: Joe Elliott <number101010@gmail.com> * todo Signed-off-by: Joe Elliott <number101010@gmail.com> * add Signed-off-by: Joe Elliott <number101010@gmail.com> * cli ho! Signed-off-by: Joe Elliott <number101010@gmail.com> * Search technically works Signed-off-by: Joe Elliott <number101010@gmail.com> * revert Signed-off-by: Joe Elliott <number101010@gmail.com> * todos Signed-off-by: Joe Elliott <number101010@gmail.com> * layouts Signed-off-by: Joe Elliott <number101010@gmail.com> * todo Signed-off-by: Joe Elliott <number101010@gmail.com> * Added label scan Signed-off-by: Joe Elliott <number101010@gmail.com> * docs Signed-off-by: Joe Elliott <number101010@gmail.com> * lint Signed-off-by: Joe Elliott <number101010@gmail.com> * changelog Signed-off-by: Joe Elliott <number101010@gmail.com> * fixed datarace Signed-off-by: Joe Elliott <number101010@gmail.com> * added max/min objid Signed-off-by: Joe Elliott <number101010@gmail.com>

view details

Zach Leslie

commit sha 2498d5be15d227a2094cabd625908cea5bece6d2

Add support for vulture sending long running traces (#951) * Add support for sending long running traces * Update changelog * Update test fixture and epoch * Refactor to clean up chance logic for long writes * Only query/search traces when we expect that all write have taken place

view details

Annanay Agarwal

commit sha 1a46c46609e6e61a6915391a6701fd739e3505e0

Support global denylist and per-tenant allowlist of tags for search data (#960) * [search] Add global deny list of tags Signed-off-by: Annanay <annanayagarwal@gmail.com> * Move CHANGELOG entries around, add docs Signed-off-by: Annanay <annanayagarwal@gmail.com> * Fix entry Signed-off-by: Annanay <annanayagarwal@gmail.com> * Rework around adding a custom type and yaml/json marshaller Signed-off-by: Annanay <annanayagarwal@gmail.com> * Checkpoint: Implemented marshal funcs and added test Signed-off-by: Annanay <annanayagarwal@gmail.com> * Improve tests, handle edge cases Signed-off-by: Annanay <annanayagarwal@gmail.com> * Address comments Signed-off-by: Annanay <annanayagarwal@gmail.com> * Test denylist/allowlist in docker-compose tempo-search example Signed-off-by: Annanay <annanayagarwal@gmail.com> * Fix single-tenant overrides Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

Joe Elliott

commit sha 050e6d45828aaff04906ee926348ad4c58ac9ef0

Compression Fix #3 (#980) * maybe Signed-off-by: Joe Elliott <number101010@gmail.com> * closes everywhere Signed-off-by: Joe Elliott <number101010@gmail.com> * thing Signed-off-by: Joe Elliott <number101010@gmail.com> * concurrency 1 Signed-off-by: Joe Elliott <number101010@gmail.com> * 10k block size Signed-off-by: Joe Elliott <number101010@gmail.com> * vendor update Signed-off-by: Joe Elliott <number101010@gmail.com> * comment Signed-off-by: Joe Elliott <number101010@gmail.com> * s2 => snappy Signed-off-by: Joe Elliott <number101010@gmail.com> * errcheck Signed-off-by: Joe Elliott <number101010@gmail.com>

view details

push time in 18 hours

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

pull request commentgrafana/tempo

Register `/ingester/ring` endpoint when running ingester target

Thanks for the contribution. We have a rather flakey test which is why CI is failing: https://github.com/grafana/tempo/issues/983

Regarding the change: The reason that the /ingester/ring endpoint does not work currently is because the ring client exposes that endpoint and an ingester has no need for the ingester ring client. This endpoint is available on both queriers and distributors which both do have need for the ring client. However, I do see some value in reducing confusion because a lot of people check the ingesters to see the ring state.

However, I believe this will introduce unnecessary overhead b/c every ingester will now start polling etcd/consul and place extra load on the kv store used to propagate the ring. @bboreham is an expert on the ring and may have more insight on whether this is appropriate or not.

chaudum

comment created time in 21 hours

issue openedgrafana/tempo

Fix Flakey Test: TestWalReplayDeletesLocalBlocks

Describe the bug TestWalReplayDeletesLocalBlocks occassionally fails in CI but never/rarely locally. Fix.

--- FAIL: TestWalReplayDeletesLocalBlocks (0.10s)
    ingester_test.go:253: 
        	Error Trace:	ingester_test.go:253
        	Error:      	"[]" should have 1 item(s), but has 0
        	Test:       	TestWalReplayDeletesLocalBlocks
FAIL

To Reproduce Steps to reproduce the behavior:

  1. Submit PRs
  2. See CI fail on this test (sometimes)

Expected behavior CI passes this test

Additional Context Of course, this may be hinting at an actual issue so please consider while fixing.

created time in a day

push eventgrafana/tempo

Joe Elliott

commit sha 050e6d45828aaff04906ee926348ad4c58ac9ef0

Compression Fix #3 (#980) * maybe Signed-off-by: Joe Elliott <number101010@gmail.com> * closes everywhere Signed-off-by: Joe Elliott <number101010@gmail.com> * thing Signed-off-by: Joe Elliott <number101010@gmail.com> * concurrency 1 Signed-off-by: Joe Elliott <number101010@gmail.com> * 10k block size Signed-off-by: Joe Elliott <number101010@gmail.com> * vendor update Signed-off-by: Joe Elliott <number101010@gmail.com> * comment Signed-off-by: Joe Elliott <number101010@gmail.com> * s2 => snappy Signed-off-by: Joe Elliott <number101010@gmail.com> * errcheck Signed-off-by: Joe Elliott <number101010@gmail.com>

view details

push time in a day

PR merged grafana/tempo

Reviewers
Compression Fix #3

What this PR does: Ok, this one is going to stick.

  • Reverts snappy to continue using golang/snappy. Although klauspost/snappy outperformed on benchmarks in a real world scenario there is either no difference or golang/snappy is better. This is likely b/c our workload is to compress a large number of small byte slices. I believe with tuning klauspost/s2 could outperform golang/snappy.
  • Keeps klauspost/s2. This allows us to experiment with a potentially better compression for the wal while not risking stability by swapping the snappy implementation out from under us.
  • Explicitly .Close() snappy/s2 writes on Put
  • Found and fixed the unclosed appender.
+14 -290

0 comment

12 changed files

joe-elliott

pr closed time in a day

issue commentgrafana/tempo

No objects neither traces see in the s3 bucket with single binary helm deployment.

do you mean in the container? node? or on my local windows machine? Whatever filesystem Tempo has access to.

sunnynehar56

comment created time in 2 days

Pull request review commentgrafana/tempo

Support global denylist and per-tenant allowlist of tags for search data

+package overrides++import (+	"encoding/json"++	"gopkg.in/yaml.v2"+)++type ListToMap struct {

this can be simpler, but not required.

type ListToMap map[string]struct{}

should work

annanay25

comment created time in 2 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentgrafana/tempo

Support global denylist and per-tenant allowlist of tags for search data

 func (o *Overrides) IngestionBurstSizeBytes(userID string) int { 	return o.getOverridesForUser(userID).IngestionBurstSizeBytes } +// SearchTagsAllowList is the list of tags to be extracted for search, for this tenant

if I'm wrong about the above suggestion then this should just return a map so calling code doesn't have to know what a ListToMap is:

// SearchTagsAllowList is the list of tags to be extracted for search, for this tenant
func (o *Overrides) SearchTagsAllowList(userID string) map[string]struct{} {
	return &o.getOverridesForUser(userID).SearchTagsAllowList.GetMap()
}
annanay25

comment created time in 2 days

Pull request review commentgrafana/tempo

Add received and ingested bytes metrics in ingesters

 func (i *instance) Push(ctx context.Context, req *tempopb.PushRequest) error {  // PushBytes is used to push an unmarshalled tempopb.Trace to the instance func (i *instance) PushBytes(ctx context.Context, id []byte, traceBytes []byte, searchData []byte) error {+	// measure received bytes as sum of slice capacities+	// type byte is guaranteed to be 1 byte in size+	// ref: https://golang.org/ref/spec#Size_and_alignment_guarantees+	size := cap(id) + cap(traceBytes) + cap(searchData)

use len() (not cap) and don't count the id, just the traceBytes/searchData

mapno

comment created time in 2 days

PullRequestReviewEvent

Pull request review commentgrafana/tempo

Add received and ingested bytes metrics in ingesters

 * [ENHANCEMENT] Compression updates: Added s2, improved snappy performance [#961](https://github.com/grafana/tempo/pull/961) (@joe-elliott) * [ENHANCEMENT] Add search block headers [#943](https://github.com/grafana/tempo/pull/943) (@mdisibio) * [ENHANCEMENT] Add search block headers for wal blocks [#963](https://github.com/grafana/tempo/pull/963) (@mdisibio)+* [ENHANCEMENT] Add received bytes metric in ingesters [#979](https://github.com/grafana/tempo/pull/979) (@mapno)

Please put the new metric name in the changelog.

mapno

comment created time in 2 days

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentgrafana/tempo

Include simple e2e test to test searching

 func TestMicroservices(t *testing.T) { 	// query an in-memory trace 	queryAndAssertTrace(t, "http://"+tempoQueryFrontend.Endpoint(3200)+"/api/traces/"+hexID, "my operation", 1) +	// search an in-memory trace+	searchAndAssertTrace(t, "http://"+tempoQueryFrontend.Endpoint(3200), batch.Spans[0].Tags[0].Key, batch.Spans[0].Tags[0].GetVStr(), hexID)

the blind [0] deferences is kind of gorpy, but honestly this whole test is.

zalegrala

comment created time in 2 days

Pull request review commentgrafana/tempo

Include simple e2e test to test searching

 func makeThriftBatchWithSpanCount(n int) *thrift.Batch { 			Flags:         0, 			StartTime:     time.Now().Unix(), 			Duration:      1,-			Tags:          nil,-			Logs:          nil,+			Tags: []*thrift.Tag{+				{+					Key:  "x",+					VStr: stringPointer("y"),+				},+			},+			Logs: nil, 		}) 	} 	return &thrift.Batch{Spans: spans} } +func stringPointer(s string) *string {

this seems unnecessary. that has to be a better way to inline a string pointer than this

zalegrala

comment created time in 2 days

PullRequestReviewEvent

push eventgrafana/tempo

Zach Leslie

commit sha 2498d5be15d227a2094cabd625908cea5bece6d2

Add support for vulture sending long running traces (#951) * Add support for sending long running traces * Update changelog * Update test fixture and epoch * Refactor to clean up chance logic for long writes * Only query/search traces when we expect that all write have taken place

view details

push time in 2 days

PR merged grafana/tempo

Reviewers
Add support for vulture sending long running traces

What this PR does:

Here we implement the ability for the vulture to send long-running traces, which will enable the vulture verify additional infrastructure components required to resolve the trace.

Which issue(s) this PR fixes: Fixes #791

Checklist

  • [x] Tests updated
  • [ ] Documentation added
  • [x] CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]
+178 -88

2 comments

4 changed files

zalegrala

pr closed time in 2 days

issue closedgrafana/tempo

Extend vulture to generate long-running traces

Is your feature request related to a problem? Please describe. Vulture currently generates random traces and sends them in a single batch to Tempo. To improve our test coverage we'd also like to send long-running traces, i.e. traces that will span multiple blocks in Tempo.

The ingester considers a trace complete after a configurable idle period (by default 30s). If vulture can send spans with a longer delay between them they should end up in different blocks and Tempo will have to do additional work to combine all spans into a single trace when requested.

Describe the solution you'd like Vulture is currently stateless and ideally it stays stateless.

The following should probably be configurable:

  • a setting to enable/disable long-running traces
  • the delay between send requests

Describe alternatives you've considered

Additional context

closed time in 2 days

kvrhdn
PullRequestReviewEvent

push eventjoe-elliott/tempo

Joe Elliott

commit sha 5b282dc284ee36f5b773eba38912b45234985206

errcheck Signed-off-by: Joe Elliott <number101010@gmail.com>

view details

push time in 2 days

issue openedgrafana/tempo

Add WAL/AppendBlock Benchmarks

Is your feature request related to a problem? Please describe. The wal is not benchmarked. Currently we are using snappy b/c benchmarks on other workloads suggested it was the most performant. However, the wal has a strange workload of writing a large number of very small blocks of data. Let's do a little legwork to confirm that snappy is indeed best.

I'd also like to see some benchmarking around s2 with configuration parameters tuned for the WAL workload.

created time in 2 days

push eventjoe-elliott/tempo

Joe Elliott

commit sha 838dbe06076aa3933e41566c689ee0d5be8a5c44

comment Signed-off-by: Joe Elliott <number101010@gmail.com>

view details

Joe Elliott

commit sha 784843e4ca7fa39f2c6239c86f1d4342dad75259

s2 => snappy Signed-off-by: Joe Elliott <number101010@gmail.com>

view details

push time in 2 days