profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/annanay25/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Annanay Agarwal annanay25 Grafana Labs (@grafana) Bangalore twitter.com/mrannanay @grafana Tempo maintainer

annanay25/go-DHT 2

A DHT library based on BitTorrent and built on Go.

annanay25/analyse 1

Prediction model based on Collaborative Filtering.

annanay25/agent 0

Prometheus Metrics, Loki Logs, and Tempo Traces, optimized for Grafana Cloud.

annanay25/annanay25.github.io 0

Build a Jekyll blog in minutes, without touching the command line.

annanay25/atom 0

The hackable editor

annanay25/bitcoin 0

Bitcoin Core integration/staging tree

annanay25/blockchain 0

Implementation of a crypto currency

annanay25/caddy 0

Fast, cross-platform HTTP/2 web server with automatic HTTPS

annanay25/common 0

Libraries used in multiple Weave projects

annanay25/concurrency-control-algorithms 0

BOCC and FOCC implementations from concurrency control algorithms.

push eventgrafana/tempo

Annanay Agarwal

commit sha 1a46c46609e6e61a6915391a6701fd739e3505e0

Support global denylist and per-tenant allowlist of tags for search data (#960) * [search] Add global deny list of tags Signed-off-by: Annanay <annanayagarwal@gmail.com> * Move CHANGELOG entries around, add docs Signed-off-by: Annanay <annanayagarwal@gmail.com> * Fix entry Signed-off-by: Annanay <annanayagarwal@gmail.com> * Rework around adding a custom type and yaml/json marshaller Signed-off-by: Annanay <annanayagarwal@gmail.com> * Checkpoint: Implemented marshal funcs and added test Signed-off-by: Annanay <annanayagarwal@gmail.com> * Improve tests, handle edge cases Signed-off-by: Annanay <annanayagarwal@gmail.com> * Address comments Signed-off-by: Annanay <annanayagarwal@gmail.com> * Test denylist/allowlist in docker-compose tempo-search example Signed-off-by: Annanay <annanayagarwal@gmail.com> * Fix single-tenant overrides Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

push time in a day

PR merged grafana/tempo

Reviewers
Support global denylist and per-tenant allowlist of tags for search data

Signed-off-by: Annanay annanayagarwal@gmail.com

<!-- Thanks for sending a pull request! Before submitting:

  1. Read our CONTRIBUTING.md guide
  2. Rebase your PR if it gets out of sync with main -->

What this PR does:

  • Support global denylist of tags to be dropped from search data. The denylist will be applied at the distributors and only filtered tags will be added to the flatbuffers.
  • Support per-tenant allowlist of tags that will override the global denylist.

Which issue(s) this PR fixes: Fixes part of #932

Checklist

  • [x] Tests updated
  • [x] Documentation added
  • [x] CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]
+366 -34

1 comment

14 changed files

annanay25

pr closed time in a day

PullRequestReviewEvent

push eventannanay25/tempo

Joe Elliott

commit sha b68d7d73ad884abdc03e9e7aa3e1f502c774b83b

[Tempo CLI] Additional Features (#972) * blockslurp test Signed-off-by: Joe Elliott <number101010@gmail.com> * blockslurp Signed-off-by: Joe Elliott <number101010@gmail.com> * pod Signed-off-by: Joe Elliott <number101010@gmail.com> * added slurp buffer Signed-off-by: Joe Elliott <number101010@gmail.com> * prefetch iter Signed-off-by: Joe Elliott <number101010@gmail.com> * queueimpl7 Signed-off-by: Joe Elliott <number101010@gmail.com> * ballast Signed-off-by: Joe Elliott <number101010@gmail.com> * megaslurp Signed-off-by: Joe Elliott <number101010@gmail.com> * remove megaslurp Signed-off-by: Joe Elliott <number101010@gmail.com> * dumbness Signed-off-by: Joe Elliott <number101010@gmail.com> * done with this garbage Signed-off-by: Joe Elliott <number101010@gmail.com> * mega deslurp, iteratorchannel Signed-off-by: Joe Elliott <number101010@gmail.com> * added parse Signed-off-by: Joe Elliott <number101010@gmail.com> * increasing chunk sizes Signed-off-by: Joe Elliott <number101010@gmail.com> * increase base chunk Signed-off-by: Joe Elliott <number101010@gmail.com> * todo Signed-off-by: Joe Elliott <number101010@gmail.com> * add Signed-off-by: Joe Elliott <number101010@gmail.com> * cli ho! Signed-off-by: Joe Elliott <number101010@gmail.com> * Search technically works Signed-off-by: Joe Elliott <number101010@gmail.com> * revert Signed-off-by: Joe Elliott <number101010@gmail.com> * todos Signed-off-by: Joe Elliott <number101010@gmail.com> * layouts Signed-off-by: Joe Elliott <number101010@gmail.com> * todo Signed-off-by: Joe Elliott <number101010@gmail.com> * Added label scan Signed-off-by: Joe Elliott <number101010@gmail.com> * docs Signed-off-by: Joe Elliott <number101010@gmail.com> * lint Signed-off-by: Joe Elliott <number101010@gmail.com> * changelog Signed-off-by: Joe Elliott <number101010@gmail.com> * fixed datarace Signed-off-by: Joe Elliott <number101010@gmail.com> * added max/min objid Signed-off-by: Joe Elliott <number101010@gmail.com>

view details

Zach Leslie

commit sha 2498d5be15d227a2094cabd625908cea5bece6d2

Add support for vulture sending long running traces (#951) * Add support for sending long running traces * Update changelog * Update test fixture and epoch * Refactor to clean up chance logic for long writes * Only query/search traces when we expect that all write have taken place

view details

Annanay

commit sha 6b2a3ecc8ebe6d51ce5b4fa7aa565f103c090e20

Merge branch 'main' into search-tag-denylist Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

push time in a day

pull request commentgrafana/tempo

Support global denylist and per-tenant allowlist of tags for search data

Thanks @joe-elliott, addressed both.

annanay25

comment created time in a day

push eventannanay25/tempo

Annanay

commit sha b06385a5e025d84a5d0aea367eb1309e100bc671

Address comments Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

Annanay

commit sha d3745b21c718332f608321ea2b79afe71254dbb2

Test denylist/allowlist in docker-compose tempo-search example Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

Annanay

commit sha a87ae2042eedcaf43507f67184ce3abd85fb7f51

Fix single-tenant overrides Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

push time in a day

issue commentgrafana/tempo

No objects neither traces see in the s3 bucket with single binary helm deployment.

Hi @sunnynehar56, since the WAL folder is empty and enabling log_received_traces is not logging anything, it looks like Tempo is unable to receive any traces at all.

Which client are you using to send traces to Tempo? And what URL is configured on that app to report traces?

Please check this troubleshooting document detailing some remediation methods: https://grafana.com/docs/tempo/latest/troubleshooting/unable-to-see-trace/

If that does not work please get back and we will assist you with further debugging.

sunnynehar56

comment created time in a day

PullRequestReviewEvent

Pull request review commentgrafana/tempo

Include simple e2e test to test searching

 func TestAllInOne(t *testing.T) { 			require.NoError(t, tempo.WaitSumMetrics(cortex_e2e.Equals(1), "tempo_ingester_traces_created_total"))  			// query an in-memory trace-			queryAndAssertTrace(t, "http://"+tempo.Endpoint(3200)+"/api/traces/"+hexID, "my operation", 1)+			apiClient := tempoUtil.NewClient("http://"+tempo.Endpoint(3200), "")+			queryAndAssertTrace(t, apiClient, hexID, "my operation", 1)

Add tag search here as well?

zalegrala

comment created time in a day

PullRequestReviewEvent

push eventannanay25/tempo

Annanay

commit sha 0266c3990c1e770ca6a4f52155f0cab731e0c3e3

Improve tests, handle edge cases Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

push time in 2 days

issue openedgrafana/tempo

Compactors require high memory as traces combine and grow in the backend

Describe the bug <!-- A clear and concise description of what the bug is.--> Compactors require high memory as traces combine and grow in the backend. This opens the possibility of crafting a long running trace with low spans-per-second that can grow and eventually OOM compactors.

To Reproduce Steps to reproduce the behavior:

  1. Start Tempo (SHA or version): all versions till e5f7ded
  2. Perform Operations (Read/Write/Others): Carefully craft super-long running traces with a few spans every second that will eventually be combined by the compactors into a MEGA trace (the largest we are seeing so far are 1.3GB)

Expected behavior <!-- A clear and concise description of what you expected to happen. --> Compactors do not keep OOMing.

Environment:

  • Infrastructure: [e.g., Kubernetes, bare-metal, laptop]
  • Deployment tool: [e.g., helm, jsonnet]

Additional Context <!-- Additional relevant info which can help us debug this issue easily like Logs, Configuration etc. --> Some possibilities considered:

  • write multiple splits of a trace into the same block (might be harder than it sounds)
  • Do not compact blocks that have very large traces

created time in 2 days

PullRequestReviewEvent

Pull request review commentgrafana/tempo

Set span.kind to client in query-frontend

 func (r retryWare) Do(req *http.Request) (*http.Response, error) { 	ctx := req.Context() 	span, ctx := opentracing.StartSpanFromContext(ctx, "frontend.Retry") 	defer span.Finish()+	span.SetTag("span.kind", "client")

I think it might be better to use the opentracing-go/ext which has the right set of tags to add Ex: https://github.com/weaveworks/common/blob/284892f3c53779d1a5956460b7fc8843b02ae7a9/instrument/instrument.go#L157

mapno

comment created time in 2 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventannanay25/tempo

Annanay Agarwal

commit sha e5f7dedd96c9ee9f0dda73ff65c9e217ea7f0f65

Patch WAL dedupe when search is disabled (#968) * Patch WAL dedupe when search is disabled Signed-off-by: Annanay <annanayagarwal@gmail.com> * Lint Signed-off-by: Annanay <annanayagarwal@gmail.com> * Chanelog Signed-off-by: Annanay <annanayagarwal@gmail.com> * Add comment around guard code Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

Annanay

commit sha 5a1b8bdb04948119b55f2713135093e2a9c2d0de

Merge branch 'main' into search-tag-denylist Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

push time in 2 days

push eventannanay25/tempo

Annanay

commit sha c97995c44fb574f569eb22312cb3628016a61e73

Checkpoint: Implemented marshal funcs and added test Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

push time in 2 days

PullRequestReviewEvent

Pull request review commentgrafana/tempo

[Tempo CLI] Additional Features

 func dumpBlock(r tempodb_backend.Reader, c tempodb_backend.Compactor, tenantID s 		maxObjSize := 0 		minObjSize := 0 +		allKVP := kvPairs{} 		printStats := func() { 			fmt.Println() 			fmt.Println("Scanning results:") 			fmt.Println("Objects scanned : ", i) 			fmt.Println("Duplicates      : ", dupe) 			fmt.Println("Smallest object : ", humanize.Bytes(uint64(minObjSize))) 			fmt.Println("Largest object  : ", humanize.Bytes(uint64(maxObjSize)))

While you're in here - can we print the ID for the largest object? Currently trying to debug some OOMing compactors and it would help to know if they are trying to combine a super large trace across multiple blocks

joe-elliott

comment created time in 2 days

PullRequestReviewEvent

issue commentgrafana/tempo

Index out of range at `completing block` log message

Thanks for reporting this @zalegrala, caught an evasive bug!

zalegrala

comment created time in 3 days

push eventgrafana/tempo

Annanay Agarwal

commit sha e5f7dedd96c9ee9f0dda73ff65c9e217ea7f0f65

Patch WAL dedupe when search is disabled (#968) * Patch WAL dedupe when search is disabled Signed-off-by: Annanay <annanayagarwal@gmail.com> * Lint Signed-off-by: Annanay <annanayagarwal@gmail.com> * Chanelog Signed-off-by: Annanay <annanayagarwal@gmail.com> * Add comment around guard code Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

push time in 3 days

PR merged grafana/tempo

Reviewers
Patch WAL dedupe when search is disabled

<!-- Thanks for sending a pull request! Before submitting:

  1. Read our CONTRIBUTING.md guide
  2. Rebase your PR if it gets out of sync with main -->

What this PR does: Fixes an issue with WAL dedupe (SearchDataCombiner) when search is disabled. This happens because we still create search WAL files on the ingester with empty entries.

Which issue(s) this PR fixes: Fixes #967

Checklist

  • [x] Tests updated
  • [ ] Documentation added
  • [x] CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]
+84 -1

2 comments

4 changed files

annanay25

pr closed time in 3 days

issue closedgrafana/tempo

Index out of range at `completing block` log message

Describe the bug

It appears that when a block is completing, we have an index out of range panic.

level=info ts=2021-09-17T15:08:48.719429979Z caller=compactor.go:104 msg="enabling compaction"
level=info ts=2021-09-17T15:08:48.719564092Z caller=tempodb.go:368 msg="compaction and retention enabled."
level=info ts=2021-09-17T15:09:50.54346294Z caller=frontend.go:109 tenant=single-tenant method=GET traceID=49c4f409f3d3da11 url=/api/traces/5cfd75829f7636034774f39fd6294859 duration=58.658035ms response_size=12225 status=200
level=info ts=2021-09-17T15:10:20.505573808Z caller=frontend.go:109 tenant=single-tenant method=GET traceID=34b75bfdabe97a08 url=/api/traces/2640357a799d655265fe5791c5734dbb duration=23.859058ms response_size=7949 status=200
level=info ts=2021-09-17T15:10:50.522667639Z caller=frontend.go:109 tenant=single-tenant method=GET traceID=528a543f686516df url=/api/traces/0c360f4ee129bece3f5a3fab9884c732 duration=42.164202ms response_size=19854 status=200
level=info ts=2021-09-17T15:11:08.726815404Z caller=flush.go:157 msg="head block cut. enqueueing flush op" userid=single-tenant block=5ec589ad-0528-4b9c-84bd-13f99a994b5f
level=info ts=2021-09-17T15:11:17.624818984Z caller=flush.go:232 msg="completing block" userid=single-tenant blockID=5ec589ad-0528-4b9c-84bd-13f99a994b5f
panic: runtime error: index out of range [3] with length 0

goroutine 473 [running]:
github.com/google/flatbuffers/go.GetUint32(...)
	/home/zach/go/src/github.com/grafana/tempo/vendor/github.com/google/flatbuffers/go/encode.go:47
github.com/google/flatbuffers/go.GetUOffsetT(...)
	/home/zach/go/src/github.com/grafana/tempo/vendor/github.com/google/flatbuffers/go/encode.go:121
github.com/grafana/tempo/pkg/tempofb.GetRootAsSearchEntry(...)
	/home/zach/go/src/github.com/grafana/tempo/pkg/tempofb/SearchEntry.go:14
github.com/grafana/tempo/pkg/tempofb.SearchEntryFromBytes(...)
	/home/zach/go/src/github.com/grafana/tempo/pkg/tempofb/searchdata_util.go:257
github.com/grafana/tempo/tempodb/search.(*DataCombiner).Combine(0xc00257b230, {0x28, 0x266f400}, {0xc00257b260, 0x2, 0x185d9df})
	/home/zach/go/src/github.com/grafana/tempo/tempodb/search/data_combiner.go:28 +0x3a6
github.com/grafana/tempo/tempodb/encoding.(*dedupingIterator).Next(0xc0022bc660, {0x26b7098, 0xc000052038})
	/home/zach/go/src/github.com/grafana/tempo/tempodb/encoding/iterator_deduping.go:61 +0x237
github.com/grafana/tempo/tempodb/search.NewBackendSearchBlock(0xc0004fef60, 0xc00013c4d0, {0x5e, 0xc5, 0x89, 0xad, 0x5, 0x28, 0x4b, 0x9c, ...}, ...)
	/home/zach/go/src/github.com/grafana/tempo/tempodb/search/backend_search_block.go:64 +0x2f1
github.com/grafana/tempo/modules/ingester.(*instance).CompleteBlock(0xc0018b4480, {0x5e, 0xc5, 0x89, 0xad, 0x5, 0x28, 0x4b, 0x9c, 0x84, ...})
	/home/zach/go/src/github.com/grafana/tempo/modules/ingester/instance.go:272 +0x269
github.com/grafana/tempo/modules/ingester.(*Ingester).handleComplete(0xc0004c2c00, 0xc0017f7540)
	/home/zach/go/src/github.com/grafana/tempo/modules/ingester/flush.go:238 +0x25c
github.com/grafana/tempo/modules/ingester.(*Ingester).flushLoop(0xc0004c2c00, 0x1)
	/home/zach/go/src/github.com/grafana/tempo/modules/ingester/flush.go:195 +0xc7
created by github.com/grafana/tempo/modules/ingester.New
	/home/zach/go/src/github.com/grafana/tempo/modules/ingester/ingester.go:78 +0x1e9

To Reproduce Steps to reproduce the behavior:

  1. Start Tempo 3108978d configured with memberlist and s3 backend pointed to minio
  2. Start Vulture to send and query traces
  3. Wait for completing block to happen

Expected behavior Not panic.

Environment:

  • Infrastructure: laptop
  • Deployment tool: docker-compose

Additional Context

closed time in 3 days

zalegrala

push eventannanay25/tempo

Annanay

commit sha 84f3f5935e48decea9a6e1e40da17d07565a280d

Add comment around guard code Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

push time in 3 days

issue openedgrafana/tempo

Add configuration documentation for overrides module

The configuration page (https://grafana.com/docs/tempo/latest/configuration/) needs a section on the overrides module and the various per-tenant override settings.

created time in 3 days

push eventannanay25/tempo

Joe Elliott

commit sha e976a081ade98e54437e2f2b4881570131909027

Zstd decoder (#948) * blerg Signed-off-by: Joe Elliott <number101010@gmail.com> sadness Signed-off-by: Joe Elliott <number101010@gmail.com> removed encoding Signed-off-by: Joe Elliott <number101010@gmail.com> duh Signed-off-by: Joe Elliott <number101010@gmail.com> * updated compression library Signed-off-by: Joe Elliott <number101010@gmail.com> * different horribleness Signed-off-by: Joe Elliott <number101010@gmail.com> * klauspost Signed-off-by: Joe Elliott <number101010@gmail.com> * cgo zstd * Revert "cgo zstd" This reverts commit 04bae4ad0497c981b318a84dada5fc73cf1bc6e0. * http conn mucking Signed-off-by: Joe Elliott <number101010@gmail.com> * encoder Signed-off-by: Joe Elliott <number101010@gmail.com> * Revert "http conn mucking" This reverts commit 4b1e3cd870e3a628539ce4f051f9b617e5c705f3. * Revert "encoder" This reverts commit b081970a9ba9403a38f6952daf2cc3c82a9656f6. * comments Signed-off-by: Joe Elliott <number101010@gmail.com> * changelog Signed-off-by: Joe Elliott <number101010@gmail.com> * remove test code Signed-off-by: Joe Elliott <number101010@gmail.com> * todo Signed-off-by: Joe Elliott <number101010@gmail.com>

view details

Joe Elliott

commit sha b89f2ae1269095f7601e5848d89cd224ac1b5778

Compression Updates (#961) * fixed write benchmark Signed-off-by: Joe Elliott <number101010@gmail.com> * benchmarks work Signed-off-by: Joe Elliott <number101010@gmail.com> * klauspost snappy Signed-off-by: Joe Elliott <number101010@gmail.com> * add s2 Signed-off-by: Joe Elliott <number101010@gmail.com> * disable compression Signed-off-by: Joe Elliott <number101010@gmail.com> * Revert "disable compression" This reverts commit b68b0380d87239114b245bcec94c2cb9bfb1aa92. * removed jpes Signed-off-by: Joe Elliott <number101010@gmail.com> * docs/changelog Signed-off-by: Joe Elliott <number101010@gmail.com> * vendor-check Signed-off-by: Joe Elliott <number101010@gmail.com> * fixed test Signed-off-by: Joe Elliott <number101010@gmail.com> * maxEncoding? Signed-off-by: Joe Elliott <number101010@gmail.com>

view details

Zach Leslie

commit sha 3108978d915c2cbd09f0debb56dfe80600fe6789

Drop incorrect vulture flag from operations manifests (#962)

view details

Nick Pillitteri

commit sha 4def3a089772123e03630743dbc4ae4f0a588a50

Update to latest dskit to remove Etcd server dependency (#969) Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>

view details

Martin Disibio

commit sha af0f76f8aee206c0e344ccb750ec81e2d82312d1

Add search block headers for wal blocks (#963) * Add search block headers for wal blocks Signed-off-by: Martin Disibio <mdisibio@gmail.com> * changelog Signed-off-by: Martin Disibio <mdisibio@gmail.com>

view details

Annanay

commit sha a8f1d7d60d1512325b3fe3d28321fb6a1f05cb4d

Merge branch 'main' into search-tag-denylist Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

push time in 3 days

push eventannanay25/tempo

Annanay

commit sha 7409d4bb05d4531597a6fd0448ef1741a4d0f54c

Rework around adding a custom type and yaml/json marshaller Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

push time in 3 days

push eventgrafana/tempo

Martin Disibio

commit sha af0f76f8aee206c0e344ccb750ec81e2d82312d1

Add search block headers for wal blocks (#963) * Add search block headers for wal blocks Signed-off-by: Martin Disibio <mdisibio@gmail.com> * changelog Signed-off-by: Martin Disibio <mdisibio@gmail.com>

view details

push time in 3 days