profile
viewpoint

grafana/loki 9527

Like Prometheus, but for logs.

slim-bean/leafbus 13

capturing data from a nissan leaf

slim-bean/cortex 1

A horizontally scalable, highly available, multi-tenant, long term Prometheus.

slim-bean/grafana 1

The tool for beautiful monitoring and metric analytics & dashboards for Graphite, InfluxDB & Prometheus & More

slim-bean/grafana-tail-datasource 1

Tail files into grafana

slim-bean/jsonnet-bundler 1

A jsonnet package manager.

geowa4/learn-loki 0

Learning Loki with Osquery

grafana/loki-canary 0

Writes logs and then reads them back from loki to create metrics/alerts when log files go missing

push eventgrafana/loki

cyriltovena

commit sha 0b7f0b259e3349b0bd437881e6470f8e24731181

[skip ci] Publishing helm charts: 5e0fe09f697e7eda0a2fea451eb72c80f7643500

view details

push time in an hour

push eventgrafana/loki

beorn7

commit sha 550af159c4f760d61b3ceb68b386a0f74914935d

[skip ci] Publishing helm charts: a5aa34137a48b69ec2b005719e134fbeb68af6ec

view details

push time in 4 hours

push eventgrafana/loki

owen-d

commit sha 7260b09e57f38ce1c3026dbccdfad64c8fa72307

[skip ci] Publishing helm charts: 89d80a6fc469daba5302bdf8f58dfc40595c4422

view details

push time in 17 hours

push eventgrafana/loki

cyriltovena

commit sha 3bce9eec31da4c49746a1eee6a0099c469b9e79e

[skip ci] Publishing helm charts: 156023ae9abc4acbdbd5d6e32dc9146b42700827

view details

push time in 18 hours

push eventgrafana/loki

cyriltovena

commit sha cce55d52debbd7b43bdb3b1f18b8e9a0e722dd9c

[skip ci] Publishing helm charts: 665712ff181db865bed5b1c79170e667fb4a4cc5

view details

push time in 18 hours

push eventgrafana/loki

cyriltovena

commit sha d59e3f5d81d2c8cc87aaa1b4543b6a8d86270c5d

[skip ci] Publishing helm charts: d175fa58c881f23a3ad71d508cc2e7db05937dc8

view details

push time in 18 hours

push eventgrafana/loki

cyriltovena

commit sha fe4b886035d1230cba9c2a2252a5916ad3d5520a

[skip ci] Publishing helm charts: b9f276cfed85a8c569cc58f46fd2127756df1ea0

view details

push time in 18 hours

push eventgrafana/loki

cyriltovena

commit sha 168a36802b7bbed49e3b5e075a5d19674cfdc1c9

[skip ci] Publishing helm charts: ff0a4c35f4103803682f105adb28b834f3dff88a

view details

push time in 18 hours

push eventgrafana/loki

cyriltovena

commit sha c77abdb79cf15b23a6898bb0b2e91af14c00fe45

[skip ci] Publishing helm charts: 8cbeb53cf12581ea02038a87b2bad1ac0d071b42

view details

push time in 2 days

push eventgrafana/loki

cyriltovena

commit sha 0b3d3fccc3466f2a3a645d4ec433d72db2632cd4

[skip ci] Publishing helm charts: a0546a3bc33b0c6bf0a1b2eef1091b21cf6b2f82

view details

push time in 2 days

push eventgrafana/loki

rfratto

commit sha 4aa531ccfdb595b3d1e311112444d3e650bfdafb

[skip ci] Publishing helm charts: f984e71192d41723c16f01c83f04d191b0bb2a8c

view details

push time in 5 days

issue commentgrafana/loki

loki 1.5.0 failed parsing config: not a valid duration string: "0"

Thanks for reporting this though @adacaccia I suspect many others will run into this too. I will update the upgrade guide release notes to mention this

adacaccia

comment created time in 5 days

Pull request review commentcortexproject/cortex

Fix remote read route registration

 func (a *API) RegisterQuerier(queryable storage.Queryable, engine *promql.Engine 	cacheGenHeaderMiddleware := getHTTPCacheGenNumberHeaderSetterMiddleware(tombstonesLoader) 	promHandler := fakeRemoteAddr(cacheGenHeaderMiddleware.Wrap(promRouter)) -	a.registerRouteWithRouter(router, a.cfg.PrometheusHTTPPrefix+"/api/v1/read", querier.RemoteReadHandler(queryable), true, "GET")+	a.registerRouteWithRouter(router, a.cfg.PrometheusHTTPPrefix+"/api/v1/read", querier.RemoteReadHandler(queryable), true, "POST")

is there any harm in supporting both GET and POST?

jtlisi

comment created time in 6 days

push eventgrafana/loki

cyriltovena

commit sha b5575e45572c3a9ac407aa16158e7f3158f6e760

[skip ci] Publishing helm charts: 360d9a7f97c30031733190a9c52b7ba43381ebd5

view details

push time in 6 days

push eventgrafana/loki

slim-bean

commit sha dd03839bc0da8e152edfaadd7f7310dbfe291abf

[skip ci] Publishing helm charts: 26a4c53cbd1e172ce8441574f6f3fc32fd290ed1

view details

push time in 7 days

push eventgrafana/loki

Ed Welch

commit sha 26a4c53cbd1e172ce8441574f6f3fc32fd290ed1

updating all the images to 1.5.0 release (#2100) * updating all the images to 1.5.0 release Signed-off-by: Ed Welch <edward.welch@grafana.com> * duration must have a unit Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 7 days

delete branch grafana/loki

delete branch : update-1.5.0-release

delete time in 7 days

PR merged grafana/loki

updating all the images to 1.5.0 release size/M

Signed-off-by: Ed Welch edward.welch@grafana.com

+19 -19

1 comment

10 changed files

slim-bean

pr closed time in 7 days

push eventgrafana/loki

Ed Welch

commit sha acaaa38c2af02195c46f8098485955ffc4849822

duration must have a unit Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 7 days

PR opened grafana/loki

updating all the images to 1.5.0 release

Signed-off-by: Ed Welch edward.welch@grafana.com

+18 -18

0 comment

10 changed files

pr created time in 7 days

create barnchgrafana/loki

branch : update-1.5.0-release

created branch time in 7 days

push eventgrafana/loki

slim-bean

commit sha eb37db2ec9f8535e47566ef512b57e456638cf7c

[skip ci] Publishing helm charts: 12c7eab8bb94fd82b184c1c222200e37f2ca050a

view details

push time in 7 days

created taggrafana/loki

tagv1.5.0

Like Prometheus, but for logs.

created time in 7 days

push eventgrafana/loki

slim-bean

commit sha bdfee47f6cbe38582b7b1e4aeabdfe7cfd3538f8

[skip ci] Publishing helm charts: 12c7eab8bb94fd82b184c1c222200e37f2ca050a

view details

push time in 7 days

push eventgrafana/loki

Ed Welch

commit sha 12c7eab8bb94fd82b184c1c222200e37f2ca050a

Prep 1.5.0 release (#2098) * Updating the Changelog and Upgrade guide for 1.5.0 release. Signed-off-by: Ed Welch <edward.welch@grafana.com> * Changing release number in all the docs Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 7 days

delete branch grafana/loki

delete branch : release-v1.5.0

delete time in 7 days

PR merged grafana/loki

Prep 1.5.0 release size/L

Signed-off-by: Ed Welch edward.welch@grafana.com

+369 -29

1 comment

7 changed files

slim-bean

pr closed time in 7 days

push eventgrafana/loki

Ed Welch

commit sha 0963b6ec31d35d1a50482fcc3558b236af201403

Changing release number in all the docs Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 7 days

Pull request review commentgrafana/loki

Prep 1.5.0 release

+## 1.5.0 (2020-05-20)++It's been a busy month and a half since 1.4.0 was released, a lot has happened!++Be prepared for some configuration changes that may cause some bumps when upgrading, +we apologize for this but are always striving to reach the right compromise of code simplicity and user/operating experience. ++In this case we opted to keep a simplified configuration inline with Cortex rather than a more complicated and error prone internal config mapping or difficult to implement support for multiple config names for the same feature.++This does result in breaking config changes for some configurations, however, these will fail fast and with the [list of diffs](https://cortexmetrics.io/docs/changelog/#config-file-breaking-changes) from the Cortex project should be quick to fix.++### Important Notes++**Be prepared for breaking config changes.**  Loki 1.5.0 vendors cortex [v1.0.1-0.20200430170006-3462eb63f324](https://github.com/cortexproject/cortex/commit/3462eb63f324c649bbaa122933bc591b710f4e48), +there were substantial breaking config changes in Cortex 1.0 which standardized config options, and fixed typos.++**The Loki docker image user has changed to no longer be root**++Check the [upgrade guide](https://github.com/grafana/loki/blob/master/docs/operations/upgrade.md#150) for more detailed information on these changes.++### Notable Features and Fixes++There are quite a few we want to mention listed in order they were merged (mostly)++* [1837](https://github.com/grafana/loki/pull/1837) **sandeepsukhani**: flush boltdb to object store++This is perhaps the most exciting feature of 1.5.0, the first steps in removing a dependency on a separate index store!  This feature is still very new and experimental, however, we want this to be the future for Loki.  Only requiring just an object store.++If you want to test this new feature, and help us find any bugs, check out the [docs](docs/operations/storage/boltdb-shipper.md) to learn more and get started.++* [2073](https://github.com/grafana/loki/pull/2073) **slim-bean**: Loki: Allow configuring query_store_max_look_back_period when running a filesystem store and boltdb-shipper++This is even more experimental than the previous feature mentioned however also pretty exciting for Loki users who use the filesystem storage. We can leverage changes made in [1837](https://github.com/grafana/loki/pull/1837) to now allow Loki to run in a clustered mode with individual filesystem stores!++Please check out the last section in the [filesystem docs](docs/operations/storage/filesystem.md) for more details on how this works and how to use it!++* [2095](https://github.com/grafana/loki/pull/2095) **cyriltovena**: Adds backtick for the quoted string token lexer.++This will come as a big win to anyone who is writing complicated reqular expressions in either their Label matchers or Filter Expressions.  Starting now you can use the backtick to encapsulate your regex **and not have to do any escaping of special characters!!**++Examples:++```+{name="cassandra"} |~ `error=\w+`+{name!~`mysql-\d+`}+```++* [2055](https://github.com/grafana/loki/pull/2055) **aknuds1**: Chore: Fix spelling of per second in code++This is technically a breaking change for anyone who wrote code to processes the new statistics output in the query result added in 1.4.0, we apologize to anyone in this situation but if we don't fix this kind of error now it will be there forever.+And at the same time we didn't feel it was appropriate to make any major api revision changes for such a new feature and simple change.  We are always trying to use our best judgement in cases like this.++* [2031](https://github.com/grafana/loki/pull/2031) **cyriltovena**: Improve protobuf serialization++Thanks @cyriltovena for another big performance improvement in Loki, this time around protbuf's!++* [2021](https://github.com/grafana/loki/pull/2021) **slim-bean**: Loki: refactor validation and improve error messages+* [2012](https://github.com/grafana/loki/pull/2012) **slim-bean**: Loki: Improve logging and add metrics to streams dropped by stream limit++These two changes standardize the metrics used to report when a tenant hits a limit, now all discarded samples should be reported under `loki_discarded_samples_total` and you no longer need to also reference `cortex_discarded_samples_total`.+Additionally error messages were improved to help clients take better action when hitting limits.++** [1970](https://github.com/grafana/loki/pull/1970) **cyriltovena**: Allow to aggregate binary operations.++Another nice improvement to the query language which allows queries like this to work now:++```+sum by (job) (count_over_time({namespace="tns"}[5m] |= "level=error") / count_over_time({namespace="tns"}[5m]))+```++** [1713](https://github.com/grafana/loki/pull/1713) **adityacs**: Log error message for invalid checksum++In the event something went wrong with a stored chunk, rather than fail the query we ignore the chunk and return the rest.++** [2066](https://github.com/grafana/loki/pull/2066) **slim-bean**: Promtail: metrics stage can also count line bytes++This is a nice extension to a previous feature which let you add a metric to count log lines per stream, you can now count log bytes per stream.++Check out [this example](docs/clients/promtail/configuration.md#counter) to configure this in your promtail pipelines.++** [1935](https://github.com/grafana/loki/pull/1935) **cyriltovena**: Support stdin target via flag instead of automatic detection.++Third times a charm!  With 1.4.0 we allowed sending logs directly to promtail via stdin, with 1.4.1 we released a patch for this feature which wasn't detecting stdin correctly on some operating systems.+Unfortunately after a few more bug reports it seems this change caused some more undesired side effects so we decided to not try to autodetect stdin at all, instead now you must pass the `--stdin` flag if you want Promtail to listen for logs on stdin.++** [2076](https://github.com/grafana/loki/pull/2076) **cyriltovena**: Allows to pass inlined pipeline stages to the docker driver.++It's always been challenging to configure pipeline stages for the docker driver, with this PR there are now a few easier ways to do this!++** [1864](https://github.com/grafana/loki/pull/1864) **cyriltovena**: Sign helm package with GPG.++We now GPG sign helm packages!++### All Changes++#### Loki++* [2097](https://github.com/grafana/loki/pull/2097) **owen-d**: simplifies/updates some of our configuration examples+* [2095](https://github.com/grafana/loki/pull/2095) **cyriltovena**: Adds backtick for the quoted string token lexer.+* [2093](https://github.com/grafana/loki/pull/2093) **cyriltovena**: Fixes unit in stats request log.+* [2088](https://github.com/grafana/loki/pull/2088) **slim-bean**: Loki: allow no encoding/compression on chunks+* [2078](https://github.com/grafana/loki/pull/2078) **owen-d**: removes yolostring +* [2073](https://github.com/grafana/loki/pull/2073) **slim-bean**: Loki: Allow configuring query_store_max_look_back_period when running a filesystem store and boltdb-shipper+* [2064](https://github.com/grafana/loki/pull/2064) **cyriltovena**: Reverse entry iterator pool+* [2059](https://github.com/grafana/loki/pull/2059) **cyriltovena**: Recover from panic in http and grpc handlers.+* [2058](https://github.com/grafana/loki/pull/2058) **cyriltovena**: Fix a bug in range vector skipping data.+* [2055](https://github.com/grafana/loki/pull/2055) **aknuds1**: Chore: Fix spelling of per second in code+* [2046](https://github.com/grafana/loki/pull/2046) **gouthamve**: Fix bug in logql parsing that leads to crash.+* [2050](https://github.com/grafana/loki/pull/2050) **aknuds1**: Chore: Correct typo "per seconds"+* [2034](https://github.com/grafana/loki/pull/2034) **sandeepsukhani**: some metrics for measuring performance and failures in boltdb shipper+* [2031](https://github.com/grafana/loki/pull/2031) **cyriltovena**: Improve protobuf serialization+* [2030](https://github.com/grafana/loki/pull/2030) **adityacs**: Update loki to cortex master+* [2023](https://github.com/grafana/loki/pull/2023) **cyriltovena**: Support post requests in the frontend queryrange handler.+* [2021](https://github.com/grafana/loki/pull/2021) **slim-bean**: Loki: refactor validation and improve error messages+* [2019](https://github.com/grafana/loki/pull/2019) **slim-bean**: make `loki_ingester_memory_streams` Gauge per tenant.+* [2012](https://github.com/grafana/loki/pull/2012) **slim-bean**: Loki: Improve logging and add metrics to streams dropped by stream limit+* [2010](https://github.com/grafana/loki/pull/2010) **cyriltovena**: Update lz4 library to latest to ensure deterministic output.+* [2001](https://github.com/grafana/loki/pull/2001) **sandeepsukhani**: table client for boltdb shipper to enforce retention+* [1995](https://github.com/grafana/loki/pull/1995) **sandeepsukhani**: make boltdb shipper singleton and some other minor refactoring+* [1987](https://github.com/grafana/loki/pull/1987) **slim-bean**: Loki: Add a missing method to facade which is called by the metrics storage client in cortex+* [1982](https://github.com/grafana/loki/pull/1982) **cyriltovena**: Update cortex to latest.+* [1977](https://github.com/grafana/loki/pull/1977) **cyriltovena**: Ensure trace propagation in our logs.+* [1976](https://github.com/grafana/loki/pull/1976) **slim-bean**: incorporate some better defaults into table-manager configs+* [1975](https://github.com/grafana/loki/pull/1975) **slim-bean**: Update cortex vendoring to latest master+* [1970](https://github.com/grafana/loki/pull/1970) **cyriltovena**: Allow to aggregate binary operations.+* [1965](https://github.com/grafana/loki/pull/1965) **slim-bean**: Loki: Adds an `interval` paramater to query_range queries allowing a sampling of events to be returned based on the provided interval+* [1964](https://github.com/grafana/loki/pull/1964) **owen-d**: chunk bounds metric now records 8h range in 1h increments+* [1963](https://github.com/grafana/loki/pull/1963) **cyriltovena**: Improve the local config to work locally and inside docker.+* [1961](https://github.com/grafana/loki/pull/1961) **jpmcb**: [Bug] Workaround for broken etcd gomod import+* [1958](https://github.com/grafana/loki/pull/1958) **owen-d**: chunk lifespan histogram+* [1956](https://github.com/grafana/loki/pull/1956) **sandeepsukhani**: update cortex to latest master+* [1953](https://github.com/grafana/loki/pull/1953) **jpmcb**: Go mod: explicit golang.org/x/net replace+* [1950](https://github.com/grafana/loki/pull/1950) **cyriltovena**: Fixes case handling in regex simplification.+* [1949](https://github.com/grafana/loki/pull/1949) **SerialVelocity**: [Loki]: Cleanup dockerfile+* [1946](https://github.com/grafana/loki/pull/1946) **slim-bean**: Loki Update the cut block size counter when creating a memchunk from byte slice+* [1939](https://github.com/grafana/loki/pull/1939) **owen-d**: adds config validation, similar to cortex+* [1916](https://github.com/grafana/loki/pull/1916) **cyriltovena**: Add cap_net_bind_service linux capabilities to Loki.+* [1914](https://github.com/grafana/loki/pull/1914) **owen-d**: only fetches one chunk per series in /series+* [1875](https://github.com/grafana/loki/pull/1875) **owen-d**: support `match[]` encoding+* [1869](https://github.com/grafana/loki/pull/1869) **pstibrany**: Update Cortex to latest master+* [1846](https://github.com/grafana/loki/pull/1846) **owen-d**: Sharding optimizations I: AST mapping+* [1838](https://github.com/grafana/loki/pull/1838) **cyriltovena**: Move default port for Loki to 3100 everywhere.+* [1837](https://github.com/grafana/loki/pull/1837) **sandeepsukhani**: flush boltdb to object store+* [1834](https://github.com/grafana/loki/pull/1834) **Mario-Hofstaetter**: Loki/Change local storage directory to /loki/ and fix permissions (#1833)+* [1819](https://github.com/grafana/loki/pull/1819) **cyriltovena**: Adds a counter for total flushed chunks per reason.+* [1816](https://github.com/grafana/loki/pull/1816) **sdojjy**: loki can not be started with loki-local-config.yaml+* [1810](https://github.com/grafana/loki/pull/1810) **cyriltovena**: Optimize empty filter queries.+* [1809](https://github.com/grafana/loki/pull/1809) **cyriltovena**: Test stats memchunk+* [1804](https://github.com/grafana/loki/pull/1804) **pstibrany**: Convert Loki modules to services+* [1799](https://github.com/grafana/loki/pull/1799) **pstibrany**: loki: update Cortex to master+* [1798](https://github.com/grafana/loki/pull/1798) **adityacs**: Support configurable maximum of the limits parameter+* [1713](https://github.com/grafana/loki/pull/1713) **adityacs**: Log error message for invalid checksum+* [1706](https://github.com/grafana/loki/pull/1706) **cyriltovena**: Non-root user docker image for Loki.++#### Logcli+* [2027](https://github.com/grafana/loki/pull/2027) **pstibrany**: logcli: Query needs to be stored into url.RawQuery, and not url.Path+* [2000](https://github.com/grafana/loki/pull/2000) **cyriltovena**: Improve URL building in the logcli to strip trailing /.+* [1922](https://github.com/grafana/loki/pull/1922) **bavarianbidi**: logcli: org-id/tls-skip-verify set via env var+* [1861](https://github.com/grafana/loki/pull/1861) **yeya24**: Support series API in logcli+* [1850](https://github.com/grafana/loki/pull/1850) **chrischdi**: BugFix: Fix logcli client to use OrgID in LiveTail+* [1814](https://github.com/grafana/loki/pull/1814) **cyriltovena**: Logcli remote storage.+* [1712](https://github.com/grafana/loki/pull/1712) **rfratto**: clarify logcli commands and output++#### Promtail+* [2069](https://github.com/grafana/loki/pull/2069) **slim-bean**: Promtail: log at debug level when nothing matches the specified path for a file target+* [2066](https://github.com/grafana/loki/pull/2066) **slim-bean**: Promtail: metrics stage can also count line bytes+* [2049](https://github.com/grafana/loki/pull/2049) **adityacs**: Fix promtail client default values+* [2075](https://github.com/grafana/loki/pull/2075) **cyriltovena**: Fixes a panic in dry-run when using external labels.+* [2026](https://github.com/grafana/loki/pull/2026) **adityacs**: Targets not required in promtail config+* [2004](https://github.com/grafana/loki/pull/2004) **cyriltovena**: Adds config to disable HTTP and GRPC server in Promtail.+* [1935](https://github.com/grafana/loki/pull/1935) **cyriltovena**: Support stdin target via flag instead of automatic detection.+* [1920](https://github.com/grafana/loki/pull/1920) **alexanderGalushka**: feat: tms readiness check bypass implementation+* [1894](https://github.com/grafana/loki/pull/1894) **cyriltovena**: Fixes possible panic in json pipeline stage.+* [1865](https://github.com/grafana/loki/pull/1865) **adityacs**: Fix flaky promtail test+* [1815](https://github.com/grafana/loki/pull/1815) **adityacs**: Log error message when source does not exist in extracted values+* [1627](https://github.com/grafana/loki/pull/1627) **rfratto**: Proposal: Promtail Push API++#### Docker Driver+* [2076](https://github.com/grafana/loki/pull/2076) **cyriltovena**: Allows to pass inlined pipeline stages to the docker driver.+* [2054](https://github.com/grafana/loki/pull/2054) **bkmit**: Docker driver: Allow to provision external pipeline files to plugin+* [1906](https://github.com/grafana/loki/pull/1906) **cyriltovena**: Add no-file and keep-file log option for docker driver.

done

slim-bean

comment created time in 7 days

push eventgrafana/loki

Ed Welch

commit sha 3406dce2ce1b02f10fd744b2e9b62f8dd36f3e82

Updating the Changelog and Upgrade guide for 1.5.0 release. Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 7 days

push eventgrafana/loki

Ed Welch

commit sha 77a93dbe3a5f692381a526b676952783a4529b09

adding more notes for upgrading Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 7 days

push eventgrafana/loki

Ed Welch

commit sha 4c768ace19dd1ca2eb66e44242af16bd6cb0b528

fix some of the bullets callout the improved docker driver file handling as a notable feature Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 7 days

push eventgrafana/loki

Ed Welch

commit sha 3d0c56ebed1bc72b6f52f70cbf05058de9a2e233

improve wording Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 7 days

push eventgrafana/loki

Ed Welch

commit sha e04376718da0852cd53ff0d6646abffbd740b208

on second thought, adding back the relative path and full binary name Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 7 days

push eventgrafana/loki

Ed Welch

commit sha 07ecb8063c2770666138f6e106db63aeaafda566

Update docs/operations/upgrade.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com>

view details

push time in 7 days

Pull request review commentgrafana/loki

Prep 1.5.0 release

 Cortex has done lot of cleanup in the configuration files, and you are strongly  Following fields were removed from YAML configuration completely: `claim_on_rollout` (always true), `normalise_tokens` (always true). +#### Test Your Config++To see if your config needs to change, one way to quickly test is to download a 1.5.0 (or newer) binary from the [release page](https://github.com/grafana/loki/releases/tag/v1.5.0)++Then run the binary providing your config file `./loki-linux-amd64 -config.file=myconfig.yaml`++If there are configs which are no longer valid you will see errors immediately:++```shell+./loki -config.file=loki-local-config.yaml+failed parsing config: loki-local-config.yaml: yaml: unmarshal errors:+  line 35: field dynamodbconfig not found in type aws.StorageConfig+```++Referencing the [list of diffs](https://cortexmetrics.io/docs/changelog/#config-file-breaking-changes) I can see this config changed:++```diff+-  dynamodbconfig:++  dynamodb:+```++Also several other AWS related configs changed and would need to udpate those as well.+++### Loki Docker Image User Changes++To improve security concerns, in 1.5.0 the Docker container no longer runs the loki process as `root` and instead the process runs as user `loki` with UID `10001` and GID `10001`++This may affect people in a couple ways:++#### Loki Port++If you are running Loki with a config that opens a port number above 1000 (which is the default, 3100 for HTTP and 9095 for GRPC) everything should work fine in regards to ports.

was this default actually changed or is this just the weave default? (If it's the weave default we can't really change this in a clean way)

The config file that ships in the docker image sets it to 3100 so it's the "default" in docker

slim-bean

comment created time in 7 days

PR opened grafana/loki

Prep 1.5.0 release

Signed-off-by: Ed Welch edward.welch@grafana.com

+301 -0

0 comment

2 changed files

pr created time in 7 days

create barnchgrafana/loki

branch : release-v1.5.0

created branch time in 7 days

Pull request review commentgrafana/loki

Spelling

 kvstore:       - <string>      # The Dial timeout for the ETCD connection.-    [dial_tmeout: <duration> | default = 10s]+    [dial_timeout: <duration> | default = 10s]

are we 100% sure this is a typo in the docs and not a config actually spelled wrong?

jsoref

comment created time in 8 days

Pull request review commentgrafana/loki

Spelling

 A **huge** thanks to the **36 contributors** who submitted **148 PR's** since 0. * **Build** [1121](https://github.com/grafana/loki/pull/1121) **pracucci**: Publish loki-canary binaries on release * **Build** [1054](https://github.com/grafana/loki/pull/1054) **pstibrany**: Fix dep check warnings by running dep ensure * **Build/release** [1018](https://github.com/grafana/loki/pull/1018) **slim-bean**: updating the image version for loki-canary and adding the version increment to the release_prepare script-* **Build/CI** [997](https://github.com/grafana/loki/pull/997) **slim-bean**: full cirlce+* **Build/CI** [997](https://github.com/grafana/loki/pull/997) **slim-bean**: full circle

lol this misspelling was intentional FWIW :wink:

jsoref

comment created time in 8 days

startedslim-bean/leafbus

started time in 8 days

push eventgrafana/loki

owen-d

commit sha c3efe190568588a5d131e9144b8046290d650d14

[skip ci] Publishing helm charts: 13424a20ce2bde13f521ba8b7608ba5a2a07e64f

view details

push time in 8 days

push eventgrafana/loki

cyriltovena

commit sha fc3d6e4725766d6dac9973d6f587c0bf88cf2463

[skip ci] Publishing helm charts: f9a2c436aca705792c4e04483313614897042cfd

view details

push time in 8 days

Pull request review commentgrafana/loki

Adds backtick for the quoted string token lexer.

 func TestParse(t *testing.T) { 		exp Expr 		err error 	}{+		{+			// raw string+			in: "count_over_time({foo=~`bar\\w+`}[12h] |~ `error\\`)",

yeah, ok, i was more just asking if this was a mistake or if it was a requirement of how to get the value in the test... no worries LGTM

cyriltovena

comment created time in 8 days

Pull request review commentgrafana/loki

Adds backtick for the quoted string token lexer.

 func TestParse(t *testing.T) { 		exp Expr 		err error 	}{+		{+			// raw string+			in: "count_over_time({foo=~`bar\\w+`}[12h] |~ `error\\`)",

should this be:

"count_over_time({foo=~`bar\w+`}[12h] |~ `error\`)",

With only a single backlash because it's inside the new backticks?

cyriltovena

comment created time in 8 days

issue commentgrafana/loki

Metric name enforcement should be ignored

you got your wish, when I ported over the remainder of the validation code from cortex I removed this validation completely.

The config still exists though, it's just not used anywhere

owen-d

comment created time in 8 days

push eventgrafana/loki

slim-bean

commit sha cdad7e2625a9650293457a78b2ec5fe1a6e2f673

[skip ci] Publishing helm charts: c81be6603ebb89b3fab100b30a48780c959af09c

view details

push time in 8 days

PR merged grafana/loki

Fixes unit in stats request log. size/XS

Mistakes were made

/cc @slim-bean

+3 -3

3 comments

2 changed files

cyriltovena

pr closed time in 8 days

push eventgrafana/loki

Cyril Tovena

commit sha c81be6603ebb89b3fab100b30a48780c959af09c

Fixes unit in stats request log. (#2093) * Fix unit conversion in stats log. 🤦 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * Fix the tests because you know ! you know ? Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

view details

push time in 8 days

pull request commentgrafana/loki

Fixes unit in stats request log.

LOL at least you were consistent in your tests being wrong too :)

cyriltovena

comment created time in 8 days

push eventgrafana/loki

owen-d

commit sha e60bbeea96afff8988f3cc2818f05e325ce1fe96

[skip ci] Publishing helm charts: 0520a0ac1c74578ae6fc5d21c24e375e253854af

view details

push time in 9 days

push eventgrafana/loki

beorn7

commit sha 29ab804286a96231ab32e911cf62e83e33b6fa7a

[skip ci] Publishing helm charts: d343c09aa9095c78b1b02fbf9053b00702356834

view details

push time in 9 days

push eventslim-bean/leafbus

Ed Welch

commit sha 3936e0137e0579b34c0a8a6389fa7e9638c0ab65

cleaning up improving playback

view details

push time in 10 days

push eventslim-bean/grafana

Ed Welch

commit sha b579235b11779424c2907167ef2024bbe330f6af

tweaking up makefile a bit

view details

push time in 10 days

push eventslim-bean/cortex

Ed Welch

commit sha 7bb4ceffc0108ec23bf3e9b221129b62f6c523fd

rebased, updated my thanos fork, also ran go mod vendor and tidy which I think jacked some stuff up. Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 10 days

create barnchslim-bean/thanos

branch : chunk-max-size

created branch time in 10 days

push eventslim-bean/thanos

Lucas Servén Marín

commit sha c983b0b51c894b201db9ed240eff81186a0556aa

store: proxy: fix queries never timing out bug (#2411) (#2443) * store: proxy: add test for deadlocking problem Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com> * store: proxy: add fix for timeouts Checking here if the series context has ended is the correct fix here. We want to check it because if any of the other Series() calls error out then the context is canceled. So, it is equal to checking for errors "downstream", in `mergedSeriesSet`. Also, `handleErr()` here is the correct function to use because in such a case we want to set `s.err` -- if `io.EOF` still hasn't been received then it means that StoreAPI still has some data that it wants to send but hasn't yet. With this, the previously added test passes. Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com> Co-authored-by: Giedrius Statkevičius <giedriuswork@gmail.com>

view details

Peter Štibraný

commit sha 6452f5dac969d06985a86e969e6a7454d6c80000

store: Added regex-set optimization to ExpandedPostings (#2450) * Added regex-set optimization to ExpandedPostings Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fixed capitalization. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * CHANGELOG.md Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Removed unnecessary change. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Remove whitespace Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use testutil instead of testify. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Added copyright header, from original Prometheus querier.go Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use Thanos copyright header. :facepalm: Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Added · at the end of the sentence. :exploding_head:. I will randomly add emojis and GitHub emoji markup to commit messages that fix frustrating checks like this one. And intentionally not break the line. Let's see how lint deals with that! Ha. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

view details

Davor Kapsa

commit sha 762d1ad4d136d7e4b3c90879c9c46d33017e33cf

docs/contributing: use Before for IsExpired (#2456) Signed-off-by: Davor Kapsa <davor.kapsa@gmail.com>

view details

Davor Kapsa

commit sha b5f96ee41ec780245115401d918023403b51cb5c

cmd/thanos: clean gosimple S1039 (#2464) Signed-off-by: Davor Kapsa <davor.kapsa@gmail.com>

view details

Ranjith Kumar

commit sha 392c843429ff16ecf2e6d0f7b5e4af74ad7967e1

docs: Update CONTRIBUTING.md with DCO (#2465) * docs: Update CONTRIBUTING.md with DCO Signed-off-by: ranjithkumar007 <ranjith.dakshana2015@gmail.com> * Update CONTRIBUTING.md Co-Authored-By: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: ranjithkumar007 <ranjith.dakshana2015@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>

view details

Bartlomiej Plotka

commit sha 7c5bea680a5cdebbf127fa4f318f53c0818bd31c

Added tests to reproduce #2459. (#2462) Related to: https://github.com/thanos-io/thanos/issues/2459 Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

view details

Yash Sharma

commit sha 853175a5b74d9ba347746a8078f396fb47c764d5

Added a page for documenting beginner issues (#2461) * Added some documentation for beginner issues Signed-off-by: Yash <yashrsharma44@gmail.com> * Edited some lines Signed-off-by: Yash <yashrsharma44@gmail.com> * Update docs/operating/troubleshooting.md Co-Authored-By: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Yash <yashrsharma44@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>

view details

Lucas Servén Marín

commit sha 7250c6bb1685f766d7df7f2b3022ea7a8c5e2c3a

pkg/block/fetcher: fix concurrent map usage (#2474) Fixes: #2471 This commit fixes an issue where multiple goroutines in the block fetcher filtering were concurrently accessing the same map. The goroutines were concurrently writing AND reading to the shared metas map. This commit guards this concurrent access by giving the DeduplicateFilter struct a mutex. Signed-off-by: Lucas Servén Marín <lserven@gmail.com>

view details

Bartlomiej Plotka

commit sha 2654a10c3f02a89fa34a01c6af5bb28540ce6c40

Reverted addition of deletion mark for partial uploads. (#2472) Fixes https://github.com/thanos-io/thanos/issues/2459 (quick fix). This keeps the logic from the 0.11.0 which was good enough. Some improvement for future: https://github.com/thanos-io/thanos/issues/2470 Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

view details

Peter Štibraný

commit sha 4d41d36ea6642a1fecbfa3ec72e848e209f3a981

Remove optimizations for label=~".*" and label!~".*". (#2475) * Remove optimizations for label=~".*" and label!~".*". They are not correct. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * CHANGELOG.md Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

view details

Lucas Servén Marín

commit sha 6ebdb92ee8919446315033105afb8b36c27f28b0

cut v0.12.1 (#2476) Signed-off-by: Lucas Servén Marín <lserven@gmail.com>

view details

Lucas Servén Marín

commit sha 60a2c9e4f09e760b678b9f39c731d3a6ccb650bc

fix thanos web route prefix register twice (#2489) Signed-off-by: yeya24 <yb532204897@gmail.com> Signed-off-by: Lucas Servén Marín <lserven@gmail.com> Co-authored-by: yeya24 <yb532204897@gmail.com>

view details

Marco Pracucci

commit sha 822bc7c111a1620e2bb837e994cb74716b33dfb1

Do not lock DNS Provider.Address() while Resolve() is running (#2492) Signed-off-by: Marco Pracucci <marco@pracucci.com>

view details

Lucas Servén Marín

commit sha 45a6bc4ed39e7f7b14f99fedbc62ac414b745bb4

Merge pull request #2500 from thanos-io/release-0.12 Merge v0.12.1 into master

view details

Johnathan Falk

commit sha d8c2a332a7ebaecf9b1b08bfa3428ea8f063f6b1

Compact: Update compact documentation to better clarify dedupeReplicaLabels. (#2481) * Update compact documentation to better clarify dedupeReplicaLabels. Signed-off-by: Johnathan Falk <johnathan.falk@gmail.com> * Fix capitalization. Signed-off-by: Johnathan Falk <johnathan.falk@gmail.com>

view details

Marco Pracucci

commit sha 0bb67bcbc3a75fb274c1649dcfdb13a8c2db4024

Gracefully handle additional oneof fields in SeriesResponse (#2501) * Gracefully handle additional oneof fields in SeriesResponse Signed-off-by: Marco Pracucci <marco@pracucci.com> * Removed unnecessary continue Signed-off-by: Marco Pracucci <marco@pracucci.com> * Updated CHANGELOG Signed-off-by: Marco Pracucci <marco@pracucci.com>

view details

Arthur

commit sha bafd90ed201d66976dc814e5c07eb219cc7a6c01

fix typo (#2509) Signed-off-by: arthur yang <yang_yapo@126.com>

view details

Kemal Akkoyun

commit sha ebee995dc24987f58e43157d48ef51f451d66a4e

Adjust memcached operation buckets (#2504) Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

view details

Simon Pasquier

commit sha 66dd2fbbcc2ab85236a56e45f01f5c2932e34543

pkg/query: remove obsolete 'thanos_store_node_info' metric (#2505) Signed-off-by: Simon Pasquier <spasquie@redhat.com>

view details

Povilas Versockas

commit sha 28ea33b5d3d396f45f231a7879b25646b45dbcec

Add Community information (#2510) * Add Community information Signed-off-by: Povilas Versockas <p.versockas@gmail.com> * Fixes after review Signed-off-by: Povilas Versockas <p.versockas@gmail.com> * Move to contributing menu Signed-off-by: Povilas Versockas <p.versockas@gmail.com>

view details

push time in 10 days

push eventslim-bean/thanos

Paul Gier

commit sha f332ece9bbe3a77e292326bf28c401479e00c105

tutorials: fix typo in image version (#2223) Signed-off-by: Paul Gier <pgier@redhat.com>

view details

Bartlomiej Plotka

commit sha d23b6a339af18b644f1398bdb5aa92c884432d67

Blocked classic prometheus constructors, moved all to promauto; Removed unnecessary printfs. (#2228) Fixes: https://github.com/thanos-io/thanos/issues/2102 Also blocked them on CI side, thanks to https://github.com/fatih/faillint/pull/8 Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

view details

Robin Clarke-Williams

commit sha 45d6bb8d83c927b1e490172a2cb66307b0db2546

ruler: Fix #2204 bug where alert queue is unpoppable causing full queue and dropped alerts (#2238) * Add test for alert queue Pop after multiple Push Signed-off-by: Robin Clarke-Williams <43950815+robincw-gr@users.noreply.github.com> * Fix alert queue bug by resignal after Pop (#2204) Signed-off-by: Robin Clarke-Williams <43950815+robincw-gr@users.noreply.github.com> * Fix alert queue test and simplify Signed-off-by: Robin Clarke-Williams <43950815+robincw-gr@users.noreply.github.com> * Update CHANGELOG.md Signed-off-by: Robin Clarke-Williams <43950815+robincw-gr@users.noreply.github.com> * Link to thanos-io/thanos PR in CHANGELOG.md Signed-off-by: Robin Clarke-Williams <43950815+robincw-gr@users.noreply.github.com>

view details

Jacob Colvin

commit sha 50ffaaaa013ae229e2274a2ead50363b94391f8b

bucket: improve shard label handling (#2219) Signed-off-by: Jacob Colvin <Jacob.Colvin@8451.com>

view details

Rajesh Rajendran

commit sha 54bfaea70075bbab960eeed04d04222b515e6166

fixing querier deployment kube manifest example 404 error (#2229) Signed-off-by: Rajesh Rajendran <rjshrjndrn@gmail.com>

view details

Kemal Akkoyun

commit sha e6566606ac2f56aac4c0681ef3bf107eb15c59c7

*: Fix misuse of pkg/errors.Errorf and error directive (#2253) * Fix pkg/errors error directive issues Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com> * Fix misuse of Errorf Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

view details

Kemal Akkoyun

commit sha 9c0d4676b666643976d70c35485d44022d8f1e08

Fix false metric name in Store GW e2e test (#2256) Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

view details

Lawrence Li

commit sha ab2a0cf2698e7b9fdcef1e36736ac12ea98fa3b3

Add scheme to the alertmanagers.url in ruler example (#2255) Signed-off-by: gitlawr <lawrleegle@gmail.com>

view details

Paul Traylor

commit sha 739191c4a7aea1521acd974e923794dd18ca5bee

Sort chunks by thanos.downsample.resolution for better grouping (#2231) Signed-off-by: Paul Traylor <paul.traylor@linecorp.com>

view details

Richard Poole

commit sha b08c0ea62abfe4dcf1400da0e37598f0cd8fa8cf

Remove duplicate log.level arg in quickstart.sh (#2148) Signed-off-by: Richard Poole <richard.poole@cudoventures.com>

view details

John Chen

commit sha 8a6746298ec39ccb40108e5517535e944d3920d3

tutorials: fix incorrect query (#2239) You would have to query `prometheus_tsdb_head_series` instead of `sum(prometheus_tsdb_head_series)` in order to get the 5 results when deduplicating. Signed-off-by: John Chen <johnchen456@gmail.com>

view details

Kemal Akkoyun

commit sha c0418a2e21fbcbd3599a8f2024d7d5f6180daf44

Use new go jsonnet formatter (#2258) Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

view details

Xiang Dai

commit sha 081d7493473a7e108c9bbfeba6be7876e19259d9

docs: Document Thanos Sharding (#1922) * docs: Document Thanos Sharding Signed-off-by: Xiang Dai <764524258@qq.com> * Add time partitioning Signed-off-by: Xiang Dai <764524258@qq.com> * feedback Signed-off-by: Xiang Dai <764524258@qq.com>

view details

Xiang Dai

commit sha b475d33d858cc03b40ee2489310bfd39f711f9d3

Sharding: document supported relabel action and add store gateway backgroud (#2272) * Sharding: document supported relabel action and add store gateway background Signed-off-by: Xiang Dai <764524258@qq.com> * add hashmod Signed-off-by: Xiang Dai <764524258@qq.com>

view details

Kemal Akkoyun

commit sha cf7c8e07178d85fbe1f6764abb2bcd1f0276fdf9

Add wait-interval flag (#2265) Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

view details

Bartlomiej Plotka

commit sha 60b9b9542f043c6403f06110bf1178e308009d07

store: Optimized labels conversion on store.Series; Added unsafe labels conversion. (#2230) ## Changes * method TranslateLables CPU Optimized (streamed sorting). * All store GW label conversation to []storepb.Label are now alloc-less. ``` go test -bench=BenchmarkUnsafeVSSafeLabelsConversion -run=^$ -benchmem -timeout 2h -benchtime 10s ./pkg/store/storepb/... goos: linux goarch: amd64 pkg: github.com/thanos-io/thanos/pkg/store/storepb BenchmarkUnsafeVSSafeLabelsConversion/safe-12 34822 339076 ns/op 655368 B/op 2 allocs/op BenchmarkUnsafeVSSafeLabelsConversion/unsafe-12 1000000000 2.32 ns/op 0 B/op 0 allocs/op PASS ``` TODO: Do the same on Querier. Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

view details

Kush Trivedi

commit sha de3f2825d556404756cd7135f121e0acc0f24287

fix: Ignore the OS-X Trash (#2274) Signed-off-by: kushthedude <kushthedude@gmail.com>

view details

Xiang Dai

commit sha f6921dcd9ad563d2422822bd82cf7c6f63016bea

docs/sharding.md: fix a typo (#2273) Signed-off-by: Xiang Dai <764524258@qq.com>

view details

Ben Ye

commit sha 2a95d7a7fd924c6b6e7135a0d477e24a8d30f939

fix replicate duplicate metrics (#2254) Signed-off-by: yeya24 <yb532204897@gmail.com>

view details

Xiang Dai

commit sha cc0b7e62b8b2c6f0879e5bfdd9881a1f7d699fc4

Document downsample component (#2090) * scripts/genflagdocs.sh: Generate downsample flag Signed-off-by: Xiang Dai <764524258@qq.com> * Document downsample component Signed-off-by: Xiang Dai <764524258@qq.com> * Move downsample as bucket sub-command Signed-off-by: Xiang Dai <764524258@qq.com> * update docs Signed-off-by: Xiang Dai <764524258@qq.com> * feedback Signed-off-by: Xiang Dai <764524258@qq.com>

view details

push time in 10 days

push eventslim-bean/cortex

Goutham Veeramachaneni

commit sha 1bd32df09a00e5b45c4f6fa6497ba4cf4a575bbe

Ensure queries return correctly during rolling upgrades of stateful cluster with RF 3 and only 3 nodes. (#2503) (#2512) * Use a real ring with mock KV when testing distributor. This is to teast out errors in the replication logic. Signed-off-by: Tom Wilkie <tom@grafana.com> * Extend distributor test to cover the case RF=3 with 2 ingesters. Signed-off-by: Tom Wilkie <tom@grafana.com> * Ensure ring correctly calculates the number of allowed failures when RF=3 and #ingesters=2. Signed-off-by: Tom Wilkie <tom@grafana.com> * Add changelog and review feedback. Signed-off-by: Tom Wilkie <tom@grafana.com> * Refactor some distributor tests to try and get them to pass. Signed-off-by: Tom Wilkie <tom@grafana.com> * Speed up tests but polling more frequently. Signed-off-by: Tom Wilkie <tom@grafana.com> * Fix same bug on the write path. Signed-off-by: Tom Wilkie <tom@grafana.com> * Tidy up the distributor tests. Signed-off-by: Tom Wilkie <tom@grafana.com> * Make test correctly handle RF3 and 2 ingesters - previously was succeeding when it shouldn't Signed-off-by: Tom Wilkie <tom@grafana.com> * Update pkg/ring/ring.go Co-Authored-By: Jacob Lisi <jacob.t.lisi@gmail.com> Signed-off-by: Tom Wilkie <tom@grafana.com> Co-authored-by: Jacob Lisi <jacob.t.lisi@gmail.com> Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> Co-authored-by: Tom Wilkie <tomwilkie@users.noreply.github.com> Co-authored-by: Jacob Lisi <jacob.t.lisi@gmail.com>

view details

Goutham Veeramachaneni

commit sha 6d7270057498f9cbb18f9bc77c9c8c30e06e2754

Release 1.0.1 (#2513) Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

view details

Dmitry Shmulevich

commit sha d4d7320a8764a9d8a208826c234ca728c202dbd0

fix integration test (#2534) Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com>

view details

Jay Batra

commit sha 3dfbe2f80b2e49d67b8a01d339adc8931a8e3de6

Proposal: Versioning of Documentation (#2349) * Proposal: Versioning of Documentation Let's start a discussion about how to version documenation. I have listed a few tools that can be used with their respective pros and cons. Signed-off-by: Jay Batra <jaybatra73@gmail.com> * Update Version Documentation Proposal Updates the proposal as per discussion on slack. Signed-off-by: Jay Batra <jaybatra73@gmail.com> * Fixed markdown Signed-off-by: Marco Pracucci <marco@pracucci.com> * Cleaned white noise Signed-off-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

view details

Samit

commit sha fe56f1420099aa1bf1ce09316c186e05bddee879

Fix a typo in architecture doc (#2536) s/rules/ruler Signed-off-by: Samit Pal <samit04@gmail.com>

view details

Patrick DeVivo

commit sha f2f4d757b42d83b4080bbd1326796625be03656b

Add TODOs badge to README (#2533) Closes #2529 Signed-off-by: Patrick DeVivo <patrick.devivo@gmail.com>

view details

Artem Z

commit sha 55e50b34fed8c63c466dad396e2f31e69ae537f8

query-frontend: fix trace propagation when downstream_url is specifie… (#2539) * query-frontend: fix trace propagation when downstream_url is specified (#2538) Signed-off-by: Artem Zavatskiy <r@nextus.me> * Update CHANGELOG.md Signed-off-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

view details

Jan-Philip Gehrcke

commit sha b51a84a14493c119309a6a6c9419724a781f1a81

metrics: ingester_chunk_size_bytes: add 2 buckets (#2540) Signed-off-by: Dr. Jan-Philip Gehrcke <jp@opstrace.com>

view details

Bryan Boreham

commit sha 058d03f538270cae9a5c8d4fd2d08fad07c4344d

Re-use buffers when serialising chunks. (#2505) Improves performance of ingester handover and WAL checkpoint by reducing garbage. Signed-off-by: Bryan Boreham <bryan@weave.works>

view details

Dmitry Shmulevich

commit sha a6840b5e525bf8a9d15ba13598d336602799e6ff

allow human-readable memory size in FifoCache config (#2527) * allow human-readable memory size in FifoCache config Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * update CHANGELOG, usage message Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * addressed comments Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * update tests Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * fixed integration test failure Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * added unit test Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> Co-authored-by: Dmitry Shmulevich <dima@dev.local>

view details

Sandeep Sukhani

commit sha d4f86ba86948f7240a9953acd277ec9c8bf7f38b

Handling of empty directories in filesystem object client (#2537) * handling of empty directories in filesystem object client Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * calling os.Remove instead of os.RemoveAll while removing empty directory Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * some minor improvement Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * negative test for isNotEmptyErr Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * some changes suggested from PR review Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * fixed comment Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * changes suggested from PR review Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>

view details

Jacob Lisi

commit sha d94cdce0aa46b69fff7cb40d1ad57e77b11b4ab0

Handle forward slashes in ruler mapper/manager (#2506) * handle forward slashes in ruler mapper/manager Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update unit tests Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * Update Changelog Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * fix linting error in tests Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * use url escaping instead of base64 Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * move changelog entry from enhancement to CHANGE Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * fix typo Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update changelog Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update comment Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>

view details

Peter Štibraný

commit sha ef8dc06c7ae24c1bbdd5341ce744ea37ff515b90

Only wait for other modules if underlying service is still running. (#2547) Otherwise there is no point in waiting. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

view details

Ganesh Vernekar

commit sha 7a8848c272d95cb66b819834ec941198e37dad74

Better spread of checkpointing (#2531) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

view details

Jacob Lisi

commit sha 0a8fc3a79ae7de1fd8208ebf928c7595b3c3c197

add documentation for the ruler API (#2544) Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>

view details

Annanay Agarwal

commit sha ef28d343a129846323aaba66f0c9ba88517c6644

Update weaveworks/common dependency version (#2535) * Update weaveworks/common dependency version Signed-off-by: Annanay <annanayagarwal@gmail.com> * Update CHANGELOG.md Signed-off-by: Annanay <annanayagarwal@gmail.com> * go mod tidy Signed-off-by: Annanay <annanayagarwal@gmail.com> * Revert CHANGELOG entry, make doc Signed-off-by: Annanay <annanayagarwal@gmail.com> * Add grpc.WithBlock() DialOption according to etcd upgrade guide Signed-off-by: Annanay <annanayagarwal@gmail.com> * Update ww/common after latest change to use min TLS v1.2 Signed-off-by: Annanay <annanayagarwal@gmail.com> * Add CHANGELOG entries Signed-off-by: Annanay <annanayagarwal@gmail.com> * Fix lint Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

Aditya C S

commit sha 65d3b343cc6bbdeb52cb021fd9d3cecc1a7337de

Create index table from local table client for boltdb (#2441) Signed-off-by: Aditya C S <aditya.gnu@gmail.com>

view details

Joe Elliott

commit sha ba0dc4ff6d7d9da930fe1c5ab573c4be632ea03c

Proposal: Scalable Query Frontend (#2528) * Scalable Query Frontend Proposal Signed-off-by: Joe Elliott <number101010@gmail.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

view details

Marco Pracucci

commit sha 8a797c955a291eb63c2c26ee1b1ee23d6c766399

Fixed tests (#2551) Signed-off-by: Marco Pracucci <marco@pracucci.com>

view details

gotjosh

commit sha 3462eb63f324c649bbaa122933bc591b710f4e48

Implement /api/v1/metadata (#2549) * Handle /api/v1/metadata in the queriers This commit handles the call to /api/v1/metadata from Prometheus. It looks for all metric metadata of a given metadata and exposes it as a map of slices. Signed-off-by: gotjosh <josue@grafana.com> * Add changelog entry Signed-off-by: gotjosh <josue@grafana.com> * Wordsmithing on comments Signed-off-by: gotjosh <josue@grafana.com> * Update CHANGELOG.md Signed-off-by: gotjosh <josue@grafana.com> * Add a note about this custom handler being temporary Signed-off-by: gotjosh <josue@grafana.com>

view details

push time in 10 days

push eventslim-bean/cortex

Goutham Veeramachaneni

commit sha 1bd32df09a00e5b45c4f6fa6497ba4cf4a575bbe

Ensure queries return correctly during rolling upgrades of stateful cluster with RF 3 and only 3 nodes. (#2503) (#2512) * Use a real ring with mock KV when testing distributor. This is to teast out errors in the replication logic. Signed-off-by: Tom Wilkie <tom@grafana.com> * Extend distributor test to cover the case RF=3 with 2 ingesters. Signed-off-by: Tom Wilkie <tom@grafana.com> * Ensure ring correctly calculates the number of allowed failures when RF=3 and #ingesters=2. Signed-off-by: Tom Wilkie <tom@grafana.com> * Add changelog and review feedback. Signed-off-by: Tom Wilkie <tom@grafana.com> * Refactor some distributor tests to try and get them to pass. Signed-off-by: Tom Wilkie <tom@grafana.com> * Speed up tests but polling more frequently. Signed-off-by: Tom Wilkie <tom@grafana.com> * Fix same bug on the write path. Signed-off-by: Tom Wilkie <tom@grafana.com> * Tidy up the distributor tests. Signed-off-by: Tom Wilkie <tom@grafana.com> * Make test correctly handle RF3 and 2 ingesters - previously was succeeding when it shouldn't Signed-off-by: Tom Wilkie <tom@grafana.com> * Update pkg/ring/ring.go Co-Authored-By: Jacob Lisi <jacob.t.lisi@gmail.com> Signed-off-by: Tom Wilkie <tom@grafana.com> Co-authored-by: Jacob Lisi <jacob.t.lisi@gmail.com> Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> Co-authored-by: Tom Wilkie <tomwilkie@users.noreply.github.com> Co-authored-by: Jacob Lisi <jacob.t.lisi@gmail.com>

view details

Goutham Veeramachaneni

commit sha 6d7270057498f9cbb18f9bc77c9c8c30e06e2754

Release 1.0.1 (#2513) Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

view details

Dmitry Shmulevich

commit sha d4d7320a8764a9d8a208826c234ca728c202dbd0

fix integration test (#2534) Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com>

view details

Jay Batra

commit sha 3dfbe2f80b2e49d67b8a01d339adc8931a8e3de6

Proposal: Versioning of Documentation (#2349) * Proposal: Versioning of Documentation Let's start a discussion about how to version documenation. I have listed a few tools that can be used with their respective pros and cons. Signed-off-by: Jay Batra <jaybatra73@gmail.com> * Update Version Documentation Proposal Updates the proposal as per discussion on slack. Signed-off-by: Jay Batra <jaybatra73@gmail.com> * Fixed markdown Signed-off-by: Marco Pracucci <marco@pracucci.com> * Cleaned white noise Signed-off-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

view details

Samit

commit sha fe56f1420099aa1bf1ce09316c186e05bddee879

Fix a typo in architecture doc (#2536) s/rules/ruler Signed-off-by: Samit Pal <samit04@gmail.com>

view details

Patrick DeVivo

commit sha f2f4d757b42d83b4080bbd1326796625be03656b

Add TODOs badge to README (#2533) Closes #2529 Signed-off-by: Patrick DeVivo <patrick.devivo@gmail.com>

view details

Artem Z

commit sha 55e50b34fed8c63c466dad396e2f31e69ae537f8

query-frontend: fix trace propagation when downstream_url is specifie… (#2539) * query-frontend: fix trace propagation when downstream_url is specified (#2538) Signed-off-by: Artem Zavatskiy <r@nextus.me> * Update CHANGELOG.md Signed-off-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

view details

Jan-Philip Gehrcke

commit sha b51a84a14493c119309a6a6c9419724a781f1a81

metrics: ingester_chunk_size_bytes: add 2 buckets (#2540) Signed-off-by: Dr. Jan-Philip Gehrcke <jp@opstrace.com>

view details

Bryan Boreham

commit sha 058d03f538270cae9a5c8d4fd2d08fad07c4344d

Re-use buffers when serialising chunks. (#2505) Improves performance of ingester handover and WAL checkpoint by reducing garbage. Signed-off-by: Bryan Boreham <bryan@weave.works>

view details

Dmitry Shmulevich

commit sha a6840b5e525bf8a9d15ba13598d336602799e6ff

allow human-readable memory size in FifoCache config (#2527) * allow human-readable memory size in FifoCache config Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * update CHANGELOG, usage message Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * addressed comments Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * update tests Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * fixed integration test failure Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * added unit test Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> Co-authored-by: Dmitry Shmulevich <dima@dev.local>

view details

Sandeep Sukhani

commit sha d4f86ba86948f7240a9953acd277ec9c8bf7f38b

Handling of empty directories in filesystem object client (#2537) * handling of empty directories in filesystem object client Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * calling os.Remove instead of os.RemoveAll while removing empty directory Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * some minor improvement Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * negative test for isNotEmptyErr Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * some changes suggested from PR review Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * fixed comment Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * changes suggested from PR review Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>

view details

Jacob Lisi

commit sha d94cdce0aa46b69fff7cb40d1ad57e77b11b4ab0

Handle forward slashes in ruler mapper/manager (#2506) * handle forward slashes in ruler mapper/manager Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update unit tests Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * Update Changelog Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * fix linting error in tests Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * use url escaping instead of base64 Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * move changelog entry from enhancement to CHANGE Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * fix typo Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update changelog Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update comment Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>

view details

Peter Štibraný

commit sha ef8dc06c7ae24c1bbdd5341ce744ea37ff515b90

Only wait for other modules if underlying service is still running. (#2547) Otherwise there is no point in waiting. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

view details

Ganesh Vernekar

commit sha 7a8848c272d95cb66b819834ec941198e37dad74

Better spread of checkpointing (#2531) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

view details

Jacob Lisi

commit sha 0a8fc3a79ae7de1fd8208ebf928c7595b3c3c197

add documentation for the ruler API (#2544) Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>

view details

Annanay Agarwal

commit sha ef28d343a129846323aaba66f0c9ba88517c6644

Update weaveworks/common dependency version (#2535) * Update weaveworks/common dependency version Signed-off-by: Annanay <annanayagarwal@gmail.com> * Update CHANGELOG.md Signed-off-by: Annanay <annanayagarwal@gmail.com> * go mod tidy Signed-off-by: Annanay <annanayagarwal@gmail.com> * Revert CHANGELOG entry, make doc Signed-off-by: Annanay <annanayagarwal@gmail.com> * Add grpc.WithBlock() DialOption according to etcd upgrade guide Signed-off-by: Annanay <annanayagarwal@gmail.com> * Update ww/common after latest change to use min TLS v1.2 Signed-off-by: Annanay <annanayagarwal@gmail.com> * Add CHANGELOG entries Signed-off-by: Annanay <annanayagarwal@gmail.com> * Fix lint Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

Aditya C S

commit sha 65d3b343cc6bbdeb52cb021fd9d3cecc1a7337de

Create index table from local table client for boltdb (#2441) Signed-off-by: Aditya C S <aditya.gnu@gmail.com>

view details

Joe Elliott

commit sha ba0dc4ff6d7d9da930fe1c5ab573c4be632ea03c

Proposal: Scalable Query Frontend (#2528) * Scalable Query Frontend Proposal Signed-off-by: Joe Elliott <number101010@gmail.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

view details

Marco Pracucci

commit sha 8a797c955a291eb63c2c26ee1b1ee23d6c766399

Fixed tests (#2551) Signed-off-by: Marco Pracucci <marco@pracucci.com>

view details

gotjosh

commit sha 3462eb63f324c649bbaa122933bc591b710f4e48

Implement /api/v1/metadata (#2549) * Handle /api/v1/metadata in the queriers This commit handles the call to /api/v1/metadata from Prometheus. It looks for all metric metadata of a given metadata and exposes it as a map of slices. Signed-off-by: gotjosh <josue@grafana.com> * Add changelog entry Signed-off-by: gotjosh <josue@grafana.com> * Wordsmithing on comments Signed-off-by: gotjosh <josue@grafana.com> * Update CHANGELOG.md Signed-off-by: gotjosh <josue@grafana.com> * Add a note about this custom handler being temporary Signed-off-by: gotjosh <josue@grafana.com>

view details

push time in 10 days

push eventslim-bean/cortex

Goutham Veeramachaneni

commit sha 1bd32df09a00e5b45c4f6fa6497ba4cf4a575bbe

Ensure queries return correctly during rolling upgrades of stateful cluster with RF 3 and only 3 nodes. (#2503) (#2512) * Use a real ring with mock KV when testing distributor. This is to teast out errors in the replication logic. Signed-off-by: Tom Wilkie <tom@grafana.com> * Extend distributor test to cover the case RF=3 with 2 ingesters. Signed-off-by: Tom Wilkie <tom@grafana.com> * Ensure ring correctly calculates the number of allowed failures when RF=3 and #ingesters=2. Signed-off-by: Tom Wilkie <tom@grafana.com> * Add changelog and review feedback. Signed-off-by: Tom Wilkie <tom@grafana.com> * Refactor some distributor tests to try and get them to pass. Signed-off-by: Tom Wilkie <tom@grafana.com> * Speed up tests but polling more frequently. Signed-off-by: Tom Wilkie <tom@grafana.com> * Fix same bug on the write path. Signed-off-by: Tom Wilkie <tom@grafana.com> * Tidy up the distributor tests. Signed-off-by: Tom Wilkie <tom@grafana.com> * Make test correctly handle RF3 and 2 ingesters - previously was succeeding when it shouldn't Signed-off-by: Tom Wilkie <tom@grafana.com> * Update pkg/ring/ring.go Co-Authored-By: Jacob Lisi <jacob.t.lisi@gmail.com> Signed-off-by: Tom Wilkie <tom@grafana.com> Co-authored-by: Jacob Lisi <jacob.t.lisi@gmail.com> Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com> Co-authored-by: Tom Wilkie <tomwilkie@users.noreply.github.com> Co-authored-by: Jacob Lisi <jacob.t.lisi@gmail.com>

view details

Goutham Veeramachaneni

commit sha 6d7270057498f9cbb18f9bc77c9c8c30e06e2754

Release 1.0.1 (#2513) Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

view details

Dmitry Shmulevich

commit sha d4d7320a8764a9d8a208826c234ca728c202dbd0

fix integration test (#2534) Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com>

view details

Jay Batra

commit sha 3dfbe2f80b2e49d67b8a01d339adc8931a8e3de6

Proposal: Versioning of Documentation (#2349) * Proposal: Versioning of Documentation Let's start a discussion about how to version documenation. I have listed a few tools that can be used with their respective pros and cons. Signed-off-by: Jay Batra <jaybatra73@gmail.com> * Update Version Documentation Proposal Updates the proposal as per discussion on slack. Signed-off-by: Jay Batra <jaybatra73@gmail.com> * Fixed markdown Signed-off-by: Marco Pracucci <marco@pracucci.com> * Cleaned white noise Signed-off-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

view details

Samit

commit sha fe56f1420099aa1bf1ce09316c186e05bddee879

Fix a typo in architecture doc (#2536) s/rules/ruler Signed-off-by: Samit Pal <samit04@gmail.com>

view details

Patrick DeVivo

commit sha f2f4d757b42d83b4080bbd1326796625be03656b

Add TODOs badge to README (#2533) Closes #2529 Signed-off-by: Patrick DeVivo <patrick.devivo@gmail.com>

view details

Artem Z

commit sha 55e50b34fed8c63c466dad396e2f31e69ae537f8

query-frontend: fix trace propagation when downstream_url is specifie… (#2539) * query-frontend: fix trace propagation when downstream_url is specified (#2538) Signed-off-by: Artem Zavatskiy <r@nextus.me> * Update CHANGELOG.md Signed-off-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

view details

Jan-Philip Gehrcke

commit sha b51a84a14493c119309a6a6c9419724a781f1a81

metrics: ingester_chunk_size_bytes: add 2 buckets (#2540) Signed-off-by: Dr. Jan-Philip Gehrcke <jp@opstrace.com>

view details

Bryan Boreham

commit sha 058d03f538270cae9a5c8d4fd2d08fad07c4344d

Re-use buffers when serialising chunks. (#2505) Improves performance of ingester handover and WAL checkpoint by reducing garbage. Signed-off-by: Bryan Boreham <bryan@weave.works>

view details

Dmitry Shmulevich

commit sha a6840b5e525bf8a9d15ba13598d336602799e6ff

allow human-readable memory size in FifoCache config (#2527) * allow human-readable memory size in FifoCache config Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * update CHANGELOG, usage message Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * addressed comments Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * update tests Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * fixed integration test failure Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> * added unit test Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@sysdig.com> Co-authored-by: Dmitry Shmulevich <dima@dev.local>

view details

Sandeep Sukhani

commit sha d4f86ba86948f7240a9953acd277ec9c8bf7f38b

Handling of empty directories in filesystem object client (#2537) * handling of empty directories in filesystem object client Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * calling os.Remove instead of os.RemoveAll while removing empty directory Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * some minor improvement Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * negative test for isNotEmptyErr Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * some changes suggested from PR review Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * fixed comment Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * changes suggested from PR review Signed-off-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>

view details

Jacob Lisi

commit sha d94cdce0aa46b69fff7cb40d1ad57e77b11b4ab0

Handle forward slashes in ruler mapper/manager (#2506) * handle forward slashes in ruler mapper/manager Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update unit tests Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * Update Changelog Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * fix linting error in tests Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * use url escaping instead of base64 Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * move changelog entry from enhancement to CHANGE Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * fix typo Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update changelog Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * update comment Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>

view details

Peter Štibraný

commit sha ef8dc06c7ae24c1bbdd5341ce744ea37ff515b90

Only wait for other modules if underlying service is still running. (#2547) Otherwise there is no point in waiting. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

view details

Ganesh Vernekar

commit sha 7a8848c272d95cb66b819834ec941198e37dad74

Better spread of checkpointing (#2531) Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

view details

Jacob Lisi

commit sha 0a8fc3a79ae7de1fd8208ebf928c7595b3c3c197

add documentation for the ruler API (#2544) Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>

view details

Annanay Agarwal

commit sha ef28d343a129846323aaba66f0c9ba88517c6644

Update weaveworks/common dependency version (#2535) * Update weaveworks/common dependency version Signed-off-by: Annanay <annanayagarwal@gmail.com> * Update CHANGELOG.md Signed-off-by: Annanay <annanayagarwal@gmail.com> * go mod tidy Signed-off-by: Annanay <annanayagarwal@gmail.com> * Revert CHANGELOG entry, make doc Signed-off-by: Annanay <annanayagarwal@gmail.com> * Add grpc.WithBlock() DialOption according to etcd upgrade guide Signed-off-by: Annanay <annanayagarwal@gmail.com> * Update ww/common after latest change to use min TLS v1.2 Signed-off-by: Annanay <annanayagarwal@gmail.com> * Add CHANGELOG entries Signed-off-by: Annanay <annanayagarwal@gmail.com> * Fix lint Signed-off-by: Annanay <annanayagarwal@gmail.com>

view details

Aditya C S

commit sha 65d3b343cc6bbdeb52cb021fd9d3cecc1a7337de

Create index table from local table client for boltdb (#2441) Signed-off-by: Aditya C S <aditya.gnu@gmail.com>

view details

Joe Elliott

commit sha ba0dc4ff6d7d9da930fe1c5ab573c4be632ea03c

Proposal: Scalable Query Frontend (#2528) * Scalable Query Frontend Proposal Signed-off-by: Joe Elliott <number101010@gmail.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

view details

Marco Pracucci

commit sha 8a797c955a291eb63c2c26ee1b1ee23d6c766399

Fixed tests (#2551) Signed-off-by: Marco Pracucci <marco@pracucci.com>

view details

gotjosh

commit sha 3462eb63f324c649bbaa122933bc591b710f4e48

Implement /api/v1/metadata (#2549) * Handle /api/v1/metadata in the queriers This commit handles the call to /api/v1/metadata from Prometheus. It looks for all metric metadata of a given metadata and exposes it as a map of slices. Signed-off-by: gotjosh <josue@grafana.com> * Add changelog entry Signed-off-by: gotjosh <josue@grafana.com> * Wordsmithing on comments Signed-off-by: gotjosh <josue@grafana.com> * Update CHANGELOG.md Signed-off-by: gotjosh <josue@grafana.com> * Add a note about this custom handler being temporary Signed-off-by: gotjosh <josue@grafana.com>

view details

push time in 10 days

PR opened grafana/loki

Loki: allow no encoding/compression on chunks

We didn't export this originally, I found a use case for it, I suspect others might too so I'm exporting it. Signed-off-by: Ed Welch edward.welch@grafana.com

+1 -0

0 comment

1 changed file

pr created time in 10 days

create barnchgrafana/loki

branch : allow-no-encoding

created branch time in 10 days

push eventslim-bean/grafana

Ed Welch

commit sha 4bca1e4ddd22bc5268005b0192342491eaa19046

cleaning up the tool tip a bit

view details

push time in 11 days

issue commentgrafana/loki

CI: plugins/manifest is flaking

The error messages is likely correct, we prune old master-***** image from dockerhub after 60 days, they are meant to be ephemeral.

Are you deploying with master-***** images? Or does this just create a noisy log? I'm afraid I don't really have any suggestions/solutions.

rfratto

comment created time in 12 days

push eventgrafana/loki

Owen Diehl

commit sha e0dacd2795ebc4179e0b34250b170bb706c9ee5e

removes yolostring (#2078)

view details

push time in 13 days

push eventgrafana/loki

owen-d

commit sha 991f2b02b6dbfe2e79993bcb82dc773881636852

[skip ci] Publishing helm charts: e0dacd2795ebc4179e0b34250b170bb706c9ee5e

view details

push time in 13 days

push eventgrafana/loki

Ed Welch

commit sha 9803eab94749ad7bcb5fb647f76ab5fc85eeb5ea

Loki: Allow configuring query_store_max_look_back_period when running a filesystem store and boltdb-shipper (#2073) * Allow configuring query_store_max_look_back_period when running a filesystem store, which allows for boltdb-shipper to horizontally scale filesystem store. Signed-off-by: Ed Welch <edward.welch@grafana.com> * Update docs/operations/storage/boltdb-shipper.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * add some more precision to the timestamp used to make the name unique just to make sure there are not collisions if for some reason the ID is empty (which can happen if the hostname doesn't resolve for some reason) Signed-off-by: Ed Welch <edward.welch@grafana.com> * Allow the ingester to query the store for labels if query store is enabled. Signed-off-by: Ed Welch <edward.welch@grafana.com> * make queriers aware of the fact that ingesters may be querying the store so they don't duplicate query efforts Signed-off-by: Ed Welch <edward.welch@grafana.com> * improve the docs Signed-off-by: Ed Welch <edward.welch@grafana.com> * add tests to the calculateMaxLookBack Signed-off-by: Ed Welch <edward.welch@grafana.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>

view details

Cyril Tovena

commit sha c2f100c33424dc305e5ecf7e62839a4dbe950799

Allows to pass inlined pipeline stages to the docker driver. (#2076) * Allows to pass inlined pipeline stages to the docker driver. Fixes #1296 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * doc. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

view details

Cyril Tovena

commit sha 0de1a42406bdb8953e03c85736ea27f88c1deb81

Fixes a panic in dry-run when using external labels. (#2075) Fixes #1972. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

view details

Or Sela

commit sha 835dd293167fc125066dd78b318045d406355f7e

fluentd plugin: support placeholders in tenant field (#2074)

view details

push time in 13 days

push eventgrafana/loki

slim-bean

commit sha a080e7a3198ab4798bdc3632cad8467e7f63329c

[skip ci] Publishing helm charts: 0de1a42406bdb8953e03c85736ea27f88c1deb81

view details

push time in 13 days

push eventgrafana/loki

slim-bean

commit sha 853c2e48056392447ab43a05fd7209bac9961834

[skip ci] Publishing helm charts: 835dd293167fc125066dd78b318045d406355f7e

view details

push time in 13 days

push eventgrafana/loki

slim-bean

commit sha 296d70dd0170fe58f5b7359cf27eb43ecba7d11d

[skip ci] Publishing helm charts: c2f100c33424dc305e5ecf7e62839a4dbe950799

view details

push time in 13 days

push eventgrafana/loki

slim-bean

commit sha 0f36d57340cff41869dab661949ce4078dc9d985

[skip ci] Publishing helm charts: 9803eab94749ad7bcb5fb647f76ab5fc85eeb5ea

view details

push time in 13 days

push eventgrafana/loki

Or Sela

commit sha 835dd293167fc125066dd78b318045d406355f7e

fluentd plugin: support placeholders in tenant field (#2074)

view details

push time in 13 days

PR merged grafana/loki

fluentd plugin: support placeholders in tenant field size/M

What this PR does / why we need it:

When running in a multi-tenant environment with a single fluentd and loki in multi-tenant mode, it's currently impossible to use a single loki output match because the tenant field is static. You basically need to have a separate match for every tenant and handle the routing yourself.

This PR adds placeholder support to the tenant field, so each chunk can be sent to the right tenant based on tag or record values while still using a single loki output match.

Checklist

  • [x] Documentation added
  • [ ] Tests updated
+26 -6

1 comment

3 changed files

osela

pr closed time in 13 days

push eventgrafana/loki

Cyril Tovena

commit sha 0de1a42406bdb8953e03c85736ea27f88c1deb81

Fixes a panic in dry-run when using external labels. (#2075) Fixes #1972. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

view details

push time in 13 days

PR merged grafana/loki

Fixes a panic in dry-run when using external labels. size/S

Fixes #1972.

Signed-off-by: Cyril Tovena cyril.tovena@gmail.com

+12 -0

0 comment

1 changed file

cyriltovena

pr closed time in 13 days

issue closedgrafana/loki

Promtail dry-run crashes with external_labels in client config

Describe the bug Promtail crashes if client config has any external_labels specified and -dry-run option is used.

To Reproduce Consider following clients section in config.yaml:

clients:
  - url: http://loki.instance:3100/loki/api/v1/push
    external_labels:
      my_label: my

Launching promtail in dry-run mode fails:

C:\promtail\promtail-windows-amd64-v1.4.1.exe -config.file=C:\promtail\config.yaml -dry-run
Clients configured:
panic: Option ,inline needs a map with string keys in struct flagext.LabelSet [recovered]
        panic: Option ,inline needs a map with string keys in struct flagext.LabelSet

goroutine 1 [running]:
gopkg.in/yaml%2ev2.handleErr(0xc0005ef290)
        /src/loki/vendor/gopkg.in/yaml.v2/yaml.go:249 +0xa1
panic(0x2188ce0, 0xc0000bd840)
        /usr/local/go/src/runtime/panic.go:679 +0x1c0
gopkg.in/yaml%2ev2.(*encoder).structv(0xc00001e2c0, 0x0, 0x0, 0x242cd40, 0xc000106728, 0x99)
        /src/loki/vendor/gopkg.in/yaml.v2/encode.go:211 +0x11f
gopkg.in/yaml%2ev2.(*encoder).marshal(0xc00001e2c0, 0x0, 0x0, 0x242cd40, 0xc000106728, 0x99)
        /src/loki/vendor/gopkg.in/yaml.v2/encode.go:160 +0x767
gopkg.in/yaml%2ev2.(*encoder).structv.func1()
        /src/loki/vendor/gopkg.in/yaml.v2/encode.go:226 +0x55f
gopkg.in/yaml%2ev2.(*encoder).mappingv(0xc00001e2c0, 0x0, 0x0, 0xc0005eecc0)
        /src/loki/vendor/gopkg.in/yaml.v2/encode.go:256 +0x14d
gopkg.in/yaml%2ev2.(*encoder).structv(0xc00001e2c0, 0x0, 0x0, 0x2461300, 0xc000106680, 0x99)
        /src/loki/vendor/gopkg.in/yaml.v2/encode.go:213 +0xfe
gopkg.in/yaml%2ev2.(*encoder).marshal(0xc00001e2c0, 0x0, 0x0, 0x2461300, 0xc000106680, 0x99)
        /src/loki/vendor/gopkg.in/yaml.v2/encode.go:160 +0x767
gopkg.in/yaml%2ev2.(*encoder).marshalDoc(0xc00001e2c0, 0x0, 0x0, 0x2461300, 0xc000106680, 0x99)
        /src/loki/vendor/gopkg.in/yaml.v2/encode.go:93 +0x115
gopkg.in/yaml%2ev2.Marshal(0x2461300, 0xc000106680, 0x0, 0x0, 0x0, 0x0, 0x0)
        /src/loki/vendor/gopkg.in/yaml.v2/yaml.go:203 +0x377
github.com/grafana/loki/pkg/promtail/client.NewLogger(0xc000106410, 0x1, 0x1, 0x1, 0xc0000063f8, 0x30, 0x30)
        /src/loki/pkg/promtail/client/logger.go:45 +0x240
github.com/grafana/loki/pkg/promtail.New(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /src/loki/pkg/promtail/promtail.go:35 +0x178
main.main()
        /src/loki/cmd/promtail/main.go:61 +0x2c8

Enclosing into double quotes like this "my_label": "my" doesn't make any difference. Launching promtail without -dry-run option works normally and do not crash application. Removing single line from config.yaml fixes the issue:

clients:
  - url: http://loki.instance:3100/loki/api/v1/push
    external_labels:

Expected behavior Dry-run should work with any valid configuration file.

Environment: promtail-windows-amd64-v1.4.1.exe -version promtail, version 1.4.1 (branch: HEAD, revision: 2445ccd2) build user: root@bb5e00b6b7f4 build date: 2020-04-06T20:29:26Z go version: go1.13.4

closed time in 13 days

SDedik

push eventgrafana/loki

Cyril Tovena

commit sha c2f100c33424dc305e5ecf7e62839a4dbe950799

Allows to pass inlined pipeline stages to the docker driver. (#2076) * Allows to pass inlined pipeline stages to the docker driver. Fixes #1296 Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com> * doc. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

view details

push time in 13 days

PR merged grafana/loki

Allows to pass inlined pipeline stages to the docker driver. size/L

Fixes #1296

Long overdue, this will help to use pipeline stages with the docker driver. Turns out it's even better when using yaml compose file.

A bit more complicated when using bash.

Signed-off-by: Cyril Tovena cyril.tovena@gmail.com

+190 -26

0 comment

4 changed files

cyriltovena

pr closed time in 13 days

issue closedgrafana/loki

Support for inlined pipeline config for docker logger

Is your feature request related to a problem? Please describe. When there's need to rebuild plugin or to copy file with pipeline to pluigin directory, it's big problem.

options:
    loki-url: ...
    loki-external-labels: container_name={{.Name}}
    loki-pipeline-stage-file: /ag/app.yaml

The problem is /ag/app.yaml is should be inside plugin container, not in root fs!

Describe the solution you'd like Allow to inline pipeline stage config in config parameter. Like this:

options:
    loki-url: ...
    loki-external-labels: container_name={{.Name}}
    loki-pipeline-stages: 
      - json:
        // ...
      - output:
       // ..

Describe alternatives you've considered Only injecting (copying) pipeline file to plugin directory.

closed time in 13 days

wapmorgan

delete branch grafana/loki

delete branch : boltdb-shipper-separate-filesystem

delete time in 13 days

push eventgrafana/loki

Ed Welch

commit sha 9803eab94749ad7bcb5fb647f76ab5fc85eeb5ea

Loki: Allow configuring query_store_max_look_back_period when running a filesystem store and boltdb-shipper (#2073) * Allow configuring query_store_max_look_back_period when running a filesystem store, which allows for boltdb-shipper to horizontally scale filesystem store. Signed-off-by: Ed Welch <edward.welch@grafana.com> * Update docs/operations/storage/boltdb-shipper.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com> * add some more precision to the timestamp used to make the name unique just to make sure there are not collisions if for some reason the ID is empty (which can happen if the hostname doesn't resolve for some reason) Signed-off-by: Ed Welch <edward.welch@grafana.com> * Allow the ingester to query the store for labels if query store is enabled. Signed-off-by: Ed Welch <edward.welch@grafana.com> * make queriers aware of the fact that ingesters may be querying the store so they don't duplicate query efforts Signed-off-by: Ed Welch <edward.welch@grafana.com> * improve the docs Signed-off-by: Ed Welch <edward.welch@grafana.com> * add tests to the calculateMaxLookBack Signed-off-by: Ed Welch <edward.welch@grafana.com> Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>

view details

push time in 13 days

PR merged grafana/loki

Loki: Allow configuring query_store_max_look_back_period when running a filesystem store and boltdb-shipper size/L

This should allow horizontal scaling of a filesystem store on separate nodes as the ingesters can query the store

Signed-off-by: Ed Welch edward.welch@grafana.com

+410 -74

1 comment

10 changed files

slim-bean

pr closed time in 13 days

pull request commentgrafana/loki

Loki: Allow configuring query_store_max_look_back_period when running a filesystem store and boltdb-shipper

I will do a follow up to remove QueryStore as well as refactor the Select method to be able to test the logic within. If I don't do this in 30 days please fire me from the project

slim-bean

comment created time in 13 days

push eventgrafana/loki

Ed Welch

commit sha c15f5a9f2e3d1ba095e2ac6fc57eeb74d7afedc9

add tests to the calculateMaxLookBack Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 13 days

Pull request review commentgrafana/loki

Loki: Allow configuring query_store_max_look_back_period when running a filesystem store and boltdb-shipper

 var modules = map[moduleName]module{ 	}, } -// activeIndexType type returns index type which would be applicable to logs that would be pushed starting now+// activePeriodConfig type returns index type which would be applicable to logs that would be pushed starting now // Note: Another periodic config can be applicable in future which can change index type-func activeIndexType(cfg chunk.SchemaConfig) string {+func activePeriodConfig(cfg chunk.SchemaConfig) chunk.PeriodConfig { 	now := model.Now() 	i := sort.Search(len(cfg.Configs), func(i int) bool { 		return cfg.Configs[i].From.Time > now 	}) 	if i > 0 { 		i-- 	}-	return cfg.Configs[i].IndexType+	return cfg.Configs[i]+}++func calculateMaxLookBack(pc chunk.PeriodConfig, maxLookBackConfig, maxChunkAge time.Duration) (time.Duration, error) {

lol, yeah, i thought i put in the description or somewhere I wanted to test this but hadn't done it yet

slim-bean

comment created time in 13 days

push eventgrafana/loki

Ed Welch

commit sha e9442ea16ad90656efeeeb0486f94d1106467ba5

improve the docs Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 13 days

push eventgrafana/loki

Ed Welch

commit sha af2ac6d94b6b8a3fe11cb0158eda9e712fa8bdb8

make queriers aware of the fact that ingesters may be querying the store so they don't duplicate query efforts Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 13 days

push eventgrafana/loki

Ed Welch

commit sha 46b495fc58dbe76e885e64218c480b359c47bd88

Allow the ingester to query the store for labels if query store is enabled. Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 13 days

push eventgrafana/loki

Ed Welch

commit sha 820f9097a3b97ca75728bb747fba63a24fdaf9a2

add some more precision to the timestamp used to make the name unique just to make sure there are not collisions if for some reason the ID is empty (which can happen if the hostname doesn't resolve for some reason) Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 13 days

push eventgrafana/loki

Ed Welch

commit sha 56fe4e20a35f5ba7ee81243c3870660d7630838b

Update docs/operations/storage/boltdb-shipper.md Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>

view details

push time in 13 days

Pull request review commentgrafana/loki

Loki: Allow configuring query_store_max_look_back_period when running a filesystem store and boltdb-shipper

 Frequency for checking updates can be configured with `resync_interval` config. To avoid keeping downloaded index files forever there is a ttl for them which defaults to 24 hours, which means if index files for a period are not used for 24 hours they would be removed from cache location. ttl can be configured using `cache_ttl` config. +## Horizontal scaling of non-shared filesystem stores++Using the boltdb-shipper also allows running the single binary Loki (or really just the ingesters) with a `filesystem` object store also using a shared hash ring.++If you configure a shared ring via etcd/consul/memberlist you can run multiple instances of Loki on separate machines with separate filesystems,+this now works because the ingesters are able to query the store directly.++To enable this configuration in the ingester config you must set `query_store_max_look_back_period` according to how far back you want to store data, or use a value of -1 for infinite.++At query time, any Loki instance can field the query, the instance will then use the ring to ask every other Loki instance for relevant data, +and because they ingesters can each query their store as far back as `query_store_max_look_back_period` allows, the correct data can be returned.++Scaling up is as easy as adding more loki instances and letting them talk to the same ring.++Scaling down is possible but manual, you would need to shutdown the loki instance and then physically copy the chunks directory and its index files in their entirety to another Loki instance.

Looking at how this works, there is a real chicken and egg problem here when initializing the shipper and trying to get the ingester ID to use in the name file.

I'm not sure it's possible to do this, but maybe you have some ideas?

I'm wondering instead if we should try a different strategy and use the hostname or allow for the name to be configured in the yaml directly?

slim-bean

comment created time in 13 days

Pull request review commentgrafana/loki

Loki: Allow configuring query_store_max_look_back_period when running a filesystem store and boltdb-shipper

 func (cfg *Config) RegisterFlags(f *flag.FlagSet) { 	f.Float64Var(&cfg.SyncMinUtilization, "ingester.sync-min-utilization", 0, "Minimum utilization of chunk when doing synchronization.") 	f.IntVar(&cfg.MaxReturnedErrors, "ingester.max-ignored-stream-errors", 10, "Maximum number of ignored stream errors to return. 0 to return all errors.") 	f.DurationVar(&cfg.MaxChunkAge, "ingester.max-chunk-age", time.Hour, "Maximum chunk age before flushing.")+	f.DurationVar(&cfg.QueryStoreMaxLookBackPeriod, "ingester.query-store-max-look-back-period", 0, "How far back should an ingester be allowed to query the store for data, for use only with boltdb-shipper index and filesystem object store. -1 for infinite.")

If my above assumption is true, that the file names are unique, I specified boltdb_shipper so that you could copy the files between ingesters to allow scaling down, otherwise what you said is true that there isn't much reason to use boltdb_shipper for this rather than boltdb.

With the exception of one other reason I could think of, it would be easier to take backups or even move the Loki to another host because the index files can be directly copied with the chunks directory while Loki is still running, I don't think this is true for boltdb I don't think you can safely copy an open boltdb file.

slim-bean

comment created time in 13 days

Pull request review commentgrafana/loki

Loki: Allow configuring query_store_max_look_back_period when running a filesystem store and boltdb-shipper

 Frequency for checking updates can be configured with `resync_interval` config. To avoid keeping downloaded index files forever there is a ttl for them which defaults to 24 hours, which means if index files for a period are not used for 24 hours they would be removed from cache location. ttl can be configured using `cache_ttl` config. +## Horizontal scaling of non-shared filesystem stores++Using the boltdb-shipper also allows running the single binary Loki (or really just the ingesters) with a `filesystem` object store also using a shared hash ring.++If you configure a shared ring via etcd/consul/memberlist you can run multiple instances of Loki on separate machines with separate filesystems,+this now works because the ingesters are able to query the store directly.++To enable this configuration in the ingester config you must set `query_store_max_look_back_period` according to how far back you want to store data, or use a value of -1 for infinite.++At query time, any Loki instance can field the query, the instance will then use the ring to ask every other Loki instance for relevant data, +and because they ingesters can each query their store as far back as `query_store_max_look_back_period` allows, the correct data can be returned.++Scaling up is as easy as adding more loki instances and letting them talk to the same ring.++Scaling down is possible but manual, you would need to shutdown the loki instance and then physically copy the chunks directory and its index files in their entirety to another Loki instance.

Each ingester should be writing files with a unique name such that you should be able to copy ones file to another without conflicts

uploader := fmt.Sprintf("%s-%d", s.cfg.IngesterName, time.Now().Unix())

However, I think I may have found a bug here, this is what I see for my current test ingesters, it looks like the lifecycler ID was empty when they created their files

ed@ed-VirtualBox:/tmp$ ls loki1/chunks/index/index_2628
-1589422114
ed@ed-VirtualBox:/tmp$ ls loki2/chunks/index/index_2628
-1589422113

These still wouldn't collide because the timestamps were different but they should also have the unique lifecycler ID in front

My suspicion here is they created the files before joining the ring? Can you take a look at this?

slim-bean

comment created time in 13 days

PR opened grafana/loki

Loki: Allow configuring query_store_max_look_back_period when running a filesystem store and boltdb-shipper

This should allow horizontal scaling of a filesystem store on separate nodes as the ingesters can query the store

Signed-off-by: Ed Welch edward.welch@grafana.com

+65 -12

0 comment

6 changed files

pr created time in 14 days

create barnchgrafana/loki

branch : boltdb-shipper-separate-filesystem

created branch time in 14 days

create barnchgrafana/loki

branch : k17

created branch time in 14 days

push eventgrafana/loki

slim-bean

commit sha 6a5d106222fe1e90ebeba45b0a252456d68b64ab

[skip ci] Publishing helm charts: 7dd109763c84f1653ab96e9312cf00e8799b9a89

view details

push time in 14 days

push eventgrafana/loki

Ed Welch

commit sha 7dd109763c84f1653ab96e9312cf00e8799b9a89

expand the match_all to also optionally allow counting bytes per stream (#2066) Signed-off-by: Ed Welch <edward.welch@grafana.com>

view details

push time in 14 days

more