Why can't we set the same partition for different nodes like with kafka-mdm-in "partitions"

IIRC you can. that said I would advise against running a cluster with carbon input. to me cluster implies HA. but the problem with carbon is, as soon as a MT instance needs to restart it needs to replay data to fill its chunks, but you can't do that with carbon. kafka as a WAL solves that problem.

strange, according to "movingWindow": {NewMovingWindow, true} it should work. the other reason why it might proxy is if you have this

# proxy to graphite when metrictank considers the request bad
proxy-bad-requests = true

AND metrictank considers the query invalid (e.g. bad parameters not matching signature). the logs should say

strangely the behavior requested in #5212 is not how vim works at all... the behavior asked for here, is, however.

This adds the long-needed ability use mt-index-cat on Bigtable.

also this check here is redundant since you already check every time after showing and we only call processDefs once.

note that by printing to stdout here, and below, we break the output format. a common use case is to pipe the output of mt-index-cat into other tool, so i'm hesitant of adding these new log messages. (note that all other/pre-existing log calls are error messages or help output

		fmt.Printf("idxtype: 'cass' (cassandra) or 'bt' (bigtable)\n\n")

	globalFlags.IntVar(&btTotalPartitions, "bt-total-partitions", -1, "total number of partitions (when using bigtable and partitions='*')")

I would like to import and use these functions somewhere else, would you mind if we export them?

see docs for explanation :)

oops. i'm sorry i was too trigger happy. was not expected, actually the builds have been broken since last week but yeah let's fix it!

if you do this, you should clarify at least in the config files that these settings will be ignored for query mode

Let's just update everything (tests+ build) to the latest Go version.

is it possible to let it run without aggregator? reviewing the changelog, seems most changes for the 0.13.0 release was related to aggregators. I suspect either that, or something to do with matchers (anything that has a prefix/substring/regex condition on it, such as a route), but more likely aggregator.

ping @shanson7 your thoughts on the above?

e.g. /metrics/index.json

Hi i also have this problem with version 1.2.1 Out of the blue, parcellite just refuses to start when I start I get errors like this:

<stuff i copied from primary selection buffer>No magic! Assume no history.
Attempt to unlock mutex that was not locked
[1]    316828 abort (core dumped)  parcellite

and this:

<stuff i copied from primary selection buffer>No magic! Assume no history.
Attempt to unlock mutex that was not locked
[1]    316828 abort (core dumped)  parcellite
No magic! Assume no history.
Attempt to unlock mutex that was not locked
[1]    317334 abort (core dumped)  parcellite

Edited help descriptions

strange, i can't reproduce the issue anymore with format = "{ssid} {signal_strength} {speed_up} {speed_down}" either for both format strings i disconnect from wifi using network manager and it all seems to work fine. when the issue originally appeared it wasn't after simply manually disconnecting, i was having wifi issues where i lost connectivity due to a still undiagnosed issue.

Hi Jason, with that format string it seems to work fine.

interesting. I have

[[block]]
block = "net"
device = "wlp3s0"
format = "{ssid} {signal_strength} {speed_up} {speed_down}"
interval = 5
use_bits = false

I note you have a signal_strength setting which per https://github.com/greshake/i3status-rust/blob/master/blocks.md#deprecated-options is deprecated.

which version do you use? i have it with latest master.

Error in block 'net': Signal strength is only available for connected wireless devices.

we probably just need a small error handling tweak.

This PR is basically how i wanted #1743 to be all along :)

update graphite.md to clarify how we pick the consolidator, which takes storage-aggregation into account.

while we're at it, i think we can mention the response metadata as well.

seems like a very niche problem. let's close

Hello @gosuri , thanks for this nice library. What do you think of this PR? It fixes a bug for me and seems to work well to me. thanks.

hi @ammgws i have addressed the feedback. thanks!

Hello, first of all, thanks for the work that's been put into guru. It's certainly been quite helpful to me over the years. I know guru is no longer actively maintained (and I don't expect a resolution), but considering gopls is still in alpha, i thought it might be helpful to others (or to my future self) to document this issue.

What version of Go are you using (go version)?

<pre> go version go1.15.1 linux/amd64 </pre>

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> Output</summary><br><pre> go env GO111MODULE="" GOARCH="amd64" GOBIN="" GOCACHE="/home/dieter/.cache/go-build" GOENV="/home/dieter/.config/go/env" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOINSECURE="" GOMODCACHE="/home/dieter/go/pkg/mod" GONOPROXY="" GONOSUMDB="" GOOS="linux" GOPATH="/home/dieter/go" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/usr/lib/go" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64" GCCGO="gccgo" AR="ar" CC="gcc" CXX="g++" CGO_ENABLED="1" GOMOD="" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/dieter/go-build095063144=/tmp/go-build -gno-record-gcc-switches" </pre></details>

I found out that based on the working directory when guru implements is run, it can take an extreme amount of cpu and take a very long time.

In particular, consider these 3 types of runs:

1) cd /home/dieter && \
    guru -json implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'
2) cd /home/dieter/go/src/github.com/prometheus/prometheus && \
    guru -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
3) cd /home/dieter/go/src/github.com/prometheus/prometheus && \
    guru -json -scope github.com/prometheus/prometheus/... implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'

only 1) returns in a reasonable amount of time. 2) is how visual studio code invokes it (when i'm inside of the prometheus project), and 3) is my attempt to fix the problem with 2) by constraining the scope.

More info about each run:

Note: I moved guru to guru-real and replaced guru with this script:

$ cat /home/dieter/go/bin/guru
#!/bin/bash
file="/home/dieter/guru-$(date '+%F_%T')"
echo "guru $@" > $file
env >> $file
guru-real "$@"

this allows me to track the exact environment of the process each time it is run (particularly when the parent is vscode and not my terminal), but it turns out the issue was reproducible perfectly from a terminal, so at this point it's no longer useful, but it does explain why i have mentions of "guru-real" in the pstree output below.

run 1

time /home/dieter/go/bin/guru -json implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'
{
	"type": {
		"name": "github.com/prometheus/prometheus/storage.Storage",
		"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:43:6",
		"kind": "interface"
	},
	"to": [
		{
			"name": "*github.com/prometheus/prometheus/cmd/prometheus.readyStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:898:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/storage.fanout",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout.go:33:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/storage/remote.Storage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/remote/storage.go:48:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/tsdb.DB",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/tsdb/db.go:120:6",
			"kind": "pointer"
		},
		{
			"name": "github.com/prometheus/prometheus/storage/fanout.errStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout/fanout_test.go:152:6",
			"kind": "struct"
		},
		{
			"name": "github.com/prometheus/prometheus/util/teststorage.testStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/util/teststorage/storage.go:46:6",
			"kind": "struct"
		}
	],
	"from": [
		{
			"name": "github.com/cortexproject/cortex/vendor/github.com/prometheus/common/expfmt.Closer",
			"pos": "/home/dieter/go/src/github.com/cortexproject/cortex/vendor/github.com/prometheus/common/expfmt/encode.go:40:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/storage.Appendable",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:35:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/storage.Queryable",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:56:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt.Closer",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt/encode.go:40:6",
			"kind": "interface"
		},
		{
			"name": "io.Closer",
			"pos": "/usr/lib/go/src/io/io.go:98:6",
			"kind": "interface"
		}
	]
}
/home/dieter/go/bin/guru -json implements   95.38s user 26.11s system 733% cpu 16.554 total

pstree -a -t -l | less
  |   |   `-guru /home/dieter/go/bin/guru -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |       `-guru-real -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |           `-13*[{guru-real}]

run 2

I didn't let it complete. I killed it after it ran for hours.

pstree -a -t -l | less
  |   |   `-guru /home/dieter/go/bin/guru -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |       `-guru-real -json implements /home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/share
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/hello
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/shape
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/peter-rabbit
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/showdeps
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/rog-go/cmd/share2
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/go.tools/godoc/redirect
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/go.tools/godoc/static
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/unsafeunmarshaler
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/marshalto
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/populate
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/stringer
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- code.google.com/p/gogoprotobuf/plugin/unmarshal
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/addons/kube-proxy
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/gce/manifests
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/juju/prereqs
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cluster/kubemark/pre-existing
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/docs/yaml/kubectl
  |   |           |   `-10*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cmd/kube-apiserver/app
  |   |           |   `-11*[{go}]
  |   |           |-go list -e -compiler=gc -tags= -installsuffix= -f={{.Dir}}\012{{.ImportPath}}\012{{.Root}}\012{{.Goroot}}\012{{if .Error}}{{.Error}}{{end}}\012 -- k8s.io/kubernetes/cmd/kube-controller-manager/app
  |   |           |   `-11*[{go}]
  |   |           `-35*[{guru-real}]

run 3

time /home/dieter/go/bin/guru -json -scope github.com/prometheus/prometheus/... implements '/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:#1503'
{
	"type": {
		"name": "github.com/prometheus/prometheus/storage.Storage",
		"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:43:6",
		"kind": "interface"
	},
	"to": [
		{
			"name": "*github.com/prometheus/prometheus/cmd/prometheus.readyStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:898:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/storage.fanout",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout.go:33:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/storage/remote.Storage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/remote/storage.go:48:6",
			"kind": "pointer"
		},
		{
			"name": "*github.com/prometheus/prometheus/tsdb.DB",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/tsdb/db.go:120:6",
			"kind": "pointer"
		},
		{
			"name": "github.com/prometheus/prometheus/storage/fanout.errStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/fanout/fanout_test.go:152:6",
			"kind": "struct"
		},
		{
			"name": "github.com/prometheus/prometheus/util/teststorage.testStorage",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/util/teststorage/storage.go:46:6",
			"kind": "struct"
		}
	],
	"from": [
		{
			"name": "github.com/prometheus/common/expfmt.Closer",
			"pos": "/home/dieter/go/pkg/mod/github.com/prometheus/common@v0.10.0/expfmt/encode.go:40:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/storage.Appendable",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:35:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/storage.Queryable",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/storage/interface.go:56:6",
			"kind": "interface"
		},
		{
			"name": "github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt.Closer",
			"pos": "/home/dieter/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/common/expfmt/encode.go:40:6",
			"kind": "interface"
		},
		{
			"name": "io.Closer",
			"pos": "/usr/lib/go/src/io/io.go:98:6",
			"kind": "interface"
		}
	]
}
/home/dieter/go/bin/guru -json -scope github.com/prometheus/prometheus/...    5478.10s user 886.50s system 246% cpu 42:57.99 total

Once again, feature asked from years ... still no implementation

Out of all the items of work needed to achieve an implementation, the asking for it part is fairly minor.

If you want this to happen, you can make a concrete proposal or implementation, or pay someone (e.g. Grafana Labs) to do it for you.

ok, will change

Not sure. I see that block and button are used interchangeably in this file. e.g. for the cpu usage block on_click also refers to a button. should it be block everywhere consistently? I could do that in a followup PR.

couldn't find much in the style guide about this specific tradeoff of using map_filter + next for processing Option's; but since @themadprofessor 's suggestion definitely seems more in line with the rest of the code in general, i've applied the tweak in a new commit. does this look good to merge now?

you're fine with the current code in this PR or with @themadprofessor 's version? aside from "fine", is there a concensus which is "better"? Also @ammgws seems you're a de facto maintainer, perhaps you should be added to https://github.com/greshake/i3status-rust/blob/master/CONTRIBUTING.md#maintainership ?

I think the people best suited to review this are @robert-milan and @shanson7 ?

@themadprofessor very interesting. thanks for sharing. so, basically use filter_map as a way to "absorb" intermediate None values and turn them into a "final" None value through next().

Interestingly, in the go world we tend to favor simple code even when it's a bit more verbose, but if the prevalent style in rust is to favor a bit more magic to create more concise code, that's fine with me too. I'm happy to add a commit that changes it to your style. is that what @ammgws would prefer?

that results in an error on the line below:

error[E0599]: no method named `as_bytes` found for struct `regex::re_bytes::Captures<'_>` in the current scope
   --> src/blocks/net.rs:999:41
    |
999 |     maybe_ssid_convert(result.map(|x| x.as_bytes()))
    |                                         ^^^^^^^^ method not found in `regex::re_bytes::Captures<'_>`

it's proven to be beneficial

Might make sense to mention somewhere what format of the dump it expects

as the doc and the help text say:

reads metric names from stdin

could be as simple as "querystart needs to be between serieStart and serieEnd for the loop below to work (this should always be the case)" . at least that's my interpretation. maybe you can explain it better.

I see. well given how we adjust context.from in FuncMovingWindow.Context() (we reduce from), serieStart will always be before querystart. but ok, i have no objection against making sure.I do think these lines of code need some clarification about why they're checking this, what the concern is, how things might break if the condition is not met, etc

so, the fact that we only allow a window size as interval string, not as a integer number of points, is an exception to document. if, that aside, everything else behaves exactly like graphite does - to your knowledge - then yes you can mark it as stable.

yes but the Signature() function is what is used to validate the query (function arguments) provided by the user. if we already set the func at construction time, we shouldn't look for an input parameter for it. See NewAggregate() / FuncAggregate.Signature(), it's the same idea.

see below

i couldn't find a way to do it more elegantly. i needed to convert the match object to to a &[u8], so the best way i saw was to retrieve it out of the Option, and call as_bytes() on it. And to be able to do that, i similarly had to pull out the captures out of the option.

I did verify though, that just calling the same processing chain, using unwrap() and removing the is_none() checks, and passing the result to maybe_ssid_convert, leads to a panic in case the output from iw does not match the expected format

1. don't include extraneous 'SSID: ' prefix. see #814
2. an SSID is allowed to contain any character. update regex accordingly. in particular mine is of the form foo-1234 and my ssid was reported as 'foo'.

I'm a rust beginner so any feedback on the code is welcome. I think more can be fixed in this block (e.g. adjusting the other regexes etc), but one small step at a time :)