profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/pdf/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Peter Fern pdf Melbourne, Australia

laher/goxc 1669

a build tool for Go, with a focus on cross-compiling, packaging and deployment

bryanwb/chef-rewind 194

monkeypatch chef to edit existing resources in place

Feu-Secret/Tokenmagic 19

A Foundry VTT module that allows you to add animations and graphic effects to tokens, tiles, templates and drawings.

pdf/2dc_jqgrid 4

This plugin allows you to add jQuery datagrids into your applications.

pdf/bower-vendor 4

Vendor the bower assets you want for Ruby on Rails. Inspired by bower-installer. DEPRECATED: I recommend using rails-assets instead

pdf/activeldap 1

ActiveLdap provides an object oriented interface to LDAP.

pdf/active_admin 1

The administration framework for Ruby on Rails applications.

pdf/bettertabs 1

Bettertabs for Rails

pdf/chef 1

A systems integration framework, built to bring the benefits of configuration management to your entire infrastructure.

pdf/alertmanager 0

Prometheus Alertmanager

push eventpdf/zfs_exporter

Peter Fern

commit sha 20182da452d497ff7f050eded4b3e58367e6312e

fix(docs): Document web.disable-exporter-metrics flag in README

view details

Peter Fern

commit sha e6fbdf523dc4151037f0ea8986a3ba48d8577a2a

fix(collector): Avoid race on upstream channel close, tidy sync points

view details

push time in 13 days

pull request commentrspamd/rspamd

[Fix] Namespace and add metadata for OpenMetrics, fix interleaving

Thank you for the PR! The only comment is that there is a clear memory leak of the strings you have introduced. But I will fix that, no worries.

Ah right, I should have called rspamd_fstring_free on the temporary strings after appending them to the output? Apologies.

pdf

comment created time in 18 days

push eventpdf/rspamd

Peter Fern

commit sha b6f2a0a6acce95ef6e2339d529b8b598ef35ad99

[Fix] Namespace and add metadata for OpenMetrics, fix interleaving

view details

push time in 18 days

PR opened rspamd/rspamd

[Fix] Namespace and add metadata for OpenMetrics, fix interleaving

Let me preface this PR by saying that I don't write C, so reviewers may wish to handle with care.

Per discussion in #3754 this PR makes the following changes:

  • All metrics have HELP and TYPE defined (with varying usefulness of HELP, since some of the stats are non-obvious to me)
  • Metrics are now successfully parsed as the specified openmetrics-text content-type (spurious spaces between labels were causing Prometheus to barf)
  • Metrics are no longer interleaved for statfiles_*
  • rspamd-specific metrics are prefixed with rspamd_

Some possibly contentious decisions:

  1. Metrics that appear to me to be monotonically increasing values have been marked as counter type, and consequently have _total appended to their metric names. If these values are not in fact monotonically increasing, these metrics will need to be changed to gauge type, but if that's the case it would be preferable for those metrics to contain a delta of the value between collections (though that is not critical).
  2. The statfiles_total metric was renamed to rspamd_statfiles_totals (note the trailing s), since the _total suffix is reserved for counter values, and I'm not sure whether that value meets the requirements for a counter (it's 0 on my instances, and I have no idea what it denotes).
+114 -27

0 comment

1 changed file

pr created time in 18 days

create barnchpdf/rspamd

branch : fix_openmetrics

created branch time in 18 days

fork pdf/rspamd

Rapid spam filtering system.

fork in 18 days

pull request commentrspamd/rspamd

Implement an endpoint for OpenMetrics / Prometheus

Also, there are some problems with the metrics - OpenMetrics specifies that metrics MUST NOT be interleaved, however the statfiles_* metrics are interleaved, also there's no metadata included, and Prometheus fails to parse the metrics, with: expected label name, got "INVALID". Also, it would be greatly appreciated if these metrics were prefixed with rspamd_.

mrueg

comment created time in 20 days

pull request commentrspamd/rspamd

Implement an endpoint for OpenMetrics / Prometheus

Is there documentation somewhere on how to allow metrics collection from an address other than localhost?

mrueg

comment created time in 20 days

issue closedpdf/zfs_exporter

Disabling go metric collection

Is there a way to not collect certain metrics, like go metrics?

# HELP go_gc_duration_seconds A summary of the GC invocation durations.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 0
go_gc_duration_seconds{quantile="0.25"} 0
go_gc_duration_seconds{quantile="0.5"} 0
go_gc_duration_seconds{quantile="0.75"} 0
go_gc_duration_seconds{quantile="1"} 0
...

I think this is possible in Node Exporter (see https://github.com/prometheus/node_exporter/pull/1148).

closed time in 21 days

martinvelez

push eventpdf/zfs_exporter

Peter Fern

commit sha 066c7d294e7589b0219ba6679c9950972bcec1af

fix(docs): Correct misspelling

view details

Peter Fern

commit sha 1ca87176e10b1aa338506319d38e6a8edd4b5176

feat(metrics): Allow disabling exporter metrics Fixes #2

view details

Peter Fern

commit sha b84092b8d9d7be5671e82c93da7a4fa8757617f2

refactor(zfs): Allow testing collector output TODO: Test caching

view details

push time in 21 days

issue commentpdf/kdeconnect-chrome-extension

KDE sees my device but not the Firefox extention...

Then I have no idea why one native host would work, and the other wouldn't. Extremely confusing :confused:

Jiminimonka

comment created time in a month

issue commentpdf/kdeconnect-chrome-extension

KDE sees my device but not the Firefox extention...

I am using another NativeMessaging plugin (https://github.com/smorks/keepassnatmsg) that resides in the folder you mention and works from there.

That's very confusing then. Are you certain you restarted the browser after install?

Jiminimonka

comment created time in a month

issue commentpdf/kdeconnect-chrome-extension

KDE sees my device but not the Firefox extention...

@HarHarLinks if vivaldi-snapshot stores its user data in a different location to standard vivaldi, you'll need to copy ~/.config/vivaldi/NativeMessagingHosts/ to the equivalent config directory. I'm afraid I can't test if that's the case though, and I'm not certain how it's packaged.

Jiminimonka

comment created time in a month

push eventpdf/zfs_exporter

Peter Fern

commit sha f47b69a8f40326492a0dbc484c64dd12be38cfda

fix(build): Update to Go 1.17 for crossbuild, and enable all platforms

view details

Peter Fern

commit sha b39382b5486aff77e17ae51ba6b0dc092a216355

fix(core): Update dependencies

view details

push time in a month

pull request commentprometheus/golang-builder

Bump Go version

Can someone explain to me what the process is here? prombot just keeps closing and opening new PRs on loop, makes it very hard to track when this change goes live.

prombot

comment created time in a month

issue commentpdf/zfs_exporter

Renormalized values are counterintuitive (case compressratio)

I went with retaining them as ratios, but as a ratio of e.g. compressed vs uncompressed size, and tried to clarify the help strings to indicate more clearly what the value means.

riyad

comment created time in a month

push eventpdf/zfs_exporter

Peter Fern

commit sha 1a7bc3a55e311ab4048ebb182e998a782d556028

fix(metrics): Invert ratio for multiplier fields, and clarify their docs Fixes #11

view details

Peter Fern

commit sha 8e14a7bd72f7f676d02395a97d345168bdf7b9bc

chore(ci): Add test action for PRs

view details

Peter Fern

commit sha 5d5c6eb2f0b5871c80b4374b999f415a93fffb26

chore(docs): Add badges to README

view details

Peter Fern

commit sha 1760a4a5e63df6d157245a652fe14c772c354e9c

fix(logging): Include collector in warning for unsupported properties

view details

Peter Fern

commit sha b64115c4ae9a1bbbcf5f4a09854090ea449bf449

feat(build): Update to Go 1.17

view details

push time in a month

issue closedpdf/zfs_exporter

Renormalized values are counterintuitive (case compressratio)

It seems like the changes made in version 2.0.0 to renormalize values to be between 0..1 seems to produce wrong(?) results:

case in point compressratio:

$ zfs list -po name,refcompressratio,logicalreferenced,referenced rpool/crypt/var/log
NAME                 REFRATIO      LREFER       REFER
rpool/crypt/var/log      4.26  6032219136  1445105664

It's clear how the ratio is calculated: REFRATIO = LREFER/REFER and it's clear what it means: the "real" data is 4.26 times larger than what's written to disk ... i.e. compression shrunk the data to roughly 1/4 (~24%) of it's original size.

But activating the new metric gives numbers like this:

zfs_dataset_compression_ratio{name="rpool/crypt/var/log",pool="rpool",type="filesystem"} 0.7830802603036877

I was confused :confused: After looking at the formula (i.e. transformMultiplier()) I could rationalize that this is the ratio of data that was "compressed away" (i.e. ~ 1 - 0.24) ... which is "not so useful". I'm ok with renormalizing percentages, but "reformulating" ratios is maybe going to far ... while mathematically correct, they change their "meaning" (i.e. label <-> number correspondence).

My suggestion would be to remove ratios completely (when they can be calculated from other data: compression, capacity, etc.) or to report them "as is" (i.e. to keep the mental model close to what the CLI reports). Not all ratios are between 0 and 1, nor do they need to be. Whoever uses those ratios will probably know what to do with them (including transformations). :wink:

closed time in a month

riyad

issue commentpdf/zfs_exporter

Renormalized values are kind of wrong (case compressratio)

  1. I feel a little like like this is kicking the can down the road, and inviting issues asking why these props are not supported. The general recommendation is indeed to avoid exposing ratio metrics when the consituent parts are available, however I don't believe there's a way to calculate dedupratio, which means this probably needs solving, at which point we may as well just expose them all.
  2. I suppose if I was to do this, I would want to rename the metrics to _multiplier or something, since ratio has a particular meaning as a base unit for Prometheus/OpenMetrics.

the current formula is like looking at things mirrored (1/x) and from the "back" (1-x)

If I was to keep these as ratios, I think I'd change them to be simply "mirrored", rather than both.

I can't say I feel particularly enthusiastic about any of the available options :-/

riyad

comment created time in a month

issue commentpdf/zfs_exporter

Renormalized values are kind of wrong (case compressratio)

I think I'm leaning towards using the ratio of compressed space to uncompressed, and updating the help text to reflect that this is what is reported, thoughts?

riyad

comment created time in a month

issue commentminio/minio

Prometheus metrics path /minio/v2/metrics/cluster returns 403 with generated bearer token

not sure where you found the idea to use it in this manner.

I didn't "find the idea", I just happened to be authenticating from the CLI via a service token, which does not seem that unusual. If there are requirements for a feature to work, either documenting what they are, or including error handling to prevent non-functional operation would greatly improve the user experience.

pdf

comment created time in a month

issue commentminio/minio

Prometheus metrics path /minio/v2/metrics/cluster returns 403 with generated bearer token

Service Accounts cannot be used and they are never to be used here.

Again, entirely undocumented, and very confusing.

pdf

comment created time in a month

issue commentminio/minio

Prometheus metrics path /minio/v2/metrics/cluster returns 403 with generated bearer token

The user that is configured mc alias ls <alias> needs to have sufficient permissions to access Prometheus if you need a user who needs to have permissions with Actions: "admin:Prometheus" in your policy.

This is not mentioned in the documentation anywhere, that should probably be fixed. Not only because this permission is required, but also because it's now clear to me that the bearer token does not have a policy applied to it to restrict permissions to only what's required for the metrics endpoint, making it a security concern.

That said, I've done further testing based on your feedback, and the problem appears to be that if you're logged in via a service account, no matter what permissions you have, the generated bearer token is unable to authenticate.

pdf

comment created time in a month

issue openedminio/minio

Prometheus metrics path /minio/v2/metrics/cluster returns 403 with generated bearer token

Expected Behavior

As documented in the README, the default authentication mode for this path is JWT, and a bearer token with config snippet can be generated via mc admin prometheus generate <alias>, to authenticate against the endpoint when JWT auth is enabled.

Current Behavior

After generating the config snippet and adding it to the prometheus configuration, the scraper fails, indicating a 403 error. Attempting to authenticate using the generated bearer token via the CLI confirms that it is not possible to authenticate with this token (sanitized):

$ curl -v -sSL -H "Authorization: Bearer ${TOKEN}" http://localhost:9000/minio/v2/metrics/cluster
*   Trying ::1:9000...
* TCP_NODELAY set
* Connected to localhost (::1) port 9000 (#0)
> GET /minio/v2/metrics/cluster HTTP/1.1
> Host: localhost:9000
> User-Agent: curl/7.68.0
> Accept: */*
> Authorization: Bearer <token>
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 403 Forbidden
< Content-Security-Policy: block-all-mixed-content
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Vary: Origin
< X-Amz-Request-Id: 169BC0953F19E447
< X-Content-Type-Options: nosniff
< X-Xss-Protection: 1; mode=block
< Date: Mon, 16 Aug 2021 09:50:37 GMT
< Content-Length: 0
< 
* Connection #0 to host localhost left intact

Possible Solution

Setting MINIO_PROMETHEUS_AUTH_TYPE="public" allows access to the metrics by disabling authorization, but this is undesirable.

Steps to Reproduce (for bugs)

  1. Install minio cluster
  2. Leave MINIO_PROMETHEUS_AUTH_TYPE unset, or explicitly set to jwt
  3. Generate bearer token via mc admin prometheus generate
  4. Attempt to authenticate against /minio/v2/metrics/cluster using token
  5. Receive 403 error

Your Environment

<!--- Include as many relevant details about the environment you experienced the bug in -->

  • Version used: RELEASE.2021-08-05T22-01-19Z
  • Operating System: Ubuntu 20.04 Linux 5.4.0-77-generic SMP x86_64 x86_64 x86_64 GNU/Linux

created time in a month

issue commentpdf/zfs_exporter

Renormalized values are kind of wrong (case compressratio)

The Prometheus docs recommend that ratios are expressed in the range 0-1, in the same way that bytes is the default data unit, which results in queries being easily applied based on the metric type. In this case, the result is that the compression rate is 78%, though I suppose an argument might be made that this should be expressed as the ratio of compressed size to uncompressed, ie 22%.

Properties like capacity are already reported by ZFS as percentages, it's only compression and dedup ratios that are reported in this multiplier format (which seems like a really odd way of expressing this).

Honestly, I'm not certain what the correct answer is here, I'm going to sleep on it.

riyad

comment created time in a month

issue commentpdf/zfs_exporter

Can we have more dataset properties?

Please take v2.0.0 for a spin - I've added most of the numeric properties, and the collected properties are now available to be configured per-collector via command-line flags on the exporter (it defaults to the properties that used to be available).

If you find that there are any properties that are still unsupported, I'd welcome a PR for them ;-)

riyad

comment created time in a month

Pull request review commentpdf/zfs_exporter

Refactor collector by DRYing code

 func registerCollector(collector string, isDefaultEnabled bool, factory factoryF func expandMetricName(prefix string, context ...string) string { 	return strings.Join(append(context, prefix), `-`) }++func newDesc(subsystem string, metric_name string, help_text string, labels []string) desc {+	var name = prometheus.BuildFQName(namespace, subsystem, metric_name)+	return desc{+		name: name,+		prometheus: prometheus.NewDesc(+			name,+			help_text,+			labels,+			nil,+		),+	}+}++func newMetric(metric_desc *desc, value float64, labels []string) metric {+	return metric{+		name: expandMetricName(metric_desc.name, labels...),+		prometheus: prometheus.MustNewConstMetric(+			metric_desc.prometheus,+			prometheus.GaugeValue,

I ended up agreeing with you on this in my refactor.

riyad

comment created time in a month

PullRequestReviewEvent

PR closed pdf/zfs_exporter

Reviewers
Refactor collector by DRYing code

I extracted two helper functions to reduce repetition when creating metrics descriptions (i.e. newDesc()) as well as metrics data points themselves (i.e. newMetric()) ... they're named after the data structures they return (no creative reinterpretation there). :smile:

Also I tried to sort metrics (alphabetically) where possible. Otherwise it's difficult to keep track of them. :sweat_smile:

+252 -353

3 comments

3 changed files

riyad

pr closed time in a month

pull request commentpdf/zfs_exporter

Refactor collector by DRYing code

Closed in favor of 53b0e9814f66f6b3d1de03c139594d14f6396cbe

riyad

comment created time in a month