profile
viewpoint
sh0rez sh0rez @Grafana Germany https://shorez.de Cargo ship pilot at Grafana Labs. Creator of Tanka

grafana/tanka 1114

Flexible, reusable and concise configuration for Kubernetes

jsonnet-bundler/jsonnet-bundler 235

A jsonnet package manager.

grafana/grizzly 57

A utility for managing Jsonnet dashboards against the Grafana API

monitoring-mixins/mixtool 25

mixtool is a helper for easily working with jsonnet mixins.

sh0rez/awesome-libsonnet 13

Curated list of Kubernetes focused Jsonnet resources

jsonnet-libs/docsonnet 11

Experimental Jsonnet docs generator

sh0rez/tkbyexample 4

Tanka by Example

sh0rez/asciimatrix 2

golang library to plot ascii text to a coordinate matrix in the terminal

sh0rez/klint 2

Clint Eastwood validates your Kubernetes configs

sh0rez/jsonnet-builder 1

Write Jsonnet code from Go

Pull request review commentjsonnet-libs/k8s

Generalised configuration, CRD resources support

 specs:-  1.14: https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.14/api/openapi-spec/swagger.json-  1.15: https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.15/api/openapi-spec/swagger.json-  1.16: https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.16/api/openapi-spec/swagger.json-  1.17: https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.17/api/openapi-spec/swagger.json-  1.18: https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.18/api/openapi-spec/swagger.json++# Kubernetes core APIs+- output: 'gen/local'+  openapi: http://localhost:8001/openapi/v2+  prefix: io.k8s.api+  patchDir: custom/core+  extensionDir: extensions/core

Generally happy with this! Could you re-include the Kubernetes swagger urls for 1.14 to 1.18, so that default behavior of this repository stays the same?

For CRD libraries, we could instead create new repositories on GitHub and have both the respective config and the artifacts in there. wdty?

xvzf

comment created time in 2 days

PullRequestReviewEvent

issue commentgrafana/tanka

tanka fmt tries to format files in vendor/ on windows

@nlowe Awesome debugging, thanks for that! As you already figured out the root cause, I'd like to invite you to file a PR to fix this :)

nlowe

comment created time in 3 days

Pull request review commentgrafana/tanka

feat: inline Environment

 func New() *Config { // Config holds the configuration variables for config version v1alpha1 // ApiVersion and Kind are currently unused, this may change in the future. type Config struct {-	APIVersion string   `json:"apiVersion"`-	Kind       string   `json:"kind"`-	Metadata   Metadata `json:"metadata"`-	Spec       Spec     `json:"spec"`+	APIVersion string      `json:"apiVersion"`+	Kind       string      `json:"kind"`+	Metadata   Metadata    `json:"metadata"`+	Spec       Spec        `json:"spec"`+	Data       interface{} `json:"data"`

I'd like us to think about this a bit more.

Generally we seem to have two ways here:

  1. Having tanka.dev/Environment as yet another top level field that is recognized and filtered out by Tanka
  2. What this PR currently implements, understanding Environment effectively as a data holder type

I would not worry too much about compatibility here. If an older Tanka version encounters an empty spec.json / no at all, apply will inevitably fail, so no harm can be done.

That being said, Option 1 feels more lightweight to me, it requires less changes to your code (you just embed or import spec.json).

Ultimately both solutions are fine for me, but I would like hear some arguments pro / contra those :)

/cc @malcolmholmes @jdbaldry @igorwwwwwwwwwwwwwwwwwwww @mikulas <-- You two were interested in this before

Duologic

comment created time in 3 days

PullRequestReviewEvent

push eventgrafana/tanka

Justin Walz

commit sha 1220d149a4f3210d8932f41f415599e88cbbee41

fix(helm): Detect already pulled charts (#402) No longer attempts to download Helm Charts that are already locally in the `charts` folder. To re-download a chart, `rm` it from the `charts` folder and run `tk tool charts vendor` again!

view details

push time in 3 days

PR merged grafana/tanka

fix(helm): Check prior vendoring of helm charts

Fixes https://github.com/grafana/tanka/issues/401

+15 -1

0 comment

1 changed file

justinwalz

pr closed time in 3 days

issue closedgrafana/tanka

Calling vendor twice shouldn't error the second time if the chart already exists locally

Mentioned this on slack, and a GitHub issue was requested. Copied from thread:

Output from two vendor calls

# tk-0.12.0 tool charts vendor
Syncing Repositories ...
Pulling Charts ...
 stable/mysql@1.6.7
 eks/aws-node-termination-handler@0.10.0
# tk-0.12.0 tool charts vendor
Syncing Repositories ...
Pulling Charts ...
Error: failed to untar: a file or directory with the name /home/ME/MY/PATH/charts/mysql already exists
exit status 1

closed time in 3 days

justinwalz
PullRequestReviewEvent

Pull request review commentgrafana/tanka

fix(helm): Check prior vendoring of helm charts

 func parseReq(s string) (*Requirement, error) { 		Version: *ver, 	}, nil }++// parseReqName parses a name from a string of the format `repo/name` or `name`+func parseReqName(s string) string {

Thanks @justinwalz!

justinwalz

comment created time in 3 days

PullRequestReviewEvent

delete branch grafana/tanka

delete branch : duologic/allow_alternative_entrypoint

delete time in 3 days

push eventgrafana/tanka

Jeroen Op 't Eynde

commit sha 59a5f5f8b9eaabe3d13e1de28920bff9d4a3f769

feat(jsonnet): allow other entrypoints (#389) So far, Tanka always evaluates `main.jsonnet` whatever directory it receives from the user. Going on, this is still supported and the default behavior. When passed a plain file instead of a directory instead however, Tanka evaluates that file now, choosing the enclosing directory as the environments directory. This makes `tk show` and `tk eval` more lightweight, requiring only a single file, not a whole directory.

view details

push time in 3 days

PR merged grafana/tanka

feat: allow other entrypoints kind/enhancement

The change allows the user to run tanka with alternative entrypoints, not limiting to main.jsonnet anymore.

This opens the door to using tk eval for purposes outside of the Tanka/K8s Environments paradigm. For example, one can write a library with a test.jsonnet and run tk eval test.jsonnet in that library. Caveat: it needs to have a tkrc.yaml or jsonnetfile.json so Tanka can resolve the JPATH. It also works for tk export and tk show, beware that it will add default namespace to kubernetes objects, even though a spec.json might not exist.

This change is backwards compatible and still defaults to main.jsonnet if no file is given, tk diff will still want a spec.json to connect to a cluster.

Feel free to give feedback/pushback on the idea here too.

I've done a grep on main.jsonnet in the codebase to set this up, but there are many other valid references to main.jsonnet left that don't need replacing.

+66 -38

3 comments

5 changed files

Duologic

pr closed time in 3 days

PullRequestReviewEvent

push eventgrafana/tanka

Gene Liverman

commit sha 7aad7b98eb353bb5b25aa540dc915f240dc7868b

doc(install): Jsonnet Bundler via brew (#405) This updates the docs to install jb in the same way tanka is instealled: via brew on macOS.

view details

push time in 3 days

push eventgrafana/tanka

Gene Liverman

commit sha 321f9a8e0d85bb8941d538014521fcdbd426edfa

Jsonnet Bundler via brew (#405) This updates the docs to install jb in the same way tanka is instealled: via brew on macOS.

view details

push time in 3 days

PR merged grafana/tanka

Jsonnet Bundler via brew

I am not 100% sure this is all that is needed, but I noticed today while reading your docs that you didn't indicate jb could be installed through brew right along with tanka so I thought I'd take a stab at fixing that. Hope this helps.

+12 -2

3 comments

3 changed files

genebean

pr closed time in 3 days

PullRequestReviewEvent

Pull request review commentgrafana/tanka

Adds `tk fmt` documentation

+---+name: "Formatting"+route: "/formatting"+menu: "References"+---++# File Formatting++Tanka supports formatting for all `jsonnet` and `libsonnet` files using the `tk fmt` command.++By default, the command excludes all `vendor` directories and any files in the immediate directory.

and any files in the immediate directory

Could you explain what you mean by this sentence? To me it reads like tk fmt . would not format the files in the current directory, which is not true afaict

heuristicAL

comment created time in 3 days

Pull request review commentgrafana/tanka

Adds `tk fmt` documentation

+---+name: "Formatting"+route: "/formatting"+menu: "References"+---++# File Formatting++Tanka supports formatting for all `jsonnet` and `libsonnet` files using the `tk fmt` command.++By default, the command excludes all `vendor` directories and any files in the immediate directory.++```bash+# Run for current and child directories. Run this in the root of the project to format all your files.+$ tk fmt .

I'd love to see some more examples here:

  1. Formatting single files (tk fmt myFile.jsonnet)
  2. Testing instead if formatted (tk fmt -t myFile.jsonnet)
  3. Verbose mode (tk fmt -v)
heuristicAL

comment created time in 3 days

PullRequestReviewEvent
PullRequestReviewEvent

issue commentmonitoring-mixins/mixtool

Can't retrieve mixtool using go get

Try to omit -u, this sometimes causes issues

nlamirault

comment created time in 4 days

issue commentgoogle/go-jsonnet

install of jsonnet-deps using GOMODULES failing

Where do these instructions come from? What are you trying to install?

gedw99

comment created time in 7 days

issue commentgoogle/go-jsonnet

Figure out what code-modifying tools should do about formatting

It is! Try tk fmt!

sbarzowski

comment created time in 11 days

push eventsh0rez/cortex

sh0rez

commit sha 8103b4aa11d354f34b0ecb840db8e6f4b0004d3f

fix(ruler): properly read OrgID from context Previously the `/prometheus/api/v1/{rules|alerts}` endpoints used `user.ExtractOrgIDFromHTTPRequest` to read the tenant id directly from the HTTP request headers. This however only works in multi-tenant mode. When auth is disabled, no such headers are needed to be set and above function returns no value. By directly reading from the request, the HTTP auth middleware which usually catches such cases is bypassed. This PR changes the behavior to always read from the context instead of the request, which always holds the correct org id as set by the middleware. Signed-off-by: sh0rez <me@shorez.de>

view details

sh0rez

commit sha 10baad945532f437db29a1a8513eff69e3553859

fix(alertmanager): read org id from context Ports fixes done to the ruler api also to alertmanager Signed-off-by: sh0rez <me@shorez.de>

view details

sh0rez

commit sha 6720e8361008136136f189fdc59f1d29e3ef000b

doc: add CHANGELOG entry Signed-off-by: sh0rez <me@shorez.de>

view details

push time in 12 days

push eventsh0rez/cortex

sh0rez

commit sha 7eb2a241f1069b2500e9d8159cc3425394f3fa2e

doc: add CHANGELOG entry

view details

push time in 12 days

PR opened cortexproject/cortex

fix(ruler): properly read OrgID from context

What this PR does:
Previously the /prometheus/api/v1/{rules|alerts} endpoints used user.ExtractOrgIDFromHTTPRequest to read the tenant id directly from the HTTP request headers.

This however only works in multi-tenant mode. When auth is disabled, no such headers are needed to be set and above function returns no value.

By directly reading from the request, the HTTP auth middleware which usually catches such cases is bypassed.

This PR changes the behavior to always read from the context instead of the request, which always holds the correct org id as set by the middleware.

Also does these changes for the alertmanager routes.

Thanks @cyriltovena for helping me figure this out :heart:

Which issue(s) this PR fixes: Fixes #3300 Fixes #3260

Checklist

  • [ ] Tests updated
  • [ ] Documentation added
  • [ ] CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

/cc @gotjosh @pstibrany

+7 -7

0 comment

2 changed files

pr created time in 12 days

push eventsh0rez/cortex

Jacob Lisi

commit sha 69ffb56c03c3991ae2e854b89d8cc240bf6baf3d

Add injectable middlewares to bucket client (#3278) * feat: add middleware config field Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * fix: rename Middleware to Middlewares Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * fix: rename Middleware to Middlewares Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * address PR comments Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com>

view details

Marco Pracucci

commit sha 56bbaba7423cacc0b3fcddd0985c3bda47d54812

Increased -<prefix>.redis.timeout default from 100ms to 500ms (#3301) Signed-off-by: Marco Pracucci <marco@pracucci.com>

view details

Igor Novgorodov

commit sha 1734c75459714a55cd6b97d9491b483079c4f8dc

Add an ability to load an arbitrary list of Cortex modules (#3275) * Add an ability to load an arbitrary list of Cortex modules Signed-off-by: Igor Novgorodov <igor@novg.net> * Fix module loading logic Signed-off-by: Igor Novgorodov <igor@novg.net> * Improve multiple module loading Signed-off-by: Igor Novgorodov <igor@novg.net> * Add forgotten files Signed-off-by: Igor Novgorodov <igor@novg.net> * Fix linter errors Signed-off-by: Igor Novgorodov <igor@novg.net> * Reduce state in ModuleManager Signed-off-by: Igor Novgorodov <igor@novg.net> * Fix pkg/cortex & pkg/util/modules tests, make initModules() stateless, update docs Signed-off-by: Igor Novgorodov <igor@novg.net> * Fix missed mess with &c.Target pointer cast Signed-off-by: Igor Novgorodov <igor@novg.net> * Cosmetic changes to minimize diff Signed-off-by: Igor Novgorodov <igor@novg.net> * Fix problems with PR #3287 Signed-off-by: Igor Novgorodov <igor@novg.net> * Update CHANGELOG.md Signed-off-by: Marco Pracucci <marco@pracucci.com> * Update pkg/cortex/cortex_test.go Signed-off-by: Marco Pracucci <marco@pracucci.com> * Fixed linter Signed-off-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

view details

Peter Štibraný

commit sha bcc8199f2856c20746c778536c86789aff480fd0

Respect context cancellation faster. (#3309) Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

view details

gotjosh

commit sha ef2c7a5725c0226a23fbf5f9a15853de685d201b

[BUGFIX] Fix reporting of `cortex_prometheus_rule_group_duration_seconds` (#3310) * [BUGFIX] Fix reporting of `cortex_prometheus_rule_group_duration_seconds` metric Fixes a small bug in the ruler metrics, where we used the wrong name to match against the upstream metric. This caused the metric to not report any data at all. Signed-off-by: gotjosh <josue@grafana.com> * Changelog entry Signed-off-by: gotjosh <josue@grafana.com>

view details

Michel Hollands

commit sha 9d4fb4c15c32294241376d989dac991e868e7327

Use valid grpc header (#3307) * Use valid grpc header Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rename grpc metadata key to be more unique and not binary Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

view details

Marco Pracucci

commit sha 110828ed0e8d651bf4d2ceaf62adfc36ed165ab0

Fixed gRPC connections leaking in rulers when rulers sharding is enabled and APIs called (#3314) Signed-off-by: Marco Pracucci <marco@pracucci.com>

view details

Peter Štibraný

commit sha 3f8e5fc367cbf7af84c93ae768aeb981420a58fa

Auth injector defaults (#3312) * Change default local address to :8080. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Print listening address. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Warn about missing remote address. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

view details

Marco Pracucci

commit sha cd52d6d1c377c152a788a9d3d067465f4dd8d4b8

Clarify how to configure auth for BigTable and GCS (#3311) Signed-off-by: Marco Pracucci <marco@pracucci.com>

view details

Marco Pracucci

commit sha 28bf0fa7ccf4fdb53561de15dfa2575e70444753

Fixed shuffle sharding consistency when zone-awareness is enabled and the shard size is increased or instances in a new zone are added (#3299) * Fixed shuffle sharding consistency when zone-awareness is enabled and the shard size is increased or instances in a new zone are added Signed-off-by: Marco Pracucci <marco@pracucci.com> * Slightly more permissive deviance Signed-off-by: Marco Pracucci <marco@pracucci.com> * Cleaned up ring unit tests Signed-off-by: Marco Pracucci <marco@pracucci.com> * Use YoloBuf to reduce allocations in ShuffleShardSeed() Signed-off-by: Marco Pracucci <marco@pracucci.com>

view details

Peter Štibraný

commit sha e8d5028e6721dfd4459d88fd88b430ded4184b27

Fix flaky test. (#3313) * Get log for failed test. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Make more CAS attempts, and add a bit of random delay between them to reduce CAS collisions. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix cleanup of lifecycler. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix cleanup of lifecycler. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix another cleanup function. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Remove racy buffering of log messages. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

view details

Azfaar Qureshi

commit sha 491e60715986ddf6947fb866e16a54ab8b7f809a

Migrate CircleCI workflows to GitHub Actions (1/3) (#3302) * Added lint job Signed-off-by: Azfaar Qureshi <azfaarq@amazon.com> * Added test and build jobs Signed-off-by: Azfaar Qureshi <azfaarq@amazon.com> * adding README Signed-off-by: Azfaar Qureshi <azfaarq@amazon.com> * adding table to README and removing vestigial lines from workflow Signed-off-by: Azfaar Qureshi <azfaarq@amazon.com> Co-authored-by: Shovnik Bhattacharya <shovnik@amazon.com>

view details

Marco Pracucci

commit sha 437c7fae29dbb722ac8435f8f2b21b1c1392a384

Remove artifacts upload from GitHub actions (#3324) Signed-off-by: Marco Pracucci <marco@pracucci.com>

view details

Marco Pracucci

commit sha 7a66f21bdf76c8ab544faa3ce733678beb7b7121

Upgrade build-image from Debian stretch to buster (#3323) * Upgrade build-image from Debian stretch to buster Signed-off-by: Marco Pracucci <marco@pracucci.com> * Rebuilt protos Signed-off-by: Marco Pracucci <marco@pracucci.com> * Removed 'Checkout version' note from GitHub actions workflow doc Signed-off-by: Marco Pracucci <marco@pracucci.com>

view details

Jacob Lisi

commit sha f122c7c847506acaefa855c53e40141b31fe73e5

test: add integration test for ruler evaluation delay (#3316) * test: add integration test for ruler evaluation delay Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * better comparison Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * add rule config Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * cleanup and finalize test Signed-off-by: Jacob Lisi <jacob.t.lisi@gmail.com> * Update integration/ruler_test.go Signed-off-by: Marco Pracucci <marco@pracucci.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>

view details

Peter Štibraný

commit sha 922853d9f20c21b12cd2fa785cf937c8a0b089b2

Blocksconvert cleaner (#3283) * Extracted plan_processor_service.go from builder.go. To be used by cleaner, which is also processing a plan. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Comments. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Moved fetcher back, since cleaner doesn't use it. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Added new "cleaner" target to blocksconvert. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Remove forgotten gauge from builder. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Register cleaner flags. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Set userID into context for store methods to work. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Parse 'finished' status without block ID. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Cleaner now removes entries / chunks directly, without using store. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Ignore object not found error when deleting chunks. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Don't delete chunks starting outside of the plan range. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Delete index entries first, chunks second. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Delete index entries first, chunks second. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Improve comment. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Count number of deleted index entries. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Use correct From/Through when getting chunk write entries. Remove label entries as well. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Fix test. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Make lint happy. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Make lint happy. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Make lint happy again. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * Moved additional config options to NewService method. Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com> * CHANGELOG.md Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

view details

Marco Pracucci

commit sha 9e8beee8cacb972a2b246c348be209dcd02c0e11

Fix repo main README (#3326) Signed-off-by: Marco Pracucci <marco@pracucci.com>

view details

Bryan Boreham

commit sha fcd9723aac33ae17c3ffbd22d00c55664006ceec

Don't return raw buffer from util.ParseProtoReader() (#3328) It's never used. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

view details

Stan Kwong

commit sha 4500accd13e1e0ea6c8cb111c323b95a308c6e4f

Add more documentation on AM templates (#3273) Signed-off-by: Stan Kwong <stanley.kwong@robinhood.com>

view details

Marco Pracucci

commit sha 46786b2cd855aa8a2c497525538d7c4d5b5545c3

Increase integration tests timeout (#3335) Signed-off-by: Marco Pracucci <marco@pracucci.com>

view details

push time in 12 days

create barnchsh0rez/cortex

branch : ruler-orgid-ctx

created branch time in 13 days

fork sh0rez/cortex

A horizontally scalable, highly available, multi-tenant, long term Prometheus.

https://cortexmetrics.io/

fork in 13 days

Pull request review commentmonitoring-mixins/mixtool

Look for a vendor directory along side mixin.libsonnet.

 func main() {  // If no jPath is given, we check if ./vendor exists in the current directory and use it. // filename should be the path to the mixin root file (ie 'mixin.libsonnet').-func availableVendor(filename string, jPathsFlag []string) []string {+func availableVendor(filename string, jPathsFlag []string) ([]string, error) { 	if len(jPathsFlag) == 0 {-		// Look for a vendor dir along side the root mixin file.-		basedir := path.Dir(filename)-		vendor := path.Join(basedir, "vendor")-		_, err := os.Stat(vendor)+		workdir, err := filepath.Abs(filename)+		if err != nil {+			return nil, err+		}++		workdir = filepath.Dir(workdir)+		root, err := jpath.FindParentFile("jsonnetfile.json", workdir, "/")

This could use jpath.FindRoot, which is cross-platform (C:\\ for windows)

tomwilkie

comment created time in 13 days

PullRequestReviewEvent

Pull request review commentmonitoring-mixins/mixtool

Look for a vendor directory along side mixin.libsonnet.

 func main() {  // If no jPath is given, we check if ./vendor exists in the current directory and use it. // filename should be the path to the mixin root file (ie 'mixin.libsonnet').-func availableVendor(filename string, jPathsFlag []string) []string {+func availableVendor(filename string, jPathsFlag []string) ([]string, error) { 	if len(jPathsFlag) == 0 {-		// Look for a vendor dir along side the root mixin file.-		basedir := path.Dir(filename)-		vendor := path.Join(basedir, "vendor")-		_, err := os.Stat(vendor)+		workdir, err := filepath.Abs(filename)+		if err != nil {+			return nil, err+		}++		workdir = filepath.Dir(workdir)+		root, err := jpath.FindParentFile("jsonnetfile.json", workdir, "/")+		if err != nil {+			if _, ok := err.(jpath.ErrorFileNotFound); ok {+				return jPathsFlag, nil+			}+			return nil, err+		}++		vendor := path.Join(root, "vendor")
		vendor := filepath.Join(root, "vendor")

path is not cross platform

tomwilkie

comment created time in 13 days

PullRequestReviewEvent

issue commentjsonnet-bundler/jsonnet-bundler

Access different branches in import

(this is a very bad solution using legacy features)

Try this:

{
  "version": 1,
  "dependencies": [
    {
      "source": {
        "git": {
          "remote": "AAA",
          "subdir": ""
        }
      }
    },
    {
+     "name: "foo-develop"
      "source": {
        "git": {
          "remote": "AAA",
          "subdir": ""
        }
      },
      "version": "develop"
    }
  ],
  "legacyImports": true
}

This installs the dependency using a different name. This is not something that should work to be honest, and expect it to be removed, but it is a stop-gap.

We should still consider implementing this properly imo

simonfrey

comment created time in 13 days

issue commentjsonnet-bundler/jsonnet-bundler

legacyImports: true is not always set to true

@metalmatze legacyImports is exclusively read from jsonnetfile.json, the value is the lockFile is silently ignored:

https://github.com/jsonnet-bundler/jsonnet-bundler/blob/ada055a225fa6fc37b05bdbd838ed91b04727a6e/pkg/packages.go#L97

direct in this case comes from here:

https://github.com/jsonnet-bundler/jsonnet-bundler/blob/ada055a225fa6fc37b05bdbd838ed91b04727a6e/cmd/jb/install.go#L76

lilic

comment created time in 13 days

issue closedjsonnet-bundler/jsonnet-bundler

O

closed time in 13 days

Modnxs

issue commentjsonnet-bundler/jsonnet-bundler

O

Closing, as probably opened by mistake. Feel free to reopen in case there actually is an issue / question :)

Modnxs

comment created time in 13 days

Pull request review commentgrafana/jsonnet-libs

Use mixtool to lint mixins

 local g = (import 'grafana-builder/grafana.libsonnet') + { {   grafanaDashboards+: {     'jaeger-write.json':-      g.dashboard('Jaeger / Write')+      (g.dashboard('Jaeger / Write') + { uid: '5f26222aa7a3fb734f0cd4072cab43b3' })

I've seen us using hashed uids in multiple places. Why? If they were human readable, they could be easy identifiers for dashboards, considering you might want to do something as mixtool apply -t someday

tomwilkie

comment created time in 13 days

Pull request review commentgrafana/jsonnet-libs

Use mixtool to lint mixins

 lint: 					RESULT=1; \ 				fi; \ 		done; \+		for d in $$(find . -name '*-mixin' -a -type d -print); do \+			if [ -e "$$d/jsonnetfile.json" ]; then \+				echo "Installing dependancies for $$d"; \
				echo "Installing dependencies for $$d"; \

In case you care about this ;)

tomwilkie

comment created time in 13 days

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

issue closedgrafana/tanka

Environment name in different env subcommands is either the last element of the path or a path relative to the root dir

There seems to be some inconsistency between different tk env subcommands around what is treated as the environment name. tk env add environments/jaeger/production will create spec.json with metadata.name: production, but tk env set environments/jaeger/production --server-from-context minikube will set it to metadata.name: environments/jaeger/production.

Should tk env add perhaps be switched to using a relative path for inferring the name of the namespace, something similar to https://github.com/grafana/tanka/blob/master/pkg/tanka/parse.go#L112 instead of cfg.Metadata.Name = filepath.Base(path) (https://github.com/grafana/tanka/blob/master/cmd/tk/env.go#L160 ) ?

closed time in 14 days

mwasilew2

push eventgrafana/tanka

Michal

commit sha 111cbd5f688d02df0f5bbc42dbb856faf3365415

fix(cli): Environment name at env creation (#404) Environment name is the relative path of an env dir. tk env add is incorrectly using the base dir name. This PR fixes it. jpath pkg is used to benefit from the existing implementation. FindRoot function is made public instead of using Resolve because Resolve does many other things as well, some of which require files that don't exist at that time. Signed-off-by: Michal Wasilewski <mwasilewski@gmx.com>

view details

push time in 14 days

PR merged grafana/tanka

fix(cli): Environment name at env creation (#397)

Environment name is the relative path of an env dir. tk env add is incorrectly using the base dir name. This PR fixes it.

jpath pkg is used to benefit from the existing implementation. FindRoot function is made public instead of using Resolve because Resolve does many other things as well, some of which require files that don't exist at that time.

Signed-off-by: Michal Wasilewski mwasilewski@gmx.com

+9 -4

0 comment

2 changed files

mwasilew2

pr closed time in 14 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentgrafana/tanka

feat: allow other entrypoints

 func dirContainsFile(files []os.FileInfo, filename string) bool { 	} 	return false }++func Entrypoint(dir string) (string, error) {+	filename := "main.jsonnet"

Please make this an exported const, so the meaning is more clear:

const DEFAULT_ENTRYPOINT = "main.jsonnet"
Duologic

comment created time in 14 days

Pull request review commentgrafana/tanka

feat: allow other entrypoints

 func MakeVM(opts Opts) *jsonnet.VM { // result in JSON form. It disregards opts.ImportPaths in favor of automatically // resolving these according to the specified file. func EvaluateFile(jsonnetFile string, opts Opts) (string, error) {-	jpath, _, _, err := jpath.Resolve(filepath.Dir(jsonnetFile))+	jpath, _, _, err := jpath.Resolve(jsonnetFile)

You call jpath.Resolve here with jsonnetFile, but from what I can see it requires a workdir (directory), right?

Duologic

comment created time in 14 days

PullRequestReviewEvent

Pull request review commentgrafana/tanka

fix(helm): Check prior vendoring of helm charts

 func parseReq(s string) (*Requirement, error) { 		Version: *ver, 	}, nil }++// parseReqName parses a name from a string of the format `repo/name` or `name`+func parseReqName(s string) string {

It seems like parseReqName is only ever called with an argument of the form name@repo as r.Chart looks like this.

Any reason this function also handles plain name?

justinwalz

comment created time in 14 days

PullRequestReviewEvent
PullRequestReviewEvent

issue commentcortexproject/cortex

/prometheus/api/v1/rules auth middleware behaving incorrectly when auth disabled

@gotjosh this one seems to be related to Alertmanager, while this is about the Prometheus compatible API itself. These might still be very well related.

sh0rez

comment created time in 15 days

startedorlangure/gnomock

started time in 15 days

issue commentorlangure/gnomock

Kubernetes preset

Hey @orlangure!

As a use-case, I am primarily looking at automatically testing tools like Grafana Tanka, which does application definition for Kubernetes. It uses a special language (Jsonnet) to define k8s objects and calls kubectl to apply afterwards.

I'd like to do some basic testing in CI, but many operations require a Kubernetes cluster, which is rather hard to get. It would be incredibly helpful if my tests would just spin one up using docker, run my tests against it and tear it all down again.

A single-node cluster with basic api-access would be all I need.

Does this sound like something worth trying?

sh0rez

comment created time in 15 days

issue commentcortexproject/cortex

/prometheus/api/v1/rules auth middleware behaving incorrectly when auth disabled

Also note I'd be willing to give it a try to fix this myself, but I would need a bit of hints where to start looking

sh0rez

comment created time in 18 days

issue openedcortexproject/cortex

/prometheus/api/v1/rules auth middleware behaving incorrectly when auth disabled

Describe the bug

When running using auth_enabled: false, calling /prometheus/api/v1/rules without X-Scope-OrgID yields 500 Internal Server Error: no valid org id found.

To Reproduce Steps to reproduce the behavior:

  1. Start Cortex (used 8d30827):
    go run go run ./cmd/cortex --config.file ./docs/configuration/single-process-config.yaml
  2. GET the endpoint (used HTTPie, but curl works as well:
$ http localhost:9009/prometheus/api/v1/rules
HTTP/1.1 500 Internal Server Error
Content-Length: 89
Content-Type: text/plain; charset=utf-8
Date: Thu, 08 Oct 2020 15:06:10 GMT

{
    "data": null,
    "error": "no valid org id found",
    "errorType": "server_error",
    "status": "error"
}

Expected behavior

Instead of failing, I expected Cortex to silently assume fake as the org id, as specified by https://cortexmetrics.io/docs/production/auth/:

To disable the multi-tenant functionality, you can pass the argument -auth.enabled=false to every Cortex component, which will set the OrgID to the string fake for every request.

Environment:

  • Infrastructure: laptop
  • Deployment tool: go run? go version go1.15.2 linux/amd64

@pstibrany @owen-d

created time in 18 days

Pull request review commentmonitoring-mixins/mixtool

Make lint subcommand exit with non-zero error code when errors are found.

 type LintOptions struct { }  func Lint(w io.Writer, filename string, options LintOptions) error {-	vm := NewVM(options.JPaths)

I've tried hard in the past to make it break, and I didn't. Ran ~20 evals on the same vm in parallel, seemed to work

tomwilkie

comment created time in 19 days

PullRequestReviewEvent

Pull request review commentmonitoring-mixins/mixtool

Make lint subcommand exit with non-zero error code when errors are found.

 type LintOptions struct { }  func Lint(w io.Writer, filename string, options LintOptions) error {-	vm := NewVM(options.JPaths)

Probably out of scope of this PR however

tomwilkie

comment created time in 19 days

PullRequestReviewEvent

Pull request review commentmonitoring-mixins/mixtool

Make lint subcommand exit with non-zero error code when errors are found.

 type LintOptions struct { }  func Lint(w io.Writer, filename string, options LintOptions) error {-	vm := NewVM(options.JPaths)

Reusing would actually be quite beneficial, because both evaluations would share cache then. This means that fields the first evaluation already evaluates don't need to be re-evaluated for the second run

tomwilkie

comment created time in 19 days

PullRequestReviewEvent

issue openedtinygo-org/tinygo

ESP8266 i2c support

The docs (https://tinygo.org/microcontrollers/esp8266/) list that the ESP8266 i2c feature is not yet implemented.

To interface sensors like the BME280, I2C is required. What needs to be done so i2c can be used?

created time in 20 days

issue openedtinygo-org/tinygo

ESP8266 WiFi support

The docs (https://tinygo.org/microcontrollers/esp8266/) list that the ESP8266 WiFi feature is not supported by TinyGo.

Imo having this support is crucial for this board to make sense. What needs to be done so this can happen?

created time in 20 days

issue commentgrafana/tanka

Environment name in different env subcommands is either the last element of the path or a path relative to the root dir

Hi, good spot!

Yes, the code in pkg/tanka/parse.go is correct. The environment name shall be the relative path from the project root. Actually, the environment name is never read from the file, but inferred from the directory structure when loading the config.

Still it makes perfect sense to fix tk env add. Would you like to do that?

mwasilew2

comment created time in 20 days

issue commentgrafana/tanka

Question: Tanka vision/roadmap

Hey @bewest, let me try to answer some of your questions.

How does tanka play with gitops and ci/cd or other tools in general? Is export meant to output the files for another tool to consume?

Yes it does! The tk export command is specifically meant to be used for handing the YAML resources to some sort of CD tooling. We pilot https://fluxcd.io/ internally and have promising results. @captncraig can tell more on that.

kubernetes says the objects are managed by kubectl; does this suggest its part of a classic unix pipeline? However piping show to other tools requires a long switch; does this suggest tanka prefers some prioritized relationship to managing the workflow

Tanka uses kubectl, not the go-library for cluster communication. Piping however is discouraged for safety reasons: kubectl must be pointed at a given cluster, which resources are sent do when doing kubectl apply. Not being careful about this can have fatal outcomes, e.g. when you apply to the wrong cluster. This happened before and wiped a cluster.
To combat this, tk apply and tk diff invoke kubectl with the --context flag, to force a context that matches the cluster of spec.json. However, when you pipe tk show to kubectl apply, Tanka can't do this for you. The long switch makes this responsibility clear.
In day-to-day operations there should be no need for piping tk show. If you want to export for CD, use tk export. For everything else (diff, apply, delete) Tanka has controlled commands. Does this make sense?

Wondering how much to expect tanka to interop more with the system around it vs focusing on an interactive terminal, vs maintaining the utility, backwards compatibility

Not sure whether I understand this question fully.

The Tanka workflow is primarily interactive: You edit your Jsonnet, tk diff it to verify the outcome and tk apply to make it happen. For automated deployments, tooling is available, notably FluxCD and ArgoCD. tk export provides a bridge to those.
We strongly value backwards compatibility, no core area of Tanka should ever break when upgrading.

and workflow surrounding jsonnet-libs, k, et al?

Let's try to clarify on the Kubernetes libraries.

In the beginning it was ksonnet-lib, which was a Jsonnet library for quickly piecing together Kubernetes objects. When the Heptio was acquired by VMWare, ksonnet was deprecated and thus all development stopped.

Due to a lack of alternative, we stayed on this library, even after we released Tanka as a replacement for ksonnet. Around February 2020, during FOSDEM, @metalmatze and I forked ksonnet-lib to https://github.com/kube-jsonnet/k, aiming to maintain it. Unfortunately we realized that the code-base was far more complex than we had time for to understand to be able to fix issues.

We re-wrote the generator, which formed https://github.com/jsonnet-libs/k8s. This library serves us well for over 2 months in prod now, and will be default in Tanka by the time we declare it GA.

A notable thing here is kausal.libsonnet, which is an extension on top of ksonnet-lib to do some new things we needed for ourselves. Most of what kausal provides is already merged into jsonnet-libs/k8s, so soon there will be only one library left.

kubecfg doesn't seem to work with the jsonnet-lib stuff, but does facilitate getting started with components using yaml or json files as input (kubecfg show -o json previously_formatted.yaml is a good way to seed new jsonnet lib ready for factorizing). It's also been tricky dealing with different components that will create or consume different namespaces within the same environment. Is cloning lots of duplicate environments to manage different components expected?

I can't tell too much about kubecfg, never really used it.

On Environments, they should usually be subset of a namespace. Say a environments/loki/us-central1 would deploy into the loki namespace of a us-central1 cluster. The reason resources are allowed into different namespaces is the fact that some operators require resources in certain namespaces they watch.

As this goes quite into detail, also requiring more information on your setup to answer properly, I suggest you join the #tanka channel on https://grafana.slack.com and ping the there!

olegmayko

comment created time in 20 days

issue commentgrafana/tanka

unable to update/deploy default serviceAccount on first apply

It is indeed running apply

ghostsquad

comment created time in 20 days

issue openedorlangure/gnomock

Kubernetes preset

It would be incredibly useful to have a Kubernetes preset, that spins up an empty cluster (maybe https://k3s.io/) and gives code access to it in some sort (https://github.com/kubernetes/client-go)?

created time in 20 days

push eventgrafana/jsonnet-libs

sh0rez

commit sha d03c4a9f2828d6e99423c1da9c898efcbc134638

doc(helm-util): Use docsonnet for README.md (#332) * doc(helm-util): Use docsonnet for README.md Uses the github.com/sh0rez/docsonnet to generate the README.md

view details

push time in 20 days

delete branch grafana/jsonnet-libs

delete branch : helm-docsonnet

delete time in 20 days

PR merged grafana/jsonnet-libs

Reviewers
doc(helm-util): Use docsonnet for README.md

Uses the github.com/sh0rez/docsonnet to generate the README.md

+161 -39

0 comment

5 changed files

sh0rez

pr closed time in 20 days

push eventgrafana/jsonnet-libs

sh0rez

commit sha b9344569324072af61f5502b781037d52a96ef73

refactor: readme.md.tpl

view details

push time in 20 days

Pull request review commentmonitoring-mixins/mixtool

Make lint subcommand exist with non-zero error code when errors are found.

 import ( )  func evaluatePrometheusAlerts(vm *jsonnet.VM, filename string) (string, error) {-	snippet := fmt.Sprintf("(import '%s').prometheusAlerts", filename)+	snippet := fmt.Sprintf(`+local mixin = (import %q);++if std.objectHas(mixin, "prometheusAlerts")+then mixin.prometheusAlerts+else {}+`, filename)+ 	return vm.EvaluateSnippet("", snippet) }  func evaluatePrometheusRules(vm *jsonnet.VM, filename string) (string, error) {-	snippet := fmt.Sprintf("(import '%s').prometheusRules", filename)+	snippet := fmt.Sprintf(`+local mixin = (import %q);++if std.objectHas(mixin, "prometheusRules")+then mixin.prometheusRules+else {}+`, filename)+ 	return vm.EvaluateSnippet("", snippet) }  func evaluateGrafanaDashboards(vm *jsonnet.VM, filename string) (string, error) {-	snippet := fmt.Sprintf("(import '%s').grafanaDashboards", filename)+	snippet := fmt.Sprintf(`+local mixin = (import %q);++if std.objectHas(mixin, "grafanaDashboards")+then mixin.grafanaDashboards+else {}+`, filename)+ 	return vm.EvaluateSnippet("", snippet)

Done

tomwilkie

comment created time in 20 days

PullRequestReviewEvent

issue commentmonitoring-mixins/mixtool

Find a better name for the mixtool

I must say mixtool is not too bad, because it reflects the aim of the project – tooling for mixins. Whether this means Jsonnet, Prometheus, whatever is really a question of what mixins are and this tool should adapt.

Currently mixins are written in Jsonnet and provide Prometheus and Grafana data, so this tool does exactly that.

To do less typing on the command line, it might make sense to rename the command to mix.

metalmatze

comment created time in 20 days

issue openedmonitoring-mixins/mixtool

Use jsonnet.EvaluateAnonymousSnippet

Jsonnet recently introduced a new function class for evaluating snippets that are not actually files on disk. https://github.com/google/go-jsonnet/pull/447

We have several places this applies, so we should adopt this: https://github.com/monitoring-mixins/mixtool/blob/f4e131da6cb67de07357510c9a947ffe02ad17a9/pkg/mixer/eval.go#L25 https://github.com/monitoring-mixins/mixtool/blob/f4e131da6cb67de07357510c9a947ffe02ad17a9/pkg/mixer/eval.go#L30 https://github.com/monitoring-mixins/mixtool/blob/f4e131da6cb67de07357510c9a947ffe02ad17a9/pkg/mixer/eval.go#L35

created time in 20 days

issue openedmonitoring-mixins/mixtool

Remove vendor

go.mod should provide sufficient stability guarantees for not keeping vendor around. We don't have a vendor folder on Grafana Tanka and did not run into any issues.

Not having vendor benefits pull requests, because it keeps them smaller and eaiser to review.

Thoughts?

created time in 20 days

created taggrafana/tanka

tagv0.12.0

Flexible, reusable and concise configuration for Kubernetes

created time in 20 days

created taggrafana/tanka

tagv0.12-alpha3

Flexible, reusable and concise configuration for Kubernetes

created time in 20 days

push eventgrafana/tanka

Dominik Süß

commit sha a18afbfc561cd8ef94f42195df752561a9ee99ee

fix(cli): invalid formatting in tk status (#399)

view details

push time in 20 days

PR merged grafana/tanka

switch format verb for status output

Currently when running tk status the output contains the following:

  InjectLabels: %!s(bool=false)

This is caused by using the %s format verb which only works on string

By specifying %v, go will use the default format for the corresponding datatype

+1 -1

0 comment

1 changed file

theSuess

pr closed time in 20 days

PullRequestReviewEvent

push eventgrafana/tanka

sh0rez

commit sha 73101c46d986e1336b4e99e636712e93197755f3

doc: helm.template() (#388) Co-authored-by: Jack Baldry <jack.baldry@grafana.com>

view details

push time in 20 days

delete branch grafana/tanka

delete branch : helm-docs

delete time in 20 days

PR merged grafana/tanka

Reviewers
doc: helm.template() component/docs

Adds docs for the upcoming Helm support:

  • Reorganizes the "loose" docs pages into categories
  • Adds Helm support to "Advanced features"

Also needs https://github.com/grafana/jsonnet-libs/pull/332

I've tagged a lot of people here. If you got time, please read those and think of a less involved user. Is it clear how to use this feature?

+239 -22

0 comment

13 changed files

sh0rez

pr closed time in 20 days

push eventgrafana/tanka

sh0rez

commit sha 363c5f1935a968418a1f34426fb9fffcb8a24089

doc: review feedback

view details

push time in 20 days

delete tag grafana/tanka

delete tag : v0.12-alpha3

delete time in 20 days

delete tag grafana/tanka

delete tag : v0.12.0

delete time in 20 days

issue commentgrafana/tanka

unable to update/deploy default serviceAccount on first apply

Can we do something about this? Tanka attempts to sort objects to satisfy dependencies, but most of the work is done by kubectl.

ghostsquad

comment created time in 20 days

created taggrafana/tanka

tagv0.12.0

Flexible, reusable and concise configuration for Kubernetes

created time in 20 days

push eventgrafana/tanka

sh0rez

commit sha d392545cdca4bb9bc72977fbee3f0b8aed37e48b

chore(release): Changelog for 0.12 stable (#398) * chore(release): Changelog for 0.12 stable * doc: add helm beta notice

view details

push time in 20 days

more