profile
viewpoint

jsonnet-bundler/jsonnet-bundler 232

A jsonnet package manager.

jsonnet-bundler/frozen-lib 0

Dummy lib that won't change (for testing)

startedjsonnet-bundler/jsonnet-bundler

started time in 4 days

issue commentjsonnet-bundler/jsonnet-bundler

Access different branches in import

(this is a very bad solution using legacy features)

Try this:

{
  "version": 1,
  "dependencies": [
    {
      "source": {
        "git": {
          "remote": "AAA",
          "subdir": ""
        }
      }
    },
    {
+     "name: "foo-develop"
      "source": {
        "git": {
          "remote": "AAA",
          "subdir": ""
        }
      },
      "version": "develop"
    }
  ],
  "legacyImports": true
}

This installs the dependency using a different name. This is not something that should work to be honest, and expect it to be removed, but it is a stop-gap.

We should still consider implementing this properly imo

simonfrey

comment created time in 7 days

issue commentjsonnet-bundler/jsonnet-bundler

legacyImports: true is not always set to true

@metalmatze legacyImports is exclusively read from jsonnetfile.json, the value is the lockFile is silently ignored:

https://github.com/jsonnet-bundler/jsonnet-bundler/blob/ada055a225fa6fc37b05bdbd838ed91b04727a6e/pkg/packages.go#L97

direct in this case comes from here:

https://github.com/jsonnet-bundler/jsonnet-bundler/blob/ada055a225fa6fc37b05bdbd838ed91b04727a6e/cmd/jb/install.go#L76

lilic

comment created time in 7 days

issue closedjsonnet-bundler/jsonnet-bundler

O

closed time in 7 days

Modnxs

issue commentjsonnet-bundler/jsonnet-bundler

O

Closing, as probably opened by mistake. Feel free to reopen in case there actually is an issue / question :)

Modnxs

comment created time in 7 days

issue openedjsonnet-bundler/jsonnet-bundler

O

created time in 7 days

startedjsonnet-bundler/jsonnet-bundler

started time in 11 days

issue commentjsonnet-bundler/jsonnet-bundler

[Feature] Keep installation path of local sources in jsonnetfile.json

So sorry for the noise, had github link in commit message and forgot it links through :facepalm: :pray: This was related to the different branch but same leaf node comment above. Appreciate the ease of use for jsonnet management jb affords us.

hangxie

comment created time in 16 days

startedjsonnet-bundler/jsonnet-bundler

started time in 18 days

PR opened jsonnet-bundler/jsonnet-bundler

Add README package install

Reference to packages for Arch Linux and Homebrew

+5 -0

0 comment

1 changed file

pr created time in 20 days

pull request commentjsonnet-bundler/jsonnet-bundler

Add support for azure devops https git address

Totally agree with what @sc250024. We really shouldn't have to hardcode something for each and every platform individually to support them. That being said, I like the idea of following what other projects have done in that area.

tehho

comment created time in 20 days

fork megian/jsonnet-bundler

A jsonnet package manager.

fork in 21 days

startedjsonnet-bundler/jsonnet-bundler

started time in 21 days

startedjsonnet-bundler/jsonnet-bundler

started time in 22 days

issue commentjsonnet-bundler/jsonnet-bundler

panic during `jb install` over ssh

this apparently doesn't work either for non-master branches, see #130

ghostsquad

comment created time in a month

issue openedjsonnet-bundler/jsonnet-bundler

unable to ssh install from non-master branch

❯ jb install git+ssh://git@github.com/tunein/spinnaker-libsonnet.git@main
Initialized empty Git repository in /Users/wmcnamee/projects/kubernetes/vendor/.tmp/jsonnetpkg-github.com-tunein-spinnaker-libsonnet-main212901404/.git/
remote: Enumerating objects: 8, done.
remote: Counting objects: 100% (8/8), done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 8 (delta 1), reused 5 (delta 1), pack-reused 0
Unpacking objects: 100% (8/8), 3.87 KiB | 991.00 KiB/s, done.
From ssh://github.com/tunein/spinnaker-libsonnet
 * branch            main       -> FETCH_HEAD
 * [new branch]      main       -> origin/main
Branch 'main' set up to track remote branch 'main' from 'origin'.
Switched to a new branch 'main'
Initialized empty Git repository in /Users/wmcnamee/projects/kubernetes/vendor/.tmp/jsonnetpkg-github.com-tunein-util-libsonnet-master180338891/.git/
fatal: couldn't find remote ref master
remote: Enumerating objects: 6, done.
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 0), reused 2 (delta 0), pack-reused 0
Unpacking objects: 100% (6/6), 1.48 KiB | 378.00 KiB/s, done.
From ssh://github.com/tunein/util-libsonnet
 * [new branch]      main       -> origin/main
error: pathspec 'master' did not match any file(s) known to git
jb: error: failed to install packages: downloading: exit status 1

created time in a month

startedjsonnet-bundler/jsonnet-bundler

started time in a month

issue commentjsonnet-bundler/jsonnet-bundler

legacyImports: true is not always set to true

If one does jb install do we even look at the jsonnetfile.json though? Maybe it's still worth having in the jsonnetfile.lock.json, but I'll double-check the implementation details.

lilic

comment created time in a month

issue commentjsonnet-bundler/jsonnet-bundler

legacyImports: true is not always set to true

It's not needed at all! Didn't find the time to separate the Jsonnetfile and Lockfile structs back then

lilic

comment created time in a month

issue commentjsonnet-bundler/jsonnet-bundler

legacyImports: true is not always set to true

@sh0rez do we need it in the lock file at all or is it something we can remove to prevent further confusion?

lilic

comment created time in a month

issue commentjsonnet-bundler/jsonnet-bundler

legacyImports: true is not always set to true

It's confusing UX. Both files share the same Schema, but this value is only read from the jsonnetfile, not the lockfile

lilic

comment created time in a month

issue commentjsonnet-bundler/jsonnet-bundler

legacyImports: true is not always set to true

I don't think that's expected and I've seen similar things while creating these PRs: https://github.com/brancz/kubernetes-grafana/pull/98 https://github.com/prometheus-operator/kube-prometheus/pull/675 https://github.com/kubernetes-monitoring/kubernetes-mixin/pull/502

lilic

comment created time in a month

issue openedjsonnet-bundler/jsonnet-bundler

legacyImports: true is not always set to true

When "legacyImports": true is set in jsonnetfile.json file, somehow "legacyImports": false is set instead in the jsonnetfile.lock.json file. Not sure if this is the expected thing to happen? If yes, seems a bit confusing from UX side of things, I would expect both to be set the same?

I haven't looked much into the code, I can do if no one knows the problem straight away. Thanks!

See example on PR https://github.com/kubernetes/kube-state-metrics/pull/1224.

created time in a month

startedjsonnet-bundler/jsonnet-bundler

started time in a month

startedjsonnet-bundler/jsonnet-bundler

started time in a month

startedjsonnet-bundler/jsonnet-bundler

started time in 2 months

startedjsonnet-bundler/jsonnet-bundler

started time in 2 months

startedjsonnet-bundler/jsonnet-bundler

started time in 2 months

startedjsonnet-bundler/jsonnet-bundler

started time in 2 months

startedjsonnet-bundler/jsonnet-bundler

started time in 2 months

fork FreeK/jsonnet-bundler

A jsonnet package manager.

fork in 2 months

startedjsonnet-bundler/jsonnet-bundler

started time in 2 months

issue commentjsonnet-bundler/jsonnet-bundler

Access different branches in import

Does anyone have an idea how to do that?

simonfrey

comment created time in 3 months

startedjsonnet-bundler/jsonnet-bundler

started time in 3 months

PR closed jsonnet-bundler/jsonnet-bundler

Reviewers
feat: Adding GoReleaser to handle releases enhancement

Enables this repository to use GoReleaser to handle all releases. This PR is basically ready, it just needs some feedback from the maintainers.

Description

By merging this PR, this repository will use GoReleaser for releasing of all artifacts. This includes, but is not limited to:

  • Docker images
  • GitHub releases with Go binaries pre-packaged
  • Homebrew formulas

There are a number of issues to be addressed / questions to be answered by the maintainers of this project in order to merge this PR. See Generic questions.

The following is an example of the _output directory after running make cross, which triggers goreleaser ultimately:

$ tree _output

_output
├── config.yaml
├── goreleaserdocker799593599
│   ├── Dockerfile
│   └── jb
├── jb-darwin-amd64.tar.gz
├── jb-linux-amd64.tar.gz
├── jb-linux-arm64.tar.gz
├── jb-linux-armv6.tar.gz
├── jb-linux-armv7.tar.gz
├── jb-windows-amd64.zip
├── jb.rb
├── jb_darwin_amd64
│   └── jb
├── jb_linux_amd64
│   └── jb
├── jb_linux_arm64
│   └── jb
├── jb_linux_arm_6
│   └── jb
├── jb_linux_arm_7
│   └── jb
├── jb_v0.4.0-SNAPSHOT-7a629bf_checksums.txt
└── jb_windows_amd64
    └── jb.exe

7 directories, 17 files

Motivation and Context

This fixes issue https://github.com/jsonnet-bundler/jsonnet-bundler/issues/64, which has been open for a while. Essentially, it enables jsonnet-bundler to be packaged more easily so it can be distributed amongst the main channels, i.e. Docker, Homebrew, etc.

Items to Address

Generic questions

  1. Should GoReleaser be triggered automatically by Drone on a tag push, or should a maintainer perform the releasing manually?
  2. Are Docker images wanted? See Docker registry below.
  3. Is a Homebrew formula wanted? See Homebrew formula below.
  4. What artifacts should be attached to the GitHub release? Binaries only? Archives only? Both? See GitHub releases.

Docker registry

In order for this to be merged, the repository maintainers should decide where to put Docker images, if at all. If it's desired to not have Docker images generated, that's also possible. If it's decided that this is wanted, credentials would also need to be generated to push to the Docker registry.

Essentially, the GoReleaser builds tags according to semver specifications. For example, if this project is hosted on DockerHub, the following images would be generated:

  • docker.io/jsonnet-bundler/jb:latest
  • docker.io/jsonnet-bundler/jb:v0.4.0
  • docker.io/jsonnet-bundler/jb:v0.4
  • docker.io/jsonnet-bundler/jb:v0

Other options are also possible.

Homebrew formula

In a similar fashion, if it's decided to create Homebrew formulas, the following options are available:

  • Option 1: Host the Homebrew tap from this repository, which means creating a Formula folder at the root.
  • Option 2: Host the Homebrew tap from another repository in the jsonnet-bunlder organization; in which case, a Github token would need to be provided by the maintainers which has access to write to that repository.
  • Option 3: Don't host any Homebrew tap, and later manually create a formula in the https://github.com/Homebrew/homebrew-core repository.

GitHub releases

Currently the jb binary is added to GitHub releases directly without packaging. This is also possible with GoReleaser in the exact same way, but if Homebrew is being used as well, then Homebrew needs archives to be built. In other words, Homebrew does not download binaries directly; it always first downloads an archive, and then extracts it.

Coupled with the Homebrew options above are the following options for GitHub releases:

  • Option 1: Don't use Homebrew at all; attach binaries to the GitHub release as it happens right now.
  • Option 2: Use Homebrew in some way; attach archives only to the GitHub release.
  • Option 3: Use Homebrew in some way; attach both archives and binaries to the GitHub release, and let people choose.

How Has This Been Tested?

To test locally, you can either run goreleaser --rm-dist --skip-publish --snapshot or make cross; they do the same thing, in the end.

Someone who has write access to this repository should probably do more testing related to releases in order to make sure it works.

Checklist:

<!--- Go over all the following points, and put an x in all the boxes that apply. --> <!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->

  • [x] My code follows the code style of this project.
  • [x] My change requires a change to the documentation.
  • [x] I have updated the documentation accordingly.
+146 -18

17 comments

9 changed files

sc250024

pr closed time in 3 months

startedjsonnet-bundler/jsonnet-bundler

started time in 3 months

startedjsonnet-bundler/jsonnet-bundler

started time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

I'll make some of the "final" changes tonight. I'll also open an issue for multi-arch builds. It should be straightforward, but I would need to do some testing.

By the way, jsonnet-bundler is now in Homebrew => https://github.com/Homebrew/homebrew-core/pull/58487 🥳

sc250024

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

That works for me. In that case I would prefer to keep all container images out of this PR and have the next PR do all the docker things, just so we don't end up in a somewhat "in progress" state.

sc250024

comment created time in 3 months

startedjsonnet-bundler/jsonnet-bundler

started time in 3 months

fork isgasho/jsonnet-bundler

A jsonnet package manager.

fork in 3 months

startedjsonnet-bundler/jsonnet-bundler

started time in 3 months

startedjsonnet-bundler/jsonnet-bundler

started time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

We do this in Tanka and it works well.

Agree it's outside of this PR, let's keep this focused!

If that's the case, and there's already an existing example that you all know works, I can open another PR after this to take care of the multi-arch build.

sc250024

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

We do this in Tanka and it works well.

Agree it's outside of this PR, let's keep this focused!

sc250024

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

@brancz I did some research, and while implementing multi-architecture Docker builds with Drone is possible, it means expanding the scope of this PR quite a bit.

See for yourself how Drone themselves, and Gitea implement multi-arch builds:

  • Drone: Uses the Drone manifest plugin and multiple Dockerfiles
    • https://github.com/drone/drone/blob/master/.drone.yml
    • https://github.com/drone/drone/tree/master/docker
  • Gitea:
    • https://github.com/go-gitea/gitea/blob/master/docker/manifest.tmpl
    • https://github.com/go-gitea/gitea/blob/master/Dockerfile
    • https://github.com/go-gitea/gitea/blob/master/docker/manifest.tmpl

This would mean scope-creeping a bit, and I would need more time to implement this since this really needs testing with Drone locally.

sc250024

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

Judging by demand in other projects, I expect that if we have containers, then people will soon ask for multi arch. Is there a good way that we can combine these concerns? With these changes we will already be building multi-arch.

I'll add them.

sc250024

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

Judging by demand in other projects, I expect that if we have containers, then people will soon ask for multi arch. Is there a good way that we can combine these concerns? With these changes we will already be building multi-arch.

sc250024

comment created time in 3 months

startedjsonnet-bundler/jsonnet-bundler

started time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

Updated docs to reflect Docker and Homebrew. Homebrew-core PR is here: https://github.com/Homebrew/homebrew-core/pull/58487

sc250024

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

I just want the jsonnet folks to be ok with it, and not invade their space. Obviously I'm happy with it being more official :)

sc250024

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

PS @sc250024: You seem to be force-pushing. This makes it slightly harder for us to review your PRs, because addressed review suggestions can't be viewed in their own commit, but must be identified in the big changeset, which is now different than it was before. It might be that you do this for having a clean commit history, but given we squash merge, this extra care is not neccessary :)

@sh0rez Ahh sorry, I'll stop doing that from now on. I just made the last force-push just now. Basically, I took your advice, and used Drone's native Docker building: http://plugins.drone.io/drone-plugins/drone-docker/

sc250024

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

Hey, so afaict the last open question is the location of the docker hub image? I personally like jsonnet/bundler, because it would allow to co-colocate with other popular jsonnet tools like jsonnet/fmt or jsonnet/jsonnet.

Citing a Jsonnet maintainer, I think it's fair enough to do this:

The jsonnet-bundler project is almost-official. It was consulted with @sparkprime and broader community and it has everyone's blessing. So we are counting on jsonnet-bundler becoming the standard for jsonnet dependency management. However, as you mentioned, it's still pretty alpha.

I can create the jsonnet/bundler repo if everyone is okay with this.

PS @sc250024: You seem to be force-pushing. This makes it slightly harder for us to review your PRs, because addressed review suggestions can't be viewed in their own commit, but must be identified in the big changeset, which is now different than it was before. It might be that you do this for having a clean commit history, but given we squash merge, this extra care is not neccessary :)

sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

 go 1.12 require ( 	github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc // indirect 	github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf // indirect-	github.com/campoy/embedmd v1.0.0 // indirect

Oh right, I changed that line some time ago. Let's go ahead, it shouldn't break. Fingers crossed

sc250024

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

That makes this look more official than it currently is, I think it should be jsonnet-bundler/jsonnet-bundler. As I said I don't feel too strongly, but docker containers don't seem necessary, but again if someone wants to maintain all of this I'm not against having it.

I still need to look into the Docker part of Drone CI, but assuming that it works, I would just need a decision as to where to put it.

sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

+archives:+  - builds:+      - jb+    format: tar.gz+    format_overrides:+      - format: zip+        goos: windows+    name_template: '{{ .ProjectName }}-{{ .Os }}-{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}{{ if .Mips }}-{{ .Mips }}{{ end }}'+before:+  hooks:+    - go mod tidy+    - go generate ./...+brews:+  - description: The jsonnet-bundler is a package manager for Jsonnet.+    homepage: https://github.com/jsonnet-bundler/jsonnet-bundler+    skip_upload: true+    tap:+      name: jsonnet-bundler+      owner: jsonnet-bundler+builds:+  - binary: jb+    env:+      - CGO_ENABLED=0+      - GO111MODULE=on+    goarch:+      - amd64+      - arm+      - arm64+    goarm:+      - 6+      - 7+    goos:+      - darwin+      - linux+      - windows

Pushed just now.

sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

     ],   }, +  local release(version) = std.mergePatch(+    golang(version) { commands: ['curl -sL https://git.io/goreleaser | bash'] },+    ({+      environment: { GITHUB_TOKEN: { from_secret: 'github_token' } },+      when: { event: 'tag' },+    })+  ),

@sh0rez Nice! Pushed just now.

sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

+archives:+  - builds:+      - jb+    format: tar.gz+    format_overrides:+      - format: zip+        goos: windows+    name_template: '{{ .ProjectName }}-{{ .Os }}-{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}{{ if .Mips }}-{{ .Mips }}{{ end }}'+before:+  hooks:+    - go mod tidy+    - go generate ./...+brews:+  - description: The jsonnet-bundler is a package manager for Jsonnet.

Pushed just now.

sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

+archives:+  - builds:+      - jb+    format: tar.gz+    format_overrides:+      - format: zip+        goos: windows+    name_template: '{{ .ProjectName }}-{{ .Os }}-{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}{{ if .Mips }}-{{ .Mips }}{{ end }}'+before:+  hooks:+    - go mod tidy+    - go generate ./...+brews:+  - description: The jsonnet-bundler is a package manager for Jsonnet.

This section will go away entirely since there's no self-hosting of the Homebrew formula.

sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

 go 1.12 require ( 	github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc // indirect 	github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf // indirect-	github.com/campoy/embedmd v1.0.0 // indirect

These lines are removed because of the before.hooks sections in the .goreleaser.yml:

before:
  hooks:
    - go mod tidy
    - go generate ./...

I dug into this a bit, and I think they're still not needed. You make a good point that embedmd is needed for generating documentation, but if you look at the Makefile, it actually tries to avoid putting that library in the go.mod file:

embedmd:
	pushd /tmp && GO111MODULES=on go get github.com/campoy/embedmd && popd

As far as I know, using that trick basically avoids putting it into the go.mod and go.sum files, right?

sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

-FROM busybox:1.28+FROM busybox:1.32

Will change in the next commit.

sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

+archives:+  - builds:+      - jb+    format: tar.gz+    format_overrides:+      - format: zip+        goos: windows+    name_template: '{{ .ProjectName }}-{{ .Os }}-{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}{{ if .Mips }}-{{ .Mips }}{{ end }}'+before:+  hooks:+    - go mod tidy+    - go generate ./...+brews:+  - description: The jsonnet-bundler is a package manager for Jsonnet.+    homepage: https://github.com/jsonnet-bundler/jsonnet-bundler+    skip_upload: true+    tap:+      name: jsonnet-bundler+      owner: jsonnet-bundler+builds:+  - binary: jb+    env:+      - CGO_ENABLED=0+      - GO111MODULE=on+    goarch:+      - amd64+      - arm+      - arm64+    goarm:+      - 6+      - 7+    goos:+      - darwin+      - linux+      - windows

For now, it's only generating the ones you mentioned. If you look at the tree output above in the PR comments, it generates the following archives:

jb-darwin-amd64.tar.gz
jb-linux-amd64.tar.gz
jb-linux-arm64.tar.gz
jb-linux-armv6.tar.gz
jb-linux-armv7.tar.gz
jb-windows-amd64.zip
sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

     ],   }, +  local release(version) = std.mergePatch(+    golang(version) { commands: ['curl -sL https://git.io/goreleaser | bash'] },+    ({+      environment: { GITHUB_TOKEN: { from_secret: 'github_token' } },+      when: { event: 'tag' },+    })+  ),

https://tanka.dev/jsonnet/overview#merging

sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

     ],   }, +  local release(version) = std.mergePatch(+    golang(version) { commands: ['curl -sL https://git.io/goreleaser | bash'] },+    ({+      environment: { GITHUB_TOKEN: { from_secret: 'github_token' } },+      when: { event: 'tag' },+    })+  ),

Ah damn, I always wondered what the +: syntax meant. I will fix in next commit.

sc250024

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

No strong opinion on all of this, obviously +1 on automation, all I want to make sure and emphasize is I want to ensure that go get will always continue to work.

Given I got https://hub.docker.com/orgs/jsonnet, docker.io/jsonnet/bundler sounds neat to me

That makes this look more official than it currently is, I think it should be jsonnet-bundler/jsonnet-bundler. As I said I don't feel too strongly, but docker containers don't seem necessary, but again if someone wants to maintain all of this I'm not against having it.

sc250024

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

It can. See here: https://github.com/sc250024/jsonnet-bundler/blob/feat-GoReleaser/.goreleaser.yml#L41-L51.

Only if it has a docker daemon in CI, right? And I'm not quite sure you can get one without the plugin

Where should the Docker repo be hosted?

Given I got https://hub.docker.com/orgs/jsonnet, docker.io/jsonnet/bundler sounds neat to me

sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

 go 1.12 require ( 	github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc // indirect 	github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf // indirect-	github.com/campoy/embedmd v1.0.0 // indirect

go.mod setup is a bit borked here, this is required for generating the markdown file (go generate)

sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

-FROM busybox:1.28+FROM busybox:1.32

If nobody objects, let's use alpine here

sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

+archives:+  - builds:+      - jb+    format: tar.gz+    format_overrides:+      - format: zip+        goos: windows+    name_template: '{{ .ProjectName }}-{{ .Os }}-{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}{{ if .Mips }}-{{ .Mips }}{{ end }}'+before:+  hooks:+    - go mod tidy+    - go generate ./...+brews:+  - description: The jsonnet-bundler is a package manager for Jsonnet.+    homepage: https://github.com/jsonnet-bundler/jsonnet-bundler+    skip_upload: true+    tap:+      name: jsonnet-bundler+      owner: jsonnet-bundler+builds:+  - binary: jb+    env:+      - CGO_ENABLED=0+      - GO111MODULE=on+    goarch:+      - amd64+      - arm+      - arm64+    goarm:+      - 6+      - 7+    goos:+      - darwin+      - linux+      - windows

Will that also attempt windows/arm builds, etc? I think we only need:

linux/amd64
linux/arm64
linux/arm
darwin/amd64
windows/amd64

and due to apple silicon maybe soon darwin/arm64

sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

+archives:+  - builds:+      - jb+    format: tar.gz+    format_overrides:+      - format: zip+        goos: windows+    name_template: '{{ .ProjectName }}-{{ .Os }}-{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}{{ if .Mips }}-{{ .Mips }}{{ end }}'+before:+  hooks:+    - go mod tidy+    - go generate ./...+brews:+  - description: The jsonnet-bundler is a package manager for Jsonnet.
  - description: jsonnet-bundler is a package manager for Jsonnet.

I think we usually refer to it without a "the"

sc250024

comment created time in 3 months

Pull request review commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

     ],   }, +  local release(version) = std.mergePatch(+    golang(version) { commands: ['curl -sL https://git.io/goreleaser | bash'] },+    ({+      environment: { GITHUB_TOKEN: { from_secret: 'github_token' } },+      when: { event: 'tag' },+    })+  ),

std.mergePatch is not required to be used manually in Jsonnet. Instead, you can use syntax features:

  local release(version) = golang(version) {
    commands: ['curl -sL https://git.io/goreleaser | bash'] },
    environment+: { GITHUB_TOKEN: { from_secret: 'github_token' } },
    when: { event: 'tag' },
  },

Notice the use of +: (merge) and : (override)

sc250024

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

@sh0rez Perfect. I will have to update the .goreleaser.yml file based on the changes you mentioned, assuming everyone else doesn't have further comments.

To reply to some of the items:

Yes! All artifacts should be made in a reproducible by the CI. The GH release should only be a Draft and require manual publishing.

GoReleaser supports this.

Not sure whether it's actually possible to use goreleaser for building the docker images.

It can. See here: https://github.com/sc250024/jsonnet-bundler/blob/feat-GoReleaser/.goreleaser.yml#L41-L51

I'd really favor option 3, having it in homebrew-core. Updating there is fairly straight-forward and it would be provided to every user as-is, without needing taps.

Alright, that means we don't need a brews section here, and therefore, Tar / ZIP archives are not needed then. I'll make those changes.

sc250024

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

Hi @sc250024, thanks a lot for setting this up!

Let me answer your questions from my personal perspective. This is not authoritative for all maintainers, but it's what I consider good and hopefully aligns with the others:

Should GoReleaser be triggered automatically by Drone on a tag push, or should a maintainer perform the releasing manually?

Yes! All artifacts should be made in a reproducible by the CI. The GH release should only be a Draft and require manual publishing.

Are Docker images wanted? See Docker registry below.

Yes! Having docker images allows other Dockerfiles to use FROM jb; COPY /usr/local/bin/jb ., which is real handy.

Is a Homebrew formula wanted? See Homebrew formula below.

Yes!

What artifacts should be attached to the GitHub release? Binaries only? Archives only? Both? See GitHub releases.

jb is fairly small, I'd vote for plain binaries as it requires least tooling on the consumer side


Docker registry

Not sure whether it's actually possible to use goreleaser for building the docker images – Drone CI is fairly restrictive with root permissions, you can't just run a docker daemon. You need to use their docker plugin, which is whitelisted server-side to elevate it's permissions.

Unless I miss something, we need to keep this in mind :)

Homebrew formula

I'd really favor option 3, having it in homebrew-core. Updating there is fairly straight-forward and it would be provided to every user as-is, without needing taps.

GitHub releases

Plain binaries should work just fine. We do the same for Tanka, and we have a working brew Formula as well: https://formulae.brew.sh/formula/tanka


Again, thanks for taking the time to look at this. This is super cool!!

sc250024

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

/ping @metalmatze @sh0rez

sc250024

comment created time in 3 months

issue commentjsonnet-bundler/jsonnet-bundler

feature: Distribute jsonnet-bundler through mainstream package registries

@sc250024 Wow, that was fast 🚀 It looks solid to me, let see what maintainers think.

kakkoyun

comment created time in 3 months

issue commentjsonnet-bundler/jsonnet-bundler

feature: Distribute jsonnet-bundler through mainstream package registries

@kakkoyun Would you mind taking a peek? It's a big one => https://github.com/jsonnet-bundler/jsonnet-bundler/pull/128

kakkoyun

comment created time in 3 months

PR opened jsonnet-bundler/jsonnet-bundler

feat: Adding GoReleaser to handle releases

Enables this repository to use GoReleaser to handle all releases. This PR is basically ready, it just needs some feedback from the maintainers.

Description

By merging this PR, this repository will use GoReleaser for releasing of all artifacts. This includes, but is not limited to:

  • Docker images
  • GitHub releases with Go binaries pre-packaged
  • Homebrew formulas

There are a number of issues to be addressed / questions to be answered by the maintainers of this project in order to merge this PR.

Motivation and Context

This fixes issue https://github.com/jsonnet-bundler/jsonnet-bundler/issues/64, which has been open for a while. Essentially, it enables jsonnet-bundler to be packaged more easily so it can be distributed amongst the main channels, i.e. Docker, Homebrew, etc.

Items to Address

Generic questions

  1. Should GoReleaser be triggered automatically by Drone on a tag push, or should a maintainer perform the releasing manually?
  2. Are Docker images wanted? See Docker registry.
  3. Is a Homebrew formula wanted? See Homebrew formula.
  4. What artifacts should be attached to the GitHub release? Binaries only? Archives only? Both? See GitHub releases.

Docker registry

In order for this to be merged, the repository maintainers should decide where to put Docker images, if at all. If it's desired to not have Docker images generated, that's also possible. If it's decided that this is wanted, credentials would also need to be generated to push to the Docker registry.

Essentially, the GoReleaser builds tags according to semver specifications. For example, if this project is hosted on DockerHub, the following images would be generated:

  • docker.io/jsonnet-bundler/jb:latest
  • docker.io/jsonnet-bundler/jb:v0.4.0
  • docker.io/jsonnet-bundler/jb:v0.4
  • docker.io/jsonnet-bundler/jb:v0

Other options are also possible.

Homebrew formula

In a similar fashion, if it's decided to create Homebrew formulas, the following options are available:

  • Option 1: Host the Homebrew tap from this repository, which means creating a Formula folder at the root.
  • Option 2: Host the Homebrew tap from another repository in the jsonnet-bunlder organization; in which case, a Github token would need to be provided by the maintainers which has access to write to that repository.
  • Option 3: Don't host any Homebrew tap, and later manually create a formula in the https://github.com/Homebrew/homebrew-core repository.

GitHub releases

Currently the jb binary is added to GitHub releases directly without packaging. This is also possible with GoReleaser in the exact same way, but if Homebrew is being used as well, then Homebrew needs archives to be built. In other words, Homebrew does not download binaries directly; it always first downloads an archive, and then extracts it.

Coupled with the Homebrew options above are the following options for GitHub releases:

  • Option 1: Don't use Homebrew at all; attach binaries to the GitHub release as it happens right now.
  • Option 2: Use Homebrew in some way; attach archives only to the GitHub release.
  • Option 3: Use Homebrew in some way; attach both archives and binaries to the GitHub release, and let people choose.

How Has This Been Tested?

To test locally, you can either run goreleaser --rm-dist --skip-publish --snapshot or make cross; they do the same thing, in the end.

Someone who has write access to this repository should probably do more testing related to releases in order to make sure it works.

Checklist:

<!--- Go over all the following points, and put an x in all the boxes that apply. --> <!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->

  • [x] My code follows the code style of this project.
  • [x] My change requires a change to the documentation.
  • [x] I have updated the documentation accordingly.
+89 -12

0 comment

7 changed files

pr created time in 3 months

issue commentjsonnet-bundler/jsonnet-bundler

feature: Distribute jsonnet-bundler through mainstream package registries

@kakkoyun I'll spin up a PR, and let you know.

kakkoyun

comment created time in 3 months

issue commentjsonnet-bundler/jsonnet-bundler

feature: Distribute jsonnet-bundler through mainstream package registries

I don't see anybody disagrees. Just somebody needs to the work. I have been planning to send a PR for quite some time, I just couldn't get to it. So feel free to contribute.

kakkoyun

comment created time in 3 months

pull request commentjsonnet-bundler/jsonnet-bundler

Add support for azure devops https git address

@tehho Sorry to piggyback on your PR, but I must ask the repo maintainers: why does a PR like this exist in the first place? I commented on this issue (https://github.com/jsonnet-bundler/jsonnet-bundler/issues/72) a while back; at that time it was about GitLab support, but this PR is also along the same lines.

It seems to me this problem can be solved in a similar manner as how Terraform solves this problem. They don't deal with jsonnet files obviously, but they have to work with module sources which reference Git repositories in the same way: Terraform Module Sources.

Why not just have two options:

var gitProtoFmts = map[string]string{
	GitSchemeSSH:   GitSchemeSSH + "%s/%s/%s.git",
	GitSchemeHTTPS: GitSchemeHTTPS + "%s/%s/%s.git",
}

But instead of hardcoding the format like above, just do what Terraform does, and let a double-slash // indicate where the repository "begins", and we don't have to add more formats as time progresses.

Then, all of the following would work:

  • https://dev.azure.com/my-org/my-project/_git//some-crazy-ass-app
  • https://gitlab.com/company/team/devops//some-crazy-ass-helm-chart
  • https://github.com/your-ass//a-hole-in-the-ground
  • git@ssh.dev.azure.com:v3/fabrikam-fiber/FabrikamFiber//FabrikamFiber.git
  • git@ssh.dev.azure.com:my-org/my-project/_git//some-crazy-ass-app.git
  • git@gitlab.com:company/team/devops//some-crazy-ass-helm-chart.git
  • git@github.com:your-ass//a-hole-in-the-ground.git

How Terraform does things

All of the following work with Terraform. From their documentation:

When the source of a module is a version control repository or archive file (generically, a "package"), the module itself may be in a sub-directory relative to the root of the package.

A special double-slash syntax is interpreted by Terraform to indicate that the remaining path after that point is a sub-directory within the package. For example:

  • hashicorp/consul/aws//modules/consul-cluster
  • git::https://example.com/network.git//modules/vpc
  • https://example.com/network-module.zip//modules/vpc
  • s3::https://s3-eu-west-1.amazonaws.com/examplecorp-terraform-modules/network.zip//modules/vpc

If the source address has arguments, such as the ref argument supported for the version control sources, the sub-directory portion must be before those arguments:

git::https://example.com/network.git//modules/vpc?ref=v1.2.0

In our case, the double-slash // would not be an indication of a sub-directory as they have in their example above. Instead, it would be the indication to say "this is where the repository starts."

tehho

comment created time in 3 months

issue commentjsonnet-bundler/jsonnet-bundler

feature: Distribute jsonnet-bundler through mainstream package registries

What about Homebrew? Would anyone be opposed to using goreleaser in this repository to handle all releasing, as well as handling the creation of a Homebrew tap here?

kakkoyun

comment created time in 3 months

issue commentjsonnet-bundler/jsonnet-bundler

feature request: save `--jsonnetpkg-home` in `jsonnetfile.json`

Agreed. Would be super useful.

ghostsquad

comment created time in 3 months

startedjsonnet-bundler/jsonnet-bundler

started time in 3 months

more