profile
viewpoint
Brigade brigadecore https://brigade.sh Event-based Scripting for Kubernetes

brigadecore/brigade 1968

Event-based Scripting for Kubernetes.

brigadecore/kashti 345

Kashti is a dashboard for your Brigade pipelines.

brigadecore/buck 34

Brigade Universal Controller for Kubernetes

brigadecore/brigade-k8s-gateway 23

Send Kubernetes events into a Brigade pipeline

brigadecore/brigade-github-app 15

Brigade GitHub App Gateway for forwarding events from GitHub to Brigade

brigadecore/brigadier 8

A JavaScript library for writing and testing brigade.js files

brigadecore/brigade-eventgrid-gateway 7

Brigade gateway that responds to Azure EventGrid events using CloudEvents schema

brigadecore/brigade-utils 7

Commonly used Brigade jobs

brigadecore/charts 4

Helm charts for the Brigade project

startedbrigadecore/brigade

started time in 17 hours

startedbrigadecore/brigade

started time in a day

pull request commentbrigadecore/brigade

Update installation command to Helm 3

Ooh, I missed that there were more, good catch. Updating now.

radu-matei

comment created time in a day

pull request commentbrigadecore/brigade

Update installation command to Helm 3

Deploy preview for brigade-docs ready!

Built with commit f40bb54158819e78314bda7c896856ee1cc944bb

https://deploy-preview-1113--brigade-docs.netlify.app

radu-matei

comment created time in a day

PR opened brigadecore/brigade

Reviewers
Update installation command to Helm 3

This commit updates the instructions in the installation guide to use the correct Helm 3 command for deploying Brigade - with the name passed as the first argument, rather than with the --name flag.

Signed-off-by: Radu M root@radu.sh

+1 -1

0 comment

1 changed file

pr created time in a day

issue commentbrigadecore/brigade

Vacuum doesn't delete resources for completed builds.

I also just tried with a new 1.16 cluster and chart versionbrigade-1.6.1. To make things easier, I set the vacuum to delete all jobs older than 1 minute, and run every 5 minutes (not recommended in a real cluster - --set vacuum.schedule="*/5 * * * *" --set vacuum.age="1m"), and the job runs fine for me as well:

$ k describe cronjob brigade-brigade-vacuum
Name:                          brigade-brigade-vacuum
Namespace:                     brigade
Labels:                        app.kubernetes.io/instance=brigade
                               app.kubernetes.io/managed-by=Helm
                               app.kubernetes.io/name=brigade-brigade
                               helm.sh/chart=brigade-1.6.1
                               role=vacuum
Annotations:                   meta.helm.sh/release-name: brigade
                               meta.helm.sh/release-namespace: brigade
Schedule:                      */5 * * * *
Concurrency Policy:            Allow
Suspend:                       False
Successful Job History Limit:  10
Failed Job History Limit:      10
Starting Deadline Seconds:     <unset>
Selector:                      <unset>
Parallelism:                   <unset>
Completions:                   <unset>
Pod Template:
  Labels:           app.kubernetes.io/name=brigade-brigade
                    role=vacuum
  Service Account:  brigade-vacuum
  Containers:
   brigade-vacuum:
    Image:      brigadecore/brigade-vacuum:v1.4.0
    Port:       <none>
    Host Port:  <none>
    Environment:
      VACUUM_AGE:                  1m
      VACUUM_MAX_BUILDS:           -1
      VACUUM_SKIP_RUNNING_BUILDS:  false
      BRIGADE_NAMESPACE:            (v1:metadata.namespace)
    Mounts:                        <none>
  Volumes:                         <none>
Last Schedule Time:                Wed, 05 Aug 2020 07:10:00 -0700
Active Jobs:                       <none>
Events:
  Type    Reason            Age    From                Message
  ----    ------            ----   ----                -------
  Normal  SuccessfulCreate  8m1s   cronjob-controller  Created job brigade-brigade-vacuum-1596636300
  Normal  SawCompletedJob   7m51s  cronjob-controller  Saw completed job: brigade-brigade-vacuum-1596636300, status: Complete
  Normal  SuccessfulCreate  3m     cronjob-controller  Created job brigade-brigade-vacuum-1596636600
  Normal  SawCompletedJob   2m50s  cronjob-controller  Saw completed job: brigade-brigade-vacuum-1596636600, status: Complete

And one job:

$ k describe job brigade-brigade-vacuum-1596636300
Name:           brigade-brigade-vacuum-1596636300
Namespace:      brigade
Selector:       controller-uid=430a14f4-0fe9-4715-b462-ce8a2a164e42
Labels:         app.kubernetes.io/name=brigade-brigade
                controller-uid=430a14f4-0fe9-4715-b462-ce8a2a164e42
                job-name=brigade-brigade-vacuum-1596636300
                role=vacuum
Annotations:    <none>
Controlled By:  CronJob/brigade-brigade-vacuum
Parallelism:    1
Completions:    1
Start Time:     Wed, 05 Aug 2020 07:05:00 -0700
Completed At:   Wed, 05 Aug 2020 07:05:05 -0700
Duration:       5s
Pods Statuses:  0 Running / 1 Succeeded / 0 Failed
Pod Template:
  Labels:           app.kubernetes.io/name=brigade-brigade
                    controller-uid=430a14f4-0fe9-4715-b462-ce8a2a164e42
                    job-name=brigade-brigade-vacuum-1596636300
                    role=vacuum
  Service Account:  brigade-vacuum
  Containers:
   brigade-vacuum:
    Image:      brigadecore/brigade-vacuum:v1.4.0
    Port:       <none>
    Host Port:  <none>
    Environment:
      VACUUM_AGE:                  1m
      VACUUM_MAX_BUILDS:           -1
      VACUUM_SKIP_RUNNING_BUILDS:  false
      BRIGADE_NAMESPACE:            (v1:metadata.namespace)
    Mounts:                        <none>
  Volumes:                         <none>
Events:
  Type    Reason            Age    From            Message
  ----    ------            ----   ----            -------
  Normal  SuccessfulCreate  9m20s  job-controller  Created pod: brigade-brigade-vacuum-1596636300-6hwqm

Do you have RBAC enabled? What is the vacuum role configured like?

olomakovskyi-blackbird

comment created time in a day

issue commentbrigadecore/brigade

Vacuum doesn't delete resources for completed builds.

Thanks for trying the upgrade approach. Can we try the previous release of the vacuum via setting the tag to v1.3.0 in the values file here and upgrading the brigade release? It's a longshot, but could be worth the low effort to check.

All of the worker secrets/pods are in the same (probably default) namespace that the vacuum is configured with, right?

olomakovskyi-blackbird

comment created time in 2 days

issue commentbrigadecore/brigade

Vacuum doesn't delete resources for completed builds.

Thank you for the quick response. So I upgraded my cluster to 1.16.10 and unfortunately, it didn't help. I still can see the same log on pod created by a vacuum job but worker pods are still there.

kubectl logs brigade-brigade-vacuum-1596621600-7tfls

W0805 10:00:12.834819       1 client_config.go:543] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
2020/08/05 10:00:12 Pruning records older than 2020-08-05 09:00:12.834684911 +0000 UTC m=-3599.991204137

The updated output of kubectl version

Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4", GitCommit:"8d8aa39598534325ad77120c120a22b3a990b5ea", GitTreeState:"clean", BuildDate:"2020-03-12T21:03:42Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.10", GitCommit:"89d8075525967c7a619641fabcb267358d28bf08", GitTreeState:"clean", BuildDate:"2020-06-23T02:52:37Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

Will appreciate any new ideas :)

olomakovskyi-blackbird

comment created time in 2 days

issue commentbrigadecore/brigade

Use-case: Durable Workflow

Not sure if this really belongs here, but in addition to being able to restart failed pipelines it would be nice to be able to re-run successful pupelines. Having this functionality exposed via the Kashti UI might also be nice :)

I am more interested in being to easily restart entire pipelines since I expect that I can just chain pipelines to achieve some intermediate checkpoint if I want

mumoshu

comment created time in 2 days

issue commentbrigadecore/brigade

Vacuum doesn't delete resources for completed builds.

Hi @olomakovskyi-blackbird . Thank you for the wealth of detailed information. I'm not seeing anything that looks amiss, so I'm also confused. I was unable to reproduce on my AKS v1.17.7 cluster (secrets and pods older than 1h were successfully deleted.)

My only guess is perhaps the k8s/client-go updates in 1.4.0 (libs bumped to v1.17.x) somehow affected the secret listing/deletion logic in vacuum.go. Would it be possible to either try the v1.3.0 tag for brigade-vacuum or try upgrading the underlying k8s cluster to 1.16+?

olomakovskyi-blackbird

comment created time in 2 days

issue openedbrigadecore/brigade

Vacuum doesn't delete resources for completed builds.

Hi, I have enabled vacuum on my cluster but it looks like it doesn't clear resources according to the settings. I haven't found similar issues here so I assume that probably something is wrong in my config. What am I missing?

kubectl describe cronjob brigade-brigade-vacuum

Namespace:                     default
Labels:                        app.kubernetes.io/instance=brigade
                               app.kubernetes.io/managed-by=Helm
                               app.kubernetes.io/name=brigade-brigade
                               helm.sh/chart=brigade-1.6.0
                               role=vacuum
Annotations:                   meta.helm.sh/release-name: brigade
                               meta.helm.sh/release-namespace: default
Schedule:                      @hourly
Concurrency Policy:            Allow
Suspend:                       False
Successful Job History Limit:  10
Failed Job History Limit:      10
Starting Deadline Seconds:     <unset>
Selector:                      <unset>
Parallelism:                   <unset>
Completions:                   <unset>
Pod Template:
  Labels:           app.kubernetes.io/name=brigade-brigade
                    role=vacuum
  Service Account:  brigade-vacuum
  Containers:
   brigade-vacuum:
    Image:      brigadecore/brigade-vacuum:v1.4.0
    Port:       <none>
    Host Port:  <none>
    Environment:
      VACUUM_AGE:                  1h
      VACUUM_MAX_BUILDS:           -1
      VACUUM_SKIP_RUNNING_BUILDS:  true
      BRIGADE_NAMESPACE:            (v1:metadata.namespace)
    Mounts:                        <none>
  Volumes:                         <none>
Last Schedule Time:                Tue, 04 Aug 2020 16:00:00 +0300
Active Jobs:                       <none>
Events:
  Type    Reason            Age    From                Message
  ----    ------            ----   ----                -------
  Normal  SuccessfulCreate  7m10s  cronjob-controller  Created job brigade-brigade-vacuum-1596546000
  Normal  SawCompletedJob   7m     cronjob-controller  Saw completed job: brigade-brigade-vacuum-1596546000, status: Complete
  Normal  SuccessfulDelete  7m     cronjob-controller  Deleted job brigade-brigade-vacuum-1596510000

kubectl get job

brigade-brigade-vacuum-1596517200   1/1           2s         8h
brigade-brigade-vacuum-1596520800   1/1           3s         7h7m
brigade-brigade-vacuum-1596524400   1/1           2s         6h7m
brigade-brigade-vacuum-1596528000   1/1           1s         5h7m
brigade-brigade-vacuum-1596531600   1/1           3s         4h7m
brigade-brigade-vacuum-1596535200   1/1           3s         3h7m
brigade-brigade-vacuum-1596538800   1/1           3s         127m
brigade-brigade-vacuum-1596542400   1/1           3s         67m
brigade-brigade-vacuum-1596546000   1/1           2s         7m45s

kubectl describe job brigade-brigade-vacuum-1596546000

Namespace:      default
Selector:       controller-uid=b537daad-0191-4c08-81fc-580a871ac0a1
Labels:         app.kubernetes.io/name=brigade-brigade
                controller-uid=b537daad-0191-4c08-81fc-580a871ac0a1
                job-name=brigade-brigade-vacuum-1596546000
                role=vacuum
Annotations:    <none>
Controlled By:  CronJob/brigade-brigade-vacuum
Parallelism:    1
Completions:    1
Start Time:     Tue, 04 Aug 2020 16:00:06 +0300
Completed At:   Tue, 04 Aug 2020 16:00:08 +0300
Duration:       2s
Pods Statuses:  0 Running / 1 Succeeded / 0 Failed
Pod Template:
  Labels:           app.kubernetes.io/name=brigade-brigade
                    controller-uid=b537daad-0191-4c08-81fc-580a871ac0a1
                    job-name=brigade-brigade-vacuum-1596546000
                    role=vacuum
  Service Account:  brigade-vacuum
  Containers:
   brigade-vacuum:
    Image:      brigadecore/brigade-vacuum:v1.4.0
    Port:       <none>
    Host Port:  <none>
    Environment:
      VACUUM_AGE:                  1h
      VACUUM_MAX_BUILDS:           -1
      VACUUM_SKIP_RUNNING_BUILDS:  true
      BRIGADE_NAMESPACE:            (v1:metadata.namespace)
    Mounts:                        <none>
  Volumes:                         <none>
Events:
  Type    Reason            Age    From            Message
  ----    ------            ----   ----            -------
  Normal  SuccessfulCreate  8m33s  job-controller  Created pod: brigade-brigade-vacuum-1596546000-w4v2n

kubectl logs brigade-brigade-vacuum-1596546000-w4v2n

W0804 13:00:08.223420       1 client_config.go:543] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
2020/08/04 13:00:08 Pruning records older than 2020-08-04 12:00:08.223387171 +0000 UTC m=-3599.994107663

From my understanding vacuum should delete all resources related to completed builds that older than 1h but I can see all of them still present in my cluster. For example worker pods: kubectl get pods | grep worker

brigade-worker-01eev6jdmknj1s4avre56sy4z2   0/1     Completed    0          14h
brigade-worker-01eev7jz3qf14cn98s8qa24afq   0/1     Completed    0          14h
brigade-worker-01eev94gdvvrmyvnvmtz4v38m9   0/1     Completed    0          14h
brigade-worker-01eewk220c1v3gp8rha6pneamc   0/1     Completed    0          113m
brigade-worker-01eewk649ktmmw4hc8kc60m5f7   0/1     Completed    0          111m
brigade-worker-01eewkkr67aq6yd3ztvffx65tg   0/1     Completed    0          104m
brigade-worker-01eewnmnedghphvp78s6d6kc64   0/1     Completed    0          68m

Output of brig version: v1.3.0

Output of kubectl version: Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4", GitCommit:"8d8aa39598534325ad77120c120a22b3a990b5ea", GitTreeState:"clean", BuildDate:"2020-03-12T21:03:42Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.11", GitCommit:"ec831747a3a5896dbdf53f259eafea2a2595217c", GitTreeState:"clean", BuildDate:"2020-05-29T19:56:10Z", GoVersion:"go1.12.17", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider/Platform (AKS, GKE, Minikube etc.): AKS

created time in 3 days

push eventbrigadecore/kashti

Radu M

commit sha 6ac3069276744f21fafb219415f4af9631c75280

Return sorted observable for job service (#296) Signed-off-by: Radu M <root@radu.sh>

view details

push time in 3 days

PR merged brigadecore/kashti

Order jobs in build

closes #294

This PR returns a sorted observable from the job service, sorted by start date. Ideally in the future, this should be performed by the API, but until that happens, we can sort client-side.

+17 -8

1 comment

2 changed files

radu-matei

pr closed time in 3 days

issue closedbrigadecore/kashti

Job list in build not ordered by start date

#14 #124 both addressed some variation of this issue - leaving the chart at the top of the page aside, the jobs should be sorted by their start date, and we can do this in Kashti (without changing the API server) rather easily.

image

closed time in 3 days

radu-matei

startedbrigadecore/brigade-k8s-gateway

started time in 4 days

startedbrigadecore/brigade

started time in 4 days

startedbrigadecore/brigade

started time in 5 days

startedbrigadecore/brigade

started time in 6 days

pull request commentbrigadecore/kashti

Order jobs in build

Needs a rebase on top of #297

radu-matei

comment created time in 6 days

delete branch brigadecore/kashti

delete branch : dependabot/npm_and_yarn/elliptic-6.5.3

delete time in 6 days

push eventbrigadecore/kashti

dependabot[bot]

commit sha 9320d53877a70becac5f2f0f9df5627dd4efb264

chore(deps): bump elliptic from 6.5.1 to 6.5.3 (#297) Bumps [elliptic](https://github.com/indutny/elliptic) from 6.5.1 to 6.5.3. - [Release notes](https://github.com/indutny/elliptic/releases) - [Commits](https://github.com/indutny/elliptic/compare/v6.5.1...v6.5.3) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

view details

push time in 6 days

PR merged brigadecore/kashti

chore(deps): bump elliptic from 6.5.1 to 6.5.3 dependencies

Bumps elliptic from 6.5.1 to 6.5.3. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/indutny/elliptic/commit/8647803dc3d90506aa03021737f7b061ba959ae1"><code>8647803</code></a> 6.5.3</li> <li><a href="https://github.com/indutny/elliptic/commit/856fe4d99fe7b6200556e6400b3bf585b1721bec"><code>856fe4d</code></a> signature: prevent malleability and overflows</li> <li><a href="https://github.com/indutny/elliptic/commit/60489415e545efdfd3010ae74b9726facbf08ca8"><code>6048941</code></a> 6.5.2</li> <li><a href="https://github.com/indutny/elliptic/commit/9984964457c9f8a63b91b01ea103260417eca237"><code>9984964</code></a> package: bump dependencies</li> <li><a href="https://github.com/indutny/elliptic/commit/ec735edde187a43693197f6fa3667ceade751a3a"><code>ec735ed</code></a> utils: leak less information in <code>getNAF()</code></li> <li>See full diff in <a href="https://github.com/indutny/elliptic/compare/v6.5.1...v6.5.3">compare view</a></li> </ul> </details> <br />

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


<details> <summary>Dependabot commands and options</summary> <br />

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
  • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
  • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
  • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
  • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

</details>

+6 -6

0 comment

1 changed file

dependabot[bot]

pr closed time in 6 days

startedbrigadecore/brigade-k8s-gateway

started time in 6 days

startedbrigadecore/kashti

started time in 6 days

startedbrigadecore/brigade

started time in 7 days

PR opened brigadecore/kashti

chore(deps): bump elliptic from 6.5.1 to 6.5.3

Bumps elliptic from 6.5.1 to 6.5.3. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/indutny/elliptic/commit/8647803dc3d90506aa03021737f7b061ba959ae1"><code>8647803</code></a> 6.5.3</li> <li><a href="https://github.com/indutny/elliptic/commit/856fe4d99fe7b6200556e6400b3bf585b1721bec"><code>856fe4d</code></a> signature: prevent malleability and overflows</li> <li><a href="https://github.com/indutny/elliptic/commit/60489415e545efdfd3010ae74b9726facbf08ca8"><code>6048941</code></a> 6.5.2</li> <li><a href="https://github.com/indutny/elliptic/commit/9984964457c9f8a63b91b01ea103260417eca237"><code>9984964</code></a> package: bump dependencies</li> <li><a href="https://github.com/indutny/elliptic/commit/ec735edde187a43693197f6fa3667ceade751a3a"><code>ec735ed</code></a> utils: leak less information in <code>getNAF()</code></li> <li>See full diff in <a href="https://github.com/indutny/elliptic/compare/v6.5.1...v6.5.3">compare view</a></li> </ul> </details> <br />

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


<details> <summary>Dependabot commands and options</summary> <br />

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
  • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
  • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
  • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
  • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

</details>

+6 -6

0 comment

1 changed file

pr created time in 7 days

create barnchbrigadecore/kashti

branch : dependabot/npm_and_yarn/elliptic-6.5.3

created branch time in 7 days

PR opened brigadecore/kashti

Order jobs in build

This PR returns a sorted observable from the job service, sorted by start date. Ideally in the future, this should be performed by the API, but until that happens, we can sort client-side.

+92 -16

0 comment

4 changed files

pr created time in 7 days

startedbrigadecore/brigade

started time in 8 days

startedbrigadecore/brigade

started time in 8 days

more