caolan/highland 3320
High-level streams library for Node.js and the browser
Node PAckage Registry
ES2015 spec-compliant shim for Reflect.ownKeys
glenjamin/azure-terraform-generate 3
Generate terraform config from deployed azure resources.
Like a whiteboard, but with bytes.
Instructions and code samples for (def shef 26)
Async utilities for node and the browser
LRUG Ruby Fight Club for October
BDD framework for PHP 5.3
glenjamin/BehatMultiFormatter 1
A Behat extension adding a multipage HTML formatter with index and summary
push eventcircleci/circleci-docs
commit sha 2c2eb87fa8b6563a727fba16bb2626de0abfe645
Update executor-types.md
commit sha f05319e180d44adcd59169d84d0fe399cd86906d
Update private-images.md
commit sha 9fd84e4cdc6ba3af8623ee7b5eddd9740d54e258
Merge pull request #4882 from circleci/ProjectFrank-patch-1
push time in 2 days
PR merged circleci/circleci-docs
Document CircleCI/DockerHub partnership with regard to docker image pulls.
pr closed time in 2 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add examples of the duration format
The default value is `false`. ### runner.max_run_time -This can be used to override the default maximum time the Task Agent will run each job.+This can be used to override the default maximum duration the Task Agent will run each job.++The value is a string with the following unit identifiers `h`, `m` or `s` for hour minute and seconds respectively:++A few valid examples++* `72h` - 3 days +* `1h30m` - 1 hour 30 minutes+* `30s` - 30 seconds+* `50m` - 50 minutes+* `1h30m20s` - An overly specific (yet still valid) duration.
😂
comment created time in 2 days
Pull request review commentCircleCI-Public/runner-preview-docs
Update the launch agent config reference docs
If `%s` is present in the value, it will be replaced with a different value for This allows for control of working directory cleanup after each job. The default value is `false`.++### runner.max_run_time++This can be used to override the default maximum time the Task Agent will run each job.++The default value is 5 hours.++### runner.disable_auto_update
as discussed on slack, lets undocument this for now and merge the PR - then continue the conversation/decision afterwards
comment created time in 2 days
issue openedglenjamin/fourten
Consider _not_ using a Request prototype in the client - just use the fields we care about
created time in 3 days
pull request commentcircleci/circleci-docs
Added an instruction to configure machine/remote docker to use the proxy
Using the cloud-init script rather than the after-boot-script could cause issues in future, but it's fairly unlikely we'd need to change that in 2.x
comment created time in 10 days
push eventglenjamin/fourten
commit sha 2af4b8f712ad07118dcefd6c082cfebdf94246f1
Sketch for how custom retries will work
commit sha 80d51e3d2d852f76806dcf094fe96d07bbd0fd0d
More of the retry spec
commit sha 8fc0059677807be242fb83bac2e88716728b7176
Add License file
push time in 13 days
Pull request review commentCircleCI-Public/circleci-cli
Remove resource_class from generated config
import ( "github.com/CircleCI-Public/circleci-cli/api/runner" ) -type AgentConfig struct {- API APIConfig `yaml:"api"`- Runner RunnerConfig `yaml:"runner"`-}--func NewAgentConfig(t runner.Token) *AgentConfig {- return &AgentConfig{- API: APIConfig{+func generateConfig(t runner.Token, w io.Writer) (err error) {+ return yaml.NewEncoder(w).Encode(&agentConfig{+ API: apiConfig{ AuthToken: t.Token, },- Runner: RunnerConfig{- Name: t.Nickname,- ResourceClass: t.ResourceClass,- CommandPrefix: []string{"/opt/circleci/launch-task"},- WorkingDirectory: "/opt/circleci/workdir/%s",- CleanupWorkingDirectory: true,+ Runner: runnerConfig{+ Name: t.Nickname,
I think the token nickname should be treated as distinct from the runner name - as we expect tokens to be shared across machines but require runner names to be unique
comment created time in 14 days
push eventCircleCI-Public/runner-preview-docs
commit sha 8fc301762765866c53ffcbf358aeab3212656d2f
Supported platforms and FAQ updates
commit sha bd94662e4a2796c58a186d745544d70e23a26cd8
Fix a typo and clarify working directory a bit more
commit sha a8af36684627dc9c9321c9ed15412b208cc387b2
Merge pull request #13 from CircleCI-Public/glenjamin-patch-1
push time in 15 days
delete branch CircleCI-Public/runner-preview-docs
delete branch : glenjamin-patch-1
delete time in 15 days
PR merged CircleCI-Public/runner-preview-docs
- Mentioned macOS on the main page
- Updated FAQs based on some recent questions/answers with preview customers
I've so far left out any mentions of running the agent inside Docker, until we have a clearer story about how to do that.
pr closed time in 15 days
push eventCircleCI-Public/runner-preview-docs
commit sha bd94662e4a2796c58a186d745544d70e23a26cd8
Fix a typo and clarify working directory a bit more
push time in 15 days
PR opened CircleCI-Public/runner-preview-docs
- Mentioned macOS on the main page
- Updated FAQs based on some recent questions/answers with preview customers
I've so far left out any mentions of running the agent inside Docker, until we have a clearer story about how to do that.
pr created time in 15 days
create barnchCircleCI-Public/runner-preview-docs
created branch time in 15 days
pull request commentCircleCI-Public/circleci-cli
Generate platform-specific example runner config
I'm still not keen on having to maintain synchronised copies of recommended config in the CLI
comment created time in 16 days
pull request commentCircleCI-Public/circleci-cli
Generate platform-specific example runner config
Now that there's only one line for auth, could we limit this to just token: $token as output, and always include it after the table output?
comment created time in 16 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
+# Installing the CircleCI Runner on macOS++## Create a runner configuration+Choose a user to run the CircleCI agent as. These instructions will refer to the selected user as `USERNAME`.++Complete the below template, with the various capitalised parameters filled in, and save as `launch-agent-config.yaml`.+```yaml+api:+ auth_token: AUTH_TOKEN+runner:+ name: NAME
we could add validation to disallow NAME as a value 😁
comment created time in 16 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
+# Installing the CircleCI Runner on macOS++## Create a runner configuration+Choose a user to run the CircleCI agent as. These instructions will refer to the selected user as `USERNAME`.++Complete the below template, with the various capitalised parameters filled in, and save as `launch-agent-config.yaml`.+```yaml+api:+ auth_token: AUTH_TOKEN+runner:+ name: NAME+ command_prefix : ["sudo", "-niHu", "USERNAME", "--"]+ working_directory: /tmp/%s+ cleanup_working_directory: true+logging:+ file: /Library/Logs/com.circleci.runner.log+```++## Install the runner configuration+Create a directory as root to hold the runner configuration: +```bash+sudo mkdir -p '/Library/Preferences/com.circleci.runner'+```++Copy the previously created `launch-agent-config.yaml` into the directory:+```bash+sudo cp 'launch-agent-config.yaml' '/Library/Preferences/com.circleci.runner/launch-agent-config.yaml'+```++## Install the launchd .plist++Copy the following to `/Library/LaunchDaemons/com.circleci.runner.plist`, owned by root, with permissions `644`:+```xml+<?xml version="1.0" encoding="UTF-8"?>+<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">+<plist version="1.0">+ <dict>+ <key>Label</key>+ <string>com.circleci.runner</string>++ <key>Program</key>+ <string>/opt/circleci/circleci-launch-agent</string>++ <key>ProgramArguments</key>+ <array>+ <string>circleci-launch-agent</string>+ <string>--config</string>+ <string>/Library/Preferences/com.circleci.runner/launch-agent-config.yaml</string>+ </array>++ <key>RunAtLoad</key>+ <true/>++ <key>KeepAlive</key>+ <true/>++ <key>ProcessType</key>+ <string>Interactive</string>++ <key>ThrottleInterval</key>+ <integer>30</integer>++ <key>StandardOutPath</key>+ <string>/dev/null</string>++ <key>StandardErrorPath</key>+ <string>/dev/null</string>+ </dict>+</plist>+```++## Enable the launchd service+If you are following these instructions for a second time, you should unload the existing service:+```bash+sudo launchctl unload '/Library/LaunchDaemons/com.circleci.runner.plist'
I normally see the -w flag in these commands, but I've just read the help text and I can't make sense of it - do you know what it's for?
comment created time in 16 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
+# Installing the CircleCI Runner on Linux++## Prerequisites++### Installation Tooling++The installation process assumes that the following utilities are installed on the system:++* curl+* sha256sum (installed as part of `coreutils`)+* systemd++Installation also requires permissions to create a user, and create directories under `/opt`.
These prereqs are required for the download too I think, so putting them here is too late?
comment created time in 16 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
mkdir -p "$platform" echo "Downloading CircleCI Launch Agent: $file" curl --compressed -L "$base_url/$agent_version/$file" -o "$file" echo "Verifying CircleCI Launch Agent download"-sha256sum --check --ignore-missing checksums.txt && chmod +x "$file"; sudo mv "$file" "$prefix/circleci-launch-agent" || echo "Invalid checksum for CircleCI Launch Agent, please try download again"+sha256sum --check --ignore-missing checksums.txt && chmod +x "$file"; sudo cp "$file" "$prefix/circleci-launch-agent" || echo "Invalid checksum for CircleCI Launch Agent, please try download again"
oh right, i missed that subtlety - I originally went with mv so it would delete download file
comment created time in 17 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
This will use `/opt/circleci` as the base install location. First, set one of these variables as appropriate for for your installation target. +#### For Linux AMD64 ```bash-# For Linux x86_64 platform=linux/amd64-# For Linux ARM+prefix=/opt/circleci+```++#### For Linux ARM+```bash platform=linux/arm64+prefix=/opt/circleci+```++#### For macOS AMD64+```bash+platform=darwin/amd64+prefix=/usr/local/bin
IME it's common for brew to install in /usr/local, and sometimes for other tools to install to /usr/local/<toolname>.
The ChefDK used to install to /opt/chef IIRC.
I'd say /opt is about as common as it is on Linux?
comment created time in 17 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
api: auth_token: AUTH_TOKEN runner:
I think this example should be removed or made more minimal, and the linux-specific stuff moved into the linux install guide (as the mac default config is in the mac guide)
comment created time in 17 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
If the directory does not exist, then the Task Agent will need permissions to cr If `%s` is present in the value, it will be replaced with a different value for each job. These directories will *not* be automatically removed. -#### runner.cleanup_working_directory--This allows for control of working directory cleanup after each job.--### Create the circleci user & working directory--These will be used when executing `task-agent`.--```bash-id -u circleci &>/dev/null || adduser --uid 1500 --disabled-password --gecos GECOS circleci--mkdir -p /opt/circleci/workdir-chown -R circleci /opt/circleci/workdir-```--### Create the Launch Script--This wrapper script will be used by Launch Agent to execute the Task Agent, while ensuring appropriate sandboxing and a clean shutdown.--Create `/opt/circleci/launch-task` owned by `root` with permissions `755`--```bash-#!/bin/bash--set -euo pipefail--## This script launches the build-agent using systemd-run in order to create a-## cgroup which will capture all child processes so they're cleaned up correctly-## on exit.--# The user to run the build-agent as - must be numeric-USER_ID=$(id -u circleci)--# Give the transient systemd unit an inteligible name-unit="circleci-$CIRCLECI_LAUNCH_ID"--# When this process exits, tell the systemd unit to shut down-abort() {- systemctl stop "$unit"-}-trap abort EXIT--systemd-run \- --pipe --collect --quiet --wait \- --uid "$USER_ID" --unit "$unit" -- "$@"-```--### Create the Stop Script--This script will be used by systemd to perform an orderly shutdown of the Launch agent. It will first request that launch agent stops accepting new tasks by sending a `SIGINT` signal, and then it will follow up with a `SIGTERM` to abort the current task if it is still going for too longer.+### runner.cleanup_working_directory -The wait times in the environment variables should be used to tune how long you wish to wait for shutdown - the `DRAIN_TIMEOUT` should be set slightly longer than your jobs normally take if you want to avoid aborting any jobs early.--Create `/opt/circleci/stop-agent` owned by `root` with permissions `755`--```bash-#!/bin/bash--set -uo pipefail--## This script performs an orderly shutdown of the agents--# How long to wait for draining to complete-DRAIN_TIMEOUT=5m--# How long to wait for cancellation to complete-CANCEL_TIMEOUT=1m--# First send a SIGINT, this tells the launch-agent to stop accepting new tasks-kill -s SIGINT $MAINPID-timeout $DRAIN_TIMEOUT tail --pid=$MAINPID -f /dev/null--# If the process is still running, then SIGTERM to cancel the running task-if [ $? -eq 124 ]; then- kill -SIGTERM $MAINPID- timeout $CANCEL_TIMEOUT tail --pid=$MAINPID -f /dev/null-fi--# If the process is _still_ running at this point, we'll leave systemd to-# perform a SIGKILL and forcibly shut down the process-```--### Starting Launch Agent with Systemd--Create `/opt/circleci/circleci.service` owned by `root` with permissions `755`.--You must ensure that `TimeoutStopSec` is greater than the total amount of time the `stop-agent` script will take.--If you want to configure the CircleCI runner installation to start on boot, it is imporant to note that the Launch Agent will attempt to consume and start jobs as soon as it starts, so it should be configured appropriately before starting. The Launch Agent may be configured as a service and be managed by systemd with the following scripts:--```-[Unit]-Description=CircleCI Runner-After=network.target-[Service]-ExecStart=/opt/circleci/circleci-launch-agent --config /opt/circleci/launch-agent-config.yaml-Restart=always-User=root-NotifyAccess=exec-TimeoutStopSec=600-ExecStop=/opt/circleci/stop-agent-[Install]-WantedBy = multi-user.target-```--You can now enable and start the service--```bash-prefix=/opt/circleci-systemctl enable $prefix/circleci.service-```--#### Start the CircleCI Runner Service--When the CircleCI runner Service starts, it will immediately attempt to start running jobs, so it should be fully configured before the first start of the service.--```bash-systemctl start circleci.service-```--### Verify the Service is Running--The system reports a very basic health status through the `Status` field in `systemctl`.-This will report **Healthy** or **Unhealthy** based on connectivity to the CircleCI APIs.--You can see the status of the agent by running:--```bash-systemctl status circleci.service --no-pager-```--Which should produce output similar to:--```-● circleci.service - CircleCI Runner- Loaded: loaded (/opt/circleci/circleci.service; enabled; vendor preset: enabled)- Active: active (running) since Fri 2020-05-29 14:33:31 UTC; 18min ago- Main PID: 5592 (circleci-launch)- Status: "Healthy"- Tasks: 8 (limit: 2287)- CGroup: /system.slice/circleci.service- └─5592 /opt/circleci/circleci-launch-agent --config /opt/circleci/launch-agent-config.yaml-```--You can also see the logs for the system by running:--```bash-journalctl -u circleci-```+This allows for control of working directory cleanup after each job.
We should note that if omitted, the default is false.
comment created time in 17 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
mkdir -p "$platform" echo "Downloading CircleCI Launch Agent: $file" curl --compressed -L "$base_url/$agent_version/$file" -o "$file" echo "Verifying CircleCI Launch Agent download"-sha256sum --check --ignore-missing checksums.txt && chmod +x "$file"; sudo mv "$file" "$prefix/circleci-launch-agent" || echo "Invalid checksum for CircleCI Launch Agent, please try download again"+sha256sum --check --ignore-missing checksums.txt && chmod +x "$file"; sudo cp "$file" "$prefix/circleci-launch-agent" || echo "Invalid checksum for CircleCI Launch Agent, please try download again"
why cp over mv?
comment created time in 17 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
This will use `/opt/circleci` as the base install location. First, set one of these variables as appropriate for for your installation target. +#### For Linux AMD64
AMD64 caused a bit of confusion with one customer, as x86_64 is the more common term (I think?) could we include both in this title?
comment created time in 17 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
This will use `/opt/circleci` as the base install location. First, set one of these variables as appropriate for for your installation target. +#### For Linux AMD64 ```bash-# For Linux x86_64 platform=linux/amd64-# For Linux ARM+prefix=/opt/circleci+```++#### For Linux ARM+```bash platform=linux/arm64+prefix=/opt/circleci+```++#### For macOS AMD64+```bash+platform=darwin/amd64+prefix=/usr/local/bin
any reason to prefer /usr/local over /opt/circleci on Mac?
I generally like opt for this sort of thing as you know it's only your own stuff
comment created time in 17 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
The steps to follow are: * It is also possible to generate a runner configuration with the above command, using the `--config` switch. * e.g. `circleci runner token create --config my-namespace/my-resource-class my-token > launch-agent-config.yaml`. -#### Installation Tooling--The installation process assumes that the following utilities are installed on the system:--* curl-* sha256sum (installed as part of `coreutils`)-* systemd--Installation also requires permissions to create a user, and create directories under `/opt`.--#### Job Running Requirements+### Job Running Requirements Running jobs requires that the following tools are available on the machine:
oh, we should also recommend installing git perhaps? our go version works, but the real one is better
comment created time in 17 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
This will use `/opt/circleci` as the base install location. First, set one of these variables as appropriate for for your installation target. +#### For Linux AMD64 ```bash-# For Linux x86_64 platform=linux/amd64-# For Linux ARM+prefix=/opt/circleci+```++#### For Linux ARM
Someone asked about ARMv7 32-bit support, so lets be explicit
#### For Linux ARM64
comment created time in 17 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
+# Installing the CircleCI Runner on macOS++## Create a runner configuration+Choose a user to run the CircleCI agent as. These instructions will refer to the selected user as `USERNAME`.++Complete the below template, with the various capitalised parameters filled in, and save as `launch-agent-config.yaml`.+```yaml+api:+ auth_token: AUTH_TOKEN+runner:+ name: NAME+ command_prefix : ["sudo", "-niHu", "USERNAME", "--"]+ working_directory: /tmp/%s+ cleanup_working_directory: true+logging:+ file: /Library/Logs/com.circleci.runner.log+```++## Install the runner configuration+Create a directory as root to hold the runner configuration: +```bash+sudo mkdir -p '/Library/Preferences/com.circleci.runner'+```++Copy the previously created `launch-agent-config.yaml` into the directory:+```bash+sudo cp 'launch-agent-config.yaml' '/Library/Preferences/com.circleci.runner/launch-agent-config.yaml'+```++## Install the launchd .plist++Copy the following to `/Library/LaunchDaemons/com.circleci.runner.plist`, owned by root, with permissions `644`:+```xml+<?xml version="1.0" encoding="UTF-8"?>+<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">+<plist version="1.0">+ <dict>+ <key>Label</key>+ <string>com.circleci.runner</string>++ <key>Program</key>+ <string>/usr/local/bin/circleci-launch-agent</string>++ <key>ProgramArguments</key>+ <array>+ <string>circleci-launch-agent</string>+ <string>--config</string>+ <string>/Library/Preferences/com.circleci.runner/launch-agent-config.yaml</string>+ </array>++ <key>RunAtLoad</key>+ <true/>++ <key>KeepAlive</key>+ <dict>+ <key>SuccessfulExit</key>+ <false/>+ </dict>++ <key>ProcessType</key>+ <string>Interactive</string>++ <key>ThrottleInterval</key>+ <integer>30</integer>++ <key>StandardOutPath</key>+ <string>/dev/null</string>++ <key>StandardErrorPath</key>+ <string>/dev/null</string>
if we omit these, what's the default?
comment created time in 17 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
+# Installing the CircleCI Runner on macOS++## Create a runner configuration+Choose a user to run the CircleCI agent as. These instructions will refer to the selected user as `USERNAME`.++Complete the below template, with the various capitalised parameters filled in, and save as `launch-agent-config.yaml`.+```yaml+api:+ auth_token: AUTH_TOKEN+runner:+ name: NAME
is there an invalid YAML value we can use here to ensure the customer doesn't just copy NAME itact?
comment created time in 17 days
Pull request review commentCircleCI-Public/circleci-cli
Generate platform-specific example runner config
import ( ) type AgentConfig struct {- API APIConfig `yaml:"api"`- Runner RunnerConfig `yaml:"runner"`+ API APIConfig `yaml:"api"`+ Runner RunnerConfig `yaml:"runner"`+ Logging LoggingConfig `yaml:"logging,omitempty"` } -func NewAgentConfig(t runner.Token) *AgentConfig {- return &AgentConfig{+func NewAgentConfig(t runner.Token, platform string) (c *AgentConfig, err error) {
in particular, I wonder if we could rejig the config format so that api token and resource class are next to each other, and then have this script only spit out those 2 lines - leaving the docs (or a future installer) as the source-of-truth for the rest
comment created time in 18 days
Pull request review commentCircleCI-Public/circleci-cli
Generate platform-specific example runner config
import ( ) type AgentConfig struct {- API APIConfig `yaml:"api"`- Runner RunnerConfig `yaml:"runner"`+ API APIConfig `yaml:"api"`+ Runner RunnerConfig `yaml:"runner"`+ Logging LoggingConfig `yaml:"logging,omitempty"` } -func NewAgentConfig(t runner.Token) *AgentConfig {- return &AgentConfig{+func NewAgentConfig(t runner.Token, platform string) (c *AgentConfig, err error) {
Hrm, I'm starting to wonder if this is worth it - it's effectively another API to keep in sync
comment created time in 18 days
Pull request review commentCircleCI-Public/circleci-cli
Generate platform-specific example runner config
import ( ) type AgentConfig struct {- API APIConfig `yaml:"api"`- Runner RunnerConfig `yaml:"runner"`+ API APIConfig `yaml:"api"`+ Runner RunnerConfig `yaml:"runner"`+ Logging LoggingConfig `yaml:"logging,omitempty"` } -func NewAgentConfig(t runner.Token) *AgentConfig {- return &AgentConfig{+func NewAgentConfig(t runner.Token, platform string) (c *AgentConfig, err error) {+ c = &AgentConfig{ API: APIConfig{ AuthToken: t.Token, }, Runner: RunnerConfig{ Name: t.Nickname, ResourceClass: t.ResourceClass,- CommandPrefix: []string{"/opt/circleci/launch-task"},- WorkingDirectory: "/opt/circleci/workdir/%s", CleanupWorkingDirectory: true, }, }++ switch platform {+ default:+ return nil, fmt.Errorf("unknown platform: %q", platform)++ case "linux":+ c.Runner.CommandPrefix = []string{"/opt/circleci/launch-task"}+ c.Runner.WorkingDirectory = "/opt/circleci/workdir/%s"++ case "macos":+ c.Runner.CommandPrefix = []string{"sudo", "-niHu", "USERNAME", "--"}
seeing this here next to the linux one highlights the difference - how the linux config assumes additional steps have been taken
comment created time in 18 days
Pull request review commentCircleCI-Public/circleci-cli
Generate platform-specific example runner config
import ( ) type AgentConfig struct {- API APIConfig `yaml:"api"`- Runner RunnerConfig `yaml:"runner"`+ API APIConfig `yaml:"api"`+ Runner RunnerConfig `yaml:"runner"`+ Logging LoggingConfig `yaml:"logging,omitempty"` } -func NewAgentConfig(t runner.Token) *AgentConfig {- return &AgentConfig{+func NewAgentConfig(t runner.Token, platform string) (c *AgentConfig, err error) {+ c = &AgentConfig{ API: APIConfig{ AuthToken: t.Token, }, Runner: RunnerConfig{ Name: t.Nickname, ResourceClass: t.ResourceClass,- CommandPrefix: []string{"/opt/circleci/launch-task"},- WorkingDirectory: "/opt/circleci/workdir/%s", CleanupWorkingDirectory: true, }, }++ switch platform {+ default:+ return nil, fmt.Errorf("unknown platform: %q", platform)++ case "linux":+ c.Runner.CommandPrefix = []string{"/opt/circleci/launch-task"}
This script isn't universal, and isn't included in the CLI, so perhaps it shouldn't be included in the CLI output?
comment created time in 18 days
Pull request review commentCircleCI-Public/circleci-cli
Generate platform-specific example runner config
import ( ) type AgentConfig struct {- API APIConfig `yaml:"api"`- Runner RunnerConfig `yaml:"runner"`+ API APIConfig `yaml:"api"`+ Runner RunnerConfig `yaml:"runner"`+ Logging LoggingConfig `yaml:"logging,omitempty"` } -func NewAgentConfig(t runner.Token) *AgentConfig {- return &AgentConfig{+func NewAgentConfig(t runner.Token, platform string) (c *AgentConfig, err error) {+ c = &AgentConfig{ API: APIConfig{ AuthToken: t.Token, }, Runner: RunnerConfig{ Name: t.Nickname, ResourceClass: t.ResourceClass,- CommandPrefix: []string{"/opt/circleci/launch-task"},- WorkingDirectory: "/opt/circleci/workdir/%s", CleanupWorkingDirectory: true, }, }++ switch platform {+ default:+ return nil, fmt.Errorf("unknown platform: %q", platform)++ case "linux":+ c.Runner.CommandPrefix = []string{"/opt/circleci/launch-task"}+ c.Runner.WorkingDirectory = "/opt/circleci/workdir/%s"++ case "macos":+ c.Runner.CommandPrefix = []string{"sudo", "-niHu", "USERNAME", "--"}+ c.Runner.WorkingDirectory = "/tmp/%s"
why do we use /tmp here, but /opt/circleci on linux?
comment created time in 18 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
+#!/bin/bash++set -euo pipefail++readonly target_user="$1"+readonly config="$2"+readonly bucket="https://circleci-binary-releases.s3.amazonaws.com/circleci-launch-agent"++prepare() {+ echo "Checking needed commands are installed:"+ command -v curl+ command -v grep+ command -v tee+ command -v tar+ command -v gzip+ command -v sha256sum+ echo++ echo "Creating temp dir:"+ # declare tmp_dir+ tmp_dir="$(mktemp -d -t runner-install)"+ # shellcheck disable=SC2064+ trap "rm -rf '${tmp_dir}'" EXIT+ echo "$tmp_dir"+ echo+}++prepare_user() {+ echo "Setting up user '${1}':"++ local workdir="/Users/${1}/workdir"+ mkdir -p "${workdir}"+ chmod 0750 "${workdir}"+ chown "${1}:" "${workdir}"+}++download_agent() {
how mac-specific is this bit?
I'm keen to keep our docs fairly agnostic, even if this makes things slightly more work for installers - as right now the set of customers we've had have all been pretty comfortable scripting up server configuration management
The reason the linux script is more manual than it needs to be is because I wanted people to walk through it and mostly understand what was going on and/or add it into their own provisioning scripts at a file-level
comment created time in 21 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
-## Installing the CircleCI Runner+# Installing the CircleCI Runner -Currently only Ubuntu 18.04 running on an AMD64 or ARM64 architecture is officially supported. Running in a container is not supported at this time.+## Supported platforms -### Prerequisites+At present, the supported platforms are:+* Ubuntu 18.04 running on an AMD64 or ARM64 architecture+* macOS Catalina (10.15.6+) or above, running on and AMD64 architecture.
* macOS Catalina (10.15.6+) or above, running on an AMD64 architecture.
comment created time in 21 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
+# Installing the CircleCI Runner on Linux++## Prerequisites++### Installation Tooling++The installation process assumes that the following utilities are installed on the system:++* curl+* sha256sum (installed as part of `coreutils`)+* systemd++Installation also requires permissions to create a user, and create directories under `/opt`.++### Job Running Requirements++Running jobs requires that the following tools are available on the machine:++* tar+* gzip+* coreutils++## Download the Launch Agent Binary and Verify the Checksum++The Launch Agent can be installed using the following script in a bash shell.++This will use `/opt/circleci` as the base install location.++First, set one of these variables as appropriate for for your installation target.++```bash+# For Linux x86_64+platform=linux/amd64+# For Linux ARM+platform=linux/arm64+```++And then run the following steps to download, verify and install the binary.++```bash+prefix=/opt/circleci+sudo mkdir -p "$prefix/workdir"+base_url="https://circleci-binary-releases.s3.amazonaws.com/circleci-launch-agent"+echo "Determining latest version of CircleCI Launch Agent"+agent_version=$(curl "$base_url/release.txt")+echo "Using CircleCI Launch Agent version $agent_version"+echo "Downloading and verifying CircleCI Launch Agent Binary"+curl -sSL "$base_url/$agent_version/checksums.txt" -o checksums.txt+IFS=" " read -r -a selected <<< "$(grep -F "$platform" checksums.txt)"+checksum=${selected[0]}+file=${selected[1]:1}+mkdir -p "$platform"+echo "Downloading CircleCI Launch Agent: $file"+curl --compressed -L "$base_url/$agent_version/$file" -o "$file"+echo "Verifying CircleCI Launch Agent download"+sha256sum --check --ignore-missing checksums.txt && chmod +x "$file"; sudo mv "$file" "$prefix/circleci-launch-agent" || echo "Invalid checksum for CircleCI Launch Agent, please try download again"+```++## Configure Launch Agent
I think the config reference should be in the shared doc.
comment created time in 21 days
Pull request review commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
-## Installing the CircleCI Runner+# Installing the CircleCI Runner -Currently only Ubuntu 18.04 running on an AMD64 or ARM64 architecture is officially supported. Running in a container is not supported at this time.+## Supported platforms -### Prerequisites+At present, the supported platforms are:+* Ubuntu 18.04 running on an AMD64 or ARM64 architecture+* macOS Catalina (10.15.6+) or above, running on and AMD64 architecture. -#### Authentication+Running in a container is not supported at this time.
@appplemac in a recent GTM meeting there was some discussion about tiers of support - is that something we can start adding into the docs?
I'd like to put Docker in a "try it if you want" category
comment created time in 21 days
delete branch CircleCI-Public/circleci-cli
delete branch : runner-error-messages
delete time in 23 days
push eventCircleCI-Public/circleci-cli
commit sha 2ef38aacb5519d2fb04714a065f659039f4a74ca
Remove HTTP statuscode info from runner error messages
commit sha 026f9bff1663785f353ebff1abed3412169856b6
Merge pull request #485 from CircleCI-Public/runner-error-messages CIRCLE-29921 Remove HTTP statuscode info from runner error messages
push time in 23 days
PR merged CircleCI-Public/circleci-cli
pr closed time in 23 days
push eventCircleCI-Public/circleci-cli
commit sha 2ef38aacb5519d2fb04714a065f659039f4a74ca
Remove HTTP statuscode info from runner error messages
push time in 23 days
PR opened CircleCI-Public/circleci-cli
pr created time in 23 days
create barnchCircleCI-Public/circleci-cli
branch : runner-error-messages
created branch time in 23 days
push eventCircleCI-Public/runner-preview-docs
commit sha a1638b51baf73716227fd53d0f26da727bf6e059
Reduce repetition in the install script
commit sha 2c8debfe5de8755bd8b12abae525087e4f3040c8
Merge pull request #9 from CircleCI-Public/simpler-arch
push time in 24 days
delete branch CircleCI-Public/runner-preview-docs
delete branch : simpler-arch
delete time in 24 days
PR merged CircleCI-Public/runner-preview-docs
pr closed time in 24 days
pull request commentCircleCI-Public/runner-preview-docs
Add installation document for macOS
I think ideally this should be part of the main doc, with platform-specific sections marked out
comment created time in 24 days
pull request commentCircleCI-Public/browser-tools-orb
Cache browsers and driver packages for faster setup [semver:minor]
I like the idea of having a tarball on S3 we can share between all of our docker images.
I'm not sure how well that would work in practice though
comment created time in 24 days
pull request commentCircleCI-Public/circleci-cli
Path escape runner IDs and send proper user agent
Is this what was causing the 500?
comment created time in 25 days
Pull request review commentCircleCI-Public/circleci-cli
[CIRCLE-29501] Add commands to manage runner types
var _ = Describe("Root", func() { Describe("subcommands", func() { It("can create commands", func() { commands := cmd.MakeCommands()- Expect(len(commands.Commands())).To(Equal(17))+ Expect(len(commands.Commands())).To(Equal(18))
I was thinking just one that lists all of the commands - but I had forgotten that there isn't a help command which lists all commands, and these are hidden anyway!
comment created time in a month
Pull request review commentCircleCI-Public/circleci-cli
[CIRCLE-29501] Add commands to manage runner types
+package runner++import (+ "fmt"+ "io"+ "net/url"+ "time"++ "gopkg.in/yaml.v3"++ "github.com/CircleCI-Public/circleci-cli/api/rest"+)++type Runner struct {+ rc *rest.Client+}++func New(rc *rest.Client) *Runner {+ return &Runner{rc: rc}+}++type ResourceClass struct {+ ID string `json:"id"`+ ResourceClass string `json:"resource_class"`+ Description string `json:"description"`+}++func (r *Runner) CreateResourceClass(resourceClass, desc string) (rc *ResourceClass, err error) {+ req, err := r.rc.NewRequest("POST", &url.URL{Path: "runner/resource"}, struct {+ ResourceClass string `json:"resource_class"`+ Description string `json:"description"`+ }{+ ResourceClass: resourceClass,+ Description: desc,+ })+ if err != nil {+ return nil, err+ }++ rc = &ResourceClass{}+ _, err = r.rc.DoRequest(req, rc)+ return rc, err+}++func (r *Runner) GetResourceClassesByNamespace(namespace string) ([]ResourceClass, error) {+ query := url.Values{}+ query.Add("namespace", namespace)+ req, err := r.rc.NewRequest("GET", &url.URL{Path: "runner/resource", RawQuery: query.Encode()}, nil)+ if err != nil {+ return nil, err+ }++ resp := struct {+ Items []ResourceClass `json:"items"`+ }{}+ _, err = r.rc.DoRequest(req, &resp)+ return resp.Items, err+}++func (r *Runner) DeleteResourceClass(id string) error {+ req, err := r.rc.NewRequest("DELETE", &url.URL{Path: fmt.Sprintf("runner/resource/%s", id)}, nil)+ if err != nil {+ return err+ }++ _, err = r.rc.DoRequest(req, nil)+ return err+}++type Token struct {+ ID string `json:"id"`+ Token string `json:"token"`+ ResourceClass string `json:"resource_class"`+ Nickname string `json:"nickname"`+ CreatedAt time.Time `json:"created_at"`+}++func (t *Token) ToAgentConfig() *AgentConfig {+ return &AgentConfig{+ API: APIConfig{+ AuthToken: t.Token,+ },+ Runner: RunnerConfig{+ Name: t.Nickname,+ ResourceClass: t.ResourceClass,+ },+ }+}++func (r *Runner) CreateToken(resourceClass, nickname string) (token *Token, err error) {+ t := struct {+ ResourceClass string `json:"resource_class"`+ Nickname string `json:"nickname"`+ }{+ ResourceClass: resourceClass,+ Nickname: nickname,+ }++ req, err := r.rc.NewRequest("POST", &url.URL{Path: "runner/token"}, t)+ if err != nil {+ return nil, err+ }++ token = &Token{}+ _, err = r.rc.DoRequest(req, token)+ return token, err+}++func (r *Runner) GetRunnerTokensByResourceClass(resourceClass string) ([]Token, error) {+ query := url.Values{}+ query.Add("resource-class", resourceClass)+ req, err := r.rc.NewRequest("GET", &url.URL{Path: "runner/token", RawQuery: query.Encode()}, nil)+ if err != nil {+ return nil, err+ }++ resp := struct {+ Items []Token `json:"items"`+ }{}+ _, err = r.rc.DoRequest(req, &resp)+ return resp.Items, err+}++func (r *Runner) DeleteToken(id string) error {+ req, err := r.rc.NewRequest("DELETE", &url.URL{Path: fmt.Sprintf("runner/token/%s", id)}, nil)
id should be url-escaped
comment created time in a month
Pull request review commentCircleCI-Public/circleci-cli
[CIRCLE-29501] Add commands to manage runner types
var _ = Describe("Root", func() { Describe("subcommands", func() { It("can create commands", func() { commands := cmd.MakeCommands()- Expect(len(commands.Commands())).To(Equal(17))+ Expect(len(commands.Commands())).To(Equal(18))
maybe replace this with a golden test on --help?
comment created time in a month
Pull request review commentCircleCI-Public/circleci-cli
[CIRCLE-29501] Add commands to manage runner types
+package runner++import (+ "os"+ "time"++ "github.com/olekukonko/tablewriter"+ "github.com/spf13/cobra"++ "github.com/CircleCI-Public/circleci-cli/api/runner"+)++func newTokenCommand(r *runner.Runner, preRunE validator) *cobra.Command {+ cmd := &cobra.Command{+ Use: "token",+ Short: "Operate on runner tokens",+ }++ createOpts := struct {+ config bool+ }{}+ createCmd := &cobra.Command{+ Use: "create <resource-class> <nickname>",+ Short: "Create a token for a resource-class",+ Args: cobra.ExactArgs(2),+ PreRunE: preRunE,+ RunE: func(_ *cobra.Command, args []string) error {+ token, err := r.CreateToken(args[0], args[1])+ if err != nil {+ return err+ }++ if createOpts.config {
This is a great idea!
Maybe it should be on by default?
comment created time in a month
Pull request review commentCircleCI-Public/circleci-cli
[CIRCLE-29501] Add commands to manage runner types
+package runner++import (+ "os"++ "github.com/olekukonko/tablewriter"+ "github.com/spf13/cobra"++ "github.com/CircleCI-Public/circleci-cli/api/runner"+)++func newResourceClassCommand(r *runner.Runner, preRunE validator) *cobra.Command {+ cmd := &cobra.Command{+ Use: "resource-class",+ Short: "Operate on runner resource-classes",+ }++ cmd.AddCommand(&cobra.Command{+ Use: "create <resource-class> <description>",+ Short: "Create a resource-class",+ Args: cobra.ExactArgs(2),+ PreRunE: preRunE,+ RunE: func(_ *cobra.Command, args []string) error {+ rc, err := r.CreateResourceClass(args[0], args[1])+ if err != nil {+ return err+ }+ table := tablewriter.NewWriter(os.Stdout)+ table.SetHeader([]string{"ID", "Resource Class", "Description"})+ table.Append([]string{rc.ID, rc.ResourceClass, rc.Description})+ table.Render()+ return nil+ },+ })++ cmd.AddCommand(&cobra.Command{+ Use: "delete <resource-class>",+ Short: "Delete a resource-class",+ Aliases: []string{"rm"},+ Args: cobra.ExactArgs(1),+ PreRunE: preRunE,+ RunE: func(_ *cobra.Command, args []string) error {+ return r.DeleteResourceClass(args[0])+ },+ })++ cmd.AddCommand(&cobra.Command{+ Use: "list <namespace>",+ Short: "List resource-classes for a namespace",+ Aliases: []string{"ls"},+ Args: cobra.ExactArgs(1),+ PreRunE: preRunE,+ RunE: func(_ *cobra.Command, args []string) error {+ rcs, err := r.GetResourceClassesByNamespace(args[0])+ if err != nil {+ return err+ }++ table := tablewriter.NewWriter(os.Stdout)
~ maybe extract and share the table with create?
comment created time in a month
Pull request review commentCircleCI-Public/circleci-cli
[CIRCLE-29501] Add commands to manage runner types
+package runner++import (+ "os"+ "time"++ "github.com/olekukonko/tablewriter"+ "github.com/spf13/cobra"++ "github.com/CircleCI-Public/circleci-cli/api/runner"+)++func newTokenCommand(r *runner.Runner, preRunE validator) *cobra.Command {+ cmd := &cobra.Command{+ Use: "token",+ Short: "Operate on runner tokens",+ }++ createOpts := struct {+ config bool+ }{}+ createCmd := &cobra.Command{+ Use: "create <resource-class> <nickname>",+ Short: "Create a token for a resource-class",+ Args: cobra.ExactArgs(2),+ PreRunE: preRunE,+ RunE: func(_ *cobra.Command, args []string) error {+ token, err := r.CreateToken(args[0], args[1])+ if err != nil {+ return err+ }++ if createOpts.config {+ return token.ToAgentConfig().WriteYaml(os.Stdout)+ }++ table := tablewriter.NewWriter(os.Stdout)+ table.SetHeader([]string{"ID", "Token", "Nickname", "Created At"})+ table.Append([]string{token.ID, token.Token, token.Nickname, token.CreatedAt.Format(time.RFC3339)})+ table.Render()+ return nil+ },+ }+ createCmd.Flags().BoolVar(&createOpts.config, "config", false, "'true' to emit a CircleCI runner config template with the token")+ cmd.AddCommand(createCmd)++ cmd.AddCommand(&cobra.Command{+ Use: "delete <token-id>",+ Short: "Delete a token",+ Aliases: []string{"rm"},+ Args: cobra.ExactArgs(1),+ PreRunE: preRunE,+ RunE: func(_ *cobra.Command, args []string) error {+ return r.DeleteToken(args[0])+ },+ })++ cmd.AddCommand(&cobra.Command{+ Use: "list <resource-class>",+ Aliases: []string{"ls"},+ Short: "List tokens for a resource-class",+ Args: cobra.ExactArgs(1),+ PreRunE: preRunE,+ RunE: func(_ *cobra.Command, args []string) error {+ tokens, err := r.GetRunnerTokensByResourceClass(args[0])+ if err != nil {+ return err+ }++ table := tablewriter.NewWriter(os.Stdout)
~ same comment as resource class about sharing the table helper
comment created time in a month
Pull request review commentCircleCI-Public/circleci-cli
[CIRCLE-29501] Add commands to manage runner types
+package runner++import (+ "fmt"+ "io"+ "net/url"+ "time"++ "gopkg.in/yaml.v3"++ "github.com/CircleCI-Public/circleci-cli/api/rest"+)++type Runner struct {+ rc *rest.Client+}++func New(rc *rest.Client) *Runner {+ return &Runner{rc: rc}+}++type ResourceClass struct {+ ID string `json:"id"`+ ResourceClass string `json:"resource_class"`+ Description string `json:"description"`+}++func (r *Runner) CreateResourceClass(resourceClass, desc string) (rc *ResourceClass, err error) {+ req, err := r.rc.NewRequest("POST", &url.URL{Path: "runner/resource"}, struct {+ ResourceClass string `json:"resource_class"`+ Description string `json:"description"`+ }{+ ResourceClass: resourceClass,+ Description: desc,+ })+ if err != nil {+ return nil, err+ }++ rc = &ResourceClass{}+ _, err = r.rc.DoRequest(req, rc)+ return rc, err+}++func (r *Runner) GetResourceClassesByNamespace(namespace string) ([]ResourceClass, error) {+ query := url.Values{}+ query.Add("namespace", namespace)+ req, err := r.rc.NewRequest("GET", &url.URL{Path: "runner/resource", RawQuery: query.Encode()}, nil)+ if err != nil {+ return nil, err+ }++ resp := struct {+ Items []ResourceClass `json:"items"`+ }{}+ _, err = r.rc.DoRequest(req, &resp)+ return resp.Items, err+}++func (r *Runner) DeleteResourceClass(id string) error {+ req, err := r.rc.NewRequest("DELETE", &url.URL{Path: fmt.Sprintf("runner/resource/%s", id)}, nil)
id should be URL escaped
comment created time in a month
Pull request review commentCircleCI-Public/circleci-cli
[CIRCLE-29501] Add commands to manage runner types
+package runner++import (+ "fmt"+ "io"+ "net/url"+ "time"++ "gopkg.in/yaml.v3"++ "github.com/CircleCI-Public/circleci-cli/api/rest"+)++type Runner struct {+ rc *rest.Client+}++func New(rc *rest.Client) *Runner {+ return &Runner{rc: rc}+}++type ResourceClass struct {+ ID string `json:"id"`+ ResourceClass string `json:"resource_class"`+ Description string `json:"description"`+}++func (r *Runner) CreateResourceClass(resourceClass, desc string) (rc *ResourceClass, err error) {+ req, err := r.rc.NewRequest("POST", &url.URL{Path: "runner/resource"}, struct {+ ResourceClass string `json:"resource_class"`+ Description string `json:"description"`+ }{+ ResourceClass: resourceClass,+ Description: desc,+ })+ if err != nil {+ return nil, err+ }++ rc = &ResourceClass{}+ _, err = r.rc.DoRequest(req, rc)+ return rc, err+}++func (r *Runner) GetResourceClassesByNamespace(namespace string) ([]ResourceClass, error) {+ query := url.Values{}+ query.Add("namespace", namespace)+ req, err := r.rc.NewRequest("GET", &url.URL{Path: "runner/resource", RawQuery: query.Encode()}, nil)+ if err != nil {+ return nil, err+ }++ resp := struct {+ Items []ResourceClass `json:"items"`+ }{}+ _, err = r.rc.DoRequest(req, &resp)+ return resp.Items, err+}++func (r *Runner) DeleteResourceClass(id string) error {+ req, err := r.rc.NewRequest("DELETE", &url.URL{Path: fmt.Sprintf("runner/resource/%s", id)}, nil)+ if err != nil {+ return err+ }++ _, err = r.rc.DoRequest(req, nil)+ return err+}++type Token struct {+ ID string `json:"id"`+ Token string `json:"token"`+ ResourceClass string `json:"resource_class"`+ Nickname string `json:"nickname"`+ CreatedAt time.Time `json:"created_at"`+}++func (t *Token) ToAgentConfig() *AgentConfig {
I think this should be outside of the API package
comment created time in a month
push eventglenjamin/fourten
commit sha 49f61352b2d309b15c8bd0a38099e63b94bc5aa1
Some parameter attempts are errors
push time in a month
push eventglenjamin/fourten
commit sha 661cdd2ba54959123706d252120d2ed3fb3d569c
Add querystring and url template parameters
push time in a month
push eventglenjamin/fourten
commit sha 9f2cad1e18019b4efe961addff5dbfe8d51357de
Add querystring and url template parameters
push time in a month
push eventglenjamin/fourten
commit sha 6d52d57195ef228e6978ed7af27990e0960a4296
Set a default user agent different to Go's default
push time in a month
PR opened CircleCI-Public/runner-preview-docs
pr created time in a month
create barnchCircleCI-Public/runner-preview-docs
created branch time in a month
issue commentgoreleaser/godownloader
Something i've often wondered about is whether goreleaser could publish a standard-format manifest file, which cannot be configured differently. This could be formatted in a way that would make the install.sh script much simpler to build/maintain
comment created time in a month
pull request commentcircleci/circleci-docs
Add rate limits as motivation to use authenticated docker pulls
I think it would be good to call out multiple contexts more explicitly on this page somehow
comment created time in a month
pull request commentCircleCI-Public/runner-preview-docs
Yeah, failing on missed grep is a good thing - having a conditional which does a custom message is a good shout
comment created time in a month
pull request commentCircleCI-Public/runner-preview-docs
if we use uname in the script, we can avoid repeating the script block.
The grep will fail if it's an unsupported architecture.
comment created time in a month
issue commentterraform-providers/terraform-provider-aws
Randomness added to resource names when using name_prefix can cause names to be too long
Based on the current suggestions on this ticket and the linked ones, it seems like a viable path forwards should be:
- generate a random suffix that can be base64/32 encoded into N characters
- allow name_prefix to be at most $resource-max-length - N
Where N is 6-10 characters.
I would be happy to work on this if the maintainers are broadly in favour if such an approach?
comment created time in 2 months
delete branch CircleCI-Public/runner-preview-docs
delete branch : reasons-up-top
delete time in 2 months