brycahta/amazon-ec2-instance-qualifier 0
A CLI tool that automates benchmarking on a range of EC2 instance types.
brycahta/amazon-ec2-instance-selector 0
A CLI tool and go library which recommends instance types based on resource criteria like vcpus and memory
brycahta/amazon-ec2-metadata-mock 0
A tool to simulate Amazon EC2 instance metadata
brycahta/aws-node-termination-handler 0
A Kubernetes Daemonset to gracefully handle EC2 instance shutdown
brycahta/container-web-scraper-example 0
An example of a web scraper in a Docker container running on AWS
Amazon EKS Helm chart repository
Pull request review commentawslabs/amazon-ec2-instance-qualifier
+{
these changes will be addressed in separate PR
comment created time in 2 days
push eventbrycahta/amazon-ec2-instance-qualifier
commit sha ee3297140c74f2b2e4a348c1feb5fa82ecffb0ce
more robust cw agent installation
push time in 2 days
Pull request review commentawslabs/amazon-ec2-instance-qualifier
+{
discussed offline:
- by offering this flexibility we put onus on users to provide valid/existing metrics
- pass in
metricsas another JSON object in config - users can either provide metrics they want XOR use the default metrics in this file
comment created time in 2 days
Pull request review commentawslabs/amazon-ec2-instance-qualifier
+{+ "agent": {+ "metrics_collection_interval": 30,+ "run_as_user": "root"+ },+ "metrics": {+ "append_dimensions": {+ "AutoScalingGroupName": "${aws:AutoScalingGroupName}",+ "InstanceId": "${aws:InstanceId}",+ "InstanceType": "${aws:InstanceType}"
Agreed, but from the docs:
append_dimensions – Optional. Adds Amazon EC2 metric dimensions to all metrics collected by the agent. The only supported key-value pairs are shown in the following list. Any other key-value pairs are ignored.
"ImageID":"${aws:ImageId}" sets the instance's AMI ID as the value of the ImageID dimension.
"InstanceId":"${aws:InstanceId}" sets the instance's instance ID as the value of the InstanceID dimension.
"InstanceType":"${aws:InstanceType}" sets the instance's instance type as the value of the InstanceType dimension.
"AutoScalingGroupName":"${aws:AutoScalingGroupName}" sets the instance's Auto Scaling group name as the value of the AutoScalingGroupName dimension.
comment created time in 2 days
Pull request review commentawslabs/amazon-ec2-instance-qualifier
+{
Yeah, I think that's a good user experience, but the main concern here is communicating to IQ which metrics to benchmark (i.e. use as target-utilization) which is one reason why I wanted the metrics to be known/valid when IQ starts up.
With that said, it's also possible to create another arg/config in JSON that encapsulates metrics and it could be a string of custom-named metrics that the user knows should be emitted by CWA, but the downside being more opportunity for failure (fat finger a name, forgot to configure it, etc.).
Who knew finding a balance between flexibility, UX, and functionality would be difficult
comment created time in 2 days
Pull request review commentawslabs/amazon-ec2-instance-qualifier
for file in *; do chmod u+x "$file" fi done++wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
good catch! reading docs is hard
comment created time in 2 days
Pull request review commentawslabs/amazon-ec2-instance-qualifier
for file in *; do chmod u+x "$file" fi done++wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
This section is more or less hard-coded; wanted to get this working and get some input/feedback before making it more robust. The following can be improved:
- use regional download link
- the architecture in the download link also supports arm64-- so need to introduce logic to change this value based on $ARCHITECTURE
- no arm testing right now; should add that too
- config file
cwagent-config.jsoncould be dynamic, but I think I'll let IQ enforce the name here
comment created time in 3 days
Pull request review commentawslabs/amazon-ec2-instance-qualifier
+{
So this file should drive all the metrics being uploaded to CW via the agent, so any custom metrics or metrics captured via StatsD or collectd would also be defined here.
Looking for ideas on how to make this dynamic/configurable by the user:
- have users provide their own via another CLI arg
- interactive mode within IQ that'll basically invoke the CW Agent Configuration Wizard
- leave this file as-is (collecting Basic metrics) and introduce a different mechanism for subscribing to other metrics
comment created time in 3 days
PR opened awslabs/amazon-ec2-instance-qualifier
Issue #, if available:
- These changes make emitting metrics via CW Agent possible
Description of changes:
- Wanted to break up changes into multiple PRs, so this first one is basically just enabling the agent, but the real benchmark data/comparisons is still coming from Instance-Qualifier scripts/agent
- Added
cloud-watch-agent-config.jsonconfigured to capture Basic metrics every 30 secs- I'm treating this JSON as another 'script' in the TestSuite because that was the easiest ingress point to get the file onto the instance-- REFACTOR WILL BE SOON^tm
- Added CloudWatch Agent installation to user data
- Added CloudWatchAgentServerPolicy to IAM Role
- Updated tests
Testing:
make clean&&make testlocally
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
pr created time in 3 days
create barnchbrycahta/amazon-ec2-instance-qualifier
created branch time in 3 days
Pull request review commentaws/aws-node-termination-handler
Add AEMM mock interruption documentation
+# AWS Node Termination Handler & Amazon EC2 Metadata Mock++We have open sourced a tool called the [amazon-ec2-metadata-mock](https://github.com/aws/amazon-ec2-metadata-mock) (AEMM)+that simulates spot interruption notices and more by starting a real webserver that serves data similar to EC2 Instance+Metadata Service. The tool is easily deployed to kubernetes with a Helm chart.++Below is a short guide on how to set AEMM up with your Node Termination Handler cluster in case you'd like to verify the+behavior yourself.++## Triggering AWS Node Termination Handler with Amazon EC2 Metadata Mock++Start by installing AEMM on your cluster. For full and up to date installation instructions reference the AEMM repository.+Here's just one way to do it.++Download the latest tar ball from the releases page, at the time of writing this that was v1.6.0. Then install it using+Helm:+```+helm install amazon-ec2-metadata-mock amazon-ec2-metadata-mock-1.6.0.tgz \+ --namespace default+```++Once AEMM is installed, you need to change the instance metadata url of Node Termination Handler to point+to the location AEMM is serving from. If you use the default values of AEMM, the installation will look similar to this:+```+helm upgrade --install aws-node-termination-handler \+ --namespace kube-system \+ --set instanceMetadataURL="http://amazon-ec2-metadata-mock-service.default.svc.cluster.local:1338" \+ eks/aws-node-termination-handler+```++That's it! Instead of polling the real IMDS service endpoint, AWS Node Termination Handler will instead poll AEMM.+If you open the logs of an AWS Node Termination Handler pod you should see that it receives (mock) interruption+events from AEMM and that the nodes are cordoned and drained. Keep in mind that these nodes won't actually get terminated,+so you might need to manually uncordon the nodes if you want to reset your test cluster.++### AEMM Advanced Configuration+If you run the example above you might notice that the logs are heavily populated. Here's an example output:+```+2020/09/15 21:13:41 Sending interruption event to the interruption channel+2020/09/15 21:13:41 Got interruption event from channel {InstanceID:i-1234567890abcdef0 InstanceType:m4.xlarge PublicHostname:ec2-192-0-2-54.compute-1.amazonaws.com PublicIP:192.0.2.54 LocalHostname:ip-172-16-34-43.ec2.internal LocalIP:172.16.34.43 AvailabilityZone:us-east-1a} {EventID:spot-itn-47ddfb5e39791606bec3e91fea4cdfa86f86a60ddaf014c8b4af8e008f134b19 Kind:SPOT_ITN Description:Spot ITN received. Instance will be interrupted at 2020-09-15T21:15:41Z+ State: NodeName:ip-192-168-123-456.us-east-1.compute.internal StartTime:2020-09-15 21:15:41 +0000 UTC EndTime:0001-01-01 00:00:00 +0000 UTC Drained:false PreDrainTask:0x113c8a0 PostDrainTask:<nil>}+WARNING: ignoring DaemonSet-managed Pods: default/amazon-ec2-metadata-mock-pszj2, kube-system/aws-node-bl2bj, kube-system/aws-node-termination-handler-2pvjr, kube-system/kube-proxy-fct9f+evicting pod "coredns-67bfd975c5-rgkh7"+evicting pod "coredns-67bfd975c5-6g88n"+2020/09/15 21:13:42 Node "ip-192-168-123-456.us-east-1.compute.internal" successfully cordoned and drained.+2020/09/15 21:13:43 Sending interruption event to the interruption channel+2020/09/15 21:13:43 Got interruption event from channel {InstanceID:i-1234567890abcdef0 InstanceType:m4.xlarge PublicHostname:ec2-192-0-2-54.compute-1.amazonaws.com PublicIP:192.0.2.54 LocalHostname:ip-172-16-34-43.ec2.internal LocalIP:172.16.34.43 AvailabilityZone:us-east-1a} {EventID:spot-itn-97be476b6246aba6401ba36e54437719bfdf987773e9c83fe30336eb7fea9704 Kind:SPOT_ITN Description:Spot ITN received. Instance will be interrupted at 2020-09-15T21:15:43Z+ State: NodeName:ip-192-168-123-456.us-east-1.compute.internal StartTime:2020-09-15 21:15:43 +0000 UTC EndTime:0001-01-01 00:00:00 +0000 UTC Drained:false PreDrainTask:0x113c8a0 PostDrainTask:<nil>}+WARNING: ignoring DaemonSet-managed Pods: default/amazon-ec2-metadata-mock-pszj2, kube-system/aws-node-bl2bj, kube-system/aws-node-termination-handler-2pvjr, kube-system/kube-proxy-fct9f+2020/09/15 21:13:44 Node "ip-192-168-123-456.us-east-1.compute.internal" successfully cordoned and drained.+2020/09/15 21:13:45 Sending interruption event to the interruption channel+2020/09/15 21:13:45 Got interruption event from channel...+```++This isn't a mistake, by default AEMM will respond to any request for metadata with a spot interruption occurring 2 minutes+later than the request time.\* AWS Node Termination Handler polls for events every 2 seconds by default, so the effect is+that new interruption events are found and processed every 2 seconds. ++In reality there will only be a single interruption event, and you can mock this by setting the `spot.time` parameter of+AEMM when installing it. +```+helm install amazon-ec2-metadata-mock amazon-ec2-metadata-mock-1.6.0.tgz \+ --set aemm.spot.time="2020-09-09T22:40:47Z" \+ --namespace default+```++Now when you check the logs you should only see a single event get processed. ++For more ways of configuring AEMM check out the [Helm configuration page](https://github.com/aws/amazon-ec2-metadata-mock/tree/master/helm/amazon-ec2-metadata-mock).++## Node Termination Handler E2E Tests++AEMM started out as a test server for aws-node-termination-handler's end-to-end tests in this repo. We use AEMM throughout+our end to end tests to create interruption notices.++The e2e tests install aws-node-termination-handler using Helm and set the metadata url [here](https://github.com/aws/aws-node-termination-handler/blob/master/test/e2e/spot-interruption-test#L36).+This becomes where aws-node-termination-handler looks for metadata; other applications on the node still look at the real+EC2 metadata service.++We set the metadata url environment variable [here](https://github.com/aws/aws-node-termination-handler/blob/master/test/k8s-local-cluster-test/run-test#L18)+for the local tests that use a kind cluster, and [here](https://github.com/aws/aws-node-termination-handler/blob/master/test/eks-cluster-test/run-test#L117)+for the eks-cluster e2e tests.++Check out the [ReadMe](https://github.com/aws/aws-node-termination-handler/tree/master/test) in our test folder for more+info on the e2e tests. ++---++\* Only the first two unique IPs to request data from AEMM receive spot itn information in the response. This was introduced+in AEMM v1.6.0. This can be overridden with a configuration paramter. For previous versions there is no unique IP restriction.
was about to ask how spellcheck didn't catch it
comment created time in 4 days
Pull request review commentaws/aws-node-termination-handler
Add AEMM mock interruption documentation
+# AWS Node Termination Handler & Amazon EC2 Metadata Mock++We have open sourced a tool called the [amazon-ec2-metadata-mock](https://github.com/aws/amazon-ec2-metadata-mock) (AEMM)+that simulates spot interruption notices and more by starting a real webserver that serves data similar to EC2 Instance+Metadata Service. The tool is easily deployed to kubernetes with a Helm chart.++Below is a short guide on how to set AEMM up with your Node Termination Handler cluster in case you'd like to verify the+behavior yourself.++## Triggering AWS Node Termination Handler with Amazon EC2 Metadata Mock++Start by installing AEMM on your cluster. For full and up to date installation instructions reference the AEMM repository.+Here's just one way to do it.++Download the latest tar ball from the releases page, at the time of writing this that was v1.6.0. Then install it using+Helm:+```+helm install amazon-ec2-metadata-mock amazon-ec2-metadata-mock-1.6.0.tgz \+ --namespace default+```++Once AEMM is installed, you need to change the instance metadata url of Node Termination Handler to point+to the location AEMM is serving from. If you use the default values of AEMM, the installation will look similar to this:+```+helm upgrade --install aws-node-termination-handler \+ --namespace kube-system \+ --set instanceMetadataURL="http://amazon-ec2-metadata-mock-service.default.svc.cluster.local:1338" \+ eks/aws-node-termination-handler+```++That's it! Instead of polling the real IMDS service endpoint, AWS Node Termination Handler will instead poll AEMM.+If you open the logs of an AWS Node Termination Handler pod you should see that it receives (mock) interruption+events from AEMM and that the nodes are cordoned and drained. Keep in mind that these nodes won't actually get terminated,+so you might need to manually uncordon the nodes if you want to reset your test cluster.++### AEMM Advanced Configuration+If you run the example above you might notice that the logs are heavily populated. Here's an example output:+```+2020/09/15 21:13:41 Sending interruption event to the interruption channel+2020/09/15 21:13:41 Got interruption event from channel {InstanceID:i-1234567890abcdef0 InstanceType:m4.xlarge PublicHostname:ec2-192-0-2-54.compute-1.amazonaws.com PublicIP:192.0.2.54 LocalHostname:ip-172-16-34-43.ec2.internal LocalIP:172.16.34.43 AvailabilityZone:us-east-1a} {EventID:spot-itn-47ddfb5e39791606bec3e91fea4cdfa86f86a60ddaf014c8b4af8e008f134b19 Kind:SPOT_ITN Description:Spot ITN received. Instance will be interrupted at 2020-09-15T21:15:41Z+ State: NodeName:ip-192-168-123-456.us-east-1.compute.internal StartTime:2020-09-15 21:15:41 +0000 UTC EndTime:0001-01-01 00:00:00 +0000 UTC Drained:false PreDrainTask:0x113c8a0 PostDrainTask:<nil>}+WARNING: ignoring DaemonSet-managed Pods: default/amazon-ec2-metadata-mock-pszj2, kube-system/aws-node-bl2bj, kube-system/aws-node-termination-handler-2pvjr, kube-system/kube-proxy-fct9f+evicting pod "coredns-67bfd975c5-rgkh7"+evicting pod "coredns-67bfd975c5-6g88n"+2020/09/15 21:13:42 Node "ip-192-168-123-456.us-east-1.compute.internal" successfully cordoned and drained.+2020/09/15 21:13:43 Sending interruption event to the interruption channel+2020/09/15 21:13:43 Got interruption event from channel {InstanceID:i-1234567890abcdef0 InstanceType:m4.xlarge PublicHostname:ec2-192-0-2-54.compute-1.amazonaws.com PublicIP:192.0.2.54 LocalHostname:ip-172-16-34-43.ec2.internal LocalIP:172.16.34.43 AvailabilityZone:us-east-1a} {EventID:spot-itn-97be476b6246aba6401ba36e54437719bfdf987773e9c83fe30336eb7fea9704 Kind:SPOT_ITN Description:Spot ITN received. Instance will be interrupted at 2020-09-15T21:15:43Z+ State: NodeName:ip-192-168-123-456.us-east-1.compute.internal StartTime:2020-09-15 21:15:43 +0000 UTC EndTime:0001-01-01 00:00:00 +0000 UTC Drained:false PreDrainTask:0x113c8a0 PostDrainTask:<nil>}+WARNING: ignoring DaemonSet-managed Pods: default/amazon-ec2-metadata-mock-pszj2, kube-system/aws-node-bl2bj, kube-system/aws-node-termination-handler-2pvjr, kube-system/kube-proxy-fct9f+2020/09/15 21:13:44 Node "ip-192-168-123-456.us-east-1.compute.internal" successfully cordoned and drained.+2020/09/15 21:13:45 Sending interruption event to the interruption channel+2020/09/15 21:13:45 Got interruption event from channel...+```++This isn't a mistake, by default AEMM will respond to any request for metadata with a spot interruption occurring 2 minutes+later than the request time.\* AWS Node Termination Handler polls for events every 2 seconds by default, so the effect is+that new interruption events are found and processed every 2 seconds. ++In reality there will only be a single interruption event, and you can mock this by setting the `spot.time` parameter of+AEMM when installing it. +```+helm install amazon-ec2-metadata-mock amazon-ec2-metadata-mock-1.6.0.tgz \+ --set aemm.spot.time="2020-09-09T22:40:47Z" \+ --namespace default+```++Now when you check the logs you should only see a single event get processed. ++For more ways of configuring AEMM check out the [Helm configuration page](https://github.com/aws/amazon-ec2-metadata-mock/tree/master/helm/amazon-ec2-metadata-mock).++## Node Termination Handler E2E Tests++AEMM started out as a test server for aws-node-termination-handler's end-to-end tests in this repo. We use AEMM throughout+our end to end tests to create interruption notices.++The e2e tests install aws-node-termination-handler using Helm and set the metadata url [here](https://github.com/aws/aws-node-termination-handler/blob/master/test/e2e/spot-interruption-test#L36).+This becomes where aws-node-termination-handler looks for metadata; other applications on the node still look at the real+EC2 metadata service.++We set the metadata url environment variable [here](https://github.com/aws/aws-node-termination-handler/blob/master/test/k8s-local-cluster-test/run-test#L18)+for the local tests that use a kind cluster, and [here](https://github.com/aws/aws-node-termination-handler/blob/master/test/eks-cluster-test/run-test#L117)+for the eks-cluster e2e tests.++Check out the [ReadMe](https://github.com/aws/aws-node-termination-handler/tree/master/test) in our test folder for more+info on the e2e tests. ++---++\* Only the first two unique IPs to request data from AEMM receive spot itn information in the response. This was introduced+in AEMM v1.6.0. This can be overridden with a configuration paramter. For previous versions there is no unique IP restriction.
paramter --> parameter
comment created time in 4 days
Pull request review commentaws/aws-node-termination-handler
Reformat logs to support proper JSON logging
func addTaint(node *corev1.Node, nth Node, taintKey string, taintValue string, e } if err != nil {- log.Log().Msgf("Error while adding %v taint on node %v: %v", taintKey, node.Name, err)+ log.Log().
Got it, I just meant comparing what you currently have in code now vs. the same line but wrapping the message with Err()
comment created time in 5 days
pull request commentaws/aws-node-termination-handler
Reformat logs to support proper JSON logging
I'm learning toward making all the keys we control snake_case.
+ 1
comment created time in 5 days
Pull request review commentaws/aws-node-termination-handler
Reformat logs to support proper JSON logging
func removeTaint(node *corev1.Node, client kubernetes.Interface, taintKey string } if err != nil {- log.Log().Msgf("Error while releasing %v taint on node %v: %v", taintKey, node.Name, err)+ log.Log().+ Str("TaintKey", taintKey).+ Str("NodeName", node.Name).+ Msg("Error while releasing taint on node")
I like this format in code, makes it easier to read/understand 👍🏼
comment created time in 5 days
Pull request review commentaws/aws-node-termination-handler
Reformat logs to support proper JSON logging
func addTaint(node *corev1.Node, nth Node, taintKey string, taintValue string, e } if err != nil {- log.Log().Msgf("Error while adding %v taint on node %v: %v", taintKey, node.Name, err)+ log.Log().
out of curiosity, what does this log look like compared to the one you have posted? I'm assuming you didn't wrap in Err for formatting reasons
log.Log().Err("Error while adding taint on node").Str("TaintKey", taintKey). ...
comment created time in 5 days
push eventbrycahta/amazon-ec2-instance-qualifier
commit sha 0913e07ab997cb812a74963e68a7c793c20eb3af
pass args via config file (#8) * pass userConfig as JSON * update e2e & docs * fixing config precedence
push time in 5 days
delete branch brycahta/amazon-ec2-instance-qualifier
delete branch : config-file
delete time in 5 days
push eventawslabs/amazon-ec2-instance-qualifier
commit sha 0913e07ab997cb812a74963e68a7c793c20eb3af
pass args via config file (#8) * pass userConfig as JSON * update e2e & docs * fixing config precedence
push time in 5 days
PR merged awslabs/amazon-ec2-instance-qualifier
Issue #, if available: N/A
Description of changes:
- args can be passed as JSON via
--config-fileflag- config file will take precedence over args provided via CLI or as Env Vars
- Updated userConfig accessibility from private to public to leverage JSON marshal + unmarshal
- Updated tests and documentation
- Fix bucket creation as e2e tests sometimes stall and fail due to 'bucket already created' error
Testing:
make clean&&make testlocally
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
pr closed time in 5 days
Pull request review commentawslabs/amazon-ec2-instance-qualifier
func (t TestFixture) CfnTemplateFilename() string { return t.cfnTemplateFilename } -// AmiId returns amiId.+// AmiId returns AmiId. func (t TestFixture) AmiId() string { return t.amiId }++// SetUserConfig sets empty fields of UserConfig to reqConfig+func (UserConfig) SetUserConfig(reqConfig UserConfig) {+ if userConfig.InstanceTypes == "" {
Had a couple more if checks in here at first such as:
if userConfig.InstanceTypes == "" && reqConfig.InstanceTypes != "" , but go-report-card (mainly gocyclo) complained the whole function was too complex so had to limit to 15 if statements.
I think the right long-term move would be using Viper to handle the precedence for us, but I think this works for now
comment created time in 6 days
push eventbrycahta/amazon-ec2-instance-qualifier
commit sha 52ee8ad5b4ddee212c90a4c09121e3fb06034d88
fixing config precedence
push time in 6 days
PR opened awslabs/amazon-ec2-instance-qualifier
Issue #, if available: N/A
Description of changes:
- args can be passed as JSON via
--config-fileflag- config file will take precedence over args provided via CLI or as Env Vars
- Updated userConfig accessibility from private to public to leverage JSON marshal + unmarshal
- Updated tests and documentation
- Fix bucket creation as e2e tests sometimes stall and fail due to 'bucket already created' error
Testing:
make clean&&make testlocally
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
pr created time in 6 days
push eventbrycahta/amazon-ec2-instance-qualifier
commit sha aa7b1b0aa56c9eab8b7a8b6e6ca9623deb09b5f2
update e2e & docs
push time in 6 days
create barnchbrycahta/amazon-ec2-instance-qualifier
created branch time in 6 days
push eventbrycahta/amazon-ec2-instance-qualifier
commit sha 017f4cf09a6383a92ab061c0dcfdec0cd63e4ea0
update session-creation logic (#6) * update session-creation logic * update region selection * readme updates * go mod tidy * update licenses and attributions
push time in 6 days
fork brycahta/amazon-ec2-instance-selector
A CLI tool and go library which recommends instance types based on resource criteria like vcpus and memory
fork in 9 days
delete branch brycahta/amazon-ec2-instance-qualifier
delete branch : session-refactor
delete time in 9 days
push eventawslabs/amazon-ec2-instance-qualifier
commit sha 017f4cf09a6383a92ab061c0dcfdec0cd63e4ea0
update session-creation logic (#6) * update session-creation logic * update region selection * readme updates * go mod tidy * update licenses and attributions
push time in 9 days
PR merged awslabs/amazon-ec2-instance-qualifier
Issue #, if available: N/A
Description of changes:
- Update aws session creation logic to align with Instance-Selector
- Pass region from CLI to agent
- If users do not provide a region, then do not assign in userConfig
make clean && make testlocally
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
pr closed time in 9 days
push eventbrycahta/amazon-ec2-instance-qualifier
commit sha 1a3edbaed9835cf378886954178d6101287fc5dc
go mod tidy
commit sha 6c27ef295037d784929b85811c3be01c56e57abb
update licenses and attributions
push time in 9 days
Pull request review commentawslabs/amazon-ec2-instance-qualifier
go 1.14 require ( github.com/aws/aws-sdk-go v1.31.5+ github.com/mitchellh/go-homedir v1.1.0
good catch!
comment created time in 9 days
Pull request review commentawslabs/amazon-ec2-instance-qualifier
func TestPopulateTestFixtureForResumedRun(t *testing.T) { func TestParseCliArgsEnvSuccess(t *testing.T) { resetFlagsForTest()- os.Setenv("AWS_DEFAULT_REGION", "AWS_DEFAULT_REGION")+ os.Setenv("AWS_REGION", "us-weast-1")
not a typo
comment created time in 10 days
push eventbrycahta/amazon-ec2-instance-qualifier
commit sha 967944ecd0824c6297ae3e302c5d8cc8718a6696
allow users to provide script in userData (#7)
commit sha 7353c1324dc99493e1b35ad5a0ce73e41a21816f
update session-creation logic
commit sha 90103be145d66cf85c891b821d36d775ad39aa68
update region selection
commit sha cd44a4223dd1b59fdb236fbdff8517231a7f4fed
readme updates
push time in 10 days
push eventbrycahta/amazon-ec2-instance-qualifier
commit sha 967944ecd0824c6297ae3e302c5d8cc8718a6696
allow users to provide script in userData (#7)
push time in 10 days
delete branch brycahta/amazon-ec2-instance-qualifier
delete branch : user-bash
delete time in 10 days
push eventawslabs/amazon-ec2-instance-qualifier
commit sha 967944ecd0824c6297ae3e302c5d8cc8718a6696
allow users to provide script in userData (#7)
push time in 10 days
PR merged awslabs/amazon-ec2-instance-qualifier
Issue #, if available: N/A
Description of changes:
- add
--custom-scriptsflag- users provide path to a script which will execute before agent and before setup
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
pr closed time in 10 days
PR opened awslabs/amazon-ec2-instance-qualifier
Issue #, if available: N/A
Description of changes:
- add
--custom-scriptsflag- users provide path to a script which will execute before agent and before setup
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
pr created time in 10 days
create barnchbrycahta/amazon-ec2-instance-qualifier
created branch time in 10 days
PR opened awslabs/amazon-ec2-instance-qualifier
Issue #, if available: N/A
Description of changes:
- Update aws session creation logic to align with Instance-Selector
- Pass region from CLI to agent
- If users do not provide a region, then do not assign in userConfig
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
pr created time in 11 days
create barnchbrycahta/amazon-ec2-instance-qualifier
created branch time in 11 days
delete branch brycahta/aws-node-termination-handler
delete branch : aemm-upgrade
delete time in 12 days
push eventaws/aws-node-termination-handler
commit sha ac6f0e79c3493c7ac621dc1aac6e7f642f9b4af7
upgrade aemm to 1.6 (#251)
push time in 12 days
PR merged aws/aws-node-termination-handler
Issue #, if available: N/A
Description of changes:
- upgrade aemm to 1.6
Testing:
make e2e-testlocally
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
pr closed time in 12 days
PR opened aws/aws-node-termination-handler
Issue #, if available: N/A
Description of changes:
- upgrade aemm to 1.6
Testing:
make e2e-testlocally
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
pr created time in 12 days
create barnchbrycahta/aws-node-termination-handler
created branch time in 12 days
push eventbrycahta/aws-node-termination-handler
commit sha e68df3460c92a4db705c37319108989585b07e62
add readme for e2e tests (#218)
commit sha bad975d59e330b8af966b087f23cbf8c5692ecbb
only deploy to worker node on e2e tests (#217) * only deploy to worker node on e2e tests * fix webhook tests without using host networking * fix webhook port consistency * print gomod download logs
commit sha 91b481abceb22816019b3953ec970ae16f31877a
fix webhook test stability (#219)
commit sha a91b0e788596ceb43b8a1893573ae9c899250fdb
add missing helm chart properties (#220) (#222)
commit sha 95fdc025df8f92b725f988869854917d961cc14b
Fix Helm chart comment on enableSpotInterruptionDraining default behavior (#221) * default enableSpotInterruptionDraining to true * enableSpotInterruptionDraining default to "true" * Update README for helm chart, and roll back value change Co-authored-by: Thomas O'Neill <[email protected]>
commit sha 1943f8bce4f9a1ef73a52c8973712aac1e1c3462
fix missing new line for updateStrategy (#227)
commit sha 292e4af3844abae869fb95bcd8f37255dfbf5286
updating default updateStrategy and bumping version for helm chart release (#228)
commit sha ca45b8f8274c25be8c5180e51efdea4b52e7e847
fix manifest updating in push-docker-images (#231)
commit sha 3e355f41607f887a3ebc7e39d38137cfb193c6eb
Add Amazon EC2 Spot Instances Integration Roadmap to readme (#232)
commit sha e48c1f8b965f7e608688fb59a555f87203bd1326
upgrade to go 1.15 (#233) * upgrade to go 1.15 * remove duplicate linker flags * remove -s linker flag from windows
commit sha c685bbaf7a614305bed5e33f4aed867e15a041b3
fix identation on PodMonitor (#235)
commit sha 021b71f48a2500e54def0d04dbd59046cc086bd0
Helm Template Gen Test (#236) * add helm test to check template generation * sync enableSpotInterruption usage statement
commit sha 97f42ef9f6c656f84ccfda982be4da4fa3529e3b
Update .travis.yml go1.15 (#238)
commit sha 713db7c6b117e638500984cbe6ab26b74783b6a2
up helm chart to use v1.7.0 image (#234)
commit sha 3fa790071c86c17ad6fbb35910f982dea3072dcf
retry aemm installs (#237)
commit sha f92ed02c5a6eb7681582b8858574c9a5853d9e90
fix release script upload to github (#240)
commit sha ef0eec0e9f4cd69f873cf1fc21744ea20e799552
only build one binary to check licenses (#239)
commit sha e5705ffd6fed89b277990bf349327f09eaadc790
clear up shellcheck warnings (#242) * clear up shellcheck warnings * changes based on PR feedback * missed some spots
commit sha 24ff89c46173c56e14ed37eb26b9525bb9194e6c
fix helm test (#243)
commit sha 92fb0c2fab2dc43c335766e3288b9b1604d198b4
Add retries when reponse from IMDSv2 retruns a 401 (#244) * Add retries when reponse from IMDSv2 retruns a 401 * Revert "Add retries when reponse from IMDSv2 retruns a 401" This reverts commit c1e34774098384db6b33c0774ec7dcf7a5c9bce7. * move IMDSv2 401 retries to Request function. move Panic to main function. add tests for 401 retries. * add missing equals to duplicateErrorThershold check in main function * fix spelling of NTH * fix report card
push time in 12 days
push eventbrycahta/amazon-ec2-metadata-mock
commit sha c93b63d4c668acecb11e956216ea491023740862
🥑🤖 v1.6.0 release prep [Skip Helm E2E Tests] 🤖🥑 (#96)
push time in 12 days
created tagaws/amazon-ec2-metadata-mock
A tool to simulate Amazon EC2 instance metadata
created time in 12 days
delete branch brycahta/amazon-ec2-metadata-mock
delete branch : pr/v1.6.0-release
delete time in 12 days
push eventaws/amazon-ec2-metadata-mock
commit sha c93b63d4c668acecb11e956216ea491023740862
🥑🤖 v1.6.0 release prep [Skip Helm E2E Tests] 🤖🥑 (#96)
push time in 12 days
PR merged aws/amazon-ec2-metadata-mock
🥑🤖 Auto-generated PR for v1.6.0 release. Updating release versions in repo.
pr closed time in 12 days
PR opened aws/amazon-ec2-metadata-mock
🥑🤖 Auto-generated PR for v1.6.0 release. Updating release versions in repo.
pr created time in 12 days
create barnchbrycahta/amazon-ec2-metadata-mock
created branch time in 12 days
push eventbrycahta/amazon-ec2-metadata-mock
commit sha 715555a712a48820c9d83c3a210486cc04a4ae2c
Update README.md (#93)
commit sha 3ecba41c7c57661bd8cee442cface2edf9fc2948
adding placement paths (#95)
commit sha 38f2e3befb1e3fc00a1370f8ec7c314fce5b99e5
Adding mock-ip-count feature (#94) * convert from daemonset to deployment * adding termination-nodes feature * update travis and documentation * rename to mock-ip-count * address pr feedback * add feature to scheduled events for parity * add e2e tests for cmds * fix ReadMe description
push time in 12 days
delete branch brycahta/amazon-ec2-metadata-mock
delete branch : helm_deployment
delete time in 12 days
push eventaws/amazon-ec2-metadata-mock
commit sha 38f2e3befb1e3fc00a1370f8ec7c314fce5b99e5
Adding mock-ip-count feature (#94) * convert from daemonset to deployment * adding termination-nodes feature * update travis and documentation * rename to mock-ip-count * address pr feedback * add feature to scheduled events for parity * add e2e tests for cmds * fix ReadMe description
push time in 12 days
PR merged aws/amazon-ec2-metadata-mock
Issue #, if available:
- AEMM as a daemonset wasn't appropriate since it's exposed as a k8s service
- Prior to these changes, there was no way to limit the amount of nodes that could get a Spot Interrupt; therefore, if someone had a cluster of 5 nodes and AEMM pods running on each one and sent a request to spot path, then all nodes in cluster could be affected
Description of changes:
- Change AEMM from a daemonset to a deployment w/replicas defaulted to 1
- Add
--mock-ip-count=<int>flag to set the number of nodes that will get a valid Spot INT-- all other nodes will get a 404 - Add an e2e cluster test
- Updated Makefile, Travis, documentation
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
pr closed time in 12 days
push eventaws/amazon-ec2-metadata-mock
commit sha 3ecba41c7c57661bd8cee442cface2edf9fc2948
adding placement paths (#95)
push time in 12 days
PR merged aws/amazon-ec2-metadata-mock
Issue #, if available: N/A
Description of changes:
- Adding
placement/*paths that were launched in Aug-2020. - Update docs
- Update test golden files
Testing:
make testlocally- queried newly added paths successfully
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
pr closed time in 12 days
push eventbrycahta/amazon-ec2-metadata-mock
commit sha 71e856af010e67e492b1a674087f07578a447efb
fix ReadMe description
push time in 12 days
Pull request review commentaws/amazon-ec2-metadata-mock
Parameter | Description | Default in Helm | Default AEMM configuration `aemm.server.hostname` | hostname to run AEMM on | `""`, in order to listen on all available interfaces e.g. ClusterIP | `0.0.0.0` `aemm.mockDelaySec` | mock delay in seconds, relative to the start time of AEMM | `0` | `0` `aemm.mockTriggerTime` | mock trigger time in RFC3339 format | `""` | `""`+`aemm.mockIPCount` | number of IPs that can receive spot interrupts and/or scheduled events; subsequent requests will return 404 | `""` | `99`
yup
comment created time in 12 days
Pull request review commentaws/amazon-ec2-metadata-mock
var ( gf.ConfigFileFlag: cfg.GetDefaultCfgFileName(), gf.MockDelayInSecFlag: 0, gf.MockTriggerTimeFlag: "",+ gf.MockIPCountFlag: 99,
good catch!
comment created time in 12 days
PR opened aws/amazon-ec2-metadata-mock
Issue #, if available: N/A
Description of changes:
- Adding
placement/*paths that were launched in Aug-2020. - Update docs
- Update test golden files
Testing:
make testlocally- queried newly added paths successfully
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
pr created time in 12 days
push eventbrycahta/amazon-ec2-metadata-mock
commit sha 27659b96a28d4afa482966ceb17ef240f4b59108
fix ReadMe description
push time in 12 days
Pull request review commentaws/amazon-ec2-metadata-mock
Flags: -t, --time string time specifies the approximate time when the spot instance will receive the shutdown signal in RFC3339 format to execute instance action E.g. 2020-01-07T01:03:47Z (default: request time + 2 minutes in UTC) Global Flags:- -c, --config-file string config file for cli input parameters in json format (default: $HOME/aemm-config.json)- -n, --hostname string the HTTP hostname for the mock url (default: localhost)- -I, --imdsv2 whether to enable IMDSv2 only, requiring a session token when submitting requests (default: false, meaning both IMDS v1 and v2 are enabled)- -d, --mock-delay-sec int mock delay in seconds, relative to the application start time (default: 0 seconds)- -p, --port string the HTTP port where the mock runs (default: 1338)- -s, --save-config-to-file whether to save processed config from all input sources in .ec2-metadata-mock/.aemm-config-used.json in $HOME or working dir, if homedir is not found (default: false)+ -c, --config-file string config file for cli input parameters in json format (default: $HOME/aemm-config.json)
need to fix this. i used root --help instead of spot --help
comment created time in 12 days
push eventbrycahta/amazon-ec2-metadata-mock
commit sha 49fcae03895717ac5842628af2c4862ca493b280
rename to mock-ip-count
commit sha 2f76530e99d80cc947ebf5666e858d9c38791bd2
address pr feedback
commit sha 94a2d845a08d89fa17a7dac33dbb55559638c143
add feature to scheduled events for parity
commit sha 68b9ef61eb73a9bf69c9906e92a89965c4aa349c
add e2e tests for cmds
push time in 16 days
Pull request review commentaws/amazon-ec2-metadata-mock
Define number of nodes to receive Spot interrupts
get_chart_test_config() { echo "$config" } +test_termination_nodes() {+ if [[ $REUSE_ENV == false ]]; then+ mkdir -p $TMP_DIR+ install_kind+ create_kind_cluster+ fi++ build_and_load_image+ install_helm
:( yes
comment created time in 16 days
Pull request review commentaws/amazon-ec2-metadata-mock
Define number of nodes to receive Spot interrupts
var ( gf.ConfigFileFlag: cfg.GetDefaultCfgFileName(), gf.MockDelayInSecFlag: 0, gf.MockTriggerTimeFlag: "",+ gf.TerminationNodesFlag: 99,
- +1 to the
-1recommendation - I'm good with defaulting to
2- do you think I should default this in the helm
values.yamlfile as well even though it'll be defaulted in app code? Not sure the etiquette on that - since this will technically break backwards compat should we major release it?
- do you think I should default this in the helm
comment created time in 16 days
Pull request review commentaws/amazon-ec2-metadata-mock
Define number of nodes to receive Spot interrupts
Flags: --mock-trigger-time string mock trigger time in RFC3339 format. This takes priority over mock-delay-sec (default: none) -p, --port string the HTTP port where the mock runs (default: 1338) -s, --save-config-to-file whether to save processed config from all input sources in .ec2-metadata-mock/.aemm-config-used.json in $HOME or working dir, if homedir is not found (default: false)+ -x, --termination-nodes int number of nodes in a cluster that can receive Spot interrupt notice (default: 99, meaning unlimited nodes) (default 99)
Yeah, it definitely should. Not that I'm hyped about --termination-nodes as a name, but could apply to both if i update the description to be non-spot specific.
Thoughts on keeping -x as shorthand?
comment created time in 16 days
Pull request review commentaws/amazon-ec2-metadata-mock
Define number of nodes to receive Spot interrupts
image: tag: "v1.5.0" pullPolicy: "IfNotPresent" +replicaCount: 1+ # nameOverride overrides the name of the helm chart nameOverride: "" # fullnameOverride overrides the name of the application fullnameOverride: "" -# Create node OS specific daemonset(s). (e.g. "linux", "windows", "linux windows")+# Create node OS specific deployment(s). (e.g. "linux", "windows", "linux windows") targetNodeOs: "linux"
win..what?
comment created time in 16 days
push eventbrycahta/amazon-ec2-metadata-mock
commit sha 5eb3eb22e26803c3ccce00888e02bb9be8e31329
update travis and documentation
push time in 17 days