Democratizing GitOps
aws/aws-node-termination-handler 550
A Kubernetes Daemonset to gracefully handle EC2 instance shutdown
Collection of articles written for publication on my blog or elsewhere
Toolkit for working with AWS APIs
Chef Cookbook and Recipes for Zuul
Github mirror of Alembic
jaypipes/amazon-eks-pod-identity-webhook 0
Amazon EKS Pod Identity Webhook
jaypipes/amazon-eks-user-guide 0
The open source version of the Amazon EKS user guide. You can submit feedback & requests for changes by submitting issues in this repo or by making proposed changes & submitting a pull request.
Pull request review commentaws/aws-controllers-k8s
+#!/usr/bin/env bash++###########################################+# API+###########################################+create_http_api_and_validate() {+## create api resource+cat <<EOF | kubectl apply -f - >/dev/null 2>&1+apiVersion: apigatewayv2.services.k8s.aws/v1alpha1+kind: API+metadata:+ name: $api_name
It's cool to clean it up in a followup PR.
comment created time in 3 days
push eventaws/aws-controllers-k8s
commit sha 606f71694430fc1ac85f173f92559f587f2d07e3
Enable finegrained diffs, update() args include desired state, latest state (#299) Issue #, if available: #298 Changed the update() method args to include 'latest' state as well. Now both desired state and latest state is available inside update() method to compute finegrained diffs to make service api calls. Also, it addresses comment received on #245 to simplify the mechanism to configure custom implementation for operation on a custom resource. Following is an example to configure CustomModifyReplicationGroup as custom_implementation operation for ModifyReplicationGroup operation in elasticache generator.yaml config file: ```yaml operations: ModifyReplicationGroup: custom_implementation: CustomModifyReplicationGroup ``` Co-authored-by: Kumar Gaurav Sharma <[email protected]>
push time in 3 days
PR merged aws/aws-controllers-k8s
Issue #, if available: #298
Description of changes: Changed the update() method args to include 'latest' state as well. Now both desired state and latest state is available inside update() method to compute finegrained diffs to make service api calls.
Also, it addresses comment received on #245 to simplify
the mechanism to configure custom implementation for operation
on a custom resource. Following is an example to
configure CustomModifyReplicationGroup as custom_implementation operation
for ModifyReplicationGroup operation in elasticache generator.yaml
config file:
operations:
ModifyReplicationGroup:
custom_implementation: CustomModifyReplicationGroup
Due to open issue #296 not all services' controllers are auto generated in this
revision of the PR. Thus, make test fails for other services as of now.
This PR includes service controllers generated for
elasticacheand manually updated forpetstore,bookstoreservices to help this review. Once issue #296 is fixed, remaining service controller will be generated and this PR will include corresponding updates.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
pr closed time in 3 days
pull request commentaws/aws-controllers-k8s
Enable finegrained diffs, update() args include desired state, latest state
@kumargauravsharma just a quick note, I'm going to squash and merge this, but in the future, please do git commit --amend && git push origin <BRANCH> --force if you make changes to a commit based on a code review instead of pushing more commits with commit messages like "fixed after rebase" etc :)
comment created time in 3 days
issue closedaws/aws-controllers-k8s
Test fail after generating controllers for apigatewayv2
Describe the bug
After generating controllers for apigateway2, the make test fails with error.
Steps to reproduce
Commit level:
commit 87bd3b1338c0f2e208f5ab9062a7240b823605c3 (HEAD -> upstream-main, upstream/main)
$ make test # this passes
$ make build-controller SERVICE=apigatewayv2 # this generates changes under services/apigatewayv2
$ git status # shows many files
$ make test # this fails with error
Error:
# github.com/aws/aws-controllers-k8s/services/apigatewayv2/pkg/resource/api
services/apigatewayv2/pkg/resource/api/sdk.go:261:6: res.SetDisableExecuteApiEndpoint undefined (type *apigatewayv2.CreateApiInput has no field or method SetDisableExecuteApiEndpoint)
services/apigatewayv2/pkg/resource/api/sdk.go:417:6: res.SetDisableExecuteApiEndpoint undefined (type *apigatewayv2.UpdateApiInput has no field or method SetDisableExecuteApiEndpoint)
# github.com/aws/aws-controllers-k8s/services/apigatewayv2/pkg/resource/domain_name
services/apigatewayv2/pkg/resource/domain_name/sdk.go:190:10: undefined: apigatewayv2.MutualTlsAuthenticationInput
services/apigatewayv2/pkg/resource/domain_name/sdk.go:197:6: res.SetMutualTlsAuthentication undefined (type *apigatewayv2.CreateDomainNameInput has no field or method SetMutualTlsAuthentication)
services/apigatewayv2/pkg/resource/domain_name/sdk.go:285:10: undefined: apigatewayv2.MutualTlsAuthenticationInput
services/apigatewayv2/pkg/resource/domain_name/sdk.go:292:6: res.SetMutualTlsAuthentication undefined (type *apigatewayv2.UpdateDomainNameInput has no field or method SetMutualTlsAuthentication)
Expected outcome
A concise description of what you expected to happen.
make test passes after generating controllers for apigatewayv2
Environment
- Kubernetes version
- Using EKS (yes/no), if so version?
- AWS service targeted (S3, RDS, etc.):
apigatewayv2
closed time in 3 days
kumargauravsharma
issue commentaws/aws-controllers-k8s
Any timeline estimates on IAM provisioning via ACK? This would be an incredibly helpful feature.
Hi! No, at this time I have not thought about a timeline for IAM provisioning via an ACK service controller. If you haven't already, please feel free to +1 this issue.
Even if we agreed to follow some of the suggestions from @mksh (which are great, btw), we would need to get an IAM service controller for ACK onto the release roadmap and then determine a likely milestone for an IAM controller to go into dev preview (and then target a GA release of the controller 3 months after we release in dev preview). Earliest I see being able to get an IAM controller for ACK into dev preview would probably be end of 2020, given the backlog of services we're currently slated to deliver these next four months.
Of course, squeaky wheel gets the grease, so no worries squeaking about this feature request AFAIC :)
comment created time in 3 days
Pull request review commentaws/aws-controllers-k8s
+#!/usr/bin/env bash++###########################################+# API+###########################################+create_http_api_and_validate() {+## create api resource+cat <<EOF | kubectl apply -f - >/dev/null 2>&1+apiVersion: apigatewayv2.services.k8s.aws/v1alpha1+kind: API+metadata:+ name: $api_name+spec:+ name: $api_name+ protocolType: HTTP+EOF++sleep 10++## validate that api-id was populated in resource status+debug_msg "retrieve api-id from api/$api_name resource's status"+api_id=$(kubectl get $api_resource_name -o=json | jq -r .status.apiID)++if [[ -z "$api_id" ]];then+ echo "FAIL: $api_resource_name resource's status does not have apiID"+ exit 1+fi++## validate that api was created using apigatewayv2 get-api operation+debug_msg "apigatewayv2 get-api with api-id $api_id"+aws apigatewayv2 get-api --api-id="$api_id" > /dev/null 2>&1+assert_equal "0" "$?" "Expected success from 'apigatewayv2 get-api --api-id=$api_id' but got $?" || exit 1+}
If you wouldn't mind, could you indent the inside of this function (and the other functions in this file) :)
Makes it tough to read when there's no indentation!
comment created time in 3 days
Pull request review commentaws/aws-controllers-k8s
+#!/usr/bin/env bash++###########################################+# API+###########################################+create_http_api_and_validate() {+## create api resource+cat <<EOF | kubectl apply -f - >/dev/null 2>&1+apiVersion: apigatewayv2.services.k8s.aws/v1alpha1+kind: API+metadata:+ name: $api_name+spec:+ name: $api_name+ protocolType: HTTP+EOF++sleep 10++## validate that api-id was populated in resource status+debug_msg "retrieve api-id from api/$api_name resource's status"+api_id=$(kubectl get $api_resource_name -o=json | jq -r .status.apiID)++if [[ -z "$api_id" ]];then+ echo "FAIL: $api_resource_name resource's status does not have apiID"+ exit 1+fi++## validate that api was created using apigatewayv2 get-api operation+debug_msg "apigatewayv2 get-api with api-id $api_id"+aws apigatewayv2 get-api --api-id="$api_id" > /dev/null 2>&1+assert_equal "0" "$?" "Expected success from 'apigatewayv2 get-api --api-id=$api_id' but got $?" || exit 1+}++delete_http_api_and_validate() {+#delete api resource+debug_msg "delete api/$api_name resource"+kubectl delete $api_resource_name >/dev/null 2>&1+assert_equal "0" "$?" "Expected success from kubectl delete but got $?" || exit 1++#validate that api was deleted using apigatewayv2 get-api operation+debug_msg "get-api with api-id $api_id"+aws apigatewayv2 get-api --api-id="$api_id" > /dev/null 2>&1+assert_equal "254" "$?" "Expected not-found status code from 'apigatewayv2 get-api --api-id=$api_id' but got $?" || exit 1
keep in mind that the aws CLI version 2 and aws CLI version 1 use different return codes unfortunately for a NotFound :(
Version 1 uses 255. version 2 uses 254, if I remember correctly.
So, you'll need to either:
-
create a new
assert_infunction that accepts multiple acceptable values or -
Use an if statement like this:
https://github.com/aws/aws-controllers-k8s/blob/main/test/e2e/ecr/smoke.sh#L68-L72
comment created time in 3 days
Pull request review commentaws/aws-controllers-k8s
+#!/usr/bin/env bash++###########################################+# API+###########################################+create_http_api_and_validate() {+## create api resource+cat <<EOF | kubectl apply -f - >/dev/null 2>&1+apiVersion: apigatewayv2.services.k8s.aws/v1alpha1+kind: API+metadata:+ name: $api_name
I strongly recommend not using environment variables (and assuming that a particular env var is set to some non-nil value). Instead, I recommend using function local-scoped variables and actually pass arguments to the Bash function. Then, make a docstring comment that outlines expected parameters.
Example:
https://github.com/aws/aws-controllers-k8s/blob/main/scripts/lib/testutil.sh#L5-L23
comment created time in 3 days
issue commentaws/aws-controllers-k8s
Make easy to migrate from existing CRDs to ACK
@jaypipes sorry meant no disrespect to crossplane, and was not intending to siphone users away from crossplane. It was more meant as a possibility to support easy migration. I will remove the mention of cross plane and make the issue generic
No need to remove the mention of Crossplane, Anoop! :) It's an important question that we've been asked a few times and figured I'd expand on the question in my answer here. I will put together a document that specifically discusses Crossplane and ACK plans.
comment created time in 3 days
issue commentkubernetes-sigs/controller-runtime
https://github.com/gomodules/jsonpatch/issues/28
comment created time in 3 days
issue commentaws/aws-controllers-k8s
Migrate from crossplane when ACK is GA
Hi @anoop2811! I want to make it clear that it is not a goal of ACK to siphon users away from Crossplane.
ACK's mission is to facilitate the most Kubernetes-native way for users to interact with AWS managed services via the Kubernetes API and configuration language. While Crossplane does enable Kubernetes users to create infrastructure resources using the Kubernetes API/language, Crossplane has a much broader mission of enabling cross-cloud-provider workflows and multi-provider infrastructure needs.
Crossplane and ACK contributors are actually collaborating with each other, as we view the two projects as complementary, not competitive. In fact, I've been noodling around some ideas of using the ack-generate CLI tool to output Go code that follows the Crossplane object model/interfaces -- something that would eventually allow Crossplane's AWS cloud provider code to be replaced with code generated from ACK.
Best, -jay
comment created time in 3 days
issue commentgomodules/jsonpatch
Yes, this has broken builds of any project that depend on sigs.k8s.io/controller-runtime, such as the aws/aws-controllers-k8s project that is now unable to run go mod download with the following failure:
Building 'apigatewayv2' controller docker image with tag: ack-apigatewayv2-controller:29360d6
[+] Building 59.6s (11/14)
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 38B 0.0s
=> [internal] load metadata for docker.io/library/golang:1.14.1 0.0s
=> [internal] load metadata for docker.io/library/amazonlinux:2 0.0s
=> [builder 1/7] FROM docker.io/library/golang:1.14.1 0.0s
=> [internal] load build context 0.2s
=> => transferring context: 1.37MB 0.2s
=> [stage-1 1/3] FROM docker.io/library/amazonlinux:2 0.0s
=> CACHED [builder 2/7] WORKDIR /github.com/aws/aws-controllers-k8s 0.0s
=> CACHED [builder 3/7] COPY go.mod go.mod 0.0s
=> CACHED [builder 4/7] COPY go.sum go.sum 0.0s
=> ERROR [builder 5/7] RUN go mod download 59.3s
------
> [builder 5/7] RUN go mod download:
#11 59.04 go: sigs.k8s.io/[email protected] requires
#11 59.04 gomodules.xyz/jsonpatch/[email protected]: unrecognized import path "gomodules.xyz/jsonpatch/v2": reading https://gomodules.xyz/jsonpatch/v2?go-get=1: 404 Not Found
#11 59.04 server response: 404 page not found
------
failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c go mod download]: runc did not terminate sucessfully
make: *** [Makefile:39: build-controller-image] Error 2
comment created time in 3 days
push eventjaypipes/aws-controllers-k8s
commit sha 75692029795b0867ccdae97a199d1d61019b34ac
Add e2e tests for SQS Queues Copies the SNS Topic e2e smoke test and modified it to use the SQS `aws sqs get-queue-url` call instead of `aws sqs get-queue-attributes` since the latter requires the QueueUrl and not the ARN like get-topic-attributes does... Issue #205
commit sha 09923136c4e8616ed30062085d7fed82664dc2f8
NotFound when any required field missing in shape In #271, @vijtrip2 added methods that output Go code that checked if the Input shape for ReadOne operations for a resource had required fields and if so, checked that the CR's `Status.{Field}` had a non-nil value, otherwise the `resourceManager.sdkFind()` method would return `NotFound`, indicating to callers (in particular when trying to determine if a newly-created resource already existed) that the backend AWS resource had not been created yet. This patch makes that work generic in two ways: * adds support for when required fields in an Input shape are in the CR's `Spec` struct in addition to the `Status` struct. * adds support for any Input shape, not just the ReadOne operation's Input shape. For SQS, the GetQueueAttributes' Input shape has required fields (QueueUrl) that, if not present in the CR's `Status` struct, indicate the resource has not been created yet. Issue #287
commit sha 29360d667ddb3d898623ccb052e6179a0cbc25c9
improve documentation for authorization (#286) This patch adds a diagram and a number of improvements to the documentation describing authorization and permissions concepts in ACK. In addition, it adds recommended IAM Policy ARNs into a `services/$SERVICE/config/iam/recommended-policy-arn` file for our three checked-in ACK service controllers.
commit sha 5d899c3f9c9ac966fcdf301fa07e430efc68699c
Update controller-tools to v0.4.0
commit sha 70c14754073055a9763aac28059212ce576ddbac
Update crd, types and auto-generated code
commit sha 3a3a504137fdc1874a8d54c645dcb21156cc0286
apigatewayv2 service controller and smoke test files. (#293) * apigatewayv2 service controller and smoke test files.
commit sha 28eabf6df523210024f7fdcd520e5a576b88d91b
Enable multiple update operations support on single resource (#245) Issue #210 Description of changes: The changes in this PR are to enable custom resource update scenario where Update operation on given resource does not support fields that are supported by its Create method, and there exist specialized methods to update such fields. These specialized method response provides the Resource object indicating the Update on the resource. For example: create-replication-group API arguments include: num-node-groups replicas-per-node-group however, modify-replication-group API arguments does not, instead modify-replication-group-shard-configuration and increase-replica-count, decrease-replica-count APIs corresponds to the related Update. The approach taken in this PR is to add the details of such fields and corresponding custom update methods in the generator.yaml. For example: resources: ReplicationGroup: custom_update_operations: UpdateShardConfiguration: fields: - NumNodeGroups UpdateReplicaCount: fields: - ReplicasPerNodeGroup And custom update methods' (UpdateShardConfiguration and UpdateReplicaCount) implementation is provided in a separate code file: aws-controllers-k8s/services/elasticache/pkg/resource/replication_group/custom_update_api.go These methods are supplied with DiffReporter which helps the code in this file determine the specific Update API to invoke based on the difference in desired, current resource details. Co-authored-by: Kumar Gaurav Sharma <[email protected]>
commit sha 1b2910dcee44802f6c9fc1a9baeb2fe6598e7230
update apigwv2 service controller for Diff Recent PR changed the AWSDescriptor interface signature and this PR simply updates the API Gateway V2 service controller to abide by that new interface.
commit sha be5f830c4e69ff7eda22c3a2040273f4e3709188
`ack-generate` calls `git pull` if aws-sdk-go repository already exists
commit sha aaffb681058597dd265c9117dcc2a1513fb28ab7
Merge pull request #304 from A-Hilaly/refresh-cache `ack-generate` call `git pull` if aws-sdk-go repository already exist in the cache
commit sha 21627a478beb45e815e8cbab299ab1bd41aa650f
remove petstore and bookstore services
commit sha 5b3f2e73790e5cdc5ea678aca9e7dbed02d3adb3
Add a fake Book runtime.Object implementation for `service_controller_test.go` Use `mocks` for runtime util testing
commit sha 4546ae0cf06e81c2557b30c1fc7ff08422ecf544
Merge pull request #305 from A-Hilaly/cleanup Remove petstore and bookstore services
commit sha a5e3cb7da40f295feca6e0f117ed11da3339ebd6
Service names arguments are accepted as case insensitive for make commands
commit sha 34ed5dae3dcd8ed01d12e0a60f3d40141d6a2429
Remove petstore and bookstore integration tests
commit sha c4bc50cdd4109086e2fed41d912072e75adeab99
Merge pull request #308 from A-Hilaly/cleanup Remove petstore and bookstore integration tests
commit sha a0d2c26ed2ab72e2aa18f95cb5895f01ae3a77ff
Remove `test/integration` folder and `scripts/run-integration-tests.sh`
commit sha 8fba3fd4f8e64c59abe32deed7d433cbaf584141
Merge pull request #309 from A-Hilaly/cleanup Remove `test/integration` folder and `scripts/run-integration-tests.sh`
commit sha 6bc0fc76a7cfa15ff889d0eb1eb4efc3d85afff7
Merge branch 'main' into update-types
commit sha d0ac10fb1ef75ca9c3af7f9df03a0e3f7f1bbcf7
Update crd, rbac for S3
push time in 4 days
Pull request review commentaws/aws-controllers-k8s
fix issues with Get/SetAttributes Go code generation
resources: Topic: unpack_attributes_map:+ set_attributes_single_attribute: true
Yeah, it's an interesting question. I did it this way because this is more of a variation on the "UnpacksAttributesMap" behaviour than it is for anything else. That said, I'm going to be using the OperationConfig.CustomOperation functionality to sidestep the weirdness in the SetTopicAttributes API call entirely for now, so the point may be moot (for the timebeing at least)
comment created time in 4 days
push eventjaypipes/aws-controllers-k8s
commit sha 22b1c9302630006e42bea22946a83e30abc9b842
handle SetAttributes-based update operations This patch adds to the code generator for the SDK linkage for APIs that use the SetAttributes-flavor of updating a resource. Previously, the `sdk.go` files for these APIs had empty `sdkUpdate` operations with a big TODO that ended up breaking the service controller (quite predictably). While implementing this patch, I stumbled upon yet another unexplainable inconsistency between two SetAttributes operations within the SNS service API. The SNS `SetPlatformApplicationAttributes` API call accepts a field in its Input shape called `Attributes` that is a map of key/value pairs for attributes to set on the platform application. Unfortunately, the similarly-named SNS `SetTopicAttributes` API call apparently does *NOT* work the same way. Instead, there is a single `AttributeName` and `AttributeValue` field in the Input shape and you need to call the `SetTopicAttributes` API call once for each modified attribute. :( In fact, the [official documentation][0] for the SNS SetTopicAttributes API call says that the `AttributeName` field in the Input shape is actually a "*map* of attributes with their corresponding values". But that isn't actually the case... So, the code in this patch for now just contains a TODO in the `CRD.GoCodeSetAttributesSetInput()` method for SetAttributes APIs that can only operate on a single attribute at a time. We will use the CustomOperation functionality as a temporary workaround for these APIs while we come up with a more permanent code-generated solution. Issue #296 [0]: https://docs.aws.amazon.com/sns/latest/api/API_SetTopicAttributes.html
push time in 4 days
PR opened aws/aws-controllers-k8s
This patch adds to the code generator for the SDK linkage for APIs that
use the SetAttributes-flavor of updating a resource. Previously, the
sdk.go files for these APIs had empty sdkUpdate operations with a
big TODO that ended up breaking the service controller (quite
predictably).
While implementing this patch, I stumbled upon yet another unexplainable
inconsistency between two SetAttributes operations within the SNS
service API. The SNS SetPlatformApplicationAttributes API call accepts a
field in its Input shape called Attributes that is a map of key/value
pairs for attributes to set on the platform application. Unfortunately,
the similarly-named SNS SetTopicAttributes API call apparently does
NOT work the same way. Instead, there is a single AttributeName and
AttributeValue field in the Input shape and you need to call the
SetTopicAttributes API call once for each modified attribute. :(
In fact, the [official documentation][0] for the SNS SetTopicAttributes
API call says that the AttributeName field in the Input shape is
actually a "map of attributes with their corresponding values". But
that isn't actually the case...
So, the code in this patch for now just contains a TODO in the
CRD.GoCodeSetAttributesSetInput() method for SetAttributes APIs that
can only operate on a single attribute at a time. We will use the
CustomOperation functionality as a temporary workaround for these APIs
while we come up with a more permanent code-generated solution.
There were a series of problems that I uncovered when investigating the cause of Issue #296:
- The code returned from GoCodeGetAttributesForOutput was failing to set
Status.ACKResourceMetadata.ARNandStatus.ACKResourceMetadata.OwnerAccountIDwhen the attribute fields corresponded to the primary resource ARN or the owner ID. This was the direct cause of Issue #296 - The code in
templates/pkg/crd_sdk.go.tplthat ran at the end of thesdkCreate()call was inadvertently overwriting any setters ofStatus.ACKResourceMetadatathat had previously executed - The code returned from APIs with GetAttributes operations was always
returning a nil-guard and constructor for the ACKResourceMetadata
struct, even when no attribute fields actually set either ARN or
OwnerAccountID. When I removed the check in the template for Status
fields being required for the
respvariable to be defined, this was causing"resp" variable unusedcompilation failures
Issue #296
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
pr created time in 4 days
Pull request review commentaws/aws-controllers-k8s
func (r *reconciler) BindControllerManager(mgr ctrlrt.Manager) error { if r.rmf == nil { return ackerr.NilResourceManagerFactory }+ clusterConfig := mgr.GetConfig()+ kc, err := kubernetes.NewForConfig(clusterConfig)+ if err != nil {+ return err+ }+ r.cache = ackrtcache.New(kc, r.log) r.kc = mgr.GetClient()
hmmm. interesting... OK, soon as I'm done with a customer call I'll look into perhaps a cleaner way to do this.
comment created time in 4 days
Pull request review commentaws/aws-controllers-k8s
func (r *reconciler) BindControllerManager(mgr ctrlrt.Manager) error { if r.rmf == nil { return ackerr.NilResourceManagerFactory }+ clusterConfig := mgr.GetConfig()+ kc, err := kubernetes.NewForConfig(clusterConfig)+ if err != nil {+ return err+ }+ r.cache = ackrtcache.New(kc, r.log) r.kc = mgr.GetClient()
Sorry, you misunderstood me :) I was saying that the clientset object is already being created on line 71 with:
r.kc = mgr.GetClient()
so you don't have to create another one with the call to mgr.GetConfig() and thenkubernetes.NewForConfig(). You can just pass the reconciler'skcattribute (which is the kubernetes clientset) to yourackrtcache.New()` function.
comment created time in 4 days
push eventaws/aws-controllers-k8s
commit sha c6dd57d61380b8d2891bdbf897f6b3e53f83b4a8
Enable custom set output operations (#307) * Enable custom set output operations Co-authored-by: Kumar Gaurav Sharma <[email protected]>
push time in 4 days
PR merged aws/aws-controllers-k8s
Issue #282
Description of changes: Added support for custom set output operations using generator.yaml configuration. Example:
operations:
DescribeReplicationGroups:
custom_set_output_operation: CustomDescribeReplicationGroupsSetOutput
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
pr closed time in 4 days
Pull request review commentaws/aws-controllers-k8s
Enable custom set output operations
type Config struct { // specify the overriding values for API operation parameters type OperationConfig struct { OverrideValues map[string]string `json:"override_values"`+ // SetOutputCustomMethodName provides the name of the custom method on the+ // `resourceManager` struct that will set fields on a `resource` struct+ // depending on the output of the operation.
In a future PR, we should consider documenting here the interface / function signature of the SetOutput custom method.
comment created time in 4 days
issue commentaws/aws-controllers-k8s
service needs to be lower case ?
Thanks, @jaypipes - I'll take a gander at #199. I'm not a pro, so may pop-up with questions and can work on it only after-hours, hence may take some time (is this fine)?
Totally fine! :) We're here to help. Alternately, feel free to find me on Kubernetes Slack #provider-aws channel!
comment created time in 4 days
pull request commentaws/aws-controllers-k8s
Enable finegrained diffs, update() args include desired state, latest state
I'm confused why this particular patch is failing make test in the way it is (seems a mockery error).
comment created time in 4 days
Pull request review commentaws/aws-controllers-k8s
Enable finegrained diffs, update() args include desired state, latest state
resources: exceptions: codes: 404: CacheSubnetGroupNotFoundFault- ReplicationGroup:- custom_update_operations:- UpdateShardConfiguration:- diff_paths:- - Spec.NumNodeGroups- UpdateReplicaCount:- diff_paths:- - Spec.ReplicasPerNodeGroup operations: ModifyReplicationGroup:+ custom_implementation: CustomModifyReplicationGroup
:+1:
comment created time in 4 days
push eventaws/aws-controllers-k8s
commit sha 5d899c3f9c9ac966fcdf301fa07e430efc68699c
Update controller-tools to v0.4.0
commit sha 70c14754073055a9763aac28059212ce576ddbac
Update crd, types and auto-generated code
commit sha 6bc0fc76a7cfa15ff889d0eb1eb4efc3d85afff7
Merge branch 'main' into update-types
commit sha d0ac10fb1ef75ca9c3af7f9df03a0e3f7f1bbcf7
Update crd, rbac for S3
commit sha fca149df8cacbd67a5a98acad338c1e6c80f484b
Update types for S3BucketDestination and gen code
commit sha b6f81ac7a08d333885ad5c78a5c3dd9766f38046
Merge pull request #295 from ricardo-larosa/update-types Update crd, types and auto-generated code for S3
push time in 4 days
PR merged aws/aws-controllers-k8s
Fixes https://github.com/aws/aws-controllers-k8s/issues/294
Description of changes:
- Bump to controllers-tools
v0.4.0 - Update crd, types and auto-generated code for S3
pr closed time in 4 days
issue closedaws/aws-controllers-k8s
ValidationError(CustomResourceDefinition.status): missing required field "storedVersions"
Describe the bug CRD definitions are wrong
Steps to reproduce Error when installing S3 CRD
error: error validating "STDIN": error validating data: ValidationError(CustomResourceDefinition.status): missing required field "storedVersions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.CustomResourceDefinitionStatus; if you choose to ignore these errors, turn validation off with --validate=false
Expected outcome CRD Should successfully be applied.
Environment Local
- Kubernetes version 1.16
- Using EKS (yes/no), if so version? Yes, 1.16
- AWS service targeted (S3, RDS, etc.) S3
closed time in 4 days
ricardo-larosa
pull request commentaws/aws-controllers-k8s
Add Github action job to build AWS service controllers
Needs a rebase, @A-Hilaly now that I merged Prateek's patch. Plus, I'm going to need to push the fix for #296 before this will pass on main branch :)
comment created time in 5 days
push eventaws/aws-controllers-k8s
commit sha a5e3cb7da40f295feca6e0f117ed11da3339ebd6
Service names arguments are accepted as case insensitive for make commands
commit sha 68cf1dcb29e7f87f32f1031559d53cc3cc0e6dbb
Merge pull request #306 from prateekgogia/bug/302 Service names arguments are accepted as case insensitive for make commands
push time in 5 days
PR merged aws/aws-controllers-k8s
Issue #302
Description of changes:
Convert service name to lowercase in Makefile and helper script files - build-controllers.sh and build-controller.sh
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
pr closed time in 5 days
push eventaws/aws-controllers-k8s
commit sha a0d2c26ed2ab72e2aa18f95cb5895f01ae3a77ff
Remove `test/integration` folder and `scripts/run-integration-tests.sh`
commit sha 8fba3fd4f8e64c59abe32deed7d433cbaf584141
Merge pull request #309 from A-Hilaly/cleanup Remove `test/integration` folder and `scripts/run-integration-tests.sh`
push time in 5 days
PR merged aws/aws-controllers-k8s
Follow up to #308
Description of changes:
- Remove
test/integrationfolder andscripts/run-integration-tests.sh
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
pr closed time in 5 days
Pull request review commentaws/aws-controllers-k8s
func TestServiceController(t *testing.T) { sc := ackrt.NewServiceController("bookstore", "bookstore.services.k8s.aws") require.NotNil(sc) + zapOptions := ctrlrtzap.Options{+ Development: true,+ Level: zapcore.InfoLevel,+ }+ fakeLogger := ctrlrtzap.New(ctrlrtzap.UseFlagOptions(&zapOptions))
Maybe. Can do later :)
comment created time in 5 days
Pull request review commentaws/aws-controllers-k8s
+// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.+//+// Licensed under the Apache License, Version 2.0 (the "License"). You may+// not use this file except in compliance with the License. A copy of the+// License is located at+//+// http://aws.amazon.com/apache2.0/+//+// or in the "license" file accompanying this file. This file is distributed+// on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either+// express or implied. See the License for the specific language governing+// permissions and limitations under the License.++package cache_test++import (+ "context"+ "testing"+ "time"++ "github.com/stretchr/testify/require"+ "go.uber.org/zap/zapcore"+ corev1 "k8s.io/api/core/v1"+ metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"+ "k8s.io/apimachinery/pkg/watch"+ k8sfake "k8s.io/client-go/kubernetes/fake"+ k8stesting "k8s.io/client-go/testing"+ ctrlrtzap "sigs.k8s.io/controller-runtime/pkg/log/zap"++ ackrtcache "github.com/aws/aws-controllers-k8s/pkg/runtime/cache"+)++const (+ testNamespace = "ack-system"++ testAccount1 = "012345678912"+ testAccountARN1 = "arn:aws:iam::012345678912:role/S3Access"+ testAccount2 = "219876543210"+ testAccountARN2 = "arn:aws:iam::012345678912:role/root"+)++func TestAccountCache(t *testing.T) {+ accountsMap1 := map[string]string{+ testAccount1: testAccountARN1,+ }++ accountsMap2 := map[string]string{+ testAccount1: testAccountARN1,+ testAccount2: testAccountARN2,+ }++ // create a fake k8s client and a fake watcher+ k8sClient := k8sfake.NewSimpleClientset()+ watcher := watch.NewFake()+ k8sClient.PrependWatchReactor("configMaps", k8stesting.DefaultWatchReactor(watcher, nil))++ zapOptions := ctrlrtzap.Options{+ Development: true,+ Level: zapcore.InfoLevel,+ }+ fakeLogger := ctrlrtzap.New(ctrlrtzap.UseFlagOptions(&zapOptions))++ // initlizing account cache+ accountCache := ackrtcache.NewAccountCache(k8sClient, fakeLogger)+ stopCh := make(chan struct{})++ // This is mainly used to avoid using time.Sleep+ accountCache.Run(stopCh)++ // Test create events+ k8sClient.CoreV1().ConfigMaps(testNamespace).Create(+ context.Background(),+ &corev1.ConfigMap{+ ObjectMeta: metav1.ObjectMeta{+ Name: "random-map",+ },+ Data: accountsMap1,+ },+ metav1.CreateOptions{},+ )++ time.Sleep(time.Second)
We can certainly improve this in the future and remove the sleep calls, but this is perfectly fine for now. Nice work on this, @A-Hilaly
comment created time in 5 days
Pull request review commentaws/aws-controllers-k8s
func (r *Registry) GetResourceManagerFactories() []types.AWSResourceManagerFacto return res } -// RegisterManagerFactory registers a resource manager factory with the+// RegisterResourceManagerFactory registers a resource manager factory with the
thanks :)
comment created time in 5 days
Pull request review commentaws/aws-controllers-k8s
func (r *reconciler) BindControllerManager(mgr ctrlrt.Manager) error { if r.rmf == nil { return ackerr.NilResourceManagerFactory }+ clusterConfig := mgr.GetConfig()+ kc, err := kubernetes.NewForConfig(clusterConfig)+ if err != nil {+ return err+ }+ r.cache = ackrtcache.New(kc, r.log) r.kc = mgr.GetClient()
Any reason why you did the above instead of just:
r.kc = mgr.GetClient()
r.cache = ackrtcache.New(r.kc, r.log)
comment created time in 5 days
Pull request review commentaws/aws-controllers-k8s
+// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.+//+// Licensed under the Apache License, Version 2.0 (the "License"). You may+// not use this file except in compliance with the License. A copy of the+// License is located at+//+// http://aws.amazon.com/apache2.0/+//+// or in the "license" file accompanying this file. This file is distributed+// on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either+// express or implied. See the License for the specific language governing+// permissions and limitations under the License.++package cache++import (+ "sync"++ "github.com/go-logr/logr"+ corev1 "k8s.io/api/core/v1"+ informersv1 "k8s.io/client-go/informers/core/v1"+ kubernetes "k8s.io/client-go/kubernetes"+ k8scache "k8s.io/client-go/tools/cache"+)++const (+ // ACKRoleAccountMap is the name of the configmap map object storing+ // all the AWS Account IDs associated with their AWS Role ARNs.+ ACKRoleAccountMap = "ack-role-account-map"+)++// AccountCache is responsible for caching the CARM configmap+// data. It is listening to all the events related to the CARM map and+// make the changes accordingly.+type AccountCache struct {+ sync.RWMutex++ log logr.Logger++ // ConfigMap informer+ informer k8scache.SharedInformer+ roleARNs map[string]string+}++// NewAccountCache makes a new AccountCache from a client.Interface+// and a logr.Logger+func NewAccountCache(clientset kubernetes.Interface, log logr.Logger) *AccountCache {+ sharedInformer := informersv1.NewConfigMapInformer(+ clientset,+ currentNamespace,+ informerResyncPeriod,+ k8scache.Indexers{},+ )+ return &AccountCache{+ informer: sharedInformer,+ log: log.WithName("AccountCache"),+ roleARNs: make(map[string]string),+ }+}++// resourceMatchACKRoleAccountConfigMap verifies if a resource is+// the CARM configmap. It verifies the name, namespace and object type.+func resourceMatchACKRoleAccountsConfigMap(raw interface{}) bool {+ object, ok := raw.(*corev1.ConfigMap)+ return ok && object.ObjectMeta.Name == ACKRoleAccountMap+}++// Run adds the default event handler functions to the SharedInformer and+// runs the informer to begin processing items.+func (c *AccountCache) Run(stopCh <-chan struct{}) {+ c.informer.AddEventHandler(k8scache.ResourceEventHandlerFuncs{+ AddFunc: func(obj interface{}) {+ if resourceMatchACKRoleAccountsConfigMap(obj) {+ object := obj.(*corev1.ConfigMap).DeepCopy()+ c.log.V(1).Info("ack-role-account-map has been created")+ c.updateAccountRoleData(object.Data)+ c.log.V(1).Info("cached ack-role-account-map data")+ }+ },+ UpdateFunc: func(old, new interface{}) {+ if resourceMatchACKRoleAccountsConfigMap(new) {+ object := new.(*corev1.ConfigMap).DeepCopy()+ c.log.V(1).Info("ack-role-account-map has been updated")+ //TODO(a-hilaly): compare data checksum before updating the cache+ c.updateAccountRoleData(object.Data)+ c.log.V(1).Info("cached ack-role-account-map data")+ }+ },+ DeleteFunc: func(obj interface{}) {+ if resourceMatchACKRoleAccountsConfigMap(obj) {+ c.log.V(1).Info("ack-role-account-map has been deleted")+ newMap := make(map[string]string)+ c.updateAccountRoleData(newMap)+ c.log.V(1).Info("cleaned up role account map")+ }+ },+ })+ go c.informer.Run(stopCh)+}++// GetAccountRoleARN queries the AWS accountID associated Role ARN+// from the cached CARM configmap. This function is thread safe.+func (c *AccountCache) GetAccountRoleARN(accountID string) (string, bool) {+ c.RLock()+ defer c.RUnlock()+ roleARN, ok := c.roleARNs[accountID]+ return roleARN, ok
:+1:
comment created time in 5 days
Pull request review commentaws/aws-controllers-k8s
+// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.+//+// Licensed under the Apache License, Version 2.0 (the "License"). You may+// not use this file except in compliance with the License. A copy of the+// License is located at+//+// http://aws.amazon.com/apache2.0/+//+// or in the "license" file accompanying this file. This file is distributed+// on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either+// express or implied. See the License for the specific language governing+// permissions and limitations under the License.++package cache++import (+ "sync"++ "github.com/go-logr/logr"+ corev1 "k8s.io/api/core/v1"+ informersv1 "k8s.io/client-go/informers/core/v1"+ kubernetes "k8s.io/client-go/kubernetes"+ k8scache "k8s.io/client-go/tools/cache"+)++const (+ // ACKRoleAccountMap is the name of the configmap map object storing+ // all the AWS Account IDs associated with their AWS Role ARNs.+ ACKRoleAccountMap = "ack-role-account-map"+)++// AccountCache is responsible for caching the CARM configmap+// data. It is listening to all the events related to the CARM map and+// make the changes accordingly.+type AccountCache struct {+ sync.RWMutex++ log logr.Logger++ // ConfigMap informer+ informer k8scache.SharedInformer+ roleARNs map[string]string+}++// NewAccountCache makes a new AccountCache from a client.Interface+// and a logr.Logger+func NewAccountCache(clientset kubernetes.Interface, log logr.Logger) *AccountCache {+ sharedInformer := informersv1.NewConfigMapInformer(+ clientset,+ currentNamespace,+ informerResyncPeriod,+ k8scache.Indexers{},+ )+ return &AccountCache{+ informer: sharedInformer,+ log: log.WithName("AccountCache"),+ roleARNs: make(map[string]string),+ }+}++// resourceMatchACKRoleAccountConfigMap verifies if a resource is+// the CARM configmap. It verifies the name, namespace and object type.+func resourceMatchACKRoleAccountsConfigMap(raw interface{}) bool {+ object, ok := raw.(*corev1.ConfigMap)+ return ok && object.ObjectMeta.Name == ACKRoleAccountMap+}++// Run adds the default event handler functions to the SharedInformer and+// runs the informer to begin processing items.+func (c *AccountCache) Run(stopCh <-chan struct{}) {+ c.informer.AddEventHandler(k8scache.ResourceEventHandlerFuncs{+ AddFunc: func(obj interface{}) {+ if resourceMatchACKRoleAccountsConfigMap(obj) {+ object := obj.(*corev1.ConfigMap).DeepCopy()+ c.log.V(1).Info("ack-role-account-map has been created")+ c.updateAccountRoleData(object.Data)+ c.log.V(1).Info("cached ack-role-account-map data")+ }+ },+ UpdateFunc: func(old, new interface{}) {
teeny nit: new is a keyword in Go and naming variables the same as keywords (something called "variable shadowing" ) is generally discouraged. Might be better to name these orig and desired?
comment created time in 5 days
pull request commentaws/aws-controllers-k8s
Remove petstore and bookstore integration tests
and the scripts/run_integration_tests.sh script :)
comment created time in 5 days
pull request commentaws/aws-controllers-k8s
Remove petstore and bookstore integration tests
You can probably now remove the entire test/integration directory :)
comment created time in 5 days
push eventaws/aws-controllers-k8s
commit sha 34ed5dae3dcd8ed01d12e0a60f3d40141d6a2429
Remove petstore and bookstore integration tests
commit sha c4bc50cdd4109086e2fed41d912072e75adeab99
Merge pull request #308 from A-Hilaly/cleanup Remove petstore and bookstore integration tests
push time in 5 days
PR merged aws/aws-controllers-k8s
Follow up to #305
Description of changes:
- Remove
petstoreandbookstoreintegration tests
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
pr closed time in 5 days
Pull request review commentaws/aws-controllers-k8s
Enable custom set output operations
func (rm *resourceManager) sdkUpdate( {{ end }} {{ $setCode := GoCodeSetUpdateOutput .CRD "resp" "ko.Status" 1 }}+{{ $setCustomOutputOperation := .CRD.GetCustomSetOutputOperation .CRD.Ops.Update }}
ditto.
comment created time in 5 days
Pull request review commentaws/aws-controllers-k8s
Enable custom set output operations
func (rm *resourceManager) sdkFind( return nil, err } {{ $setCode := GoCodeSetReadManyOutput .CRD "resp" "ko" 1 }}+{{ $setCustomOutputOperation := .CRD.GetCustomSetOutputOperation .CRD.Ops.ReadMany }}
any reason to place this here instead of below where it is used?
comment created time in 5 days
Pull request review commentaws/aws-controllers-k8s
Enable custom set output operations
func (rm *resourceManager) sdkCreate( return nil, err } {{ $createCode := GoCodeSetCreateOutput .CRD "resp" "ko.Status" 1 }}+{{ $setCustomOutputOperation := .CRD.GetCustomSetOutputOperation .CRD.Ops.Create }}
ditto...
comment created time in 5 days
Pull request review commentaws/aws-controllers-k8s
Enable custom set output operations
func (r *CRD) IsPrimaryARNField(fieldName string) bool { strings.EqualFold(fieldName, r.Names.Original+"arn") } +// GetCustomSetOutputOperation returns custom set output operation as string for+// given operation on custom resource, if specified in generator config+func (r *CRD) GetCustomSetOutputOperation(
You could just call this method SetOutputCustomMethodName()
Additionally, consider making the attribute a *string instead of string and returning nil here instead of "" to indicate there is no custom method name...
comment created time in 5 days
Pull request review commentaws/aws-controllers-k8s
Enable custom set output operations
type Config struct { // specify the overriding values for API operation parameters type OperationConfig struct { OverrideValues map[string]string `json:"override_values"`+ // CustomSetOutputOperation provides custom operation to process operation's output shape.+ CustomSetOutputOperation string `json:"custom_set_output_operation,omitempty"`
Probably just call this SetOutputCustomMethod or SetOutputCustomMethodName and change the description to say:
// SetOutputCustomMethod provides the name of the custom method on the
// `resourceManager` struct that will set fields on a `resource` struct
// depending on the output of the operation.
comment created time in 5 days
pull request commentaws/aws-controllers-k8s
Update crd, types and auto-generated code for S3
Hi @ricardo-larosa, thanks very much for the PR! Any chance you might be able to rebase to main?
comment created time in 5 days
issue commentaws/aws-controllers-k8s
service needs to be lower case ?
Kind of similar problem to this one...
comment created time in 5 days
issue commentaws/aws-controllers-k8s
service needs to be lower case ?
Can I take a gander at this?
@ChaturvediSulabh looks like @pgogia beat you to it :)
Are you looking for a place to contribute? How about this one? https://github.com/aws/aws-controllers-k8s/issues/199
comment created time in 5 days
issue commentaws/aws-controllers-k8s
Handle S3 OperationAborted due to conflicting conditions
Is this the right direction with the changes to the generatorConfig? Wanted to be clear that "error codes" doesn't mean HTTP codes in this case - there are multiple 409's you can get when create a bucket and I'm assuming you wouldn't want to requeue for
BucketAlreadyExists.resources: Bucket: ... exceptions: requeue: - OperationAborted
Yeah, I realized way too late that S3 Bucket names are globally unique, and that OperationAborted: A conflicting condition operation... is returned when trying to Create an S3 Bucket with the same name as one currently being deleted.
This will need to be handled quite carefully; I'm still thinking about ways we can instruct the code generator about these types of weirdnesses.
comment created time in 5 days
Pull request review commentaws/aws-controllers-k8s
Service names arguments are accepted as case insensitive for make commands
test: | mocks ## Run code tests clean-mocks: ## Remove mocks directory rm -rf mocks -build-controller-image: ## Build container image for SERVICE- ./scripts/build-controller-image.sh -s $(SERVICE)+build-controller-image: ## Build container image for AWS_SERVICE+ ./scripts/build-controller-image.sh -s $(AWS_SERVICE) publish-controller-image: ## docker push a container image for SERVICE @echo $(DOCKER_PASSWORD) | docker login -u $(DOCKER_USERNAME) --password-stdin- ./scripts/publish-controller-image.sh -r $(DOCKER_REPOSITORY) -s $(SERVICE)+ ./scripts/publish-controller-image.sh -r $(DOCKER_REPOSITORY) -s $(AWS_SERVICE) -build-controller: build-ack-generate ## Generate controller code for SERVICE- ./scripts/build-controller.sh $(SERVICE)+build-controller: build-ack-generate ## Generate controller code for AWS_SERVICE+ ./scripts/build-controller.sh $(AWS_SERVICE) -kind-test: test ## Run functional tests for SERVICE with AWS_ROLE_ARN- ./scripts/kind-build-test.sh -s $(SERVICE) -p -r $(AWS_ROLE_ARN)+kind-test: test ## Run functional tests for AWS_SERVICE with AWS_ROLE_ARN
ditto
comment created time in 5 days
Pull request review commentaws/aws-controllers-k8s
Service names arguments are accepted as case insensitive for make commands
test: | mocks ## Run code tests clean-mocks: ## Remove mocks directory rm -rf mocks -build-controller-image: ## Build container image for SERVICE- ./scripts/build-controller-image.sh -s $(SERVICE)+build-controller-image: ## Build container image for AWS_SERVICE+ ./scripts/build-controller-image.sh -s $(AWS_SERVICE) publish-controller-image: ## docker push a container image for SERVICE @echo $(DOCKER_PASSWORD) | docker login -u $(DOCKER_USERNAME) --password-stdin- ./scripts/publish-controller-image.sh -r $(DOCKER_REPOSITORY) -s $(SERVICE)+ ./scripts/publish-controller-image.sh -r $(DOCKER_REPOSITORY) -s $(AWS_SERVICE) -build-controller: build-ack-generate ## Generate controller code for SERVICE- ./scripts/build-controller.sh $(SERVICE)+build-controller: build-ack-generate ## Generate controller code for AWS_SERVICE
ditto
comment created time in 5 days
Pull request review commentaws/aws-controllers-k8s
Service names arguments are accepted as case insensitive for make commands
test: | mocks ## Run code tests clean-mocks: ## Remove mocks directory rm -rf mocks -build-controller-image: ## Build container image for SERVICE- ./scripts/build-controller-image.sh -s $(SERVICE)+build-controller-image: ## Build container image for AWS_SERVICE
Need to change this back to SERVICE since this is output for make help (and the variable to set for the user is SERVICE, not AWS_SERVICE)
comment created time in 5 days
push eventaws/aws-controllers-k8s
commit sha 21627a478beb45e815e8cbab299ab1bd41aa650f
remove petstore and bookstore services
commit sha 5b3f2e73790e5cdc5ea678aca9e7dbed02d3adb3
Add a fake Book runtime.Object implementation for `service_controller_test.go` Use `mocks` for runtime util testing
commit sha 4546ae0cf06e81c2557b30c1fc7ff08422ecf544
Merge pull request #305 from A-Hilaly/cleanup Remove petstore and bookstore services
push time in 5 days
PR merged aws/aws-controllers-k8s
Issue #, if available: N/A
Description of changes:
- Delete
services/petstoreandservices/bookstorefolders - Use a fake
Bookruntime.Objectimplementation inpkg/runtime/service_controller_test.go
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
pr closed time in 5 days
Pull request review commentaws/aws-controllers-k8s
Enable finegrained diffs, update() args include desired state, latest state
func (rm *resourceManager) UpdateReplicaCount( if respErr != nil { return nil, respErr }- return provideUpdatedResource(r, resp.ReplicationGroup)+ return provideUpdatedResource(rd, resp.ReplicationGroup) } // newUpdate(ShardConfiguration)RequestPayload returns an SDK-specific struct for the HTTP request // payload of the Update API call for the resource func (rm *resourceManager) newUpdateShardConfigurationRequestPayload(- r *resource,+ rd *resource, // desired+ rl *resource, // latest
actually, I was originally confused why you were passing the resourceDescriptor but then realized that you were just calling the "desired" resource parameter rd. Recommend calling these parameters desired and latest :) We use "rd" to indicate the resourceDescriptor struct
comment created time in 5 days
Pull request review commentaws/aws-controllers-k8s
Enable finegrained diffs, update() args include desired state, latest state
func (rm *resourceManager) newCreateRequestPayload( // observed and decides whether to call the resource manager's Update method func (rm *resourceManager) Update( ctx context.Context,- res acktypes.AWSResource,+ resDesired acktypes.AWSResource, // desired+ resLatest acktypes.AWSResource, // latest
FYI, we're getting rid of the petstore and bookstore services here:
https://github.com/aws/aws-controllers-k8s/pull/305
So you won't need this after you rebase.
comment created time in 5 days
push eventaws/aws-controllers-k8s
commit sha be5f830c4e69ff7eda22c3a2040273f4e3709188
`ack-generate` calls `git pull` if aws-sdk-go repository already exists
commit sha aaffb681058597dd265c9117dcc2a1513fb28ab7
Merge pull request #304 from A-Hilaly/refresh-cache `ack-generate` call `git pull` if aws-sdk-go repository already exist in the cache
push time in 6 days
PR merged aws/aws-controllers-k8s
Issue #291
Description of changes:
ack-generate now will call git pull if aws-sdk-go repository already exist in the cache
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
pr closed time in 6 days
issue commentaws/aws-controllers-k8s
Hi @sebgoa! Yeah, I'm working on the fix for this here: #296. Should have a fix pushed within a few hours. Sorry for the delay!
comment created time in 6 days
PR opened aws/aws-controllers-k8s
Recent PR changed the AWSDescriptor interface signature and this PR simply updates the API Gateway V2 service controller to abide by that new interface.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
pr created time in 9 days
push eventjaypipes/ghw
commit sha d3780e365eade533674c295d1ede5a7099d84f53
Determine total physical memory from sysfs for linux as primary method.
commit sha 75d641ea8e0d6e71b75c8dbc655740fe9927e65e
Merge pull request #198 from cclerget/physical-mem-from-sysfs Determine total physical memory from sysfs for linux as primary method
push time in 9 days
PR merged jaypipes/ghw
This PR allows to first determine on Linux systems the total of physical memory from sysfs (/sys/devices/system/memory) as primary method and fallback to syslog file method if /sys/devices/system/memory is not present for some reasons
pr closed time in 9 days
Pull request review commentjaypipes/ghw
Determine total physical memory from sysfs for linux as primary method
func (i *Info) load() error { return nil } -func memTotalPhysicalBytes(paths *linuxpath.Paths) int64 {+func memTotalPhysicalBytes(paths *linuxpath.Paths) (total int64) {+ defer func() {+ // fallback to the syslog file approach in case of error+ if total < 0 {+ total = memTotalPhysicalBytesFromSyslog(paths)+ }+ }()++ // detect physical memory from /sys/devices/system/memory+ dir := paths.SysDevicesSystemMemory++ // get the memory block size in byte in hexadecimal notation+ blockSize := filepath.Join(dir, "block_size_bytes")++ d, err := ioutil.ReadFile(blockSize)+ if err != nil {+ return -1+ }+ blockSizeBytes, err := strconv.ParseUint(strings.TrimSpace(string(d)), 16, 64)+ if err != nil {+ return -1+ }++ // iterate over memory's block /sys/devices/system/memory/memory*,+ // if the memory block state is 'online' we increment the total+ // with the memory block size to determine the amount of physical+ // memory available on this system+ sysMemory, err := filepath.Glob(filepath.Join(dir, "memory*"))+ if err != nil {+ return -1+ } else if sysMemory == nil {+ return -1+ }++ for _, path := range sysMemory {+ s, err := ioutil.ReadFile(filepath.Join(path, "state"))+ if err != nil {+ return -1+ }+ if strings.TrimSpace(string(s)) != "online" {+ continue+ }+ total += int64(blockSizeBytes)+ }++ return total+}
Very, very nice. :+1:
comment created time in 9 days
push eventaws/aws-controllers-k8s
commit sha 28eabf6df523210024f7fdcd520e5a576b88d91b
Enable multiple update operations support on single resource (#245) Issue #210 Description of changes: The changes in this PR are to enable custom resource update scenario where Update operation on given resource does not support fields that are supported by its Create method, and there exist specialized methods to update such fields. These specialized method response provides the Resource object indicating the Update on the resource. For example: create-replication-group API arguments include: num-node-groups replicas-per-node-group however, modify-replication-group API arguments does not, instead modify-replication-group-shard-configuration and increase-replica-count, decrease-replica-count APIs corresponds to the related Update. The approach taken in this PR is to add the details of such fields and corresponding custom update methods in the generator.yaml. For example: resources: ReplicationGroup: custom_update_operations: UpdateShardConfiguration: fields: - NumNodeGroups UpdateReplicaCount: fields: - ReplicasPerNodeGroup And custom update methods' (UpdateShardConfiguration and UpdateReplicaCount) implementation is provided in a separate code file: aws-controllers-k8s/services/elasticache/pkg/resource/replication_group/custom_update_api.go These methods are supplied with DiffReporter which helps the code in this file determine the specific Update API to invoke based on the difference in desired, current resource details. Co-authored-by: Kumar Gaurav Sharma <[email protected]>
push time in 9 days
PR merged aws/aws-controllers-k8s
Issue #210
Description of changes: The changes in this PR are to enable custom resource update scenario where Update operation on given resource does not support fields that are supported by its Create method, and there exist specialized methods to update such fields. These specialized method response provides the Resource object indicating the Update on the resource.
For example: create-replication-group API arguments include:
num-node-groups
replicas-per-node-group
however, modify-replication-group API arguments does not, instead modify-replication-group-shard-configuration and increase-replica-count, decrease-replica-count APIs corresponds to the related Update.
The approach taken in this PR is to add the details of such fields and corresponding custom update methods in the generator.yaml. For example:
resources:
ReplicationGroup:
custom_update_operations:
UpdateShardConfiguration:
fields:
- NumNodeGroups
UpdateReplicaCount:
fields:
- ReplicasPerNodeGroup
And custom update methods' (UpdateShardConfiguration and UpdateReplicaCount) implementation is provided in a separate code file: aws-controllers-k8s/services/elasticache/pkg/resource/replication_group/custom_update_api.go
These methods are supplied with DiffReporter which helps the code in this file determine the specific Update API to invoke based on the difference in desired, current resource details.
Auto generating following:
make build-controller SERVICE=sns
make build-controller SERVICE=ecr
make build-controller SERVICE=s3
and after updating bookstore, petstore services files.
make test passed.
Changes include autogenerated sns, ect, s3, and manually updated bookstore, petstore service files per the latest APIs.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
pr closed time in 9 days
push eventaws/aws-controllers-k8s
commit sha 3a3a504137fdc1874a8d54c645dcb21156cc0286
apigatewayv2 service controller and smoke test files. (#293) * apigatewayv2 service controller and smoke test files.
push time in 9 days