profile
viewpoint
Nick Law nicklaw5 @nib-health-funds Newcastle, Australia https://www.linkedin.com/in/nicholas-law-635510165/ Software & DevOps Engineer - Go, PHP, Node, JavaScript, TypeScript, MySQL, PostgreSQL, AWS and anything cloud hosted.

nicklaw5/go-respond 35

A Go package for handling common HTTP JSON responses.

chatstatz/cluster 0

For managing the Kubernetes cluster.

chatstatz/docs 0

General documentation for the Chatstatz project.

chatstatz/gcp-iam 0

For managing GCP IAM configurations.

chatstatz/gcp-kms 0

For managing cloud cryptographic keys.

chatstatz/logger 0

A simple logging interface for Go.

chatstatz/tfstate 0

This repo is responsible for creating and configuring the GCS bucket for managing Terraform state.

chatstatz/util 0

This package provides utility support for chatstatz applications.

chatstatz/webhooks 0

The webhooks server for chatstatz

nicklaw5/aba-file-generator 0

PHP library to generate aba files for performing bulk transactions with Australian and New Zealand banks.

Pull request review commentnicklaw5/twitch-api-php

Add CS check to `make test`. Should run in CI now.

 .SILENT: ; -test: test-phpunit test-phpspec+test: cs-check test-phpunit test-phpspec++cs-check:+	vendor/bin/php-cs-fixer fix --dry-run --stop-on-violation

May I suggest the following instead?

	vendor/bin/php-cs-fixer fix --diff --diff-format=udiff --dry-run

This will prevent it stopping on the first violation and it will print out a diff of each violation.

Running this against another project of mine, looks like the following:

$ php-cs-fixer fix --diff --diff-format=udiff --dry-run ./src
Using cache file ".php_cs.cache".
Paths from configuration file have been overridden by paths provided as command arguments.
   1) src/Migrations/Version20200330175548.php
      ---------- begin diff ----------
--- Original
+++ New
@@ -12,23 +12,23 @@
  */
 final class Version20200330175548 extends AbstractMigration
 {
-    public function getDescription() : string
+    public function getDescription(): string
     {
         return 'Drop completed_at column from employer_user_invitations';
     }
 
-    public function up(Schema $schema) : void
+    public function up(Schema $schema): void
     {
         // this up() migration is auto-generated, please modify it to your needs
-        $this->abortIf($this->connection->getDatabasePlatform()->getName() !== 'postgresql', 'Migration can only be executed safely on \'postgresql\'.');
+        $this->abortIf('postgresql' !== $this->connection->getDatabasePlatform()->getName(), 'Migration can only be executed safely on \'postgresql\'.');
 
         $this->addSql('ALTER TABLE employer_user_invitations DROP completed_at');
     }
 
-    public function down(Schema $schema) : void
+    public function down(Schema $schema): void
     {
         // this down() migration is auto-generated, please modify it to your needs
-        $this->abortIf($this->connection->getDatabasePlatform()->getName() !== 'postgresql', 'Migration can only be executed safely on \'postgresql\'.');
+        $this->abortIf('postgresql' !== $this->connection->getDatabasePlatform()->getName(), 'Migration can only be executed safely on \'postgresql\'.');
 
         $this->addSql('ALTER TABLE employer_user_invitations ADD completed_at TIMESTAMP(0) WITHOUT TIME ZONE DEFAULT NULL');
     }

      ----------- end diff -----------

   2) src/Migrations/Version20200330183847.php
      ---------- begin diff ----------
--- Original
+++ New
@@ -12,23 +12,23 @@
  */
 final class Version20200330183847 extends AbstractMigration
 {
-    public function getDescription() : string
+    public function getDescription(): string
     {
         return 'Drop multi-column index for columns employer_id and email on employer_user_invitations';
     }
 
-    public function up(Schema $schema) : void
+    public function up(Schema $schema): void
     {
         // this up() migration is auto-generated, please modify it to your needs
-        $this->abortIf($this->connection->getDatabasePlatform()->getName() !== 'postgresql', 'Migration can only be executed safely on \'postgresql\'.');
+        $this->abortIf('postgresql' !== $this->connection->getDatabasePlatform()->getName(), 'Migration can only be executed safely on \'postgresql\'.');
 
         $this->addSql('DROP INDEX uniq_945ab10241cd9e7ae7927c74');
     }
 
-    public function down(Schema $schema) : void
+    public function down(Schema $schema): void
     {
         // this down() migration is auto-generated, please modify it to your needs
-        $this->abortIf($this->connection->getDatabasePlatform()->getName() !== 'postgresql', 'Migration can only be executed safely on \'postgresql\'.');
+        $this->abortIf('postgresql' !== $this->connection->getDatabasePlatform()->getName(), 'Migration can only be executed safely on \'postgresql\'.');
 
         $this->addSql('CREATE UNIQUE INDEX uniq_945ab10241cd9e7ae7927c74 ON employer_user_invitations (employer_id, email)');
     }

      ----------- end diff -----------

   3) src/Services/EmailChangeRequestManager.php
      ---------- begin diff ----------
--- Original
+++ New
@@ -45,7 +45,7 @@
             $this->em->flush();
         } catch (\Throwable $th) {
             $this->logger->error('Uneble to create new email change request: failed to remove pending email change requests', [
-                'user'=> $user->getId(),
+                'user' => $user->getId(),
             ]);
             throw $th;
         }
@@ -62,7 +62,7 @@
             }
         } catch (\Throwable $th) {
             $this->logger->error('Uneble to create new email change request: failed to remove existing employer invites', [
-                'user'=> $user->getId(),
+                'user' => $user->getId(),
                 'new_email' => $newEmail,
                 'employer_user_invitation' => $existingEmployerInvite->getId(),
             ]);
@@ -93,10 +93,9 @@
                 'bodyHtml' => $this->templating->render('emails/verify-email-change-request.html.twig', $emailParams),
                 'bodyText' => $this->templating->render('emails/verify-email-change-request.txt.twig', $emailParams),
             ]);
-
         } catch (\Throwable $th) {
             $this->logger->error('Uneble to create new email change request', [
-                'user'=> $user->getId(),
+                'user' => $user->getId(),
                 'new_email' => $newEmail,
             ]);
             throw $th;

      ----------- end diff -----------

   4) src/Services/EmployerManager.php
      ---------- begin diff ----------
--- Original
+++ New
@@ -20,6 +20,7 @@
         $this->twig = $twig;
         $this->queueManager = $queueManager;
     }
+
     /**
      * @throws \InvalidArgumentException
      */
@@ -44,6 +45,5 @@
             'bodyHtml' => $this->twig->render('emails/employer-invite.html.twig', ['invitation' => $invitation]),
             'bodyText' => $this->twig->render('emails/employer-invite.txt.twig', ['invitation' => $invitation]),
         ]);
-
     }
 }

      ----------- end diff -----------

   5) src/Repository/UserRepository.php
      ---------- begin diff ----------
--- Original
+++ New
@@ -94,7 +94,7 @@
     {
         $conn = $this->getEntityManager()->getConnection();
 
-        $sql = "DELETE FROM sessions WHERE sess_data LIKE :email";
+        $sql = 'DELETE FROM sessions WHERE sess_data LIKE :email';
         $stmt = $conn->prepare($sql);
         $stmt->execute(['email' => sprintf('%%%s%%', $user->getEmail())]);
     }

      ----------- end diff -----------

   6) src/Controller/Api/Employer/UsersController.php
      ---------- begin diff ----------
--- Original
+++ New
@@ -7,8 +7,8 @@
 use App\Entity\EmployerUserInvitation;
 use App\Entity\User;
 use App\Enum\EmployerUserState;
+use App\Form\Employer\InviteEmployerUserFormType;
 use App\Form\Employer\UpdateEmployerUserFormType;
-use App\Form\Employer\InviteEmployerUserFormType;
 use App\Repository\EmployerUserInvitationRepository;
 use App\Repository\UserRepository;
 use App\Services\EmailChangeRequestManager;

      ----------- end diff -----------


Checked all files in 0.128 seconds, 14.000 MB memory used
echosa

comment created time in 16 days

PR closed nicklaw5/twitch-api-php

Added doc blocks & switched out private where redundant

Spoken about in #41

+113 -42

1 comment

9 changed files

b3none

pr closed time in 16 days

pull request commentnicklaw5/twitch-api-php

Added doc blocks & switched out private where redundant

Stale. Please re-open once you've brought this branch up-to-date with recent changes to the repo.

b3none

comment created time in 16 days

pull request commentnicklaw5/twitch-api-php

Update New API Endpoints to force bearer token

Good work folks :clap:

Sorry, I wasn't able to assist. It's been a busy week for me.

brandinarsenault

comment created time in 16 days

startedcontainous/traefik

started time in 21 days

startedRichiCoder1/vault-action

started time in 22 days

pull request commentnicklaw5/twitch-api-php

Update New API Endpoints to force bearer token

@nicklaw5 would you be opposed to adding two environment variables to the Travis CI? You would generate the Client ID and Secret for it on the Twitch Dev Console

To be honest, I don't want to support integration tests. They are unnecessary for a library of this nature. And again we come back to the issue where I am the only person that has access to removing/replacing the tokens as I am the only one able to access Travis CI.

We simply need to refactor the unit tests. Not a lot of work really. I already started looking at it yesterday.

If you're done making changes, and if @echosa supports it also, I'm happy to merge in its current state.

brandinarsenault

comment created time in 23 days

pull request commentnicklaw5/twitch-api-php

Update New API Endpoints to force bearer token

All Travis CI is doing is calling make test, which you can do locally also. It simply calls make test on multiple versions of PHP to validate those versions are still supported. There is no special sauce.

Happy to consider other CI tools if you have a preference. GitHub Actions may be more suitable now, and it will allow others to manage the CI workflow rather than just myself.

As far as I can see, the mock that we're using may not be working, which means its trying to actually hit the Twitch API. I feel we need to refactor our tests.

@brandinarsenault please rebase you branch onto master to pull in the changes from #61 and make a best-effort approach to resolving tests. If you're unable to resolve the tests, I'm happy to merge the PR and either @echosa or myself can resolve the tests in a separate PR. Mind you we won't be releasing a new version until tests resolve.

brandinarsenault

comment created time in 23 days

delete branch nicklaw5/twitch-api-php

delete branch : php-7.4-support

delete time in 23 days

push eventnicklaw5/twitch-api-php

Brian Zwahr

commit sha 6edc470478b13d2ed55b9d74248dd8e0dcae7f69

Add support for PHP 7.4, drop support for 7.1 (#61) * Update composer.json for PHP 7.4 * Drop support for PHP 7.1, add support for 7.3 and 7.4 Co-authored-by: Nicholas Law <nick@nicholaslaw.com.au>

view details

push time in 23 days

PR merged nicklaw5/twitch-api-php

Update composer.json for PHP 7.4

As discovered in https://github.com/nicklaw5/twitch-api-php/pull/60#issuecomment-642774007 this library doesn't currently work with PHP 7.4. This PR does what I believe to be the bare minimum to fix that.

Relates to #60

+1352 -559

5 comments

5 changed files

echosa

pr closed time in 23 days

pull request commentnicklaw5/twitch-api-php

Update composer.json for PHP 7.4

Why does 7.1 need to be dropped?

Because phpunit v8.x only supports PHP 7.2 or higher. And if we were to use phpunit v7.x, it only supports up to PHP 7.3.

echosa

comment created time in 23 days

pull request commentnicklaw5/twitch-api-php

Update composer.json for PHP 7.4

@echosa Any thoughts before I merge this?

echosa

comment created time in 24 days

pull request commentnicklaw5/twitch-api-php

Update composer.json for PHP 7.4

Unfortunately, we need to drop support for PHP 7.1 if we want support for 7.4. I've added a commit to this PR to that effect.

It does mean we have to bump this library to v3 as well.

echosa

comment created time in 24 days

push eventnicklaw5/twitch-api-php

Nicholas Law

commit sha 7a7c5d22668ab002f4fbf50fa2837c63500ce853

Drop support for PHP 7.1, add support for 7.3 and 7.4

view details

push time in 24 days

pull request commentnicklaw5/twitch-api-php

Update composer.json for PHP 7.4

So that actually not going to work. phpunit v8 requires PHP 7.3 or higher, which means we lose support for 7.1 and 7.2 and that is not ideal.

I'll take a closer look at a better solution.

echosa

comment created time in 24 days

pull request commentnicklaw5/twitch-api-php

Update New API Endpoints to force bearer token

I probably won't be able to take a look until the weekend. Unless @echosa has time before then.

brandinarsenault

comment created time in 25 days

pull request commentnicklaw5/twitch-api-php

Update New API Endpoints to force bearer token

Yep - looks like we just need to update the tests.

Both getUserById and getUserByUsername, as of this PR, now accepts a minimum of 2 arguments.

Should just be a matter of passing additional arguments here and here. The response is already mocked, so that should be all that's needed.

brandinarsenault

comment created time in 25 days

delete branch nicklaw5/tonic

delete branch : patch-1

delete time in a month

startedopen-guides/og-aws

started time in a month

PR opened hyperium/tonic

chore: fix incorrect proto path in helloworld tutorial

This change reverts #366. The proto/ directory, as described in the guide, is created under the root of the crate. Not under src/.

I've also included a fix to the grpcurl endpoint.

+2 -2

0 comment

1 changed file

pr created time in a month

push eventnicklaw5/tonic

Nick Law

commit sha 172669e65d95d4a459f14d0ba20a6adb41540738

chore: fix incorrect proto path in helloworld tutorial This change reverts #366. The `proto/` directory, as described in the guide, is created under the root of the crate. Not under `src/`. I've also included a fix to the `grpcurl` endpoint.

view details

push time in a month

fork nicklaw5/tonic

A native gRPC client & server implementation with async/await support.

https://docs.rs/tonic

fork in a month

issue closednicklaw5/twitch-api-php

OAuth for Helix endpoints

As of May 1st, it appears Twitch is requiring OAuth authentication for their Helix endpoints. Is there any plan for this project to support that?

https://discuss.dev.twitch.tv/t/requiring-oauth-for-helix-twitch-api-endpoints/23916

closed time in a month

keithslater

issue commentnicklaw5/twitch-api-php

OAuth for Helix endpoints

Duplicate of #55

keithslater

comment created time in a month

startedgrpc-ecosystem/awesome-grpc

started time in a month

startednestjs/nest

started time in a month

startedxtermjs/xterm.js

started time in a month

pull request commentnicklaw5/twitch-api-php

Added Bearer for getStreamForUserId and getStreamForUsername

Thanks for the contribution @brandinarsenault 👍

brandinarsenault

comment created time in 2 months

push eventnicklaw5/twitch-api-php

Brandin Arsenault

commit sha 5292acbe77e04943c4f3b29eb570c31620984889

Added Bearer for getStreamForUserId and getStreamForUsername (#56) * Added OAuth Token (optional) to Helix Streams API * Revert "Added OAuth Token (optional) to Helix Streams API" This reverts commit f4794dab28c967eddceb06c4a82d60816ace2ee1. * Added OAuth Token (optional) to Helix Streams API * Migrate to travis-ci.com * Added OAuth Token (optional) to Helix Streams API * Revert "Added OAuth Token (optional) to Helix Streams API" This reverts commit f4794dab28c967eddceb06c4a82d60816ace2ee1. * Added OAuth Token (optional) to Helix Streams API Co-authored-by: Nick Law <nick@nicholaslaw.com.au>

view details

push time in 2 months

PR merged nicklaw5/twitch-api-php

Added Bearer for getStreamForUserId and getStreamForUsername

Modified the following Endpoints to added optional bearer token for Helix:

  • getStreamForUserId
  • getStreamForUsername
+5 -5

6 comments

2 changed files

brandinarsenault

pr closed time in 2 months

pull request commentnicklaw5/twitch-api-php

Added Bearer for getStreamForUserId and getStreamForUsername

@brandinarsenault I have migrated the tests to be run on travis-ci.com (rather than travis-ci.org). You will need to rebase your branch onto master to trigger the test suite to run.

Relates to #57

brandinarsenault

comment created time in 2 months

delete branch nicklaw5/twitch-api-php

delete branch : travis-ci-com-migration

delete time in 2 months

push eventnicklaw5/twitch-api-php

Nick Law

commit sha 21a52478ddaa2426b5c8a451a8c0fc674068c5ab

Migrate to travis-ci.com

view details

Nick Law

commit sha f4ec63849ad8d1ce3953b8f6a16644dcb8ae9063

Merge pull request #57 from nicklaw5/travis-ci-com-migration Migrate to travis-ci.com

view details

push time in 2 months

create barnchnicklaw5/twitch-api-php

branch : travis-ci-com-migration

created branch time in 2 months

pull request commentnicklaw5/twitch-api-php

Added Bearer for getStreamForUserId and getStreamForUsername

We need to migrate to travis-ci.com, currently we're using travis-ci.org. I'll take a look.

brandinarsenault

comment created time in 2 months

startedcturra/docker-ntp

started time in 2 months

starteddenoland/deno

started time in 2 months

startedhyperium/tonic

started time in 2 months

startedgrafana/loki

started time in 2 months

startedtimberio/vector

started time in 2 months

startedChocobozzz/PeerTube

started time in 2 months

created tagnicklaw5/helix

tagv0.5.9

A Twitch Helix API client written in Go.

created time in 2 months

release nicklaw5/helix

v0.5.9

released time in 2 months

push eventnicklaw5/helix

Jake Smith

commit sha aa24d02f6aba75f7e0dd10f7a259e41a6b95dd08

Add token validation support

view details

Jake Smith

commit sha d5a044323add7b4b6af0bd94ce86c5c8c68e37ad

Change the auth type if requesting token validation

view details

Jake Smith

commit sha 5673339d88e5a4c61b90940b604ec613262e66cc

Add docs for ValidateToken

view details

Jake Smith

commit sha bf6ac6d5c9c6ebe9a3b03e45b0e9d0622dded277

Check off Validate Access Token feature

view details

Nick Law

commit sha 9d0eb94e0b14167cdb3b2a58d1d9ad63ea8ad43b

Merge pull request #49 from jakefolio/feature/token-validate Validate user access token

view details

push time in 2 months

PR merged nicklaw5/helix

Validate user access token

This PR allows us to validate a user token and bring back some token details.

Since there is no way to easily avoid modifying the auth type (Bearer vs OAuth), a conditional check against the validation URL was put into setRequestHeaders. It's not the cleanest, but it's the only path that doesn't require a significant refactor.

This PR is related to an issue comment: https://github.com/nicklaw5/helix/issues/8#issuecomment-623754693

+158 -3

1 comment

5 changed files

jakefolio

pr closed time in 2 months

issue commentnicklaw5/helix

OAuth AppAccessToken seemingly not getting automatically added to request headers.

I don't fancy forking the repo just to correct that.

Then stop wasting our time, and either write your own library or use another.

Pyorot

comment created time in 2 months

issue commentnicklaw5/helix

Oauth Enforced

We should update the library to automatically grab oauth tokens

That is not an opinion that should be held by a library. A library is for providing core functionality for its intended purpose. Not for enforcing opinions on its users.

blackstar257

comment created time in 2 months

delete branch nicklaw5/helix

delete branch : update-build-badge

delete time in 2 months

push eventnicklaw5/helix

Nicholas Law

commit sha 8ad222e8eb03b118a124de0232932f6113409242

Migrate to travis-ci.com

view details

Nick Law

commit sha fec91a20a3a1f95afc38f88ba83e097bf29b0e92

Merge pull request #47 from nicklaw5/update-build-badge Migrate to travis-ci.com

view details

push time in 2 months

PR merged nicklaw5/helix

Migrate to travis-ci.com
+1 -1

1 comment

1 changed file

nicklaw5

pr closed time in 2 months

PR opened nicklaw5/helix

Migrate to travis-ci.com
+1 -1

0 comment

1 changed file

pr created time in 2 months

create barnchnicklaw5/helix

branch : update-build-badge

created branch time in 2 months

delete branch nicklaw5/helix

delete branch : travis

delete time in 2 months

PR closed nicklaw5/helix

Testing TravisCI
+1 -0

1 comment

1 changed file

nicklaw5

pr closed time in 2 months

push eventnicklaw5/helix

Nicholas Law

commit sha b57492c6ea948599bc2367a452fb31c860632d83

wip

view details

push time in 2 months

PR opened nicklaw5/helix

wip
+1 -0

0 comment

1 changed file

pr created time in 2 months

push eventnicklaw5/helix

Nicholas Law

commit sha ff5450d775c23e6b4c85889cbfda7e5094564c61

wip

view details

push time in 2 months

create barnchnicklaw5/helix

branch : travis

created branch time in 2 months

pull request commentnicklaw5/helix

Add After Param to getBroadcasterSubscribers

yes the issue raised by @BarryCarlyon says that this endpoint is missing both

Can you add the Before option as well then? See here for example.

jackmcguire1

comment created time in 2 months

issue closednicklaw5/helix

OAuth AppAccessToken seemingly not getting automatically added to request headers.

(Reposting because may be urgent). Hey, this doesn't seem to work as you say. I have this code:

if time.Now().After(authExpiry) {
	authExpiry = time.Now()
	res, err := twitch.GetAppAccessToken()
	exitIfError(err)
	authExpiry = authExpiry.Add(time.Duration(res.Data.ExpiresIn) * time.Second)
	log.Insta <- fmt.Sprintf("< | token generated: %s (for %ds)", res.Data.AccessToken, res.Data.ExpiresIn)
}
res, err := twitch.GetStreams(&getStreamsParams)
if err == nil && res.StatusCode != 200 {         // reinterpret HTTP error as actual error
	err = fmt.Errorf("HTTP %d: %s", res.StatusCode, res.ErrorMessage)
}
if err != nil {
	log.Insta <- fmt.Sprintf("x | < : %s", err)
}

and my program output

< | token generated: pfzn88vavmt9hlgy6c33aonf2imako (for 5444223s)
x | < : HTTP 401: OAuth token is missing

during the migration test window that just ended (is back to normal now). Any ideas?

Originally posted by @Pyorot in https://github.com/nicklaw5/helix/issues/42#issuecomment-622518993

closed time in 2 months

Pyorot

issue commentnicklaw5/helix

OAuth AppAccessToken seemingly not getting automatically added to request headers.

This is already documented here.

Please open a PR if you would like to add any further documentation.

Pyorot

comment created time in 2 months

startedd0p3t/fivem-js

started time in 2 months

startedTheBritishBrotherhood/tp-banking

started time in 2 months

startedprometheus/prometheus

started time in 2 months

issue closednicklaw5/helix

Provide an OAuth example since it will be required starting 1st of May 2020

Since Twitch is going to require OAuth Authentication starting from 1st of May 2020 (More here) it would be great to add an Authentication example in order to speed up initial configuration using this API wrapper.

closed time in 2 months

Fef0

issue commentnicklaw5/helix

Provide an OAuth example since it will be required starting 1st of May 2020

This is already supported. See the existing docs and examples: https://github.com/nicklaw5/helix/blob/master/docs/authentication_docs.md.

Fef0

comment created time in 2 months

issue commentaws/aws-cdk

How to call out to a custom Lambda function using AwsCustomResource and use the response to create resources?

Here is the current solution I'm leaning towards.

We would have a standalone call-api.js script which can get called using execSync from anyway in CDK. This script will call out the the API and return the stringified JSON response. We are then able to parse the JSON response into an JS object. Error handling as been left out of the below example for brevity. This seem like a bit of an anti-pattern, but its the only solution that I've liked during my testing.

call-api.js

#!/usr/bin/env node

const http = require('http')

http.get('https://my.api.com/vpcs_all', (resp) => {
  let data = ''

  // Concatenate all chunks
  resp.on('data', (chunk) => {
    data += chunk
  })

  // Send response
  resp.on('end', () => {
    console.log(data)
  })
})

my-stack.ts

import { execSync } from 'child_process'
import { App, Stack } from '@aws-cdk/core'
import { Vpc, SecurityGroup } from '@aws-cdk/aws-ec2'

class MyStack extends Stack {
  constructor(scope: App) {
    super(scope, 'MyStack')

    const resp = execSync('node ./call-api.js').toString()
    const vpcId = JSON.parse(resp).vpc_dev

    const vpc = Vpc.fromLookup(this, 'MyVpc', { vpcId })
    new SecurityGroup(this, 'MySg', { vpc })
  }
}

const app = new App()
new MyStack(app)
app.synth()

Thoughts, comments or concerns?

nicklaw5

comment created time in 3 months

startedjcolemorrison/vault-on-aws

started time in 3 months

issue commentaws/aws-cdk

How to call out to a custom Lambda function using AwsCustomResource and use the response to create resources?

Another solution could be to "script" a call to your API, write the output to a JSON file and import this file in you CDK app. You would call this script before running your CDK app.

The problem with that solution is that I don't know what values I need from the API prior to executing CDK. I could wrap the CDK executable with our own executable and have our executable run CDK as a child process. This I have tried but its not ideal.

What we are building is a Typescript library that wraps around CDK to apply our company-specific resource opinions, naming conventions, and tagging rules amongst other things. This library will be used to deploy some 300+ applications and various infrastructure resources across some 25 AWS accounts. We use a few key environment variables to determine which account, VPC, subnets and availability zones resources are deployed to. Ideally what we want is to resolve this environment variables at synth time, and use them to call out to the API to retrieve their concrete values.

We currently achieve this scale of deployment we have a custom in-house built tool. However, its become dated over the years and is difficult add functionality to or maintain. We are hoping that CDK can replace this tool and provide base framework for which to continue building from.

nicklaw5

comment created time in 3 months

issue commentaws/aws-cdk

How to call out to a custom Lambda function using AwsCustomResource and use the response to create resources?

I am currently trying to achieve a very basic actions: create a security group using an existing VPC.

When trying to fetch an existing VPC using a token with the Vpc. fromLookup function I get the following error:

All arguments to Vpc.fromLookup() must be concrete (no Tokens)
Subprocess exited with error 1
Error: Subprocess exited with error 1
    at ChildProcess.<anonymous> (/app/node_modules/aws-cdk/lib/api/cxapp/exec.ts:118:23)
    at ChildProcess.emit (events.js:310:20)
    at ChildProcess.EventEmitter.emit (domain.js:482:12)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)

Which I assume means this function is executed at synth time(?). This is why I am trying to resolve the concrete data object from the Lambda.

Is there anyway I can call out to a HTTP API at synth time only? For example:

if (isSynth) {
  const resp = callCustomApi('/my/path')

  const vpc = Vpc.fromLookup(this, 'MyVpc', {
    vpcId: resp.vpc_dev,
  })
}
nicklaw5

comment created time in 3 months

issue commentaws/aws-cdk

How to call out to a custom Lambda function using AwsCustomResource and use the response to create resources?

Unfortunately I'm still having the same issue using @jogold's suggestion of the provider framework. The problem being that the CustomResource.getAtt and CustomResource.getAttString methods, both return tokens, and not the concrete response from the Lambda function.

My code:

const onEvent = Function.fromFunctionArn(scope, 'MyFunc', 'arn:aws:lambda:ap-southeast-2:123456789:function:my-func-name')

const cr = new CustomResource(scope, 'MyCustomResource', {
  provider: CustomResourceProvider.fromLambda(onEvent),
  properties: {
    FunctionName: 'my-func-name',
    Payload: `{"path": "${path}"}`,
  },
})

cr.getAtt('Payload')        // returns token
cr.getAttString('Payload')  // returns token

Can someone please share a concrete example on how I can invoke a Lambda and return the concrete response?

nicklaw5

comment created time in 3 months

issue commentaws/aws-cdk

How to call out to a custom Lambda function using AwsCustomResource and use the response to create resources?

Ok, so I spoke too soon. Found this: https://github.com/aws/aws-cdk/issues/4810#issuecomment-548842577 - will give it a go.

nicklaw5

comment created time in 3 months

issue commentaws/aws-cdk

How to call out to a custom Lambda function using AwsCustomResource and use the response to create resources?

One issue with that approach is that I again need to know the name of the VPC before I can create the provider Lambda function, due to the fact the Lambda that hits the API needs to reside in the VPC.

Is there a way to reference a Lambda function that already exists out of the stack?

nicklaw5

comment created time in 3 months

issue commentaws/aws-cdk

How to call out to a custom Lambda function using AwsCustomResource and use the response to create resources?

Hi @NetaNir. Thanks for responding.

I have looked into Token, but I am a little confused on how to use the methods it provides. I wasn't able to find any examples either. Are you able to provide an example as to what you would do?

nicklaw5

comment created time in 3 months

issue openedaws/aws-cdk

How to call out to a custom Lambda function using AwsCustomResource and use the response to create resources?

:question: General Issue

The Question

What I want to do is call out to a custom API endpoint (which is exposed by a Lambda function), return the ID of a VPC and use it to create a security group. (This is a contrived example. I have other use cases for retrieving values from our custom API.)

I have the following basic my-func-name lambda for calling out to the API:

import axios from 'axios'

export const handler = async (event: any = {}): Promise<any> => {
  const resp = await axios(`https://my.api-example.com/${event.path}`)
  return resp.data
}

The below stack uses AwsCustomResource to invoke the above lambda and attempts to use the the response data to create a security group. But because AwsCustomResource.getResponseField returns a Token value, I'm unable to parse the JSON response into a Object, and instead receive an error due to it trying to parse the Token represented string.

const getDataByPath = (scope: Construct, path: string): string => {
  const lambdaInvokeSdkCall: AwsSdkCall = {
    service: 'Lambda',
    action: 'invoke',
    parameters: {
      FunctionName: 'my-func-name',
      Payload: `{"path": "${path}"}`,
    },
    physicalResourceId: PhysicalResourceId.of('InvokeLambdaResourceId1234'),
  }

  const acr = new AwsCustomResource(scope, 'MyAwsCustomResource', {
    policy: AwsCustomResourcePolicy.fromSdkCalls({ resources: AwsCustomResourcePolicy.ANY_RESOURCE }),
    onCreate: lambdaInvokeSdkCall,
  })

  // Example expected return payload:
  // {
  //   "vpc_dev": "vpc-1234abcd",
  //   "vpc_test": "vpc-2345bcde",
  //   "vpc_prod": "vpc-3456cdef"
  // }
  return JSON.parse(acr.getResponseField('Payload')) // !!! Fails here !!!
}

class MyStack extends Stack {
  constructor(scope: Construct, id: string) {
    super(scope, id)

    const vpcIds = getDataByPath(this, 'vpcs/all')

    const vpc = Vpc.fromLookup(this, 'MyVpc', {
      vpcId: vpcIds.vpc_dev,
    })

    new SecurityGroup(scope, 'asdasd', { vpc })
  }
}

const app = new App()
new MyStack(app, 'MyStack')

Error:

Unexpected token $ in JSON at position 0
Subprocess exited with error 1
Error: Subprocess exited with error 1
at ChildProcess.<anonymous>(/app/node_modules / aws - cdk / lib / api / cxapp / exec.ts: 118: 23)
at ChildProcess.emit(events.js: 310: 20)
at ChildProcess.EventEmitter.emit(domain.js: 482: 12)
at Process.ChildProcess._handle.onexit(internal / child_process.js: 275: 12)

So I understand the issue here. Just want to understand how to get around it. How can I use the response data from an invoked Lambda for creating arbitrary resources?

Environment

  • CDK CLI Version: 1.33.0
  • Module Version: 133.0
  • OS: all
  • Language: TypeScript

Other information

<!-- e.g. detailed explanation, stacktraces, related issues, suggestions on how to fix, links for us to have context, eg. associated pull-request, stackoverflow, gitter, etc -->

created time in 3 months

pull request commentnicklaw5/helix

Add After Param to getBroadcasterSubscribers

Do we need First too?

jackmcguire1

comment created time in 3 months

startedaws/aws-cdk-rfcs

started time in 3 months

issue commentnicklaw5/helix

Helix OAUTH token on all endpoints

Yep, we already do this by default: https://github.com/nicklaw5/helix/blob/master/helix.go#L383-L393

jackmcguire1

comment created time in 3 months

startedscrumpy/tiptap

started time in 3 months

push eventnicklaw5/stack-1

Nicholas Law

commit sha ebc612787594f0d24de1b6e70f26a1ccb6edf52d

Destroy all resources

view details

push time in 3 months

push eventnicklaw5/stack-2

Nicholas Law

commit sha 8cba0352611b6ea1783af0f6f005baca94d12f35

Destroy all provisioned resources

view details

push time in 3 months

startedgrpc-ecosystem/grpc-gateway

started time in 3 months

push eventnicklaw5/stack-3

Nicholas Law

commit sha 971a381e56ed14501e81cc59968bd1849e13ac6b

Destroy all provisioned resources

view details

push time in 3 months

startedgrpc-ecosystem/go-grpc-middleware

started time in 3 months

more