profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/ex-aws/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

startedex-aws/ex_aws_dynamo

started time in a day

delete branch ex-aws/ex_aws_s3

delete branch : add_timeout_to_upload_type

delete time in 2 days

push eventex-aws/ex_aws_s3

Bernard Duggan

commit sha d2d72fab1e85e65a340046992995087b26860d84

Add :timeout param to upload typespec

view details

Bernard Duggan

commit sha ac8d2c81cfeba9eb2bdbc0061e7d4eb2a56130cc

Merge pull request #133 from ex-aws/add_timeout_to_upload_type Add :timeout param to upload typespec

view details

push time in 2 days

PR merged ex-aws/ex_aws_s3

Add :timeout param to upload typespec

Resolves #127

+4 -1

0 comment

1 changed file

bernardd

pr closed time in 2 days

issue closedex-aws/ex_aws_s3

ExAws.S3.upload/4 is missing type spec for `:timeout` option.

  • Do not use the issues tracker for help or support (try Elixir Forum, Slack, IRC, etc.)
  • Questions about how to contribute are fine.

Environment

  • Elixir & Erlang versions (elixir --version): Elixir 1.12 / Erlang 24.0
  • ExAws version mix deps |grep ex_aws: 2.2.3
  • HTTP client version. IE for hackney do mix deps | grep hackney: hackney 1.17.4

Current behavior

Dialyzer failure with call to ExAws.S3.upload/4 because :timeout option is not specified in the typespecs.

Expected behavior

:timeout option, which is described in codedoc for upload/4 should be specified in the typespecs for that function.

closed time in 2 days

mattpolzin

issue commentex-aws/ex_aws_s3

ExAws.S3.upload/4 is missing type spec for `:timeout` option.

Fixed - thanks @mattpolzin.

mattpolzin

comment created time in 2 days

PR opened ex-aws/ex_aws_s3

Add :timeout param to upload typespec

Resolves #127

+4 -1

0 comment

1 changed file

pr created time in 2 days

push eventex-aws/ex_aws_s3

Bernard Duggan

commit sha d2d72fab1e85e65a340046992995087b26860d84

Add :timeout param to upload typespec

view details

push time in 2 days

create barnchex-aws/ex_aws_s3

branch : add_timeout_to_upload_type

created branch time in 2 days

issue commentex-aws/ex_aws

Support for LocalStack

@alexserver It's, like, right at the top of the section entitled "Configuration". I have no idea where else you think it should live: https://hexdocs.pm/ex_aws_s3/2.3.0/ExAws.S3.html#module-configuration

rhysldavis

comment created time in 2 days

pull request commentex-aws/ex_aws_s3

presigned post

Thanks very much @Monyk015 - this looks good. I've made a few comments inline, but overall it looks fine. I especially appreciate the addition of tests :)

Monyk015

comment created time in 2 days

Pull request review commentex-aws/ex_aws_s3

presigned post

 defmodule ExAws.S3Test do     assert expected == S3.delete_object_tagging(bucket, object)   end +  test "#presigned_post simple policy" do+    bucket = "my-bucket"+    key = "test.txt"++    post_data = S3.presigned_post(ExAws.Config.new(:s3), bucket, key)++    assert URI.parse(post_data.url).scheme != nil+    assert policy = post_data.fields["Policy"]++    assert {:ok, json} = Base.decode64(policy)+    assert {:ok, policy} = config.json_codec.decode(json)+

As noted above, this should be config() since it's a function call not a variable. The compiler has picked it up in this case.

Monyk015

comment created time in 2 days

PullRequestReviewEvent

Pull request review commentex-aws/ex_aws_s3

presigned post

 defmodule ExAws.S3.Utils do   def sanitized_port_component(%{port: port}) when port in @excluded_ports, do: ""   def sanitized_port_component(%{port: port}), do: ":#{port}"   def sanitized_port_component(_), do: ""++  def build_bucket_url(config, bucket_name) do+    "#{config[:scheme]}#{bucket_name}.#{config[:host]}"+  end++  def datetime_to_erlang_time(datetime) do+    {{datetime.year, datetime.month, datetime.day},+     {datetime.hour, datetime.minute, datetime.second}}+  end++  def build_amz_post_policy(+        datetime,+        expiration_date,+        bucket,+        credential,+        opts,+        exact_key \\ nil+      ) do+    key = Keyword.get(opts, :key, exact_key)+    content_length_range = Keyword.get(opts, :content_length_range, nil)+    acl = Keyword.get(opts, :acl, nil)+    custom_conditions = Keyword.get(opts, :custom_conditions, [])++    key_condition =+      case key do+        {:starts_with, starts_with} -> [["starts-with", "$key", starts_with]]+        nil -> []+        key when is_binary(key) -> [%{"key" => key}]+      end++    content_length_condition =+      case content_length_range do+        [min, max] -> [["content-length-range", min, max]]+        nil -> []+      end++    acl_condition =+      case acl do+        {:starts_with, starts_with} -> [["starts-with", "acl", starts_with]]+        nil -> []+        acl when is_binary(acl) -> %{"acl" => acl}+      end++    %{+      "expiration" => DateTime.to_iso8601(expiration_date),+      "conditions" =>+        [+          %{"X-Amz-Algorithm" => "AWS4-HMAC-SHA256"},+          %{"X-Amz-Credential" => credential},+          %{"X-Amz-Date" => ExAws.Auth.Utils.amz_date(datetime)},

See the note about about the cleaner way of doing this

Monyk015

comment created time in 2 days

PullRequestReviewEvent

Pull request review commentex-aws/ex_aws_s3

presigned post

 defmodule ExAws.S3.Utils do   def sanitized_port_component(%{port: port}) when port in @excluded_ports, do: ""   def sanitized_port_component(%{port: port}), do: ":#{port}"   def sanitized_port_component(_), do: ""++  def build_bucket_url(config, bucket_name) do+    "#{config[:scheme]}#{bucket_name}.#{config[:host]}"+  end++  def datetime_to_erlang_time(datetime) do+    {{datetime.year, datetime.month, datetime.day},+     {datetime.hour, datetime.minute, datetime.second}}+  end++  def build_amz_post_policy(+        datetime,+        expiration_date,+        bucket,+        credential,+        opts,+        exact_key \\ nil+      ) do+    key = Keyword.get(opts, :key, exact_key)+    content_length_range = Keyword.get(opts, :content_length_range, nil)+    acl = Keyword.get(opts, :acl, nil)+    custom_conditions = Keyword.get(opts, :custom_conditions, [])++    key_condition =+      case key do+        {:starts_with, starts_with} -> [["starts-with", "$key", starts_with]]+        nil -> []+        key when is_binary(key) -> [%{"key" => key}]+      end++    content_length_condition =+      case content_length_range do+        [min, max] -> [["content-length-range", min, max]]+        nil -> []+      end++    acl_condition =+      case acl do+        {:starts_with, starts_with} -> [["starts-with", "acl", starts_with]]+        nil -> []+        acl when is_binary(acl) -> %{"acl" => acl}+      end

If I was writing this, I might be inclined to break each of these three blocks out into their own functions, and then you could just do function-level pattern matching rather than needing a case block. It also makes this function a good deal shorter (I like short functions :)). I'm not going to insist on it being changed, but since you mentioned being new I figured I'd give you at least my opinion (which is all this is :)).

Example:

defp acl_condition({:start_with, starts_with}, do: [["starts-with", "acl", starts_with]]
defp acl_conditoin(nil), do: []
defp acl_condition(acl) when is_binary(acl), do: %{"acl" => acl}

then you just replace the acl_condition in the final line with acl_condition(acl)

Monyk015

comment created time in 2 days

PullRequestReviewEvent

Pull request review commentex-aws/ex_aws_s3

presigned post

 defmodule ExAws.S3 do     end   end +  @spec presigned_post(+          config :: map,+          bucket :: binary,+          key :: binary | nil,+          opts :: presigned_post_opts()+        ) :: presigned_post_result()+  def presigned_post(config, bucket, key, opts \\ []) do+    expires_in = Keyword.get(opts, :expires_in, 3600)+    {:ok, datetime} = DateTime.now("Etc/UTC")+    expiration_date = DateTime.add(datetime, expires_in, :second)++    datetime = datetime_to_erlang_time(datetime)++    credential = ExAws.Auth.Credentials.generate_credential_v4("s3", config, datetime)++    policy =+      build_amz_post_policy(datetime, expiration_date, bucket, credential, opts, key)+      |> config.json_codec.encode!

Calling functions without parenthesis is no longer The Elixir Way - the compiler generally throws a warning about this, and only fails to here because it can't tell at compile time that it's actually a function (since the module name itself is a variable).

Monyk015

comment created time in 2 days

PullRequestReviewEvent

Pull request review commentex-aws/ex_aws_s3

presigned post

 defmodule ExAws.S3 do     end   end +  @spec presigned_post(+          config :: map,+          bucket :: binary,+          key :: binary | nil,+          opts :: presigned_post_opts()+        ) :: presigned_post_result()+  def presigned_post(config, bucket, key, opts \\ []) do+    expires_in = Keyword.get(opts, :expires_in, 3600)+    {:ok, datetime} = DateTime.now("Etc/UTC")+    expiration_date = DateTime.add(datetime, expires_in, :second)++    datetime = datetime_to_erlang_time(datetime)++    credential = ExAws.Auth.Credentials.generate_credential_v4("s3", config, datetime)++    policy =+      build_amz_post_policy(datetime, expiration_date, bucket, credential, opts, key)+      |> config.json_codec.encode!+      |> Base.encode64()++    signature = ExAws.Auth.Signatures.generate_signature_v4("s3", config, datetime, policy)++    %{+      url: build_bucket_url(config, bucket),+      fields: %{+        "key" => key,+        "X-Amz-Algorithm" => "AWS4-HMAC-SHA256",+        "X-Amz-Credential" => credential,+        "X-Amz-Date" => ExAws.Auth.Utils.amz_date(datetime),

I suspect the amz_date is a holdover from when this library was originally ported from erlcloud - you can get the same effect as this now using: datetime |> DateTime.utc_now() |> DateTime.truncate(:second) |> DateTime.to_iso8601(:basic) without the need for the datetime_to_erlang_time faffing around :)

Monyk015

comment created time in 2 days

PullRequestReviewEvent

push eventex-aws/ex_aws

Jacob Smith

commit sha bf8392aa214fb4d0320668d05e57f6a411149d4f

Pass Headers to Request This change allows HTTP headers to be passed in. This was originally questioned when it was merged in here: https://github.com/ex-aws/ex_aws/pull/286#discussion_r90781530 but wasn't necessary for the use case at the time. However, attempting to use this with Cloudfront (and presumably any other service that utilizes ETag) makes this necessary.

view details

Bernard Duggan

commit sha eaaa730af49726c00b3fdc2e08d80998d70fb64b

Merge pull request #820 from jacobsmith/patch-1 Pass Headers to Request

view details

push time in 2 days

PR merged ex-aws/ex_aws

Pass Headers to Request

This change allows HTTP headers to be passed in. This was originally questioned when it was merged in here: https://github.com/ex-aws/ex_aws/pull/286#discussion_r90781530 but wasn't necessary for the use case at the time. However, attempting to use this with Cloudfront (and presumably any other service that utilizes ETag) makes this necessary.

This should add Headers if present, otherwise, leave them as an empty array. So, there should be no functional impact for existing users of this method.

+1 -1

1 comment

1 changed file

jacobsmith

pr closed time in 2 days

pull request commentex-aws/ex_aws

Pass Headers to Request

Looks good to me - thanks @jacobsmith

jacobsmith

comment created time in 2 days

issue commentex-aws/ex_aws

Support for LocalStack

I know this issue is closed, but can the repo owners to put this very important thing in the documentation ? there's no place where the documentation says you can set a port, and I only discovered this googling and coming to this issue. if you put this in the documentation, you will save a lot of lost souls please

rhysldavis

comment created time in 2 days

issue commentex-aws/ex_aws_ses

Does this library support throttling / rate limiting?

One of the quickest solutions would be throwing a rate-limiting library between your app and and an email library, something like https://github.com/ExHammer/hammer can work.

travelmassive

comment created time in 4 days

issue commentex-aws/ex_aws_ses

Does this library support throttling / rate limiting?

Not using SES templates. The emails are personalised notifications about topics the user is subscribed to (alerting to a new post on our site), so not a big deal if a few get dropped (but prefer to send them all, obviously).

travelmassive

comment created time in 4 days

issue commentex-aws/ex_aws_ses

Does this library support throttling / rate limiting?

That's tough!

Are you using SES templates? How is it important that every customer get their email (i.e. do you need retries)?

travelmassive

comment created time in 4 days

issue commentex-aws/ex_aws_ses

Does this library support throttling / rate limiting?

Got it. The challenge is "where" do you put throttling in a typical Elixir app?

For my Phoenix app, I have no less than 5 elixir libraries between my app and SES, in this order (I think).

my_app -> bamboo_phoenix -> bamboo -> bamboo_ses -> ex_aws_ses -> ex_aws-> (AWS SES)

Ideally I'd like to implement a very simple rate limit solution that others can easily replicate.

travelmassive

comment created time in 4 days

issue commentex-aws/ex_aws_ses

Does this library support throttling / rate limiting?

They also limit the number of emails per 24 hours, but I'm talking about things like account and region rotation, for example. It's not something that everyone would do, but that was the scenario that I started with. Hence I needed as much control as possible.

Still, if you feel like submitting a PR that would provide that as an optional feature, I'm keen to review it.

It might be a layer on top of the existing functionality, for example.

travelmassive

comment created time in 4 days