Although technically spec compliant, URI.decode_www_form fails to convert a %% to a %. We do receive URLs which contain %% instead of %25 from sources which we do not control.

Upon receiving a %% eg curl 'http://localhost:4000/bla?value=%%' we get an error:

** (Plug.Conn.InvalidQueryError) invalid urlencoded params, got %%

Environment

• Elixir & Erlang/OTP versions (elixir --version): 1.12.1
• Phoenix: 1.5.9
• Operating system: Mac os 11.3

Current behavior

iex(13)> URI.decode_www_form "http://localhost:4000/bla?value=%%"
** (ArgumentError) malformed URI "http://localhost:4000/bla?value=%%"
    (elixir 1.12.1) lib/uri.ex:440: URI.decode_www_form/1
iex(14)> URI.decode_www_form "http://localhost:4000/bla?value=%25"
"http://localhost:4000/pixel?value=%"

Expected behavior

It would be nice to have the following behaviour

iex(13)> URI.decode_www_form "http://localhost:4000/bla?value=%%"
"http://localhost:4000/pixel?value=%"
iex(14)> URI.decode_www_form "http://localhost:4000/bla?value=%25"
"http://localhost:4000/pixel?value=%"

Ah hadn’t thought of that, thanks that’s a great idea

On 18/06/2021, at 4:57 PM, José Valim ***@***.***> wrote:

Yes, according to the spec you are correct: https://url.spec.whatwg.org/#percent-decode

we should append % if it is not followed by digits.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

Yes, according to the spec you are correct: https://url.spec.whatwg.org/#percent-decode

we should append % if it is not followed by digits.

Although technically spec compliant, URI.decode_www_form fails to convert a %% to a %. We do receive URLs which contain %% instead of %25 from sources which we do not control.

Upon receiving a %% eg curl 'http://localhost:4000/bla?value=%%' we get an error:

** (Plug.Conn.InvalidQueryError) invalid urlencoded params, got %%

Environment

• Elixir & Erlang/OTP versions (elixir --version): 1.12.1
• Phoenix: 1.5.9
• Operating system: Mac os 11.3

Current behavior

iex(13)> URI.decode_www_form "http://localhost:4000/bla?value=%%"
** (ArgumentError) malformed URI "http://localhost:4000/bla?value=%%"
    (elixir 1.12.1) lib/uri.ex:440: URI.decode_www_form/1
iex(14)> URI.decode_www_form "http://localhost:4000/bla?value=%25"
"http://localhost:4000/pixel?value=%"

Expected behavior

It would be nice to have the following behaviour

iex(13)> URI.decode_www_form "http://localhost:4000/bla?value=%%"
"http://localhost:4000/pixel?value=%"
iex(14)> URI.decode_www_form "http://localhost:4000/bla?value=%25"
"http://localhost:4000/pixel?value=%"

:green_heart: :blue_heart: :purple_heart: :yellow_heart: :heart:

Add exception details for docs.

When using <>/2 for pattern matching, the match should always happen on the RHS (right hand side) var, if it doesn't, then you'll get an ArgumentError with:

** (ArgumentError) the left argument of <> operator inside a match should always be a literal binary because its size can't be verified. Got: x

Add exception details for docs.

When using <>/2 for pattern matching, the match should always happen on the RHS (right hand side) var, if it doesn't, then you'll get an ArgumentError with:

** (ArgumentError) the left argument of <> operator inside a match should always be a literal binary because its size can't be verified. Got: x

@WLSF , yes please! I was going to suggest to reopen this one but it is probably best to keep it as is for historical purposes.

Elixir has explicit semantics between local/unqualified calls and remote calls. A local call will never go through code reloading, a remote call will. Because of that, a local call always binds at compilation time (the called function must exist at compilation time), but a remote call can't be verified, because the remote call can be added later on, even in the same module, as long as code reloading happens.

For this reason, we don't plan to blurry the lines between those constructs. Therefore, you need to do &Mod.foo/1 if you want to refer to it at the module body, since the function &foo/1 has not yet been defined.

Environment

$ elixir --version
Erlang/OTP 23 [erts-11.2.2.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Elixir 1.11.4 (compiled with Erlang/OTP 23)

Current behavior

The following piece of code

defmodule X do
  @funs %{fun: &foo/1}
  def bar(), do: @funs
  def foo(x), do: x
end

gives a compilation error

** (CompileError) iex:64: undefined function foo/1
    (stdlib 3.14.2.1) lists.erl:1358: :lists.mapfoldl/3
    (stdlib 3.14.2.1) lists.erl:1359: :lists.mapfoldl/3
    (elixir 1.11.4) expanding macro: Kernel.@/1

but the error does not happen if the function name is prepended with the module name like @funs %{fun: &X.foo/1}.

Expected behavior

It would be nicer to not require prepending the module name. Note that this is not required when the map is defined inside a function instead of via an attribute, i.e., the following compiles

defmodule X do
  def bar(), do: %{fun: &foo/1}
  def foo(x), do: x
end

Parenthesis are not required if you wrap Keyword using <kbd>[</kbd> and <kbd>]</kbd> characters it would work as expected.

Simple example of raise:

IO.inspect :example,
  sample: case true do
    true -> true
    _ -> false
  end,
  something: true

Example solution:

IO.inspect :example, [
  sample: case true do
    true -> true
    _ -> false
  end,
  something: true
]

haha you're quick enough it's almost like you wrote the language. thanks @josevalim

Environment

elixir 1.11.2-otp-23 erlang 23.1.4

Current behaviour

'releases.exs' looks like this (truncated, original on request):

import Config

# Configure BetEpsStore using Environment variables
case System.get_env("STORAGE_ADAPTER") do
  "COCKROACH" ->
    config :bet_eps_store, :storage_adapter, Store.Storage.Cockroach
    config :bet_eps_store, Store.Storage.Cockroach.Repo,
      ssl:
        case System.get_env("COCKROACH_SSL") do
           "true" -> true
           _ -> "false"
         end,
      ssl_cert: System.get_env("COCKROACH_SSL_CERT"),
      show_sensitive_data_on_connection_error: false
  _ ->
    config :bet_eps_store, Store.Storage.Filesystem,
      dir_path: System.fetch_env!("OUTPUT_DIR")
end

Generates the following error :

** (SyntaxError) config/releases.exs:46:10: syntax error before: ','
    (elixir 1.11.2) lib/code.ex:654: Code.format_string!/2
    (mix 1.11.2) lib/mix/tasks/format.ex:418: Mix.Tasks.Format.format_file/2
    (elixir 1.11.2) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2
    (elixir 1.11.2) lib/task/supervised.ex:35: Task.Supervised.reply/5
    (stdlib 3.13.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3

Add the following parenthesis to the 'COCKROACH_SSL' block and the error no longer manifests :

 (case System.get_env("COCKROACH_SSL") do
           "true" -> true
           _ -> "false"
         end),

The syntax error is correct. It is always easier to see if you change the names, so you remove the semantics. Let's first reduce it:

config :app,
  key: case System.get_env("COCKROACH_SSL") do
    "true" -> true
    _ -> "false"
  end

Now let's rewrite it to:

config :app,
  key: foo arg do
    "true" -> true
    _ -> "false"
  end

And move everything to the same line:

config :app, key: foo arg do
  "true" -> true
  _ -> "false"
end

So now the question is. Is the code above the same as:

config(:app, key: foo(arg)) do
  "true" -> true
  _ -> "false"
end

or:

config :app, key: (foo arg do
  "true" -> true
  _ -> "false"
end)

Elixir considers the first, which is what, once you have a comma after the do/end, it raises a syntax error!

Environment

$ elixir --version
Erlang/OTP 23 [erts-11.2.2.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Elixir 1.11.4 (compiled with Erlang/OTP 23)

Current behavior

The following piece of code

defmodule X do
  @funs %{fun: &foo/1}
  def bar(), do: @funs
  def foo(x), do: x
end

gives a compilation error

** (CompileError) iex:64: undefined function foo/1
    (stdlib 3.14.2.1) lists.erl:1358: :lists.mapfoldl/3
    (stdlib 3.14.2.1) lists.erl:1359: :lists.mapfoldl/3
    (elixir 1.11.4) expanding macro: Kernel.@/1

but the error does not happen if the function name is prepended with the module name like @funs %{fun: &X.foo/1}.

Expected behavior

It would be nicer to not require prepending the module name. Note that this is not required when the map is defined inside a function instead of via an attribute, i.e., the following compiles

defmodule X do
  def bar(), do: %{fun: &foo/1}
  def foo(x), do: x
end

Environment

elixir 1.11.2-otp-23 erlang 23.1.4

Current behaviour

'releases.exs' looks like this (truncated, original on request):

import Config

# Configure BetEpsStore using Environment variables
case System.get_env("STORAGE_ADAPTER") do
  "COCKROACH" ->
    config :bet_eps_store, :storage_adapter, Store.Storage.Cockroach
    config :bet_eps_store, Store.Storage.Cockroach.Repo,
      ssl:
        case System.get_env("COCKROACH_SSL") do
           "true" -> true
           _ -> "false"
         end,
      ssl_cert: System.get_env("COCKROACH_SSL_CERT"),
      show_sensitive_data_on_connection_error: false
  _ ->
    config :bet_eps_store, Store.Storage.Filesystem,
      dir_path: System.fetch_env!("OUTPUT_DIR")
end

Generates the following error :

** (SyntaxError) config/releases.exs:46:10: syntax error before: ','
    (elixir 1.11.2) lib/code.ex:654: Code.format_string!/2
    (mix 1.11.2) lib/mix/tasks/format.ex:418: Mix.Tasks.Format.format_file/2
    (elixir 1.11.2) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2
    (elixir 1.11.2) lib/task/supervised.ex:35: Task.Supervised.reply/5
    (stdlib 3.13.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3

Add the following parenthesis to the 'COCKROACH_SSL' block and the error no longer manifests :

 (case System.get_env("COCKROACH_SSL") do
           "true" -> true
           _ -> "false"
         end),

Ok! Thanks @josevalim.

I had also updated a doc that was wrongly saying about CompileError when running LHS pattern matches on <>/2, shall I open another PR for that?

Discussion opened here: https://github.com/erlang/otp/issues/4971

Currently the concat operator has variant responses for different scenarios, for example:

# Using literals
iex> "" <> 1
** (ArgumentError) expected binary argument in <> operator but got: 1

# Using non-literals
iex> a = ""
iex> b = 3
iex> a <> B
** (ArgumentError) argument error while evaluating iex at line 2

# Using variables that are going to be solved at runtime
defmodule T do
  def f(a, b), do: a <> b  
end

iex> a = ""
iex> b = 2
iex> T.f(a, b)
# badarg error
* 1st argument: not a bitstring     This typically happens when calling Kernel.byte_size/1 with an invalid argument     or when performing binary concatenation with &lt;&gt; and one of the arguments is not a binary\

iex> T.f(b, a)
...
# just a generic error that doesn't fit any of the cases above

Because of that, I wanted to leave these changes here as a suggestion to improve the development experience.

The idea is to "double-check" the left and right side hands at the end, in order to ensure that both sides are binaries.

And since it's going add a couple of checks and validations into the generated AST, we wouldn't be able to compare concat operations with merging binaries operations, I'm not sure if this is a problem at all. (?)

Also, this is only going to affect operations that aren't in guard/match context; otherwise it's the same as it was before.

What do you think?

Adding the checks at this point slows down some operations considerably, which is why we have decided to do a post check. My hope is that, with the new Erlang/OTP 24 error_info information, perhaps we are able to improve this directly in OTP, but unfortunately we cannot take this approach.

In any case, thanks for the PR and exploring it!

Currently the concat operator has variant responses for different scenarios, for example:

# Using literals
iex> "" <> 1
** (ArgumentError) expected binary argument in <> operator but got: 1

# Using non-literals
iex> a = ""
iex> b = 3
iex> a <> B
** (ArgumentError) argument error while evaluating iex at line 2

# Using variables that are going to be solved at runtime
defmodule T do
  def f(a, b), do: a <> b  
end

iex> a = ""
iex> b = 2
iex> T.f(a, b)
# badarg error
* 1st argument: not a bitstring     This typically happens when calling Kernel.byte_size/1 with an invalid argument     or when performing binary concatenation with &lt;&gt; and one of the arguments is not a binary\

iex> T.f(b, a)
...
# just a generic error that doesn't fit any of the cases above

Because of that, I wanted to leave these changes here as a suggestion to improve the development experience.

The idea is to "double-check" the left and right side hands at the end, in order to ensure that both sides are binaries.

And since it's going add a couple of checks and validations into the generated AST, we wouldn't be able to compare concat operations with merging binaries operations, I'm not sure if this is a problem at all. (?)

Also, this is only going to affect operations that aren't in guard/match context; otherwise it's the same as it was before.

What do you think?

:green_heart: :blue_heart: :purple_heart: :yellow_heart: :heart:

Not super sure of this, but noticed #11062 and thought I'd take a stab at it.

Fixes #11062.

• Elixir & Erlang/OTP versions (elixir --version):

Erlang/OTP 24 [erts-12.0.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit] [dtrace]
Elixir 1.12.1 (compiled with Erlang/OTP 24)

• Operating system:

Darwin [...] 20.4.0 Darwin Kernel Version 20.4.0: Thu Apr 22 21:46:47 PDT 2021; root:xnu-7195.101.2~1/RELEASE_X86_64 x86_64

Current behavior

Fetching rebar3 dependencies with no version constraints in their transitive deps is parsed in a way where mix's own parsing causes it to fail.

I was trying with the following sample specification for dependencies:

  defp deps do
    [
      {:recon, ">= 0.0.0"},
      {:grpcbox, "~> 0.14.0", override: true},
      {:opentelemetry_api, git: "http://github.com/open-telemetry/opentelemetry-erlang", branch: "master", sparse: "apps/opentelemetry_api", override: true},
      {:opentelemetry, git: "http://github.com/open-telemetry/opentelemetry-erlang", branch: "master", sparse: "apps/opentelemetry", override: true},
      {:opentelemetry_exporter, git: "http://github.com/open-telemetry/opentelemetry-erlang", branch: "master", sparse: "apps/opentelemetry_exporter", override: true}
    ]
  end

This is interesting because I'm trying to get the master branch, which is defined in Erlang and has :opentelemetry as a subdirectory checkout: https://github.com/open-telemetry/opentelemetry-erlang/blob/master/apps/opentelemetry/rebar.config (the issue is gone on main, the new mainline branch, but still happens there with grpcbox as a dep), with no lockfile to further limit the dependency.

However, when this is parsed by mix, the content returns a nil version constraint:

https://github.com/elixir-lang/elixir/blob/ba78d902e3bd084edf02416f82316715eec0795b/lib/mix/lib/mix/rebar.ex#L107-L109

This is seen as invalid in the dep loader: https://github.com/elixir-lang/elixir/blob/ba78d902e3bd084edf02416f82316715eec0795b/lib/mix/lib/mix/dep/loader.ex#L162-L168

Expected behavior

The actual proper semantics would likely be to return it as {app, ">= 0.0.0", override: true}, which would match the intent behind rebar3's format, and avoid parsing dependencies in a ways that crashes itself.

I think the test you want is this:

      assert parse_dep(:git_rebar) == {:git_rebar, override: true}

:)