I would not expect the upload example to be able to accept a 389MB file considering it reads only up to 8MB. I doubt that it works even with 2.6.3, what most likely happens is simply that the way it stops after that differs.

OK I'll try it when I can and see.

I thought you meant you were behind nginx and nginx would drop the connection when it sees that status.

Is this only for HTTP/1.1 then? Because behavior like this would make little sense for HTTP/2. For HTTP/1.1 Cowboy will drop the connection if it's not worth keeping it alive (too large or undefined size incoming body). You may also set the connection:close header I believe.

@essen And then Cowboy drops the HTTP connection without fully completing it? That's the key part of my question.

Nginx ... just drops the requests

http://nginx.org/en/docs/http/request_processing.html

Go back to cowboy 2.6.3, it works.

I just tried the upload example in cowboy 2.8.0 and it failed to upload a 389mb file. The browser (firefox) comes up with a message: "The connection was reset" and in the terminal where I did: make run for the example it has a bunch of numbers, which is binary output then at the end it has:

106,198,157,216,36,38,151,176,118,124>>,

#{bindings => #{},body_length => 408836021,cert => undefined,has_body => true,headers => #{<<"accept">> => <<"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8">>,<<"accept-encoding">> => <<"gzip, deflate">>,<<"accept-language">> => <<"en-US,en;q=0.5">>,<<"connection">> => <<"keep-alive">>,<<"content-length">> => <<"408836021">>,<<"content-type">> => <<"multipart/form-data; boundary=---------------------------187645155928358590352183544006">>,<<"host">> => <<"localhost:8080">>,<<"origin">> => <<"http://localhost:8080">>,<<"referer">> => <<"http://localhost:8080/">>,<<"upgrade-insecure-requests">> => <<"1">>,<<"user-agent">> => <<"Mozilla/5.0 (X11; Linux x86_64; rv:84.0) Gecko/20100101 Firefox/84.0">>},host => <<"localhost">>,host_info => undefined,method => <<"POST">>,multipart => {<<"---------------------------187645155928358590352183544006">>,<<>>},path => <<"/upload">>,path_info => undefined,peer => {{127,0,0,1},54274},pid => <0.451.0>,port => 8080,qs => <<>>,ref => http,scheme => <<"http">>,sock => {{127,0,0,1},8080},streamid => 1,version => 'HTTP/1.1'}}} and stacktrace [{upload_h,init,2,[{file,"src/upload_h.erl"},{line,10}]},{cowboy_handler,execute,2,[{file,"src/cowboy_handler.erl"},{line,37}]},{cowboy_stream_h,execute,3,[{file,"src/cowboy_stream_h.erl"},{line,306}]},{cowboy_stream_h,request_process,3,[{file,"src/cowboy_stream_h.erl"},{line,295}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]

It'd be great for security to respond in this way to script attacks. Nginx supports the non-standard 444 which it doesn't return to the client; it just logs it and closes the connection, firewall-style.

You can already send <<"444">> instead of an integer. Or <<"444 Some text">> if you'd like. Integer status codes are just the standard ones.

It'd be great for security to respond in this way to script attacks. Nginx supports the non-standard 444 which it doesn't return to the client; it just logs it and closes the connection, firewall-style.

That's not Websocket then, you're uploading via HTTP/2. And read_part_body is about multipart as well, so not just a normal upload. It would help to have an example I can reproduce.

I just switched my project back to cowboy 2.6.3 and the ~400meg file uploads fine and I don't see anything different with the Req var outputs.

I printed out the Req var and see: streamid => 23,version => 'HTTP/2'

Is that using http/2 to upload the file?

When I dig in to my code, I see I'm using code very similar to your upload example. So something may have changed in cowboy_req:read_part_body where I'm getting the data to write to disk. Would gen_tcp params affect any of this?

No. That's not a websockets example.

Right. Sorry. I'm looking through the commits in 2.7.0 to see what I can find. Thanks! I think you have an upload example in cowboy. Try a 200 gig file on it. Maybe you'll see what I'm seeing. I can't remember if that is a websockets example, though.

I understand but you need to provide information or a reproducible test case for me to help.

Maybe there is a websockets change back in 2.7.0 I need to find. New parameters to use?

In my project, if I use 2.8.0 or 2.7.0 this issue happens. If I go back to 2.6.3, it uploads the file fine.

Is the terminate callback called, and if so what's the reason for termination? It's entirely possible that the client closes the connection.

A year ago, I was looking through the commits trying to find what might have changed in 2.7.0 from 2.6.3 to cause this issue. http/2 probably wasn't the issue.

I don't see any error. My webapp shows a progress percent as it uploads, and it just quits at 27%. Smaller files work fine. It is websockets not http/2. Maybe the original fix was a fluke, if these parameters are only for http/2.

Via Websocket? Not HTTP/2? Linked ticket is about the latter.

What error do you get?

I'm returning to this issue again:

https://github.com/ninenines/cowboy/issues/1433

I thought I had it working, but for some time now it has not. I've tried increasing the values in max_received_frame_rate, but the upload process quits at about 27%. Any idea what I can do to allow gig sized files to be uploaded to a cowboy server via websockets?

Thanks!

Hi,

I am attempting to send several JSON documents over a WebSocket to a client. This information is updated every second for example. On the client-side, I receive the onmessage events and parse the JSON doc. The only problem is that I seem to miss some documents. I have verified that they are sent on the web_socket side in Erlang. For some reason, I do not get them all in the client. These JSON docs are sent as text frames.

Am I missing something? Is there something else I should be setting?

Thanks

This is not a cowboy issue. Some 3rd party library is failing. Sorry for the trouble.

See #1494 for this issue's resolution.

Thanks @essen , sorry for the noise.

Don't know. Not heard of such a problem before.

The template name seems to be wrong per https://github.com/ninenines/cowboy/pull/1494. <details><summary>$ gmake list-templates</summary>

Available templates:
    cowboy_http
    cowboy_loop
    cowboy_rest
    cowboy_ws
    gen_fsm
    gen_server
    gen_statem
    module
    ranch_protocol
    supervisor

</details>

seems to confirm it. As a side-note, if you use the suggested template (cowboy.http) there is no error, you just get a blank file. That doesn't seem desirable, should there be an separate issue for that?