profile
viewpoint

Ask questionsIIS in-process hosting incorrectly throws ConnectionResetException for HTTP POST

Describe the bug

An ASP.NET Core 2.2.0 application using IIS in-process hosting will intermittently throw ConnectionResetException while reading the request body of an HTTP POST request, causing the request to fail, if the request does not provide an Content-Length HTTP header.

Additionally, if the UseStatusCodePages() exception handler is used, the exception handler fails with an ArgumentException due to a duplicate key.

Based on observed behaviour in our application environment in aggregate, it is possible that this may be related to #6332 and/or #4789 (particularly this).

If a Content-Length HTTP header is specified for the request or if out-of-process hosting is enabled, the exception is not observed.

Stack Trace for exception

Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware:Error: An unhandled exception has occurred while executing the request.

Microsoft.AspNetCore.Connections.ConnectionResetException: The client has disconnected ---> System.IO.EndOfStreamException: Attempted to read past the end of the stream.
   --- End of inner exception stack trace ---
   at Microsoft.AspNetCore.Server.IIS.Core.IO.AsyncIOOperation.GetResult(Int16 token)
   at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.ReadBody()
   at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
   at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
   at System.IO.Pipelines.Pipe.GetReadAsyncResult()
   at System.IO.Pipelines.Pipe.DefaultPipeReader.GetResult(Int16 token)
   at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.ReadAsync(Memory`1 memory, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.IIS.Core.HttpRequestStream.ReadAsyncInternal(Memory`1 buffer, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions.DrainAsync(Stream stream, ArrayPool`1 bytePool, Nullable`1 limit, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Mvc.Formatters.JsonInputFormatter.ReadRequestBodyAsync(InputFormatterContext context, Encoding encoding)
   at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext bindingContext)
   at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext actionContext, IModelBinder modelBinder, IValueProvider valueProvider, ParameterDescriptor parameter, ModelMetadata metadata, Object value)
   at Microsoft.AspNetCore.Mvc.Internal.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<<CreateBinderDelegate>g__Bind|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeInnerFilterAsync()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResourceFilter()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync()
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)

Stack trace when status code pages fails

Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware:Error: An exception was thrown attempting to execute the error handler.

System.ArgumentException: An item with the same key has already been added. Key: Allow
   at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
   at Microsoft.AspNetCore.HttpSys.Internal.HeaderCollection.Add(String key, StringValues value)
   at Microsoft.AspNetCore.Routing.Matching.HttpMethodMatcherPolicy.<>c__DisplayClass12_0.<CreateRejectionEndpoint>b__0(HttpContext context)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Builder.StatusCodePagesExtensions.<>c__DisplayClass6_0.<<UseStatusCodePagesWithReExecute>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)
Exception thrown: 'Microsoft.AspNetCore.Connections.ConnectionResetException' in System.Private.CoreLib.dll

Microsoft.AspNetCore.Server.IIS.Core.IISHttpServer:Error: Connection ID "18086456109425491975", Request ID "80002fc9-0001-fb00-b63f-84710c7967bb": An unhandled exception was thrown by the application.

To Reproduce

A repro application with instructions can be found here: https://github.com/martincostello/AspNetCoreReproFor6415

Expected behavior

The application correctly responds to the HTTP POST request.

aspnet/AspNetCore

Answer questions rodion-m

It doesn't seem to be related. 0x80070040 is very different error code. Is request body large? Is it possible client goes away (or page is refreshed) while request is being sent?

Hi @pakrym ! This is actually our issue that we met just now with a large body (client has a timeout of 5.000 ms, obviously it was not enough). Is there any way to prevent this server exception? It's not good that it's throwing. Here is a stack trace (.net core 2.2.5):

System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
  Module "Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketAwaitableEventArgs", in <GetResult>g__ThrowSocketException|7_0
  Module "Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketAwaitableEventArgs", in GetResult
  Module "Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketConnection", in ProcessReceives
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketConnection", in DoReceive

Microsoft.AspNetCore.Connections.ConnectionResetException: An existing connection was forcibly closed by the remote host
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.IO.Pipelines.PipeCompletion", in ThrowLatchedException
  Module "System.IO.Pipelines.Pipe", in GetReadResult
  Module "System.IO.Pipelines.Pipe", in GetReadAsyncResult
  Module "System.IO.Pipelines.Pipe+DefaultPipeReader", in GetResult
  Module "System.Runtime.CompilerServices.ValueTaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1MessageBody", in PumpAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.IO.Pipelines.PipeCompletion", in ThrowLatchedException
  Module "System.IO.Pipelines.Pipe", in GetReadResult
  Module "System.IO.Pipelines.Pipe", in GetReadAsyncResult
  Module "System.IO.Pipelines.Pipe+DefaultPipeReader", in GetResult
  Module "System.Runtime.CompilerServices.ValueTaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.MessageBody", in ReadAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.ValueTaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream", in ReadAsyncInternal
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.TaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream", in ReadAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.TaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions", in DrainAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.Formatters.JsonInputFormatter", in ReadRequestBodyAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.TaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder", in BindModelAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder", in BindModelAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.TaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Mvc.Internal.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d", in MoveNext
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker", in InvokeInnerFilterAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker", in InvokeFilterPipelineAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker", in InvokeAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Routing.EndpointMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  File "C:\Projects\Goodwins\Goodwins.Server\Extensions\UseSerilogContext.cs", line 30, col 6, in MoveNext
    Void MoveNext()
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Authentication.AuthenticationMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Server.IISIntegration.IISMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequests>d__185`1", in MoveNext
useful!

Related questions

HTTP Error 500.31 - ANCM Failed to Find Native Dependencies hot 123
The SPA default page middleware could not return the default page '/index.html' in production application hot 102
The SPA default page middleware could not return the default page '/index.html' in production application hot 96
Asp.Net MVC Core: “Error unprotecting the session cookie” exception hot 79
No DefaultChallengeScheme found from preview8 to preview9 hot 67
BadHttpRequestException: Reading the request body timed out due to data arriving too slowly hot 65
Method not found: 'Microsoft.EntityFrameworkCore.Metadata.Builders.IndexBuilder Microsoft.EntityFrameworkCore.Metadata.Builders.EntityTypeBuilder`1.HasIndex(System.Linq.Expressions.Expression`1<System.Func`2<System.__Canon,System.Object>>)'. hot 59
The library &#39;hostpolicy.dll&#39; required to execute the application was not found - AspNetCore hot 58
error CS1503: Argument 2: cannot convert from 'method group' to 'EventCallback' hot 57
Error: Invocation canceled due to the underlying connection being closed. hot 54
Unexpected end of request content k8s pod - AspNetCore hot 53
Using 'UseMvc' to configure MVC is not supported while using Endpoint Routing. To continue using 'UseMvc', please set 'MvcOptions.EnableEndpointRounting = false' inside 'ConfigureServices'. hot 50
HTTP Error 500.30 - ANCM In-Process Start Failure hot 43
Error: Server returned handshake error: Handshake was canceled. hot 43
Issue with BuildServiceProvider() in ConfigureServices() hot 42
source:https://uonfu.com/
answerer
Rodion Mostovoy rodion-m Freelance Moscow An independent software developer and musician.
Github User Rank List