profile
viewpoint

Ask questionsIIS in-process hosting incorrectly throws ConnectionResetException for HTTP POST

Describe the bug

An ASP.NET Core 2.2.0 application using IIS in-process hosting will intermittently throw ConnectionResetException while reading the request body of an HTTP POST request, causing the request to fail, if the request does not provide an Content-Length HTTP header.

Additionally, if the UseStatusCodePages() exception handler is used, the exception handler fails with an ArgumentException due to a duplicate key.

Based on observed behaviour in our application environment in aggregate, it is possible that this may be related to #6332 and/or #4789 (particularly this).

If a Content-Length HTTP header is specified for the request or if out-of-process hosting is enabled, the exception is not observed.

Stack Trace for exception

Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware:Error: An unhandled exception has occurred while executing the request.

Microsoft.AspNetCore.Connections.ConnectionResetException: The client has disconnected ---> System.IO.EndOfStreamException: Attempted to read past the end of the stream.
   --- End of inner exception stack trace ---
   at Microsoft.AspNetCore.Server.IIS.Core.IO.AsyncIOOperation.GetResult(Int16 token)
   at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.ReadBody()
   at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
   at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
   at System.IO.Pipelines.Pipe.GetReadAsyncResult()
   at System.IO.Pipelines.Pipe.DefaultPipeReader.GetResult(Int16 token)
   at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.ReadAsync(Memory`1 memory, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.IIS.Core.HttpRequestStream.ReadAsyncInternal(Memory`1 buffer, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions.DrainAsync(Stream stream, ArrayPool`1 bytePool, Nullable`1 limit, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Mvc.Formatters.JsonInputFormatter.ReadRequestBodyAsync(InputFormatterContext context, Encoding encoding)
   at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext bindingContext)
   at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext actionContext, IModelBinder modelBinder, IValueProvider valueProvider, ParameterDescriptor parameter, ModelMetadata metadata, Object value)
   at Microsoft.AspNetCore.Mvc.Internal.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<<CreateBinderDelegate>g__Bind|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeInnerFilterAsync()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResourceFilter()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync()
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)

Stack trace when status code pages fails

Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware:Error: An exception was thrown attempting to execute the error handler.

System.ArgumentException: An item with the same key has already been added. Key: Allow
   at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
   at Microsoft.AspNetCore.HttpSys.Internal.HeaderCollection.Add(String key, StringValues value)
   at Microsoft.AspNetCore.Routing.Matching.HttpMethodMatcherPolicy.<>c__DisplayClass12_0.<CreateRejectionEndpoint>b__0(HttpContext context)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Builder.StatusCodePagesExtensions.<>c__DisplayClass6_0.<<UseStatusCodePagesWithReExecute>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)
Exception thrown: 'Microsoft.AspNetCore.Connections.ConnectionResetException' in System.Private.CoreLib.dll

Microsoft.AspNetCore.Server.IIS.Core.IISHttpServer:Error: Connection ID "18086456109425491975", Request ID "80002fc9-0001-fb00-b63f-84710c7967bb": An unhandled exception was thrown by the application.

To Reproduce

A repro application with instructions can be found here: https://github.com/martincostello/AspNetCoreReproFor6415

Expected behavior

The application correctly responds to the HTTP POST request.

aspnet/AspNetCore

Answer questions rodion-m

It doesn't seem to be related. 0x80070040 is very different error code. Is request body large? Is it possible client goes away (or page is refreshed) while request is being sent?

Hi @pakrym ! This is actually our issue that we met just now with a large body (client has a timeout of 5.000 ms, obviously it was not enough). Is there any way to prevent this server exception? It's not good that it's throwing. Here is a stack trace (.net core 2.2.5):

System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
  Module "Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketAwaitableEventArgs", in <GetResult>g__ThrowSocketException|7_0
  Module "Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketAwaitableEventArgs", in GetResult
  Module "Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketConnection", in ProcessReceives
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketConnection", in DoReceive

Microsoft.AspNetCore.Connections.ConnectionResetException: An existing connection was forcibly closed by the remote host
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.IO.Pipelines.PipeCompletion", in ThrowLatchedException
  Module "System.IO.Pipelines.Pipe", in GetReadResult
  Module "System.IO.Pipelines.Pipe", in GetReadAsyncResult
  Module "System.IO.Pipelines.Pipe+DefaultPipeReader", in GetResult
  Module "System.Runtime.CompilerServices.ValueTaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1MessageBody", in PumpAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.IO.Pipelines.PipeCompletion", in ThrowLatchedException
  Module "System.IO.Pipelines.Pipe", in GetReadResult
  Module "System.IO.Pipelines.Pipe", in GetReadAsyncResult
  Module "System.IO.Pipelines.Pipe+DefaultPipeReader", in GetResult
  Module "System.Runtime.CompilerServices.ValueTaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.MessageBody", in ReadAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.ValueTaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream", in ReadAsyncInternal
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.TaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream", in ReadAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.TaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions", in DrainAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.Formatters.JsonInputFormatter", in ReadRequestBodyAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.TaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder", in BindModelAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder", in BindModelAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.TaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Mvc.Internal.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d", in MoveNext
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker", in InvokeInnerFilterAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker", in InvokeFilterPipelineAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker", in InvokeAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Routing.EndpointMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  File "C:\Projects\Goodwins\Goodwins.Server\Extensions\UseSerilogContext.cs", line 30, col 6, in MoveNext
    Void MoveNext()
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Authentication.AuthenticationMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Server.IISIntegration.IISMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequests>d__185`1", in MoveNext
useful!

Related questions

HTTP Error 500.31 - ANCM Failed to Find Native Dependencies hot 8
ERR_HTTP2_INADEQUATE_TRANSPORT_SECURITY when clean webapi template
MissingMethodException: Method not found Microsoft.EntityFrameworkCore.Metadata.Builders.IndexBuilder
Migrate Asp.Net Core 2.2 Integration Tests to 3.0 - The TestServer constructor was not called with a IWebHostBuilder so IWebHost is not available. hot 4
UseStaticFiles with StaticFileOptions/FileExtensionContentTypeProvider breaks server side Blazor hot 3
SignalR .NetCore 3.0-- HubConnectionBuilder' does not contain a definition for 'WithUrl' hot 3
HTTP Error 500.30 - ANCM In-Process Start Failure hot 3
With Asp.net core 3.0 SignInManager.IsSignedIn(User) allways reuturn false?but in 2.2 it works fine.Is that a debug? hot 3
CSS Isolation in Blazor Components hot 3
How to retrieve the HttpContext from Blazor server-side ? hot 3
Oauth2 authentication erroring with "The oauth state was missing or invalid." hot 3
Failed to gracefully shutdown application (IIS Hang) hot 3
BadHttpRequestException: Reading the request body timed out due to data arriving too slowly hot 2
HTTP Error 500.0 - ANCM In-Process Handler Load Failure hot 2
Method not found: 'Microsoft.EntityFrameworkCore.Metadata.Builders.IndexBuilder Microsoft.EntityFrameworkCore.Metadata.Builders.EntityTypeBuilder`1.HasIndex(System.Linq.Expressions.Expression`1<System.Func`2<System.__Canon,System.Object>>)'. hot 2
Github User Rank List