profile
viewpoint

Ask questionsIIS in-process hosting incorrectly throws ConnectionResetException for HTTP POST

Describe the bug

An ASP.NET Core 2.2.0 application using IIS in-process hosting will intermittently throw ConnectionResetException while reading the request body of an HTTP POST request, causing the request to fail, if the request does not provide an Content-Length HTTP header.

Additionally, if the UseStatusCodePages() exception handler is used, the exception handler fails with an ArgumentException due to a duplicate key.

Based on observed behaviour in our application environment in aggregate, it is possible that this may be related to #6332 and/or #4789 (particularly this).

If a Content-Length HTTP header is specified for the request or if out-of-process hosting is enabled, the exception is not observed.

Stack Trace for exception

Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware:Error: An unhandled exception has occurred while executing the request.

Microsoft.AspNetCore.Connections.ConnectionResetException: The client has disconnected ---> System.IO.EndOfStreamException: Attempted to read past the end of the stream.
   --- End of inner exception stack trace ---
   at Microsoft.AspNetCore.Server.IIS.Core.IO.AsyncIOOperation.GetResult(Int16 token)
   at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.ReadBody()
   at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
   at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
   at System.IO.Pipelines.Pipe.GetReadAsyncResult()
   at System.IO.Pipelines.Pipe.DefaultPipeReader.GetResult(Int16 token)
   at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.ReadAsync(Memory`1 memory, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.IIS.Core.HttpRequestStream.ReadAsyncInternal(Memory`1 buffer, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions.DrainAsync(Stream stream, ArrayPool`1 bytePool, Nullable`1 limit, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Mvc.Formatters.JsonInputFormatter.ReadRequestBodyAsync(InputFormatterContext context, Encoding encoding)
   at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext bindingContext)
   at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext actionContext, IModelBinder modelBinder, IValueProvider valueProvider, ParameterDescriptor parameter, ModelMetadata metadata, Object value)
   at Microsoft.AspNetCore.Mvc.Internal.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<<CreateBinderDelegate>g__Bind|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeInnerFilterAsync()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResourceFilter()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync()
   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync()
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)

Stack trace when status code pages fails

Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware:Error: An exception was thrown attempting to execute the error handler.

System.ArgumentException: An item with the same key has already been added. Key: Allow
   at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
   at Microsoft.AspNetCore.HttpSys.Internal.HeaderCollection.Add(String key, StringValues value)
   at Microsoft.AspNetCore.Routing.Matching.HttpMethodMatcherPolicy.<>c__DisplayClass12_0.<CreateRejectionEndpoint>b__0(HttpContext context)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Builder.StatusCodePagesExtensions.<>c__DisplayClass6_0.<<UseStatusCodePagesWithReExecute>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)
Exception thrown: 'Microsoft.AspNetCore.Connections.ConnectionResetException' in System.Private.CoreLib.dll

Microsoft.AspNetCore.Server.IIS.Core.IISHttpServer:Error: Connection ID "18086456109425491975", Request ID "80002fc9-0001-fb00-b63f-84710c7967bb": An unhandled exception was thrown by the application.

To Reproduce

A repro application with instructions can be found here: https://github.com/martincostello/AspNetCoreReproFor6415

Expected behavior

The application correctly responds to the HTTP POST request.

aspnet/AspNetCore

Answer questions Rody66

It doesn't seem to be related. 0x80070040 is very different error code. Is request body large? Is it possible client goes away (or page is refreshed) while request is being sent?

Hi @pakrym ! This is actually our issue that we met just now with a large body (client has a timeout of 5.000 ms, obviously it was not enough). Is there any way to prevent this server exception? It's not good that it's throwing. Here is a stack trace (.net core 2.2.5):

System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
  Module "Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketAwaitableEventArgs", in <GetResult>g__ThrowSocketException|7_0
  Module "Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketAwaitableEventArgs", in GetResult
  Module "Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketConnection", in ProcessReceives
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketConnection", in DoReceive

Microsoft.AspNetCore.Connections.ConnectionResetException: An existing connection was forcibly closed by the remote host
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.IO.Pipelines.PipeCompletion", in ThrowLatchedException
  Module "System.IO.Pipelines.Pipe", in GetReadResult
  Module "System.IO.Pipelines.Pipe", in GetReadAsyncResult
  Module "System.IO.Pipelines.Pipe+DefaultPipeReader", in GetResult
  Module "System.Runtime.CompilerServices.ValueTaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1MessageBody", in PumpAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.IO.Pipelines.PipeCompletion", in ThrowLatchedException
  Module "System.IO.Pipelines.Pipe", in GetReadResult
  Module "System.IO.Pipelines.Pipe", in GetReadAsyncResult
  Module "System.IO.Pipelines.Pipe+DefaultPipeReader", in GetResult
  Module "System.Runtime.CompilerServices.ValueTaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.MessageBody", in ReadAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.ValueTaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream", in ReadAsyncInternal
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.TaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream", in ReadAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.TaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions", in DrainAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.Formatters.JsonInputFormatter", in ReadRequestBodyAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.TaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder", in BindModelAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder", in BindModelAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "System.Runtime.CompilerServices.TaskAwaiter`1", in GetResult
  Module "Microsoft.AspNetCore.Mvc.Internal.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d", in MoveNext
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker", in InvokeInnerFilterAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker", in InvokeFilterPipelineAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker", in InvokeAsync
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Routing.EndpointMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  File "C:\Projects\Goodwins\Goodwins.Server\Extensions\UseSerilogContext.cs", line 30, col 6, in MoveNext
    Void MoveNext()
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Authentication.AuthenticationMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "System.Runtime.CompilerServices.TaskAwaiter", in HandleNonSuccessAndDebuggerNotification
  Module "Microsoft.AspNetCore.Server.IISIntegration.IISMiddleware", in Invoke
  Module "System.Runtime.ExceptionServices.ExceptionDispatchInfo", in Throw
  Module "Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequests>d__185`1", in MoveNext
useful!

Related questions

HTTP Error 500.31 - ANCM Failed to Find Native Dependencies hot 523
ERR_HTTP2_INADEQUATE_TRANSPORT_SECURITY when clean webapi template hot 505
Restore MvcJsonOptions to ASP.NET Core 3.0 as a type forward? hot 420
The SPA default page middleware could not return the default page '/index.html' in production application hot 310
Issue with BuildServiceProvider() in ConfigureServices() hot 285
HTTP Error 500.30 - ANCM In-Process Start Failure hot 282
HTTP Error 500.30 - ANCM In-Process Start Failure hot 249
Using 'UseMvc' to configure MVC is not supported while using Endpoint Routing. To continue using 'UseMvc', please set 'MvcOptions.EnableEndpointRounting = false' inside 'ConfigureServices'. hot 241
Calling Request.EnableRewind throw on 3.0.0-preview7 hot 219
ERR_HTTP2_INADEQUATE_TRANSPORT_SECURITY when clean webapi template hot 206
DataProtection - CryptographicException when using EntityFramework as Key Storage Provider hot 181
error CS1503: Argument 2: cannot convert from 'method group' to 'EventCallback' hot 160
Error: Server returned handshake error: Handshake was canceled. hot 152
Blazor - rzc generate exited with code 1. DotNet Core 3.0.100-preview8-013656 hot 150
NullReferenceException in IISHttpContext hot 135
Github User Rank List