Our production site becomes unreachable several times a day for roughly 1–2 minutes, then recovers without intervention. The stack is Windows Server with IIS, ASP.NET Core MVC and Web APIs, an Angular frontend, SQL Server on a separate 8 GB server, a three-node Elasticsearch cluster, and a separate monitoring server. Traffic reaches IIS directly without a reverse proxy, CDN, WAF, or load balancer.
External monitors in different geographic locations report socket connection timeouts rather than HTTP errors. The incidents occur both during busy periods and when traffic is light. DNS is fast, the hosting provider reports no infrastructure problems, Windows remains online, and neither IIS nor ASP.NET Core logs show failed requests. Previous SQL connection-pool exhaustion was addressed with caching, and current SQL metrics look healthy.
I am considering OpenTelemetry, distributed tracing, runtime metrics, HTTPERR logging, HTTP.sys and IIS queue monitoring, Windows Performance Counters, and dashboards correlating Windows, IIS, SQL Server, Elasticsearch, and application data. Before changing application code, I want to determine which layer is actually failing.
How would you investigate this systematically? What would you monitor first, and how would you distinguish a network, TCP, Windows, HTTP.sys, IIS, ASP.NET Core, SQL Server, or Elasticsearch problem? Are there specific tools or metrics that are especially useful for short socket timeouts that never appear in application logs?
3 Answers
I would check IIS and Windows events before instrumenting the whole application. Look for application-pool recycles, idle-timeout or rapid-fail protection, scheduled tasks, antivirus or filtering-driver activity, Windows updates, and resource exhaustion. Thread-pool starvation, a long garbage-collection pause, a full HTTP.sys queue, port exhaustion, or an overlapped recycle can all produce a short outage with little useful request logging.
Compare normal and incident-time values for queue length, current connections, rejected requests, process memory, CPU, GC pauses, thread-pool availability, TCP resets, and port usage. Also verify that the application pool is not being suspended after idle periods and that the host is not relying on DHCP for a critical service address.
Use separate health checks for basic reachability and dependency reachability rather than one endpoint that only returns healthy or unhealthy. A local probe can log whether the web process itself was reachable even when external monitoring failed. Similar checks for SQL and each Elasticsearch node can reveal whether the application is waiting on a dependency.
OpenTelemetry and distributed tracing are still worthwhile, but they only describe requests that IIS accepts. They will not explain a TCP handshake that never completes. If Elasticsearch is involved, verify that the client is configured with multiple nodes and can fail over instead of depending on one node.
A message saying “unable to connect” usually means the request may never have reached IIS, so application tracing cannot explain it. Capture evidence at the TCP layer during the next event. A circular packet capture using tools such as pktmon or netsh trace, synchronized with an external probe, can show whether the SYN arrived, whether the server sent a SYN-ACK, whether the connection was reset, or whether the handshake completed and TLS or HTTP stalled.
Also collect HTTPERR logs and five-second performance data for TCP connection failures and resets, active connections, HTTP.sys queue depth and rejected requests, IIS connections, application-pool recycles, process private bytes and handles, available memory, thread-pool state, garbage collection, and ephemeral-port usage. One-minute incidents disappear quickly if you only retain daily averages.

Related Questions
Can't Load PhpMyadmin On After Server Update
Redirect www to non-www in Apache Conf
How To Check If Your SSL Cert Is SHA 1
Windows TrackPad Gestures