Investigating a slow ASP.NET Core endpoint in production: traces, dotnet-counters, dotnet-trace

Measure first, don't guess.

bash
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# ======================================================================
# Q: An endpoint is slow in production. How do you investigate?
#
# Measure first, don't guess. I start with the traces (Application Insights /
# OpenTelemetry) to see where the time goes: database, external HTTP, CPU, or
# waiting for a thread. I look at p95/p99, not the average.
# - Many small DB queries in the trace: N+1 (Q7).
# - One slow query on a large table: missing index, OFFSET pagination (Q9).
# - Latency rises sharply with load while CPU stays low: thread pool starvation
#   from sync-over-async (Q4).
# - Many sequential HTTP calls: chatty integration, missing caching or
#   Task.WhenAll (Q43).
# - High GC, memory keeps growing: allocations, buffering of large files, a
#   memory leak in a static/singleton (Q44).
# - Load testing before release: k6, NBomber or Azure Load Testing, with
#   realistic data.
# - Micro-benchmarks with BenchmarkDotNet, always in Release.
# - Set a target (SLO, e.g. p95 < 300 ms) so you know when to stop optimizing.
# ======================================================================

# Live metrics from the process: thread pool queue, GC, exceptions, requests/sec
dotnet tool install -g dotnet-counters
dotnet-counters monitor -p <pid> --counters System.Runtime,Microsoft.AspNetCore.Hosting

dotnet-trace collect -p <pid>     # CPU profile: opens in PerfView / Visual Studio / speedscope
dotnet-dump collect -p <pid>      # memory dump: for hangs and memory leaks