Observability & tracing
Tracebench
A live distributed-trace waterfall you can scrub and flip into a flame graph.
How to read this lab
A corpus of 300 synthetic traces, each a tree of spans with realistic causality (some sequential, some concurrent). Jump between the median (p50) and tail (p99) trace and watch the actual bottleneck — the critical path — change to a completely different span.
Controls
- View
- Switches between a Gantt-style waterfall (bars over time, nested by depth) and a flame graph (aggregated by self time).
- Trace index
- Scrubs through the corpus one trace at a time.
- Jump to p50
- Jumps straight to the trace at the median total duration.
- Jump to p99
- Jumps straight to the trace near the top of the tail.
What to watch for
- Compare the Slowest link readout at p50 vs p99 — it names a different span each time.
- The p50 vs p99 stat shows how much slower the tail trace is, often several times the median.
- In the waterfall view, the highlighted critical path is the chain of spans the request was actually waiting on — not necessarily the widest or deepest bar.
Why it happens
The critical path is computed by always following the child span that finishes LAST at each level, not the biggest or deepest one — that's the actual chain of dependencies the parent was waiting on. A small fraction of traces get hit by a slow-tail spike on one span kind; at p50 that span is unremarkable, but at p99 it dominates the critical path. This is why "the median request is fine" tells you almost nothing about what's slow for your worst-off users — the bottleneck genuinely moves.
Try this first — Click Jump to p50, note the Slowest link, then click Jump to p99 and watch it change.
render is the critical-path bottleneck
Trace #0 finishes in 385ms. Following whichever child finishes last at each level lands on render, contributing 42.3ms of self-time (11% of the total) across a 5-span critical path.
View
- http-request
- gateway
- auth
- db-query
- cache
- render
- critical path
Trace picker
Corpus of 300 synthetic traces. A small fraction inject a db-query slow-tail spike, which is what typically dominates the p99 critical path below.
Trace duration
385ms
Trace #0
Critical path length
5spans
Root → bottleneck leaf
Slowest link
render
42.3ms self time
p50 vs p99
4.4x
269ms → 1174ms