Skip to content

Metrics

Every stage task registers its metrics in the group vc-metrics on the Kafka Streams metrics registry, next to the Streams built-ins. They reach JMX under the kafka.streams domain, every configured metrics reporter, and CausalStreams.metrics(). All of them carry the tags thread-id, task-id, and stage; the per-topic breakdowns add topic. Sensors are removed when the task closes, so metrics follow task migration.

These metrics exist because holding is invisible to ordinary lag monitoring. The adapter accepts a record before the gate decides on it, so the stage's committed consumer position advances past records that are still held. A stalled task can therefore report zero consumer lag on its own group. Hold depth and hold age are observable only here.

Gauges are sampled by the task's wall-clock punctuator every 500 ms, so a gauge can trail the event it reflects by up to half a second. Counters are recorded at their event sites.

Task metrics

These record at the INFO metrics recording level, the Kafka default.

Name Type Meaning
records-held gauge The number of records currently held across the task's hold queues.
records-held-age-max-ms gauge The wall-clock milliseconds the longest-gating head record has been held, measured from when it became its queue's head. A hold restored at restart ages from the first sample after init.
records-delivered-rate rate The per-second rate of records released through the causal gate.
records-delivered-total total The total number of records released through the causal gate since the task initialised.
replays-skipped-total total The total number of records fed at or below the frontier and discarded. Nonzero is expected only while a rescope-growth refetch replays covered offsets.
truncation-sweeps-skipped-total total The total number of truncation sweeps that failed and skipped their cycle. A climbing value means stamp-side clock entries are not being garbage-collected.
ticks-emitted-total total The total number of tick records the task has emitted to its tick channel. Absent growth on a ticking stage means the tick punctuator is not firing.
stamp-offset-entries gauge The number of offset entries in the stamp-side clock.
stamp-sequence-entries gauge The number of sequence entries in the stamp-side clock, claims not yet resolved to offsets.

Per-topic metrics

These record at the DEBUG metrics recording level. Enable them with metrics.recording.level=DEBUG in the application properties; at the default level the sensors exist but do not record.

Name Type Meaning
records-held gauge The number of records currently held on this source topic's channel.
records-held-age-max-ms gauge The wall-clock milliseconds this channel's head record has been held.

Reading the stall pair

records-held alone distinguishes nothing: any skew between cause and effect topics holds records transiently, and the gauge returns to zero as causes arrive. The stall signal is records-held-age-max-ms growing without bound, which means a head record is waiting on a cause that is not arriving. The per-topic breakdown names the convoyed channel; to name the missing cause itself, ask CausalStreams.explainHolds(), which reports each gating head's unmet claims and the local watermarks that show the gap, and which the same measurement feeds — the gauge, the accessor, and the vc-hold-* warnings cannot disagree (diagnosing held records). The cure is fixing the lagging cause, never skipping the head (the contract); retention on the cause topic bounds how long the wait can succeed.

The width gauges guard the other slow failure. Stamp-side clock entries are freed by the log-start truncation sweep (truncation), so stamp-offset-entries growing against a steady workload, or truncation-sweeps-skipped-total climbing, means clock and header overhead is accumulating.