Chat tool telemetry

Find slow, failing, and repetitive agent tools without exposing prompts, arguments, or outputs.

PeriodToolTeam

Detailed production timing active

6058 of 6123 saved tool calls include execution duration.

98.9% timing coverage

Tool calls

+600% vs previous 7d14 saved production turns

p50 latency

728 ms

+6% vs previous 7d14 timed calls

p95 latency

12 s

+1571% vs previous 7d14 timed calls

Tool failures

14.3%

+14.3 pts vs previous 7d2 excluding NoResults

No result

0.0%

No change vs previous 7d0 product outcomes

Calls / turn

1.0

No change vs previous 7d57.1% of turns at 5+ calls

What changed

Material regressions ranked by severity and affected call volume.

Current 7 days compared with the previous equal window

Stable against baseline

No material tool regressions detected

Tools with enough activity were checked for higher failure rates, available p95 latency regressions, more no-results, and increased loop pressure.

Review the tool comparison for absolute hotspots that may still need a product threshold.

Investigate

Regression checks require at least 3 calls in both windows. Latency is flagged at +20%; failure and no-result rates are flagged at +1 and +3 points respectively.

Execution volume

Successful calls, no-results, and failures combined.

Calls

Oct 1Oct 4Oct 7

Payload pressure

How often results exceed the 20 KB inline cap.

5.0%

handle rows / all calls

307 stored handles in this period. This is a payload-size proxy, not a one-to-one call classification.

Tool comparison

Each change is measured against the previous equal window; latency appears only where detailed timing exists.

Sorted by current p95 latency

Tool	Calls	p50	p95	Failures	No result
publish_flow	14+600% vs prior	728 ms+6% vs prior	12 s+1571% vs prior	14.3%+14.3 pts vs prior	0.0%No change vs prior

Failure codes

Operational failures only; NoResults is excluded.

InvalidArguments2

Interpretation notes

The data has known edges that change what a chart means.

Blocked loops are invisible: Rate-limited and identical-argument calls are rejected before telemetry fires.
NoResults is not downtime: It remains a failed tool outcome, but is separated from runtime reliability here.
Preview volume lives elsewhere: Preview and threadless chats reach PostHog but are intentionally absent from Postgres.
Latency is partially instrumented: Saved tool parts provide production outcomes and volume; duration appears only on calls with detailed telemetry.
Partial turns are retained: They remain in volume counts but are excluded from failures and latency percentiles.
Comparisons use equal windows: Every delta compares the selected period with the immediately preceding period under the same tool and team filters.

Recent sanitized calls

Arguments, outputs, user IDs, and message content are never rendered.

Privacy safe

publish_flowHJC728 msOct 6, 8:01 PM UTC

Outcome

Success

Thread

ed7becf5…

Tool call

call_2nV…

Turn sequence

2 of 27

Turn state

Completed

publish_flowBidlo304 msOct 6, 4:54 PM UTC

Outcome

InvalidArguments

Thread

20a1bcaf…

Tool call

call_Nni…

Turn sequence

3 of 5

Turn state

Completed

publish_flowBidlo284 msOct 6, 4:30 PM UTC

Outcome

InvalidArguments

Thread

20a1bcaf…

Tool call

call_RPD…

Turn sequence

7 of 8

Turn state

Completed

publish_flowHJC728 msOct 5, 11:44 PM UTC

Outcome

Success

Thread

8baa70ed…

Tool call

call_x3U…

Turn sequence

2 of 3

Turn state

Completed

publish_flowHJC823 msOct 5, 11:38 PM UTC

Outcome

Success

Thread

8baa70ed…

Tool call

call_5Lu…

Turn sequence

28 of 34

Turn state

Completed

publish_flowHJC714 msOct 5, 11:30 PM UTC

Outcome

Success

Thread

8baa70ed…

Tool call

call_TKk…

Turn sequence

3 of 5

Turn state

Completed

publish_flowHJC631 msOct 5, 9:53 PM UTC

Outcome

Success

Thread

8baa70ed…

Tool call

call_0SG…

Turn sequence

11 of 26

Turn state

Completed

publish_flowHJC675 msOct 5, 9:48 PM UTC

Outcome

Success

Thread

8baa70ed…

Tool call

call_yc0…

Turn sequence

5 of 6

Turn state

Completed

publish_flowHJC790 msOct 5, 9:43 PM UTC

Outcome

Success

Thread

8baa70ed…

Tool call

call_IGv…

Turn sequence

2 of 3

Turn state

Completed

publish_flowHJC772 msOct 5, 7:47 PM UTC

Outcome

Success

Thread

8baa70ed…

Tool call

call_qps…

Turn sequence

25 of 31

Turn state

Completed

publish_flowTRP 2820 msOct 1, 5:22 AM UTC

Outcome

Success

Thread

a4ef58fe…

Tool call

call_o2t…

Turn sequence

2 of 2

Turn state

Completed

publish_flowTRP 212 sOct 1, 5:02 AM UTC

Outcome

Success

Thread

a4ef58fe…

Tool call

call_J7K…

Turn sequence

2 of 2

Turn state

Completed

publish_flowTRP 2865 msOct 1, 1:45 AM UTC

Outcome

Success

Thread

a4ef58fe…

Tool call

call_wPB…

Turn sequence

2 of 2

Turn state

Completed

publish_flowTRP 2834 msOct 1, 1:39 AM UTC

Outcome

Success

Thread

a4ef58fe…

Tool call

call_bb6…

Turn sequence

2 of 4

Turn state

Completed