Chat tool telemetry

Find slow, failing, and repetitive agent tools without exposing prompts, arguments, or outputs.

PeriodToolTeam

Detailed production timing active

6101 of 6166 saved tool calls include execution duration.

98.9% timing coverage

Tool calls

+900% vs previous 7d9 saved production turns

p50 latency

781 ms

+78% vs previous 7d10 timed calls

p95 latency

5.4 s

+1133% vs previous 7d10 timed calls

Tool failures

0.0%

No change vs previous 7d0 excluding NoResults

No result

0.0%

No change vs previous 7d0 product outcomes

Calls / turn

1.1

+11% vs previous 7d100.0% of turns at 5+ calls

What changed

Material regressions ranked by severity and affected call volume.

Current 7 days compared with the previous equal window

Stable against baseline

No material tool regressions detected

Tools with enough activity were checked for higher failure rates, available p95 latency regressions, more no-results, and increased loop pressure.

Review the tool comparison for absolute hotspots that may still need a product threshold.

Investigate

Regression checks require at least 3 calls in both windows. Latency is flagged at +20%; failure and no-result rates are flagged at +1 and +3 points respectively.

Execution volume

Successful calls, no-results, and failures combined.

Calls

Oct 1Oct 4Oct 7

Payload pressure

How often results exceed the 20 KB inline cap.

5.0%

handle rows / all calls

309 stored handles in this period. This is a payload-size proxy, not a one-to-one call classification.

Tool comparison

Each change is measured against the previous equal window; latency appears only where detailed timing exists.

Sorted by current p95 latency

Tool	Calls	p50	p95	Failures	No result
app_delete_file	10+900% vs prior	781 ms+78% vs prior	5.4 s+1133% vs prior	0.0%No change vs prior	0.0%No change vs prior

Failure codes

Operational failures only; NoResults is excluded.

No operational failures

in the selected view

Interpretation notes

The data has known edges that change what a chart means.

Blocked loops are invisible: Rate-limited and identical-argument calls are rejected before telemetry fires.
NoResults is not downtime: It remains a failed tool outcome, but is separated from runtime reliability here.
Preview volume lives elsewhere: Preview and threadless chats reach PostHog but are intentionally absent from Postgres.
Latency is partially instrumented: Saved tool parts provide production outcomes and volume; duration appears only on calls with detailed telemetry.
Partial turns are retained: They remain in volume counts but are excluded from failures and latency percentiles.
Comparisons use equal windows: Every delta compares the selected period with the immediately preceding period under the same tool and team filters.

Recent sanitized calls

Arguments, outputs, user IDs, and message content are never rendered.

Privacy safe

app_delete_fileHJC783 msOct 6, 6:46 PM UTC

Outcome

Success

Thread

d41e378a…

Tool call

call_38P…

Turn sequence

21 of 29

Turn state

Completed

app_delete_fileHJC781 msOct 6, 6:46 PM UTC

Outcome

Success

Thread

d41e378a…

Tool call

call_kem…

Turn sequence

22 of 29

Turn state

Completed

app_delete_fileHJC5.4 sOct 6, 5:09 PM UTC

Outcome

Success

Thread

b9ce5727…

Tool call

call_x0Z…

Turn sequence

21 of 39

Turn state

Completed

app_delete_fileHJC5.3 sOct 6, 2:06 PM UTC

Outcome

Success

Thread

940968fd…

Tool call

call_JrX…

Turn sequence

30 of 40

Turn state

Completed

app_delete_fileHJC5.2 sOct 6, 1:22 PM UTC

Outcome

Success

Thread

940968fd…

Tool call

call_GME…

Turn sequence

49 of 55

Turn state

Completed

app_delete_fileHJC5.2 sOct 6, 12:17 PM UTC

Outcome

Success

Thread

940968fd…

Tool call

call_gP7…

Turn sequence

25 of 36

Turn state

Completed

app_delete_fileHJC432 msOct 6, 2:19 AM UTC

Outcome

Success

Thread

bf677805…

Tool call

call_B1d…

Turn sequence

12 of 19

Turn state

Completed

app_delete_fileHJC420 msOct 6, 2:07 AM UTC

Outcome

Success

Thread

bf677805…

Tool call

call_puE…

Turn sequence

16 of 23

Turn state

Completed

app_delete_fileHJC544 msOct 5, 5:05 PM UTC

Outcome

Success

Thread

7518cd52…

Tool call

call_aXI…

Turn sequence

24 of 32

Turn state

Completed

app_delete_fileBidlo93 msOct 2, 5:52 PM UTC

Outcome

Success

Thread

4750a76f…

Tool call

chatcmpl…

Turn sequence

2 of 6

Turn state

Completed