Tool calling that holds up in production

A tool call fails in one of four ways, and only one of them is the model's fault. The other three are ours: a schema the model cannot satisfy, a dispatcher that swallows errors, and a retry policy written by optimism. This piece walks through the loop I now ship, the one repair pass that fixed most of it, and the traces that decided where to stop.

What actually breaks

Over 30,000 calls a day in a support agent, malformed arguments accounted for 61% of failures. Timeouts were 22%. Genuinely wrong tool choice — the failure everyone worries about — was 9%. The rest was infrastructure.

Four failure modes

Two of these are visible in the transcript; two only show up in traces. The loop below handles all four, and the article's repository has the eval suite that proves it: $ pnpm eval:tools --suite retries.

14export async function runToolCall(call: ToolCall, ctx: Ctx) {
15  const parsed = TOOLS[call.name].args.safeParse(call.args);
16

17  if (!parsed.success) {
18    const repaired = await repairArgs(call, parsed.error, ctx);
19    if (!repaired) return hardFail(call, "schema");
20    call = repaired;
21  }
22

23  const result = await withTimeout(dispatch(call), 8_000);
24  if (!result.ok && result.retryable) {
25    ctx.trace.note("retry", { call: call.id });
26    return runToolCall({ ...call, attempt: 2 }, ctx);
27  }
28  return result;
29}

Schema drift

The quiet one. A field changes type in the backend, the tool schema follows a week later, and in between the model keeps sending what used to be right. Validation catches it; logs only catch it if you log the parse error with the call id.

# 200 recorded traces, replayed
$ pnpm eval:tools --suite retries
pass 187/200  ·  repaired 9  ·  hard-fail 4
schema drift: expected string, got number (call_id 41)
wrote traces to .evals/2026-08-18T09-12.jsonl

The repair pass

When validation fails, the model gets one more turn with the parse error attached — not the raw stack, the sentence a human would write. The prompt is deliberately short; long repair prompts made things worse in every run I measured.

You repair malformed tool arguments.
Return only the corrected arguments object.
Never add fields the schema does not define.

Your previous call to {{tool}} was rejected:

  {{error}}

Return corrected arguments only. If the call cannot be repaired, return {"abort": true}.

The shape of the whole path, cache included:

const MAX_REPAIRS = 1; // measured on 200 traces, not guessed

What one retry costs

Latency, mostly. A repaired call is a second model round-trip, so the p95 on repaired calls is roughly double. That is fine when repairs are 4.5% of traffic and not fine at 30% — at which point the schema is the bug.

The numbers

What I would ship tomorrow

Validate before dispatch. Repair once, with the error in plain language. Give up loudly, in a way the product can explain to a person. Then go read your traces — the interesting failures are never the ones in the bug reports.