[taciturnaxolotl]

Two ways a turn could fail to end, and one change to how timeouts are retried.

Cancelling didn't always cancel. Waiting on the network, a cancel works. But once a provider has flushed its response into memory there's nothing left to interrupt, so the turn keeps draining the backlog (thousands of chunks, for a large tool call) while you watch. Every request now goes through one guard that stops the stream at the first part after a cancel. It does this on every model, whether or not a request timeout is set, and the stream always ends by saying why it stopped. Stopping quietly looked like a stream that finished normally, which left the half-written tool call spinning forever. A tool cut off by the cancel is recorded as interrupted and shows as cancelled rather than as a red error. Tools that had already finished keep their output.

A silent connection hung forever. Providers send content-free keepalives while a model thinks, and honoring those is what keeps a long think from being cut off as a dead connection. The catch is that they reset the idle timer indefinitely, so a connection that stays open and never produces anything (a sleeping laptop, from this side) never times out. Keepalive-only streams now run against their own budget of ten times the request timeout (20 minutes at the default). The error says whether the model went quiet or the connection died, instead of blaming the model for both.

Timeouts are now retried. A request timeout used to look like a deliberate abort, so it was never retried. It now tells the retry loop directly that it is worth retrying, using a new contract in fantasy. That covers a non-streaming request that runs past its deadline, a stream that goes silent, and a stream that sends only keepalives. Silence usually means the network dropped, which is worth another try. But it also means a request that genuinely keeps timing out takes up to the retry count times longer to give up. Setting request-timeout to 0 still disables the timeouts. Cancellation keeps working either way.

Depends on charmbracelet/fantasy#438. This PR pins fantasy to a pseudo-version of that branch, which also brings in fantasy's latest changes and its go 1.27.3 requirement. Once #438 is released, the pin should move to the tag before this merges.

💘 Generated with Crush