Every remote call needs a deadline, so every codebase eventually grows timeouts. What gets lost along the way is what the timeout means. A timeout is a decision the caller makes about its own patience. It is not information about the callee.

When a request times out, three worlds are consistent with what you observed: the request never arrived, it arrived and failed, or it arrived and fully succeeded while the response was lost. Nothing in the timeout distinguishes them.

Unknown is a third outcome

Most code is written as if calls either succeed or fail. Networks add a third outcome: unknown. Treating unknown as failure is the usual bug. It is how a payment gets charged twice, an email gets sent twice, or a counter drifts a little further from the truth on every retry.

  • Success: you received a response and it said so.
  • Failure: you received a response and it said so.
  • Unknown: you received nothing, and the work may already be done.
A retry is only safe when the operation is idempotent, or when you can ask the other side what happened.

Making unknown safe

There are two honest ways out. The first is idempotency: give the operation a client-generated key so the server can recognise a repeat and return the original result instead of doing the work again. The second is a read-back: after a timeout, query the resource and let its current state decide whether to retry.

text
POST /payments   Idempotency-Key: 8f3c…
  ↳ timeout        outcome unknown
  ↳ retry same key server recognises it, returns the first result
  ↳ no key         second charge

Deadlines beat per-hop timeouts

A single request often fans out across several services. If each hop sets its own three-second timeout independently, the total wait multiplies and the user watches a spinner for far longer than anyone intended. Passing an absolute deadline down the call chain fixes this: each hop gets whatever time is left, and work whose answer nobody is waiting for gets cancelled instead of finishing into the void.

Choosing the number

Timeouts derived from measured latency age better than round numbers. Start from the high percentile of your own observed latency, add headroom, and revisit it when the distribution moves. A timeout far above the real p99 stops being a protection and becomes a queue; one below it turns healthy requests into failures and, with retries attached, into extra load exactly when the system is least able to absorb it.

The useful mental shift is small: stop reading a timeout as an answer about the server, and start reading it as a statement about your own willingness to wait. Everything after that — idempotency keys, deadline propagation, jittered backoff, retry budgets — follows from taking the unknown outcome seriously.