Prompt Caching: First vs Repeat Request Costs

<p>You've built a chatbot. Every turn, you re-send the whole conversation — the 8,000-token system prompt, the uploaded PDF, the 15 messages of history — just so the model can

<p><em>Originally published at <a href="https://theculprit.ai/blog/anthropic-prompt-caching-90-percent" rel="noopener

<p>Prompt caching is not a setting you enable. It is a property your prompt either has or does not have, and one timestamp in the wrong place removes it from every request you

<p><strong>TL;DR: We turned on vLLM's prefix cache for our agent workloads at Nexus Labs and watched TTFT drop from 480ms to 110ms on one tenant and stay exactly the same on

<p> </p> <p>A single-model session that stays well-cached is cheap. The biggest swing in a multi-model bill comes from one move — <strong>switching models</strong> — because the

<p>GPT-5.4 shipped on March 5. Seven weeks later, on April 23, 2026, OpenAI released GPT-5.5 — and the pace alone tells you something about where this race is headed. This isn't