Three Kinds of Caching: Prompt, Semantic, Result
Three distinct caches you can wire up for an LLM app. Each one wins on different workloads. Here is which to reach for, in which order, and the failure modes you only find at scale.
Apr 13, 202612 min read
