Cost and Latency: the Two Dials Users Feel
Your users will never read your model card. They will feel two things: how fast your app is and how much it costs you per answer. Here is how to reason about both without a spreadsheet.
Apr 13, 202610 min read
