How to Spot AI Hype — A Five-Question Field Guide

The last year of your life has probably looked like this: every app you already use has added an "AI" button. Every product launch mentions AI in the first sentence. Every LinkedIn post from a founder starts with "In the age of AI…" and ends with a pitch. Your inbox has four cold emails offering to 10x your productivity with AI.

The ratio of hype to substance is, to put it mildly, cosmic.

Here's the thing: some of it is real. Some of it is nonsense. And the difference matters. If you get fooled by hype, you waste money and attention. If you get cynical and dismiss everything, you'll miss the handful of things that are genuinely changing how work gets done.

The good news is that you don't need a CS degree to tell them apart. You just need five questions. This post is the payoff of everything we've covered so far in Module 1 — the map, the history, the tribes, the data story. It's the practical filter you can use starting tomorrow.

---

The five-question checklist

Five questions. Any product that can't answer them cleanly is probably hype. Let's walk through each one.

---

Question 1: What can it do that wasn't possible before?

This is the most important question, and the one that cuts through 80% of marketing noise.

Here's the trick: companies love to attach "AI" to things that already worked fine without AI. Your calendar had smart scheduling before it was "AI-powered scheduling." Your photo app could find sunsets before it was "AI photo intelligence." Your spam filter has been doing machine learning for twenty years.

When a pitch says "we use AI to…," quietly ask yourself: did we not have this ten years ago? If the honest answer is "sure, it's just a little better now," the AI is probably doing marketing work, not real work.

On the other hand, if the answer is "actually yeah, this would have been impossible before 2020" — that's interesting. Pay attention.

A few examples of things that really weren't possible before:

• Having a genuine conversation with a computer about a piece of text it's never seen.
• Generating a photorealistic image from a sentence.
• Translating speech between languages in near real-time, on-device.
• Summarising a 200-page document in ten seconds in a way that's not trivially wrong.

Things that absolutely were possible before and don't need the "AI" label:

• Suggesting products based on what you've bought.
• Routing customer service tickets to the right department.
• Detecting fraud in credit card transactions.
• Tagging photos by location or date.

None of those last four are bad products. They might even use modern ML under the hood. But they're not doing something genuinely new, and calling them "AI" is mostly about sounding current.

The question to ask: Would this have been technically impossible three years ago?

---

Question 2: Can they show a specific working demo?

Real AI products can be demoed. Hype cannot.

When a pitch goes abstract — "transforming industries," "unlocking productivity," "reimagining workflows" — and never shows you the actual thing doing the actual task, something's off. A working AI product has a very specific loop: here's an input, here's what it does with it, here's the output, here's why that's useful. If the pitch can't walk through that loop in thirty seconds, the product might not exist.

Watch out for these red flags:

• The "smart glass" demo. The presenter gestures at a slick dashboard with charts and says "this all updates in real time." What's updating? Based on what? Who knows.
• The pre-recorded-only demo. A real product can be poked at. A demo that's always a video is a demo that might fall apart under questions.
• The "imagine a world where" pitch. Anything that starts with "imagine" and ends with "our AI does this" is selling a future, not a product.
• The analogy deflection. "It's like having a senior analyst on your team" sounds great but tells you nothing about what the system actually does.

A good test: ask to try it on your own data. If the pitch can't handle an input you bring, you're looking at a demo that's been tuned to the exact examples in the slide deck.

The question to ask: Can you show me a specific task, on my data, end to end?

---

Question 3: What happens when the AI is wrong?

Every AI system makes mistakes. The interesting question is what happens then. A well-designed product plans for failure. A hype-driven one pretends failure won't happen.

Think about a good example: the auto-complete in your code editor. It's AI, it's often wrong, and when it's wrong you just ignore the suggestion and keep typing. No harm done. The product expects to be wrong sometimes, and the human cost of a bad suggestion is zero. That's a well-designed AI loop.

Now think about a bad example: a medical AI that spits out a diagnosis with no confidence score, no explanation, no "you should double-check this" warning. When that's wrong, someone could die. That's not a well-designed AI product — that's a liability waiting to happen.

Or consider a customer service chatbot that answers confidently even when it's making things up. The cost of being wrong is a refund, a lawsuit, or a reputation hit. Is the product designed to detect "I don't know," route to a human, and fail gracefully? Or does it just bluff?

When you evaluate an AI pitch, ask the awkward question: show me the failure modes. A team that's thought seriously about their product will have answers. A team that hasn't will get defensive, change the subject, or insist their system is highly accurate (which is almost always a red flag, because highly accurate and accurate enough for this use case are not the same thing).

The question to ask: What happens when the AI is wrong, and how do you catch it?

---

Question 4: Is the AI essential, or decorative?

This is the sneakiest question of the five, and the one that separates serious products from the wave of AI-washing you're drowning in.

Take any AI product and imagine you removed the AI. What's left? Two possibilities:

Option A — The product breaks. It no longer does what it promises. It can't function without the AI. Example: a translation app without the translation model is just a text box. A self-driving car without its perception stack is a very expensive brick.

Option B — The product is basically the same. Maybe a little worse. Maybe missing one feature. But the core thing still works. Example: a project management tool that "uses AI to suggest tags" — take out the AI and you have a project management tool that works fine, just without automatic tagging.

Option A is what a real AI product looks like. The AI is doing the job. Remove it and there's nothing. Option B is mostly AI-washing: a perfectly reasonable product that bolted AI onto a side feature so it could be mentioned in the press release.

Both are valid as products — Option B isn't fraud, just marketing. But if you're trying to figure out whether you're seeing a genuine AI breakthrough or a landing page update, this question cuts to it fast.

The question to ask: If you took the AI out, would this product still work?

---

Question 5: Who's accountable when it's wrong?

Software has always made mistakes, and there's always been a clear chain of responsibility: if your database corrupts, someone fixes it; if your code crashes, someone's paged. AI systems make this murkier, because the "why" of a mistake often can't be traced cleanly.

When you talk to a team building a real AI product, they know this and they've thought about it. They can tell you:

• Who reviews the AI's outputs before they affect a real decision.
• What the escalation path is when a user reports an error.
• How they test for regressions when they update the model.
• What their policy is for "the AI was wrong and a customer got hurt."

These are boring, operational answers, and they're enormously important. "We have a human in the loop for sensitive decisions." "Users can flag errors and our team reviews them within 24 hours." "We run a test suite against every model update."

If you ask who's accountable and the answer is vague — "the AI gets better over time" or "we use state-of-the-art models" or "our enterprise plan has an SLA" — it probably means the team hasn't thought about this. Which means they'll figure it out for the first time when something goes wrong, and that's usually an expensive way to learn.

The question to ask: Who's responsible when it's wrong, and what's their process?

---

Real vs hype, side by side

Here's the same test applied to two fictional pitches — one that's probably the real thing, one that's probably marketing:

Notice something: the "hype" column isn't lying. It's all technically true. It's just that the five questions shrink the claim down to what it actually is — a reasonable product with a reasonable feature, not an AI breakthrough.

What this checklist is not

Two clarifications before we wrap.

First, this checklist isn't about catching liars. Most AI-washing isn't deliberate deception — it's overclaim. Founders are enthusiastic. Marketing teams want to ride a wave. Sales decks have to sound exciting. The five questions aren't accusations; they're filters. They help you figure out what you're actually looking at.

Second, failing the checklist doesn't mean a product is worthless. Plenty of products that are mostly AI-washing are still useful products. A project management tool with a mediocre AI tagging feature is fine — buy it or don't, based on whether the project management features are good. The checklist just helps you stop overpaying for the "AI" part when the real value is somewhere else.

The goal isn't to become cynical. It's to become calibrated: to get excited by the things that are actually new, and to shrug at the things that are just marketing. That's a much more useful state than permanent skepticism.

---

What just clicked

You started Module 1 with "AI" as a single buzzword and a lot of confusion. You're ending it with:

• A map — the four nested circles of AI, ML, DL, LLM.
• A story — how we got here, with two winters and one pivotal year.
• A framework — three tribes, collaborating in every real system.
• A root cause — why 2012 happened, and what "the bitter lesson" implies.
• A filter — five questions to separate real AI products from hype.

That's the whole foundation you'll need for the rest of this course. Everything from here on builds on it.

In Module 2, we start going deeper — into how machines actually learn. Not the math, not the code, but the mental model of what learning is, what can go wrong, and why the phrase "garbage in, garbage out" is basically the whole story of the field. Module 1 was the map; Module 2 is the first real territory we're going to walk through.

See you there.

⬅️ Previous: M1.4 — Why data changed everything 🏁 You've just finished Module 1.

---

Course navigation

📚 AI Zero to Hero · Course Home — all 33 posts, six modules.

---

Cover photo via Unsplash. This post is part of the AI Zero to Hero series.