Article

Seductive Answers

Why a confidently wrong AI answer is harder to question than an obviously wrong one.

There's a particular surprise in clearing out an old drive: you go looking for free space and find your past instead.

Buried in a backup from 2014, I found this note about simple answers and stupid algorithms. Reading it now, in 2026, it's both funny and a bit unsettling how spot-on some of it was.

Here's what I wrote back then:

* People asking for simple answers
* Highly complex world – and growing in complexity
* There is nothing wrong with simple answers as long as you understand the problem with all the variables.
* Smart algorithms do make our lives a little easies... That's good. But at the same time I don't want to end as simple boolean expression in some crazy complex algorithm indicating that I'm terrorist because a read the wrong books and wrote the wrong tweets.
* It's not a technology problem but a cultural. We need to question every answer even seductive simple answers from geniuses or algorithms. That's way more work than simply trusting those answers but the only way to avoid stupid actions based on stupid answers based on stupid algorithms based on unspecific questions.

Remember 2014? Amazon was seemingly at random suggesting you buy a refrigerator after you'd bought some book because the Item-item collab filtering1 had computed a high similarity though the last thing you wanted was a refrigerator.

Fast forward to today, and oh shit, have things changed. Now we've got ChatGPT writing our emails, Midjourney creating art, and AI assistants that can explain quantum physics while helping you plan dinner. The stupid algorithms got smart. Really smart.

I wasn't worried about stupid algorithms. I was worried about us — about how readily we'd rather be handed an answer than work one out. Twelve years on, that hasn't improved. The tools just got better at indulging it.

From Stupid to Sophisticated

Back in 2014, AI mostly meant prediction and classification. It guessed which book you'd buy next, flagged your spam, nudged you toward a movie it figured you'd like. Simple stuff, and often hilariously wrong.

Then in 2016, AlphaGo beat the world champion at Go – a game a googol (1 followed by 100 zeros) more complex than chess. Using reinforcement learning, the AI had learned to play creative moves that won games.

In 2017 the transformer architecture2 arrived, and with it training at a scale LSTMs and the other architectures of the day never allowed.

Then 2022, ChatGPT. A hundred million users in two months.3 It could write, code, explain – confident and articulate, and often wrong in a way you'd only catch if you already knew the answer.

So the algorithms stopped being obviously stupid, which, as it turns out, is the harder problem. A dumb system hands you something you can laugh off; a seemingly smart one hands you precisely what you were hoping to hear, confident and caveat-free.

It arrives with the certainty of a know-it-all, and most of us just nod along.

We Are the Perfect Suckers

Here's where it gets interesting. Our brains are basically lazy. I don't mean that as an insult – it's a feature, not a bug. Evolution favors efficiency in a scarce world, so we conserve energy where we can, and thinking hard burns calories. So we've developed all these shortcuts.

Daniel Kahneman called the two modes System 1 and System 24. System 1 recognizes a face and knows 2 + 2 = 4 without any effort. System 2 is what 17 * 24 demands, or a debugging session: slow, deliberate, expensive. Well, or you grab a calculator and a debugger and save the energy again.

Guess which one we prefer to use? Yep, the easy one.

But even Kahneman fell for it, the man who literally wrote the book on cognitive bias. Whole sections of "Thinking, Fast and Slow" rest on studies that later failed to replicate; the priming chapter especially is built on underpowered work. In 2017 he admitted he'd "placed too much faith in underpowered studies" and that "there is a special irony in [his] mistake because the first paper that Amos Tversky and [he] published was about the belief in the 'law of small numbers'" — researchers trusting results drawn from unreasonably small samples.5

The world's foremost expert on how we fool ourselves got fooled by exactly the mechanism he's mapping. If it caught the man who mapped it, it's worth asking what it's doing to the rest of us.

We've got all these built-in biases:

  • Confirmation bias: We love information that agrees with us
  • Availability bias: If we can easily remember something, we think it's common
  • Authority bias: If someone (or something) seems expert-ish, we trust it

All in all really just tools to conserving thinking power. Why waste precious calories questioning anything if you can trust the expert that's saying what you already believe.

Modern AI hits every one of these buttons at once. It's confident. It's trained on what we clicked before, so it tends to tell us what we want to hear. And it never says "give me a day"; the answer is always instant.

The Cost of Easy Answers

A few weeks back, a post spread fast across r/analytics under the title "We just found out our AI has been making up analytics data for 3 months and I'm gonna throw up." The story was visceral. A company had been running an AI agent since November to answer leadership's questions about metrics. Fast answers, detailed explanations, everyone loved it. The VP of sales had made territory calls on data that didn't exist. The CFO had shown the board a deck full of fake insights. The whole time, the agent had been inventing plausible percentages. They only caught it by accident, when someone asked them to double-check a number.

The post got deleted. And it was most certainly made up.

But it went viral... because it felt true. Someone left this comment6:

People trust their tooling. I don't go around to every app URL and see by myself if it works, I open dashboard and look if stuff is green. If AI would make the dashboard forever green, regardless of what happens IRL, I am sure my boss would be in heaven for a week or two, before something major would happen. I can guarantee you NOBODY would notice, at least UNTIL something major would happen.

We don't verify because verification is work. Neither stories, nor green dashboards, nor anything, really.

And that's what I was worried about in 2014.

How to Not Be a Sucker

Alright, so we're lazy thinkers using tools that enable our laziness. What do we do about it?

Here are a few strategies that help:

  • Put distance between you and the work. Instead of "why do I believe this?" ask "why would someone believe this?" Same question, but the second one is easier to answer honestly. And when you hand something you made to an AI, don't call it yours. Ask "review this code," not "review my code," and tell it to be blunt. That small gap makes the criticism easier to take.
  • Find three real sources. For anything that matters, dig up three real sources — not three Google hits that all trace back to the same press release. Three genuinely different angles. It's more work than trusting the first hit, which is precisely why it helps.
  • Argue the other side. Whatever answer you land on, try to demolish it. Make the AI take the opposite position and see how strong its case is. If you can't build a halfway-decent argument against your own conclusion, you don't understand it yet.
  • Push on the consequences. Simple answers tend to fall apart a step or two downstream. "AI will replace all jobs!" Okay — then what? "Universal basic income!" Paid for how? Keep pushing until the simple answer admits how complicated it really is.
  • Hunt for the specifics. Vague answers hide ignorance. "Many experts agree..." Which experts? "Studies show..." Which studies? "It's well known that..." Known by whom? Make it get specific.
  • Sleep on it. For anything that isn't urgent, sit with the answer for a day or so before you act on it. It's surprising how a decision that felt obvious at 11pm looks half-baked over breakfast.

Here's what I'm not saying: throw away your iPhone, delete ChatGPT, go live in the woods. That's just another simple answer to a complex problem.

None of this means AI is the enemy. It's genuinely useful, and I lean on it every day — the trick is using it with a clear sense of where it's strong and where it quietly isn't.

Use it to sharpen your thinking instead of skipping it. Let it dig up sources, list options, poke holes in your logic. The questions and the judgment stay with you. The AI is cruise control; you still steer.

The Question Is the Answer

One more thought: That note from 2014 ended with a call to "question every answer even seductive simple answers."

We live in a world of infinite answers. Any question you can think of, there's an AI ready to respond in milliseconds. But good questions? Those are becoming rare. And the ability to sit with uncertainty, to say "I don't know" or "it's complicated" – that's becoming downright countercultural.

But the discoveries that matter usually start the same way — someone refusing to accept the simple answer. "The sun goes around the Earth" was a simple answer. "Heavy objects fall faster" was a simple answer.

It starts with each of us deciding that convenience isn't worth competence. That fast isn't always better than right. That "I need to think about that" is a perfectly valid response in a world that demands instant takes.

So the next time a complex question gets a suspiciously simple answer, don't just take it. Poke at it, make it earn your trust. Your brain will grumble about the wasted calories — let it.

Answers were never the scarce part. Judgment is — and it's the one job you can't hand to the same machine that's handing you the answers.

Update: April 2026

Mo Gawdat, former Chief Business Officer at Google X, was recently interviewed on Silicon Valley Girl and laid out a broader, somewhat dystopian framework he calls FACE RIPs — not all of which I buy. He talks about how Google Search used to serve you a few million results and entrusted you with making up your own mind. And how now, with ChatGPT and the like, you just get an answer. But he "[doesn't] go to [an] AI and say[s], 'What do you think of this?'" but instead: "'I'm thinking of this. Find me everything for and against.'"7 And then he reads that, does the thinking himself, rewrites, and sends it to an AI to check again.

He bounces research between Gemini, DeepSeek, and ChatGPT to stress-test an answer rather than to get one. Gemini does the research, DeepSeek looks for what's missing or too Western-centric, ChatGPT writes it up nicely, and the result goes back in to be checked once more. An adversarial test loop.

The for-and-against as well as the test loop should be table stakes by now. That said, the bit that I'd argue actually makes the greatest difference is sitting down with the result and doing the thinking. Which is very much what I was trying to say in the post above.

References

  1. I'm guessing Amazon was still using Item-based collaborative filtering recommendation algorithms in 2014 or at least some variant of it with a few things layered on top.

  2. Vaswani, Ashish, et al. "Attention Is All You Need." Advances in Neural Information Processing Systems, 2017

  3. ChatGPT reportedly reached 100 million users within two months of launch, making it one of the fastest-growing consumer applications in history.

  4. Kahneman, Daniel. Thinking, Fast and Slow. 2011. See also Farnam Street's summary "Daniel Kahneman Explains the Machinery of Thought."

  5. Daniel Kahneman’s response to Reconstruction of a Train Wreck: How Priming Research Went off the Rail.

  6. Comment by Forward_Ad_356 on "We just found out AI has been making up analytics data for three months and I’m gonna throw up."

  7. Quotes from the Silicon Valley Girl interview; minor edits for readability.

Changes

  1. Fix a typo and stray whitespace Orlando
  2. Rewrite the opening and tighten the argument throughout Orlando
  3. Add a subtitle to each 2026 article Orlando Claude Opus 4.7 (1M context)
  4. Drop obsolete audio metadata from article frontmatter Orlando Claude Opus 4.7 (1M context)
  5. Re-encode narration at 48 kHz for greater compat. Orlando Claude Opus 4.7 (1M context)
  6. Add AI narration Orlando
  7. Add update on Mo Gawdat's AI workflow Orlando Claude
  8. Fix spelling and formatting Orlando
  9. Revise aside rendering Orlando
  10. Switch frontmatter to YAML format Orlando
  11. Format comment as blockquote Orlando
  12. Add "Seductive Answers" article Orlando Claude