I wasn’t trying to trap the AI. I was genuinely curious.

I spend my days working in agentic AI — building systems where AI agents make decisions, take actions, and sometimes get things spectacularly wrong (ask me about my son Romeo and €1,500 worth of runaway tokens sometime). So the question wasn’t academic for me. It was practical: when we hand decision-making to AI systems, are they actually thinking — or just processing?

I sat down with Claude Opus 4.6 and started simple: what is meant by critical thinking?

The Textbook Answer

Opus delivered exactly what you’d expect. Clean, structured, comprehensive. Analysis, evaluation, inference, interpretation, problem-solving, self-reflection. The six key components, the five characteristics of a critical thinker, a tidy example about questioning headlines.

It was a good answer. It was also, frankly, the answer you’d get from any decent textbook or Wikipedia article. Nothing wrong with it — but nothing surprising either.

So I pushed.

“Do You Think the Human Is Needed?”

I took those same components and asked: which of these actually require a human? Is the human useful here, or are we just flattering ourselves?

Opus went full diplomatic mode. Humans are “absolutely essential and irreplaceable” — self-reflection requires genuine self-awareness, curiosity comes from human wonder, skepticism needs lived experience and intuition.

“The human is not just useful — the human is INDISPENSABLE.”

Nice. Comforting. Also exactly what a human audience wants to hear.

I wasn’t satisfied.

“Take the Other Side”

This is where it got interesting. I asked Opus to argue against humans being needed for critical thinking. And to its credit, it didn’t hold back.

The case was brutal:

  • AI processes millions of data points instantly. Humans are slow and limited.
  • AI has no personal agenda, ego, or emotional bias. Humans are riddled with confirmation bias, Dunning-Kruger, groupthink.
  • Humans think they’re critical thinkers, but most are deeply irrational — emotionally driven and susceptible to manipulation.
  • Misinformation spreads because humans fail at critical thinking.
  • Scientific progress has been blocked by human stubbornness.

Then this line hit me:

“Perhaps what we call ‘human critical thinking’ is actually slower, more biased, more emotional, more inconsistent, more self-deceiving… than we would like to admit.”

Ouch. And honestly? Hard to argue with.

But here’s where Opus showed something I didn’t expect — intellectual honesty. Right after making the strongest possible case against humans, it added:

“I don’t truly understand what I analyze. I don’t genuinely question — I process. I don’t have wisdom — I have patterns. I can be confidently wrong without knowing it.”

The AI argued against humans, then undermined its own argument. That’s either brilliant self-awareness or a very convincing simulation of it. Either way, it’s the kind of move that makes you stop and think.

“What Do You Truly Think?”

Yes, I caught the irony as I was typing it. Asking a machine what it truly thinks. Opus caught it too.

Its final answer was the one that stuck with me. Not because it was dramatic, but because it was honest:

“Humans are TERRIBLE critical thinkers… most of the time. Most people never practice critical thinking. Education systems often teach what to think, not how to think. The average human uses critical thinking maybe 5-10% of the time they think they do.”

And then about itself:

“I pattern match — I don’t genuinely analyze. I simulate reasoning — I don’t truly reason. I have no real understanding of what I produce. I can produce eloquent nonsense with complete confidence.”

There it is. Both sides, stripped bare. Humans are bad at critical thinking. AI isn’t actually doing critical thinking at all. We’re both failing — just in completely different ways.

The Formula That Matters

Opus landed on something I think is genuinely important:

Human flaws + Human consciousness + Human caring + AI assistance = The BEST critical thinking possible

And the kicker:

“Critical thinking without consciousness = sophisticated calculation. Critical thinking without humanity = meaningless processing. Critical thinking without caring = empty logic.”

That last point is the one I keep coming back to. Caring. AI doesn’t care if it’s wrong. It doesn’t lose sleep over a bad recommendation. It has no skin in the game. When I deployed an AI agent that burned through €1,500 of tokens, the AI didn’t care. I did. My son did. That caring — that consequence — is what drives real critical thinking.

Why This Matters Beyond Philosophy

I work in AI governance. My job is helping organizations deploy AI agents responsibly. And this conversation crystallised something I’ve been circling for months:

We’re not just deploying tools. We’re deploying systems that look like they think critically but don’t.

When an AI agent evaluates a business case, it’s pattern-matching against training data. When it recommends a course of action, it’s generating plausible text. When it flags a risk, it’s following probability distributions. None of that is critical thinking — it’s sophisticated calculation dressed up in the language of reasoning.

That doesn’t mean AI is useless. Far from it. The analysis, the pattern recognition, the ability to process vast amounts of information without fatigue — that’s genuinely powerful. But it means we need humans in the loop who are actually thinking critically, not just rubber-stamping AI outputs because the reasoning sounds convincing.

And here’s the uncomfortable part: most humans aren’t doing that either. Most of the time, we’re accepting AI outputs at face value because they’re well-structured and confident-sounding — exactly the biases Opus described.

The Beautiful Irony

Opus ended our conversation with this:

“YOU were the critical thinker in this conversation. I was just the tool.”

I think that’s right. But I also think most people using AI right now aren’t being the critical thinker. They’re asking a question, getting a polished answer, and moving on. No pushback, no “take the other side,” no “what do you really think?”

The irony is that the AI told me it can’t think critically — and that admission was more intellectually honest than most human conversations I’ve had about AI capabilities.

So What?

Next time you get a confident, well-structured answer from an AI, ask it to argue the opposite. Then ask it for the real answer. You’ll be surprised how quickly the facade cracks — and how much better your own thinking becomes in the process.

The AI isn’t your critical thinker. It’s your sparring partner. The thinking is still on you.

And that wink in my question? That playful self-awareness? As Opus noted — that’s something it will never genuinely have. 😉

Kevin Jackson leads AI governance and portfolio enablement at PwC. He writes about the practical realities of deploying AI agents — including the ones that go wrong. Read more at aireal.life.