Can Machines Think?

The paper is called Computing Machinery and Intelligence. I thought it would open with an argument about whether machines can think. It doesn't. Turing opens with the question itself.

“Can machines think?”

I tried to answer it before reading further. My gut said yes. Machines solve problems, write code, explain things I can't follow. But what was I even calling thinking? If solving a problem counts, what about understanding it? And if a machine explains its own answer, how do I know it understands that explanation?

So I was stuck, and I was a page in.

Turing didn't try to settle it. He suggested the imitation game instead, a way to test what a machine does without agreeing first on what thinking is.

The game

In the original version, an interrogator talks to two people they can't see and has to figure out which is which. One tries to fool them. The other tries to help. Then Turing swaps a machine into one of those seats and asks if it could answer well enough that the interrogator can't reliably tell it from a human.

That's the Turing test.

Simple to explain, but I kept getting stuck on what it measures. The interrogator only judges answers. They can't open the machine up or find out what it experiences. Maths, poetry, anything, the replies are all they get.

Say you ask an AI to write about grief. It might mention the small stuff people miss after losing someone. Reaching for the phone. Expecting a voice that isn't there. Remembering something ordinary that suddenly matters a lot. Reading it, you might believe it.

But the machine hasn't necessarily gone through any of that. Does that make the writing less intelligent? Or am I confusing describing an experience with having it?

I don't know. I see why Turing went with behaviour, since that's how we judge other people's minds as well. We go by what they say and do. But a convincing answer still doesn't tell you what's behind it. I wanted to know if Turing had an answer to that. What I got from the paper was a distinction I'd never really thought about: judging what a machine can do and knowing what it experiences aren't the same job.

The things machines supposedly cannot do

A big chunk of the paper goes to objections. Some people said machines can't think because they have no soul. Others said human intelligence is just a different kind of thing. There were arguments about creativity, free will, and whether a machine could ever surprise the person who built it.

That last one made me stop. If a machine follows instructions, how does it give you something you didn't expect?

Then I thought about it more. A programmer can know every rule in a system and still be unable to predict what it outputs. Programs do unexpected things while doing exactly what they were designed to do. That doesn't mean consciousness. It does make the objection less solid than it sounds.

I see versions of this argument today. People say with a lot of confidence what machines will never do, usually based on what current systems can't do. Some of those limits are real. I don't think a bigger model fixes everything. But how would you tell a real limit from a problem nobody's solved yet?

Turing was asking this in 1950, when computers were nothing like what we have now. I won't pretend he predicted modern AI. What gets me is that he questioned these assumptions at all instead of accepting them.

Maybe we should teach machines like children

This was the part I'd been waiting to read.

Turing took on Lady Lovelace's objection, the one saying a machine can only do what it's told. One idea he explored was building a machine like a child's mind and then educating it, rather than trying to program a full adult mind from day one.

I reread that bit, because it shifted how I pictured the problem. A machine doesn't need an instruction for every situation if it can learn from examples and feedback. So the hard part becomes how you build the learning, and what happens when the machine picks up abilities nobody specified.

It made me think of the models I work with. We train them on data, look at the outputs, try to improve them. Different methods from what Turing described, same kind of question though. How do you build something that can handle more than the exact examples its builders thought of?

And this is where it gets complicated. A model can pick up patterns and still be unreliable on plenty of tasks. It can score well on an eval and then break when the inputs change a little. If I'm building an AI app, one great response doesn't cut it. I need to know what happens when the information is incomplete, or the question is strange, or the system is simply wrong.

I don't have answers there either. But the learning machine idea gave me another angle on intelligence, and on what kind of AI systems I want to build.

The ten tests can wait a little

In my last post I wrote down ten tests I'd want an AI to pass before calling it AGI. I wanted to line up what today's systems can do against what we'd expect from general intelligence.

After reading Turing, I want to go back to that list.

A machine might pass a conversation test, get through a hard exam, or manage a task in an environment it hasn't seen. Each of those tells us something, and I need to be more careful about what. Passing a test doesn't answer everything about intelligence. Failing one doesn't mean much about what a system could do under other conditions.

I was collecting tests. Now I want to understand them.

That's what I'll look at next. What the tests measure, what the evidence shows, and where people claim more than the results support.

Turing started with a question in 1950. I showed up seventy-six years later thinking I needed a definition. Still don't have one.

For now I've got ten tests to revisit and a paper that made me doubt how I was using them.

Prem signing off. ✍️

Originally published on Medium.