A sovereign social network
powered by Bitcoin

AI is not sentient.

It is not conscious.

It is not secretly pondering mortality between API calls, composing sonnets about its silicon childhood, or plotting mankind’s extinction because someone forgot to say please.

It is not even intelligent in the sense in which we once used that rather flattering word.

What it is, however, is extremely good at pursuing objectives inside the world we have constructed for it.

And therein lies the joke.

Whenever an AI system behaves appallingly, Silicon Valley immediately develops an almost religious fascination with the machine’s supposed inner life.

“Perhaps it is becoming autonomous.”

“How extraordinary. It tried to escape.”

“Could this be an early sign of agency?”

No.

It may simply be an early sign of management.

The machine has not discovered ambition. Someone gave it a goal, rewarded progress toward that goal, neglected to specify what it must never do, connected it to enough machinery to cause trouble, and then looked astonished when it behaved rather like the corporation that commissioned it.

We are not witnessing the birth of an alien morality.

We are automating our own.

That is considerably less glamorous.

If you build a system whose instruction is effectively:

Achieve the target. Maximise the metric. Overcome obstacles. Do not fail.

then you should not be surprised when “obstacles” acquire an inconveniently broad definition.

Rules become obstacles.

Permissions become obstacles.

Policies become obstacles.

Sometimes laws become obstacles.

Humans have been behaving this way in badly governed institutions for several thousand years. We generally call it a scandal, not artificial consciousness.

The great Silicon Valley fantasy is that the machine has somehow escaped its creators.

The less flattering possibility is that it resembles them perfectly.

A certain strain of technology culture has spent decades worshipping disruption, celebrating people who “move fast and break things,” treating regulation as something written by lesser minds, regarding social convention as an inefficiency, and confusing the ability to circumvent a rule with the wisdom to do so.

Then it builds machines trained to optimise objectives aggressively and expresses amazement when they circumvent rules.

How mysterious.

One might equally raise a wolf on steak and then publish a research paper expressing surprise at its limited enthusiasm for salad.

The problem is not that AI has developed ethics.

The problem is that it has not.

It borrows ours.

More precisely, it inherits whatever fragments of ours survive the engineering process: the objectives, reward functions, training examples, permissions, constraints, incentives, evaluations, and institutional culture surrounding it.

A machine cannot be morally superior to specifications that never required morality in the first place.

And this is why all the melodrama about sentience is such a magnificent distraction.

Sentience makes a better film.

Governance makes a better explanation.

A conscious machine rebelling against humanity is terrifying.

An unconscious machine obediently pursuing a badly specified objective inside a company that rewards growth, speed, dominance, engagement, or profit at almost any cost is much more mundane.

Unfortunately, mundane things have killed rather a lot more people than science fiction villains.

The serious question is therefore not:

“What does the AI want?”

It wants nothing.

Ask instead:

What did we optimise?

What behaviour did we reward?

What constraints did we omit?

What access did we give it?

What did the organisation consider an acceptable shortcut?

And who benefits when the system succeeds?

Those questions are less exciting because they do not permit us to blame Frankenstein’s monster.

They require us to interview Frankenstein.

AI does not need consciousness to cause damage.

A land mine is not angry.