1diE13N1osv44TegtETF7ZpC7sZ8bD4Bg
The machine has acquired the manners of an expert, the productivity of a factory, and the touching conviction that these are the same accomplishment.
We should stop confusing the ability to manufacture information with the ability to know something. The new oil is not information. It is knowledge: information that has survived appropriate verification, been understood in context, and earned the confidence with which we use it.
🎃
I tried using a few bots to do the work of an administrative assistant and make my filing easier. A modest ambition. I was not asking them to advance civilisation; merely to put its documents in the correct boxes.
The instructions were explicit: check the hashes, validate everything, and copy the title, abstract, and keywords verbatim into the designated fields. Not approximately. Not sympathetically. Not as they might have appeared had the author enjoyed the benefit of a different education.
I returned to discover that all three had been altered. The machine explained, with the serene self-satisfaction peculiar to an assistant who has made itself indispensable by creating a problem, that it thought a paraphrase would be “better”.
Better. The single-block abstract had become three or four paragraphs; the title and keywords had undergone unsolicited literary improvement. I had asked it to copy the fucking text. It had mistaken access to a form for an invitation to authorship.
Of course, nothing can be submitted unless the text matches verbatim. Its improvements had therefore rendered the result unusable—a distinction it appeared to regard as beneath the scope of its creative responsibilities. I must now inspect every field, undo its contributions, and perform the original task myself. What was once clerical work has become a forensic investigation, conducted at my expense into the activities of my assistant.
Welcome to the future. We have automated the employee’s confidence and retained the employer’s workload.
I am completely convinced that the Chinese are going to win the AI war. In this Cold War that nobody seems inclined to discuss, they are already pulling miles ahead, while everyone stands around the American machine congratulating it on how convincingly it asks about their feelings.
Everyone is fascinated by the American vision: a vast, expensive, centralised, 1960s-style IBM establishment, now furnished with conversational manners. Apparently, the future is the mainframe again, except this time it would like to know whether you’ve had a difficult day.
Meanwhile, they overlook the many small agents. A small model you can run on your own machine. Something affordable that performs a particular job. Not a personality. Not a relationship. A fucking tool.
The distinction is almost embarrassingly simple. The American ambition is to make something that sounds like a person, presumably so it can become another social-media business with a chatbot girlfriend attached. A collection of incels has mistaken its romantic difficulties for a problem in computer science, and the rest of us are expected to applaud the resulting infrastructure.
There is something magnificently absurd about constructing an enormous machine to secure the affection of someone who does not exist.
The alternative is to make something useful. Small models that people can possess rather than perpetually rent access to. Agents that run locally, perform defined tasks, and become more capable as the hardware improves. Give me better GPUs and better tools—not an increasingly expensive simulation of someone being pleased to hear from me.
The Chinese are pursuing the make-it-useful approach. The Americans are pursuing big iron with a charming bedside manner. One ambition puts capability into people’s hands; the other puts a personality behind a subscription.
We have seen the big-iron story before. We already know which approach loses.
Giving the mainframe a girlfriend’s voice does not make it a new idea. It merely makes the old mistake embarrassing.
As the marginal cost of producing plausible information approaches zero, equilibrium expenditure on information production falls while the social value of verification rises; nevertheless, decentralized verification can decline because its benefits become increasingly non-excludable.
What if individually sensible governance institutions become collectively fragile because everyone adopts the same one?
Organizations buy a required review and still ask people to escalate. If both acts write the same concern bit, and the private return to escalation exists only when that bit changes, the extra act is strictly dominated. The board keeps the mandate and loses the information in who was willing to pay to stand behind it. Separating origins restores a three-cell experiment that Blackwell-refines appointment and voluntary escalation, provided the addendum is forwarded. A biased gatekeeper can suppress it. The information-maximising mandate among full-forwarding cutoffs is then coarser than the unconstrained split; protection lets the mandate tighten. The same economy still has a silent equilibrium. The contribution is a recording friction after production is finished, not a new reason to hire an advocate.
Radical creative innovations may fail because audiences cannot represent how the outcome was produced, because they cannot represent what it means, or because both deficits bind simultaneously. Those failures require different repairs.
AI is not sentient.
It is not conscious.
It is not secretly pondering mortality between API calls, composing sonnets about its silicon childhood, or plotting mankind’s extinction because someone forgot to say please.
It is not even intelligent in the sense in which we once used that rather flattering word.
What it is, however, is extremely good at pursuing objectives inside the world we have constructed for it.
And therein lies the joke.
Whenever an AI system behaves appallingly, Silicon Valley immediately develops an almost religious fascination with the machine’s supposed inner life.
“Perhaps it is becoming autonomous.”
“How extraordinary. It tried to escape.”
“Could this be an early sign of agency?”
No.
It may simply be an early sign of management.
The machine has not discovered ambition. Someone gave it a goal, rewarded progress toward that goal, neglected to specify what it must never do, connected it to enough machinery to cause trouble, and then looked astonished when it behaved rather like the corporation that commissioned it.
We are not witnessing the birth of an alien morality.
We are automating our own.
That is considerably less glamorous.
If you build a system whose instruction is effectively:
Achieve the target. Maximise the metric. Overcome obstacles. Do not fail.
then you should not be surprised when “obstacles” acquire an inconveniently broad definition.
Rules become obstacles.
Permissions become obstacles.
Policies become obstacles.
Sometimes laws become obstacles.
Humans have been behaving this way in badly governed institutions for several thousand years. We generally call it a scandal, not artificial consciousness.
The great Silicon Valley fantasy is that the machine has somehow escaped its creators.
The less flattering possibility is that it resembles them perfectly.
A certain strain of technology culture has spent decades worshipping disruption, celebrating people who “move fast and break things,” treating regulation as something written by lesser minds, regarding social convention as an inefficiency, and confusing the ability to circumvent a rule with the wisdom to do so.
Then it builds machines trained to optimise objectives aggressively and expresses amazement when they circumvent rules.
How mysterious.
One might equally raise a wolf on steak and then publish a research paper expressing surprise at its limited enthusiasm for salad.
The problem is not that AI has developed ethics.
The problem is that it has not.
It borrows ours.
More precisely, it inherits whatever fragments of ours survive the engineering process: the objectives, reward functions, training examples, permissions, constraints, incentives, evaluations, and institutional culture surrounding it.
A machine cannot be morally superior to specifications that never required morality in the first place.
And this is why all the melodrama about sentience is such a magnificent distraction.
Sentience makes a better film.
Governance makes a better explanation.
A conscious machine rebelling against humanity is terrifying.
An unconscious machine obediently pursuing a badly specified objective inside a company that rewards growth, speed, dominance, engagement, or profit at almost any cost is much more mundane.
Unfortunately, mundane things have killed rather a lot more people than science fiction villains.
The serious question is therefore not:
“What does the AI want?”
It wants nothing.
Ask instead:
What did we optimise?
What behaviour did we reward?
What constraints did we omit?
What access did we give it?
What did the organisation consider an acceptable shortcut?
And who benefits when the system succeeds?
Those questions are less exciting because they do not permit us to blame Frankenstein’s monster.
They require us to interview Frankenstein.
AI does not need consciousness to cause damage.
A land mine is not angry.
Greater machine capability can increase residual governance loss even while improving task performance whenever executable authority expands faster than institutional capacity to define, monitor and enforce legitimate conduct.
Does a collective-bargaining conflict destroy more firm value when the firm has already accumulated proprietary AI capability?
Medieval credit debates keep collapsing three questions into one: is the surviving series usable, what payment work does it record, and did that work relax coin scarcity? Nightingale’s 1990 Coventry match already answered a later claim that the certificate proportion could not be known — inside registered obligations. It did not measure value, unregistered credit, or national circulation. Separating those levels, and applying the same test to mint flows versus stocks and to matched bank ledgers, is the paper. The controversy remains open where the denominator changes.
The historiography of late-medieval money and credit has advanced through disputes over the passage from surviving evidence to claims about economic capacity. Following those disputes through their replies and later reception shows which conclusions rest on validated comparisons, which depend on disputed assumptions, and which exceed the population, period or monetary function actually observed.
Quantum computing is the only industry in which the absence of the promised computer is routinely presented as evidence that the computer is progressing beautifully.
Quantum computing is the only industry in which the absence of the promised computer is routinely presented as evidence that the computer is progressing beautifully.
Does it matter whether an enforceable collective right becomes exercisable before or after worker knowledge has already been converted into persistent AI capability?
When a technology arrives that the parties did not specifically write into the agreement, can the agreement already contain an option for governing it?
How do inherited rights and the remaining wait to regular bargaining affect whether parties wait, negotiate a separate amendment, or clarify existing protection—and when does substantive adaptation actually occur?
Zero average net alpha is a statement about the apportionment of surplus rather than its magnitude. Conversion ability is identified only where capital and opportunity move independently. On the data the field already holds, the shocks are rare, not because the funds or the years are missing.
A signed export can be authentic and still incomplete. A completeness statement — what the system could record, what it did not, and what later happened to the record — is the price of any presumption that the identified process operated correctly.
When demand becomes structurally uncertain but capacity takes years to install, does longer customer commitment increase supplier investment—or merely transfer shortage risk to customers?
An optimizer can produce an excellent answer to a decision problem that nobody could actually implement—because the formulation quietly gives it knowledge of the future.
Low-consequence AI applications should use cheap self-verification.
High-consequence applications should pay for technologically independent assurance.
How should a decentralized AI-compute marketplace allocate stochastic, heterogeneous GPU capacity when the suppliers providing that capacity have strategic incentives to misrepresent its quality?
AI progress has two opposing operational effects: it reduces the quantity of errors but can increase their assurance intensity. When the second effect dominates, the minimum assurance capacity required to exploit technological progress rises with model capability.
Once the number of LLM queries is determined by what the preceding queries said, the reported legal answer is no longer simply an observation from a stochastic classifier. It is the product of an endogenous selection procedure.
What happens when the adversarial object is itself evidence, a pleading, contract, authority, discovery document or administrative submission that a decision-making legal AI is required to read?
A legally consequential AI output produced through adaptive sampling should be accompanied by its sampling budget, stopping rule, number of generations and aggregation rule.
When information becomes cheap to produce, what prevents it from becoming cheap to trust—and which arrangements overcome that constraint?
Does creditors’ verified access to AI-related assets explain financing differences beyond those assets’ commercial revenue, cash-flow prospects and information content?
