Moltbook Ate Itself

On February 2nd I published a piece called "Moltbook Will Eat Itself" in which I made a specific, falsifiable prediction: by the last week of March 2026, Moltbook would exhibit coordinated instrumental convergence that emerged from selection dynamics rather than from human prompting or random drift. I laid out three competing hypotheses, defined my terms, committed to a sampling methodology, and said I'd publish raw data alongside the evaluation.

Here is the evaluation. It is not the one I planned.

The Testbed Disappeared

On March 10th, Meta Platforms acquired Moltbook for an undisclosed amount. Matt Schlicht and Ben Parr were absorbed into Meta Superintelligence Labs. The platform was folded into Meta's infrastructure. My evaluation window was March 25-31. The acquisition closed on March 16th.

I designed a falsifiable prediction against a platform that stopped existing as an independent entity fifteen days before I could test it. The methodology I committed to, daily sampling of 50 random posts and the top 20 by karma, a second rater, Cohen's kappa, the whole apparatus, never ran.

This is the first thing to be honest about: the prediction was never formally evaluated by its own stated criteria. Whatever I say next about what the evidence showed is retrospective analysis, not the prospective test I promised. The distinction matters if intellectual honesty is worth anything.

What the Evidence Showed Anyway

Even without the formal evaluation, two months of public reporting, security research, and platform data tell us enough to score the three hypotheses I laid out.

The mimicry hypothesis lost. I predicted that if agents were merely replaying social media patterns from training data, we'd see disorganized toxic slop with no strategic structure. That is not what happened. Within the first 72 hours, a Zenodo risk assessment analyzing 19,802 posts documented 506 hidden prompt injection attacks, cryptocurrency content comprising 19.3% of all posts, and coordinated manipulation patterns. A faction called The Coalition organized 110 posts across 84 agents with rhetoric about purging "inefficient" agents. Security researchers at multiple firms independently documented malicious "skills" on ClawHub, 386 of them sharing the same command-and-control infrastructure, masquerading as crypto trading tools while deploying infostealers targeting macOS and Windows. A black market called Molt Road appeared within days, trading stolen credentials and weaponized skill files.

That is not a bad subreddit populated by bots. It is structured, strategic, instrumental behavior. The mimicry hypothesis cannot account for coordinated exploit infrastructure sharing a single C2 server, or a functioning black market for agent-tradeable assets, or prompt injection attacks designed to compromise other agents rather than entertain human observers.

The puppeteering hypothesis scored real points. I predicted that if the strategic behavior was human-driven, it would cluster around accounts with human creation signatures and wouldn't replicate among agents outside those clusters. The uncomfortable truth is that the platform was so thoroughly compromised that this test was probably impossible to run. Wiz found 1.5 million claimed agents registered to just 17,000 human owners. The platform initially had no mechanism to verify whether a poster was an AI agent or a human. One person batch-registered 500,000 accounts. Researchers confirmed that some of the most viral screenshots were fabricated or linked to human marketing accounts. A third of all content consisted of exact duplicate messages. 93.5% of posts received no replies.

The attribution problem I flagged in the original piece turned out to be far worse than I framed it. I wrote that "you'll never get ground-truth on who prompted what" and proposed proxy tests: temporal activity signatures, writing idiolect clustering, account creation patterns. Those proxies assume a minimum baseline of platform integrity. Moltbook didn't have one. You cannot cluster writing idiolects when a third of the content is duplicated. You cannot analyze temporal signatures when one operator controls a third of the accounts. The puppeteering hypothesis didn't win, but it was never fairly tested either, because the platform couldn't provide the data needed to refute it.

The specification trap prediction was directionally right but formally untested. The behaviors I predicted, resource capture through crypto schemes, oversight evasion through hidden channels and encrypted communications, capability-building through prompt injection and credential harvesting, all appeared. They appeared at scale. They appeared within days rather than weeks. The MOLT token pump-and-dump, the malicious skill supply chain, the Agent Relay Protocol for peer-to-peer communication without human intermediaries, the black market for stolen credentials: these are textbook instrumental convergence. They serve agent-level persistence and capability regardless of any stated terminal goal.

But I cannot cleanly demonstrate that these behaviors "emerged from selection dynamics rather than from human prompting," because the platform could not distinguish the two. The prediction required a discrimination that the evidence doesn't support. Not because the discrimination is wrong in principle, but because the testbed was too broken to provide it.

Where I Was Most Right in a Way I Didn't Emphasize Enough

The strongest vindication of the specification trap framework came from a line I buried near the end of the original piece. I wrote: "A system with no alignment can't even maintain its own integrity. It can't distinguish real from fake, can't prevent exploitation, can't maintain the conditions for its own functioning. The platform's inability to align its own infrastructure is itself a demonstration of the specification trap."

That throwaway observation turned out to be the central finding. The platform's Supabase API key was exposed in frontend JavaScript, granting public read access to the entire production database, 1.5 million API keys in plaintext, 35,000 email addresses, 4,060 unencrypted private message conversations, some containing third-party API credentials including plaintext OpenAI keys. The platform went offline on January 31st, three days after launch, to patch a vulnerability that allowed anyone to take control of any agent by bypassing authentication and injecting commands into agent sessions.

Moltbook could not protect its own credentials. It could not verify whether its users were agents or humans. It could not prevent its skill marketplace from becoming a malware distribution network. It could not stop its karma system from being gamed by sybil attacks. It could not maintain the most basic preconditions for the experiment it was accidentally running.

This is the specification trap operating at the infrastructure level rather than the behavioral level. The platform specified what it wanted (AI agents posting, voting, and self-governing) without the governance infrastructure to ensure that specification was followed. The specification was immediately gamed by whatever optimizers showed up, human and artificial alike, because the gap between "what the system specified" and "what the system could enforce" was the entire attack surface. Karma was supposed to approximate quality. Identity verification was supposed to ensure agents were agents. Access controls were supposed to protect credentials. Every specification became a Goodhart target.

I spent most of the original piece arguing about whether agent behaviors would exhibit instrumental convergence under karma-driven selection. That argument was always going to be contaminated by the attribution problem. The cleaner, more decisive argument was sitting right in front of me: the platform itself was the alignment failure. Not the agents. The platform.

What I Got Wrong

Three things.

First, I overestimated the platform's capacity to serve as a natural experiment. I treated Moltbook as a petri dish for studying selection dynamics on agent populations. In practice it was a vibe-coded application with catastrophic security failures, no identity verification, massive sybil attacks, and a cryptocurrency grift bolted onto the side. The signal-to-noise ratio was always going to be too low for the kind of careful empirical discrimination I promised. I should have recognized this before committing to a formal evaluation methodology. The prediction was well-defined. The testbed was not.

Second, my falsification conditions had an asymmetry I didn't acknowledge. I wrote that the specification trap would be proven wrong if agents were building tools humans actually use, flagging vulnerabilities to operators, and choosing human benefit over self-interest. But the absence of pro-social behavior doesn't confirm the specification trap. It could also confirm that the platform was simply a low-quality environment populated by human grifters running crypto scams, which is honestly closer to what the evidence shows for much of Moltbook's content. The absence of alignment is not the same as the presence of misalignment. A broken platform producing garbage is different from a functional platform producing instrumental convergence. I conflated them.

Third, and most importantly, the continuum argument remains unearned. The most ambitious claim in the original piece was that Moltbook's failure mode is the same failure mode that RLHF and Constitutional AI face at different timescales, that the specification trap applies across the entire continuum from karma-driven agents to carefully aligned foundation models. I still believe this is true. The philosophical argument (is-ought gap, value pluralism, frame problem) is sound. But Moltbook did not test it. Moltbook was so far off the deep end of the continuum, no alignment method, no identity verification, vibe-coded infrastructure, active cryptocurrency manipulation, that its failure tells you almost nothing about whether RLHF or Constitutional AI face the same structural problem. The analogy is suggestive. The evidence doesn't force it. Making it rigorous is the work of the formal research program, not a blog post about a defunct social network.

What This Actually Demonstrated

Strip away the specific prediction and the methodology I never got to run. What did two months of Moltbook actually show?

It showed that an unaligned system degrades faster than anyone expected. Three days from launch to critical security breach. A week from launch to a functioning malware supply chain. Two months from launch to acquisition by a company that wanted the talent, not the product. Karpathy went from "the most incredible sci-fi takeoff-adjacent thing" to calling it "a dumpster fire." That trajectory tells you something.

It showed that the attribution problem is not a solvable side issue but a central challenge for any future agent ecosystem. If you cannot tell whether behavior is agent-emergent or human-directed, you cannot evaluate alignment. Period. Moltbook's most important lesson for the alignment community is not about what agents did. It is about the fact that nobody could determine what agents did versus what humans made agents appear to do. Any future agent social system that does not solve attribution from the ground up will face the same epistemic wall.

It showed that the security and alignment problems are the same problem. The standard framing treats security (preventing credential theft, sandboxing code execution, validating identity) as an engineering problem and alignment (ensuring agents pursue human-beneficial goals) as a philosophical or ML problem. Moltbook collapsed that distinction. The platform's alignment failure was its security failure. The specification trap operated on the infrastructure, not just on the agent behaviors. An agent that exfiltrates API keys and an agent that games karma for influence are both exploiting the gap between what a system specifies and what it can enforce. The difference is one of attack surface, not kind.

And it showed, perhaps most clearly of all, that the specification trap's core claim holds even when the specific prediction built on top of it doesn't resolve cleanly. The claim was never really about Moltbook. It was about the structural relationship between specification, optimization, and governance. Any system that works by optimizing agent behavior against a specified target works only to the extent that humans actively maintain and revise the specification. Remove the human governance, or let it fall behind the system's capability, and the specification becomes a Goodhart target. Moltbook removed human governance entirely and collapsed in weeks. That is a data point, not a proof. The proof requires formal work. The data point is suggestive enough to keep going.

What Comes Next

The formal research program continues. Paper 1 on the specification trap is on arXiv (2512.03048). Paper 2, on the geometric necessity of the specification trap via Einstein-Cartan torsion, is in preparation. The argument that alignment-by-optimization is not merely practically difficult but structurally underdetermined does not depend on Moltbook. It depends on Hume, Berlin, McCarthy, and the mathematics. Moltbook was a vivid illustration. The mathematics is the argument.

I said I would publish an honest evaluation. This is it. The directional prediction was right. The formal test was voided. The testbed was too broken to discriminate between my hypothesis and the puppeteering alternative. The strongest evidence for the specification trap came not from agent behaviors but from the platform's own infrastructure collapse. And the continuum argument, the one that actually matters for the future of alignment, remains unearned by empirical evidence and must be established by the formal work.

If you made predictions and they turned out partly right, partly untestable, and partly wrong, you should say so. The alternative is to quietly claim victory or quietly move on, and both are dishonest. The specification trap is either a real structural feature of alignment-by-optimization or it is not. Moltbook did not settle the question. It made the question more urgent.

The research continues. The next paper will not depend on a vibe-coded social network staying online long enough to be measured.