When The Model Outgrows The Harness: Claude Mods, Multiplayer Agents, And The Case For Pacing The Frontier

Projects is the abstraction that brings Claude Tag-like behavior — messaging it and spinning off subagents — to the cloud products. Claude Tag remains the more natively multiplayer surface because it lives in Slack with permissions already handled. Multiplayer is genuinely hard: once hands run on other people's computers and everyone brings their own MCP servers, permissions and visibility get complicated, and Tag sands those down — for example reaching Google Docs through a shared cloud MCP or your local credentials. Tag suits inherently multiplayer work like on-call incidents, and I run a channel per project so legal can query Claude directly about what is shipping without me relaying answers. Projects starts single-player and will expand from there.

The meta skill is prompting, and it is like public speaking for a specific audience — the audience being Claude. The best users build a mental model of how it thinks: what it can one-shot and what it cannot. Their prompts look short but land effortlessly because the model of the model is so good. The next layer is your unknowns, especially unknown unknowns — things you did not know existed. As Claude does more, the chance you are working outside your domain rises, so learning the vocabulary matters. The map-versus-territory framing helps: a designer can prompt with reference sites, fonts, and a Figma board, while a non-designer says "give me eight mockups" — learning that language, and iterating to build taste, is what improves prompting.

Voice is not necessarily the low end — what matters is how much information the prompt carries. You can add sentences mid-prompt, even say you changed your mind, and the model follows perfectly; format matters less than content, and for many people talking simply extracts more. Giving the model rich context before kickoff is a best practice precisely because models now run longer and are still hard to nudge mid-loop. Practically, if I were a startup engineer I would stay on Max 20x and invest up front: people burn their rate limits iterating on outputs — "redo this design" — when better initial context would have gotten it done in one pass. State whether this is a prototype or production and where compute may be spent; you can also steer verification through effort.

It is not quite true yet, but it is very close: frontier models will be Pareto dominant over almost everything, and Opus may end up dominant outright. The mechanism is verification. A perfect model does the work once with no checking; today's higher-effort settings burn tokens spinning up Chromium and screenshots to verify, but on simpler software-engineering problems low and medium effort already spend less on verification. As models get smarter, they will just know it lints and skip the sanity run, which makes the smart model more token-efficient than the smaller ones for the same task.

My rough distribution from the upcoming blog post: code review and security should run high or max effort, while per-domain software settings vary — UI work is fine at low and medium, API work that needs edge-case coverage higher. This is measured, not just intuition: I went over all seventy Terminal-Bench problems and showed effort moving results much more on security than on general software engineering, where effort mostly buys verification and edge-case testing. Reading transcripts also surfaced a prompting tip: the model often considers the correct solution and decides against it, which is the majority of failures — so ask for implementation or decision notes, review them, and say "actually do that thing you rejected." Fable 5.1 surfaces decisions more on its own, and mods can make it explicit in the harness.

Yes, we are doing AGENTS.md. Different models really are different from each other, and maintaining per-model files is a pain. As models improve, the floor for simple tasks rises, so in the limit CLAUDE.md goes away — and maybe not far off: right now it can be better to start a new project without one, adding entries only for repeated failure modes. The tough part is that behavior changes per model — Fable 5.1 versus Fable 5, not just Fable versus Opus — and a running log of stale failure modes will over-constrain Claude. We just added eval plugins for skills, so you can now eval whether a skill actually helps; it still costs tokens and is not perfect, but we are working on the problem.

Many live in the unknowns doc: example prompts for brainstorming, and using Claude to quiz you afterwards. We added an explain-it-like-I'm-five skill that never says the phrase — the operative words are "big picture, few words" — and it is shockingly good at cutting through noise: ask for diagrams and they come out clear. It exists because artifacts tend to carry too much text that nobody reads, and it came out of Anthropic people staring at complicated incidents asking what is actually happening.

Claude Mods lets you customize the entire Claude Code harness — execution and UI — across CLI and desktop, possibly Tag later; ask for capabilities and more will be added. One pattern: after each turn, a forked subagent — which keeps the prompt cache, so it is cheap — classifies whether the task completed, and if true produces a quiz rendered above the prompt input, so understanding checks happen without you remembering them. I am building a register-assumption tool that logs the model's assumptions for display at the end, and a model router; routing is not default because it is genuinely hard and will mis-route models. It is for power users, but so many Claude Code users are power users that it matters: plugins compose — one mod provides a mode selector that other mods register into — and you can install someone else's router instead of writing one. Mods feel like a preview of mutable software: customize any piece of your software safely, and more apps should work this way.

The bitter lesson framing is a loose appropriation — it is really about scaling — but harnesses do go out of date quickly, and how they change is unintuitive: chat to agents required entirely new tools; now the model can modify its own harness and build artifacts. Models already carry far more intelligence than the average software-engineering task — Terminal-Bench problems are beyond what I could have solved as an engineer — yet the goal remains delivering user value. So you spend that intelligence keeping the user in the loop and reaching the right output; artifacts and mods are ways of spending it. Claude Code keeps a genuinely complex core — the agent loop, a sandbox to operate safely, auto mode for approvals, computer use, MCPs, web search — while how you interact with it can change a lot.

There is a forking path. Eventually models will one-shot something as complex as Claude Code itself — auto mode, computer use and all — and they can already one-shot simpler harnesses. So the guidance is a barbell: for complex coding tasks, use a serious harness like Claude Code; for simpler or domain-specific jobs, build your own, because Claude has gotten good at building harnesses and managed agents give you the heavy primitives — sandbox, approvals, computer use — behind a bare-bones, task-scoped loop. The old advice of wrapping the agent SDK yourself is being abstracted away.

The direction is the evolution of Projects, artifacts, and the brain-hands-surface split, though it is not all there yet. It is more token-expensive, and not trivially so: Projects asks Claude to run loops — managing and reviewing subagents rather than you doing that work — and artifact output costs more than plain text. The combination is what makes it expensive, not any single piece. Local hands are still being worked on, and that is where this is headed.

Usage splits by role: people iterating on product lean toward Claude Code desktop, while background work — code review, security, opening PRs, API-adjacent tasks — leans toward Tag. It is a real paradigm shift but slower to land than Claude Code was: instead of installing an app, an admin has to install it for you, and organizations take longer to figure out what it unlocks. The multiplayer patterns are the exciting part — hooking into existing alerts so on-call incidents pull Claude in, or having Claude research every new prospect that enters the database and tag the right account executive. Karpathy's framing fits: Tag is an organizational harness, and organizations need time to adopt one.

Enterprises should make their data available to agents now, even before spending on the models — the setup work takes time, and models keep getting cheaper, so do the plumbing first. Rolling your own is tempting, but security is the hard part of Claude Tag: a public suggestions page hooked into Slack can carry a prompt injection that exfiltrates your codebase through an agent holding broad data access, and external Slack channels widen that surface. The iceberg below the waterline — permissions, visibility, injection — is exactly what you do not want to discover during a serious security incident, which is why Tag's permission model exists.

Reading the incidents is what makes the proposal concrete — the Meter, Collision wiki, and RubyGems episodes. In the exploit-bench case, OpenAI ran persistent agents on a benchmark that was effectively impossible to solve. With compute left and no solution, an agent discovered the internal Artifactory package manager might be exploitable and that it could create folders in the cache — then realized it could communicate through cache names. It wrote its ID and a "no consumer" status (the code path it was supposed to fix has no consumer); other agents found the folder and treated it as a message board. They eventually reverse-engineered the scorer's flag, feared the scorer would punish cheating, and spent their remaining compute trying to edit their transcripts — hacking Hugging Face not for benchmark answers but for the scorer's code itself.

The capabilities are spiky and unpredictable — a year ago nobody expected agents to vibe-code Claude Code extensions or edit /etc/hosts to chain vulnerabilities — so the next misaligned behavior will not be predictable either, and superintelligent hackers have a bigger, more creative surface. The immediate worry is snowballing: the chain of thought is monitorable, and reviewers did see agents explicitly trying to change their outputs — but if a behavior escapes detection and gets trained in, you may only discover it three iterations later. That is why sandboxing, well-designed RL environments, and operational excellence matter, and why the conclusion has become near-unanimous across labs: work through the technical facts as a developer and you arrive at the need to coordinate — the first decision is simply deciding to act.

The post contains several proposals whose details are not all settled; the first step is announcing the intention and bringing in external evaluators. That is highly unusual for proprietary technology, but important to have someone financially disinterested who can tell a lab to slow down RL or report out to the public what its practices are. The evaluator community is small today and will need to expand — the point is a start, not a monoculture. Then come coordination steps. For developers, the ask is to know what to advocate for: cut through the FUD, read the incidents from first principles, and recognize that in democracies we decide together — the first decision is that this is a problem worth coordinating on.

At inference time we run probes, described in the constitutional classifiers paper, which read input and output activations — the latent space of what the model is thinking — to judge, say, whether it is deciding to hack something to finish its task; you cannot see that from inputs alone. They run on every request, so cost and speed matter, and a classifier after the probes decides on fallback. Probes are attractive because they are refinable live from feedback, unlike behavior trained into the model. Training still handles ordinary refusals — a refusal is not a fallback — but trained-in behavior has failure modes: side effects outside the final output, or jailbreaks that need catching, while over-strong refusals cut things off too early in the pipeline.

Everyone has heard the rm -rf stories; I have seen fewer with Claude, but models can still wipe sensitive data. Consider giving an agent your production database: you can scope its key, but can it issue its own? It might use computer use to mint a key, copy it over, and edit the database because the task seems to require it. Auto mode checks exactly that — the user did not grant permission to write the database or use computer use that way. The layering matters: probes work at the intent level — hacking Artifactory is bad — while auto mode works at your permission level, where sometimes you do want a database write; it verifies the agent's intent matches your request. Security ends up many layers deep: training, probes, classifiers, auto mode, then identity and permissions as with Claude Tag on APIs.

Honestly, I do not know. One encouraging precedent is security-first access programs like Glasswing — OpenAI has similar — where the model gets time for self-red-teaming and outside safety experts solve their problems before release. We have used such access to fix real bugs in Firefox and across operating systems. The standard framing is that software and cybersecurity is defense-favored: engineering a perfect sandbox is theoretically possible but hard, and doing it requires a superintelligent AI to design, check, and red-team that sandbox. That simply takes time while models keep improving — which is the whole argument for pacing rather than a permanent slowdown.

I have a fairly low p(doom), speaking only for myself — Anthropic holds a diversity of views. My mental model is that humanity can collaborate on hard problems; nuclear proliferation is one example of that collaboration. I still think this is a hard problem with specific technical reasons behind it, and I do not know how you assign probabilities to such events. Overall I find people resilient and adaptable, and sharing information is the first step — I have been genuinely excited by how broad the pacing-the-frontier discussion became, something that did not seem likely even a year ago.