# The Harness Is the Product: Why Applied AI Needs Orchestration Primitives, Not Smarter Weights
The applied AI industry has a measurement problem that is masquerading as a progress problem. Leaderboards keep climbing, top models cluster within 25 Elo points of each other, and yet 80% of AI pilots fail to reach production, only 18.4% of U.S. businesses actually use AI to produce goods or services, and 48% of enterprises report that agents lack understanding of professional scenarios. The people who benefit most from continued RL on weights are the people whose work most resembles the benchmarks—AI researchers and adjacent programmers. Everyone else finds the systems less usable, not more.
This essay describes a different kind of work: harness engineering. Not "make the model smarter," but "build the orchestration layer that turns a model with limits into a system that delivers real-world value." The argument has three parts: the primitives that compose a harness, the boundary where automation ends and human judgment begins, and why the current research culture systematically undervalues both.
## Part 1: The Primitives
A harness is built from a small set of composable operations. These are not features or tools; they are structural primitives that map directly to the constraints of the base model:
**Handoff.** One agent passes control to another with a different skill set or context. This is for specialization: the base model has finite capacity, and different tasks demand different parts of that capacity. Handoff is how you route work to the part of the model (or the part of the context) best suited to it.
**Fan-out.** One task is split into independent subtasks executed in parallel. This is for token and context-window management: when a task exceeds what fits in a single context, or when independent subproblems can be solved without cross-contamination, fan-out is the primitive. Fan-out with merge is the variant where the independent results are recombined—and the merge itself is a judgment operation, not a concatenation.
**Best-of arena.** N candidates are generated for the same task, and a selection mechanism picks the winner. This is for increasing the effective sample distribution when the task sits at the edge of the model's difficulty. The model's single-sample output is high-variance there; the arena converts variance into a selection signal.
**Arena with merge.** The arena variant where the top candidates are not just selected but synthesized. This is where warm judging matters: the candidates evaluate each other against a shared criterion, and the merge extracts what each did better than the others.
These primitives are not arbitrary. Each is a logical implication of a base-model constraint. Handoff exists because capacity is finite and specialization is real. Fan-out exists because context windows are finite and independence is exploitable. Arena exists because single-sample quality is variance-limited at the capability edge. If you understand the constraints, the primitives follow.
The research literature mostly operates above this level. It describes learned orchestration policies, architectural decoupling, capability bases—without grounding them in a primitive vocabulary. That is a mistake. Without primitives, orchestration is ad hoc; with primitives, it becomes composable and analyzable.
## Part 2: The Step
The second structural concept is the **step**. A step is not an execution unit within an automatable graph. A step is a **hard boundary where human judgment is mandatory and cannot be automated**.
The engineering goal is to make steps as large as possible. A larger step means more work compressed into the automatable region, which means more leverage from the model and less attention spent by the human. But the power of a harness does not come from any single step, no matter how large. Most valuable real-world work cannot be compressed into one automatable region at all. It requires **chaining multiple optimizable steps**, with human judgment connecting them. A system with only one step is not powerful because it has no chain—there is no sequence of automatable segments linked by judgment boundaries, so there is no compounding leverage and no path from a small automatable piece to a genuinely useful outcome. The power resides in the chain: each step is optimized to its limit, and the human-in-the-loop connections between steps are placed to minimize the attention required while preserving the judgment that cannot be automated.
This is the opposite of how "human-in-the-loop" is usually framed. HITL is typically discussed as a safety mechanism, a compliance requirement, or a vague quality-control measure. In a well-engineered harness, the human boundary is a **computational boundary**: the harness has squeezed everything it can out of the model's weights, further automated optimization yields diminishing returns or noise, and the system stops and hands the compressed result to a human—who then connects it to the next step.
The form of that handoff matters. It is not an approval gate. It is not "here is the output, click approve." It is: n agents have run the arena, judged each other against the shared criterion, and produced the top 3 proposals out of n. The harness has exhausted the model's capability. The human now decides which of the 3 is best. That decision becomes the input to the next step.
This pattern has been validated across songwriting, image generation, technical writing, and web/UI. The domain changes; the structure does not. The harness reaches the limit of what it can squeeze out of the model, knows its limit, and supplements with human expert judgment at the last mile before using the winning output as input to the next step. Good orchestration is the discipline of placing those boundaries correctly, making each automatable step as large as possible, and chaining enough steps that the system as a whole delivers real-world value.
## Part 3: Warm Judgment, Not Cold
The third structural concept is **warm judgment**. Most evaluation in the literature uses cold judges: a judge model reads candidates in isolation, without the contestants' shared criterion and without the contestants' own context. This crushes judging ability. The judge reverts to base-weight behavior, and what you get is a preference signal that reflects the judge's priors, not the task's criterion.
Warm judgment works differently. Contestants offer their best output against a shared criterion. Then they warm-read each other's outputs—with their own context and the shared criterion still in cache—and either merge the best of all or vote on a winner. Given the same criterion, agents are very good at seeing where other sample seeds did better or worse compared to their own. This is the only way to gain a better-than-random distribution out of the same given weights.
The empirical failures of cold judging are documented: cross-candidate attention collapses when candidates are evaluated in isolation, self-preference bias dominates when the judge has no external anchor, and peer errors go uncorrected when no one is responsible for locating them. The warm-interaction fix appears in adjacent work—targeted peer advice, explicit cross-candidate interaction—but no one has formalized it as the core mechanism. The framing here is: warm contestants judging each other against a shared criterion produce a better-than-random distribution from the same weights. That is the protocol.
## Part 4: Why This Is Underexplored
The reason this work is underexplored is structural, not accidental. Everyone is doing weight RL and harness RL against the same benchmarks. This makes it look like the weights' priors are Pareto-optimal—but that optimality is only true relative to the popular benchmarks, not anything else.
The evidence is clear. Benchmark contamination rates reach as high as 91.8% in some audits. Training on benchmark training sets achieves nearly the same performance as training on test sets, which means benchmarks cannot reliably distinguish further progress. RL methods that score well on benchmarks perform poorly under distribution shift, difficulty variation, and counterfactual scenarios. The "Pareto frontier" is an artifact of a contaminated and Goodharted scoreboard.
For anything outside that scoreboard, the weight priors do not help—and may actively pull outputs toward the benchmark distribution rather than the task distribution. This is why harness design must move away from weight-defaults. Not because the weights are useless, but because the "optimal" direction the weights have learned is defined by a distorted measure.
The real-world consequence is the adoption gap. For tasks that resemble provider demos or benchmarks, cold sessions almost always outperform heavily contextualized harnesses. Databricks' internal benchmark found that the simplest harness—read, write, edit, shell—scored around 90%, while Claude Code at maximum effort scored slightly lower at twice the cost. The simple harness sent one-third the context per turn and managed context better. The full-factorial study of {Planning, Tools, Memory, Self-Reflection, Retrieval} found that the all-in agent was consistently suboptimal, planning's Shapley value was significantly negative, and tool use was the only component that consistently paid for itself.
For benchmark-like tasks, the model's priors are already well-calibrated, and the harness's additional context is noise competing for attention. For tasks that don't resemble benchmarks—the tasks most real-world work consists of—the priors are insufficient, and the harness is supposed to fill the gap. But current harness design assumes the task is benchmark-like: it optimizes for "letting the model perform better on known tasks" rather than "letting the model recognize what it doesn't know and hand off to humans."
## Part 5: The Work
The work described here is harness engineering as a discipline: building orchestration layers from a small set of primitives (handoff, fan-out, arena), placing steps as hard boundaries where human judgment is mandatory, making each automatable step as large as possible, **chaining enough such steps that the system delivers real-world value**, and using warm judgment to squeeze a better-than-random distribution out of fixed weights.
This is not "make the model smarter." It is "build the system that turns a model with limits into a system that delivers real-world value." The value comes from the chain, not the step. A single optimizable task is not a system, no matter how well optimized it is. The power resides in chaining multiple optimizable steps with the correct human-in-the-loop checkpoints that minimize attention needed, and in knowing where each harness's limit is so that human judgment is spent only where it is irreducible—then using that judgment to connect one optimized segment to the next.
The industry's current trajectory—continued RL on weights against contaminated benchmarks, leaderboard ranking as a proxy for progress, harnesses designed for benchmark-like tasks—benefits the people inside the training distribution and leaves everyone else with systems that are less usable, not more. The alternative is to treat the harness as the product: to design for the boundary between automation and judgment, to build primitives that compose, to chain optimized steps through human judgment into a system that does real work, and to recognize that the last mile is not a failure of the model but a structural feature of real-world work.
That is the work. It is general, it is composable, and it is what applied AI needs.
The Mortalist
Monday, November 16, 2026
Applied AI manifesto
Friday, November 13, 2026
Teach your child chromaticism!
Chromaticism as Sovereignty
“My girl is not 五音不全. She just picked up my penchant for chromaticism as a baby.”
I did not write that as a cute joke. I wrote it as a statement of fact and of intent.
Most listeners hear a child singing outside the expected scale and reach for the familiar diagnosis: tone-deaf, undeveloped ear, needs correction. 五音不全. The five notes are incomplete. The judgment is quick because the standard is deeply installed. The diatonic collection inside equal temperament has been naturalized as the measure of correctness. Anything that moves freely across the twelve pitches, or that treats the spaces between them as livable rather than transitional, registers as error.
That judgment is historically recent and philosophically loaded.
For most of musical history, in China and elsewhere, pitch systems were not equalized. The traditional generation of the twelve lü by successive fifths produced intervals of distinct character. Modes carried seasonal, directional, and ritual weight. Pure ratios felt closer to the order of things. What later ears would call “out of tune” was often simply the sound of a system that had not yet sacrificed particularity for interchangeability.
In the late Ming, Prince Zhu Zaiyu calculated the mathematical solution that would make every semitone identical: the twelfth root of two. Equal temperament. A clean, closed, universal grid. In China the achievement was largely treated as ingenious mathematical play and did not displace the older cosmological framework. In the West the same ideal became doctrine. It solved the practical problem of fixed-pitch instruments and free modulation. It also installed a new orthodoxy: all keys are equivalent, pure intervals are negotiable, and the continuum of pitch is to be measured against a single rational standard.
The Form won. The realer reality, in the Platonic sense, was declared to be the abstract mathematical equality rather than the uneven, character-rich acoustic world. What began as an enabling technique hardened into dogma. Once the grid is accepted as neutral and natural, departures from it look like failures of competence or taste.
Chromaticism, in the sense I mean it, is not an experimental decoration performed on top of that grid. It is a return to the denser field that the grid was designed to discipline. It treats the full continuum as primary. The “wrong” notes are not wrong; they are evidence that the ear has not fully consented to the standardization.
This is what my daughter absorbed. Not a stylistic quirk. Not precocious sophistication inside the system. A more primary orientation to pitch that most formal training works to overwrite. I am not correcting it. I am protecting it.
In our era that protection is not optional. The pressures toward standardization have intensified and multiplied. Algorithmic platforms reward the immediately legible and the statistically average. Educational systems still treat deviation from the equal-tempered, diatonic norm as a deficit to be remediated. Cultural institutions prize fluency inside the dominant grid while treating alternative pitch practices as ethnic, historical, or experimental special cases. The result is a quiet narrowing of perceptual sovereignty. Children learn early that their own ears are on trial and that the verdict has already been written by a system that presents itself as neutral.
Giving a child chromatic freedom is therefore an embodied lesson in several things at once.
It teaches that the self is not required to match the standardized measure in order to be legitimate.
It teaches that expression begins with the refusal to let external grids define the boundaries of the real.
It teaches that freedom is not the ability to move fluently inside the given system, but the capacity to inhabit a relationship to the world that the system was built to supersede.
This is not romantic anti-modernism. Equal temperament solved real problems and enabled extraordinary music. The issue is not the tool; it is the elevation of the tool into unspoken ontology. When the mathematical Form is mistaken for the ground of musical reality, particularity becomes error and sovereignty becomes a technical problem to be corrected.
I am not raising a child to reject every standard. I am raising her to know that standards are historical and that her own perception is not automatically subordinate to them. The chromatic habit is one concrete place where that knowledge can live in the body before it is argued in words. It is a small, daily practice of keeping the continuum open.
That is the whole statement. Not cute. Not ironic. A father transmitting, in the medium of pitch, a working definition of freedom, self, and expression. A bulwark against the ongoing reduction of the particular to the interchangeable. In an age that systematically trains the ear to accept the grid as nature, keeping the denser possibility alive in a child is not indulgence. It is necessary work.
Saturday, October 10, 2026
Bottled up thought for aristocratic f2m friends
I don't really get the transition, not from a biological perspective but from a political perspective. It just seems like there's a better way, the old way, the natural way. Back when matriliny was the norm.
See, we are evolutionary creatures. What is evolution but the triage of the slowly mutating and the fast mutating. The DNA is stable, the RNA is volatile. The X chromosome is stable the Y chromosome is volatile. The self reproducing specie is the core, the Y branch is a strategy of adapting to changing environment quickly, cheaply, expendably, so that the core can choose based on its exquisite taste.
This was the equilibrium until the external shock of the bronze payoff function. Before bronze, bones and stone cannot actuate war. There were brawls and ritual sacrifices. During the stone age, Helen and Paris would have been sacrificed to end the blood feud. After bronze, the Trojan Horse became the all or nothing gambit. The capacity for total annihilation changed the payoff across the board. Every bronze faring civilization played or got played. Non bronze faring civilizations could only get played. The only exception was the Harrapan, who dissipated into the jungle to wait it out. They tried to explain it to Siddhartha... lose if you lose, lose if you win, only way out is not play... when the payoff changes (nuclear MAD era) see who wins... who has more children, more vitality, more life. As poor as their descendants look, they won or at minimum didn't lose that bet, in spite of millenia of enslavement.
Bronze is what made patriarchy make sense. The natural order became unsustainable because bronze made it unsustainable. The greatest patriarch in history understood this and thought he was above it, because he was above all the bronze-payoff civilization. Ghengis Khan gave his daughters the empire and his sons the periphery where they would fight or die. The local princes married INTO his daughters, in his name, his heritage, his X. But the page of his will detailing this plan was ripped out. His sons and their wives quickly murdered his daughters. He was too ahead of his time because in the bronze and steel world, his daughters couldn't survive.
But what's the alternative? The Rothschilds marrying their cousins? How did that work out? The easier way would have been: let the daughter inherit, let her invite in the best Ys in the world, strengthen the gene pool, and pass the contribution of those Y down to her daughter. In Chinese, this is called the Zhui Xu, the male-bride.
I'm not talking about girls are best or boys are best. I'm not talking about righting historical wrongs or some metaphysical sense of morality. I'm saying: two portfolio dynasties, 1) male inheritance, Y proliferation, diversification through X versus 2) female inheritance X continuity, diversification through Y... which portfolio wins after 10 generations in terms of Sharpe ratio and total return?
You don't need arms and armor anymore. Your intelligence is enough. The only thing you need to do is ask him: will you seed a child for me even if we don't get along forever? The only thing you need to ask your parents is: let me be the steward of your legacy, I have the intelligence, confidence, and strength to do it.
And if you want citation, read Laozi. "The cow wins through stillness. She lowers herself and becomes the receptive. The cow wins through competition. He lowers himself to be chosen. The only thing the cow wants is to enable and grow life. The only thing the bull wants is to participate and become part of life."
It's been almost a hundred years since the bronze payoff function flipped. It's time to see the 2000+ year old habits don't make much sense anymore. You felt it, but you didn't see it.
Wednesday, October 7, 2026
What "pacing the frontier" really means
The urgency is real. The danger is real. But the danger is not AI capability.
So what if a frontier agent finds vulnerabilities while other frontier agents patch vulnerabilities nobody even thought to look for? There will never be a sustained asymmetry between attacker and defender.
In fact the most dangerous and destructive phase of agentic innovation is probably already past us. The victims are ordinary people with low education and elderly, mostly in developing countries. Where I live in Hangzhou, you cannot walk two blocks before seeing phone/internet fraud warnings. Voice cloning, identity scraping, and agent automation has enabled a massive wave of quiet, dispersed, low level crime. And that's probably as bad as it gets.
So why the panic? And why from those voices? Because what needs pacing is the capex driven equity non-correction that has replaced the Treasury as the anchor for the US Dollar. The numbers can't go down without dragging the State down with it. But it only takes a couple of bad quarters for capital flows to start changing direction.
I use Deepseek and Mimo. They work really well. I swallowed my pride and ordered a 3090 to run ComfyUI. The "backloaded" profits will not materialize. The capex and the dismantling of regulatory supervision IS the there there. So yes, "pace the frontier" is the only way to say it politely.
And whatever happens after that, the solution is a stronger American State that legitimates authority and recaptures the public as public.
Monday, August 19, 2019
A translator’s approach to understanding Chinese mindset
Saturday, June 17, 2017
Language
Of course, it would be great if we could "rectify" our languages, as Confucius says but that the task seems so unsurmountable only reminds us that we live in an age where symbolic systems are battling fiercely for supremacy. The size of our concepts have grown in proportion to the energy that we consume. These large concepts can only be wielded by collectives and most of these collectives expend immense energy erecting barriers to access these concepts and the objects that they control. Wars manifest themselves in whatever way they can, like the fruits of decay, between concepts too large for any individual to perceive. But it seems to create new concepts, old ones must be destroyed.
Perhaps it was never a good time for the task of rectifying language. Perhaps it has only ever been done the only way it could, in piecemeal. Babel, if it ever existed, must not have lasted very long anyway.
What tools do we have at our disposal?
Wednesday, June 14, 2017
Father
Father, with a capital F, is often used to signify authority or origin, both senses of which have little to do with the experience of a biological father. I think the biological father is a strange figure, an identity which is still in its experimental stages (beta tester fathers), a role which progresses towards becoming a necessity while still leaving open the possibility that he is optional so long as resources are provided to the truly necessary caretakers.
father, with a lower case f, veers toward friendship at times, a benefactor, a mentor, but other times, he could seem more like an exploiter or dominator. Not every child is lucky enough to have a benevolent father. Such children are the most likely to misuse the word to refer to something other than his or her father, to defer the man to an idea, to displace the trauma that could have been unnecessary had fatherhood not been demanded upon societies (why does he exist in such a role?). The rejection of the Father (the displaced, misperceived father) could potentially return fatherhood to its rightfully ambiguous place.
father as a symbol; realistically rendered as father, not the father or Father or fathers, is a thing very few people (only siblings) have in common. To attempt to distill universal characteristics, enough to fill a potent symbol, isn't easy. Symbols based upon fatherhood tend less to be one liners and tend more to express themselves as tomes and ouvres. The Father is one of our oldest and most entangled symbols.
More interesting than the subject of fathers is pretty much most of everything from video games to sex. Just now, I saw a group of people downstairs standing in a circle watching a small dog fuck a tiny dog. Everyone found it very amusing. Concomitantly, we are all very amusing2.