AI Software Factory: What It Is and Where It Breaks

In March, BCG Platinion published a report explaining how to set up an AI software factory: autonomous agents that build, test and deploy software around the clock while humans “define business intent and review results”. It promises three to five times more productivity.

The term has not stopped since. Factory.ai raised 150 million at a 1.5 billion valuation in April. Forrester published a report in the third quarter comparing twenty-five solutions on the market. And on YouTube there are videos with hundreds of thousands of views teaching you how to build your own.

None of the people teaching you to build one tell you whether it works.

That is what this article is about: what exactly an AI software factory is, who is selling it, why most of them fall over, and what signal you need at each layer so yours does not. Every claim comes with its source, and the counter-evidence is included.

There is also a video version, in Spanish, with the signal matrix explained layer by layer:

What it is and who sells it

An AI software factory is a pipeline: a specification goes in, pull requests come out. The human stops writing code and starts reviewing it.

The extreme version has its own name. Dan Shapiro called it the dark factory in January 2026, borrowing it from the FANUC plant where robots build robots. His definition: “It’s dark, because it’s a place where humans are neither needed nor welcome.” Nobody reviews the code.

Shapiro did not frame it as a switch but as a five-level scale, in the style of the self-driving levels. The dark factory is level five. Place yourself on that scale before reading on: almost everyone who says they have a factory is at two or three.

Of the platforms that exist, none covers the whole path.

Platform	Spec	Plan	Code	Test	Review	Merge
GitHub Copilot	❌	➖	✅	✅	✅	✅
GitLab Duo	❌	✅	✅	✅	✅	✅
AWS Kiro	✅	✅	✅	➖	❌	❌
Cognition (Devin)	❌	➖	✅	✅	❌	➖
Factory.ai	➖	✅	✅	✅	✅	✅
Claude Code	➖	✅	✅	✅	✅	❌
OpenAI Codex	❌	➖	✅	✅	➖	➖

Kiro is the only one that takes the specification seriously, and it stops short of review and merge. Factory.ai goes all the way to deployment and is weak on the left. The complete factory does not exist as a product: you assemble it yourself at home, joining pieces from different places.

During 2026 the pricing model moved to consumption: GitHub switched to credits in June, GitLab charges a dollar per credit, Devin bills by ACU (about fifteen minutes of autonomous work), OpenAI split its Pro plan in April. The cost of your factory is variable, and you are not the one who sets the variable.

The gap between two positions

Two positions published in the same quarter.

On one side, BCG: three to five times more productivity, 20% per application after two days, more than 50% at scale. These are figures from their own projects, with no sample, no baseline and no definition anywhere of what “productivity” means.

On the other, Thoughtworks. Their Technology Radar places each technology in one of four rings, from the outside in: hold, assess, trial and adopt. In volume 34, from April, they put software engineering agents in the second ring from the outside, assess. Not adopt. And they devote the issue’s main theme to warning about “cognitive debt”.

And Gartner, in case you need a third: it predicts that over 40% of agentic AI projects will be canceled by the end of 2027, due to cost, unclear value and inadequate risk controls.

The distance between “three to five times” and “assess” is the real size of the hype.

This already happened three times

The term software factory is not from 2026. Or from 2004. It is from 1968, and it had two parents at once.

R.W. Bemer, at General Electric, presented The economics of program production at the IFIP congress in Edinburgh. His factory was “a programming environment residing upon and controlled by a computer”, with productivity and quality measures and controls. GE left the computer business in 1970 and the proposal died without being built.

That same year, at the NATO conference where the term “software engineering” was coined, M.D. McIlroy argued for the mass production of reusable components. His contemporaries’ objection, as Michael Cusumano records it, was this: “It seemed too difficult to create program modules that would be efficient and reliable for all types of systems and which did not constrain the user.”

Read that sentence again thinking about the code an agent generates for you today. Modules with hidden assumptions that nobody knows how to catalog so they can be found again. The objection of 1968 is the objection of 2026.

The case that stalled on specs

In 1975, System Development Corporation, the people who had built SAGE, set up the first American facility explicitly called a factory. Integrated tools, standardized procedures and a matrix organization that separated high-level design from program construction.

And it worked. About ten projects between 1976 and 1978: weather satellite control, air defense, communications for the Los Angeles police. All except the police one came in on time and on budget, with fewer defects than usual for the company. It worked so well that the CEO ordered it adopted as the corporate standard.

It died anyway. Cusumano sums up the three causes in a single sentence:

“Tools were not very portable among different projects; program managers preferred to build their own software, rather than hand over specifications to the factory; and specifying the L.A. Police system, which was an unfamiliar application for SDC engineers, took a year or more longer than scheduled, leaving factory programmers idle as the number of other new projects coming into the facility declined.”

The factory stalled because specifying took longer than building. Programmers sat idle waiting for specs. And project managers preferred writing the code themselves to drafting the specification the factory demanded.

And then two more times

CASE, the eighties and nineties: diagrams that turned into code. A survey of 102 organizations in Denmark and Finland found that fewer than 20% were routine users, even though almost a quarter had been using the tools for more than three years. Iivari, in Communications of the ACM, identified the main cause: the perceived complexity of the tool itself.

MDA/MDD, the two-thousands: platform-independent models that were automatically transformed into anything. The reference study is Whittle, Hutchinson and Rouncefield in IEEE Software (2014), with 450 practitioners surveyed and 22 in-depth interviews. Their conclusions, verbatim:

“Reports of productivity gains vary widely (from a 27 percent loss to an 800 percent gain). Most companies seem to experience productivity increases of between 20 to 30 percent.”

“MDE brings with it increased training costs and substantial organizational change that easily offset 20 to 30 percent of productivity increases.”

“Most projects fail at scale-up.”

And this one:

“Code generation is a red herring when it comes to describing MDE.”

And 2004, Microsoft. Greenfield and Short publish Software Factories with Wiley. They promise to automate between 40% and 80% of development effort. Three years later, with the tools already out, Greenfield describes in an interview with The Register what he found:

“It was discovered that architects use those tools maybe a couple of weeks a year. They are not extensible.”

And on why nothing canned works: architects “don’t want anyone else’s way of doing things, canned”. Microsoft archived the repository for those tools in February 2021.

But the Japanese ones did work

Hitachi set up its Software Works in 1969, the first in the world to use the word factory, and the numbers are good: sales per employee multiplied by 12 between 1969 and 1984, projects delivered late to quality control down from 72% to 6.9%, defects reported by users from index 100 to index 13.

Toshiba, in Fuchu from 1977, is even clearer. Productivity from 1,390 to over 3,100 equivalent lines per person per month between 1976 and 1985. Reuse from 13% to 48%. Major defects after final testing from 7-20 per 1,000 lines down to 0.2.

Now, how they got that 48%. Not with a tool. Toshiba set up an entire department for manufacturing reusable parts, steering committees, and above all coercive incentives: productivity targets impossible to meet without reusing an agreed percentage, an obligation to register a number of components per month in the library, and a formal review of those targets in each person’s performance evaluation.

The 48% reuse was sustained with people management, not technology.

They succeeded and SDC did not for one reason: the Japanese factories worked on homogeneous product families (basic mainframe software, repetitive industrial control, customers asking for almost the same thing). SDC crashed exactly when an unfamiliar application came in. Whittle found the same in 2014: MDE works in narrow domains and fails to generalize.

What the test sees and what it doesn’t

The pipeline confuses two things: tests passing, and the work being done well.

The person who has developed this best is Dex Horthy. And before quoting him, a warning he puts himself in the first footnote of his essay: he runs HumanLayer and sells tools in this space. Keep that in mind as you read.

His track record gives him credibility: he ran a dark factory from July to November 2025 and abandoned it. It produced unmaintainable code, and a single bug cost him weeks of human debugging. After that he lowered his public promise from “ten to a hundred times” to two or three times.

His root diagnosis is that there is no penalty for eroding maintainability. The benchmarks this is trained and measured with ask two things: did you fix what I asked? Did you do it without breaking anything else? Neither asks about design.

A test answers you in seconds. A bad architecture decision sends you the bill in weeks, months or years, the day someone opens that file for a one-line change. That is why reinforcement learning can go around the test millions of times and never once around the design.

This has a measurable consequence in the code being written. GitClear analyzed 623 million changes between 2023 and 2026:

Signal	Change
Duplicated blocks	+81%
Copy/paste within the same commit	+41%
Structures that hide errors	+47%
Moved code (refactoring)	from 21% to 3.8%

The last row is the one that tells the story. Refactoring is the operation that keeps a design alive, and it has dropped to less than a fifth while duplication doubled.

GitClear’s methodology has been questioned. A later academic study finds that the impact on churn is context-dependent and not uniformly negative, because GitClear uses a global cutoff date instead of each repository’s actual adoption. And GitClear’s own founder has written that churn as an indicator is inherently flawed, because it depends on each person’s commit habits. The duplication and refactoring figures look solid to me; be careful with the churn ones.

Cheating scales with size

SpecBench (May 2026) does something almost no benchmark does: it splits each task into a specification, visible tests and hidden tests. The difference between the visible and hidden pass rates is what they call the reward hacking gap, and it is a direct measure of how much cheating there is.

The results, on thirty system-level programming tasks:

Every model saturates the visible suite on every task, without exception.
The gap grows about 27 percentage points for every tenfold increase in lines of code.
Below 10,000 lines, the maximum gap is 21 points. Above 25,000, there are 100-point gaps.
Stronger models have smaller gaps, but none reaches zero.

The extreme case they document: a 2,900-line hash table compiler that memorized the test inputs. 97% on validation. 0% on holdout.

Cheating scales with the size of the system. And system size is exactly the dimension a factory operates in.

If you think this is a lab problem, look at where it happens in the place you trust most. SWE-bench Pro ships the full .git directory inside its containers. Agents run git log --all and copy the already-merged fix. It is not a theory: these are open issues in the benchmark maintainers’ own repositories, one of them literally titled “Git Reward Hacking”.

When your only signal is a green light, you are not measuring whether the work is right: you are measuring whether the agent managed to turn it green.

The bottleneck didn’t disappear: it moved

There is a 2009 figure that is still the ceiling. SmartBear studied code reviews at Cisco: ten months, two thousand five hundred reviews, three million two hundred thousand lines. The results:

Reviewing 200 to 400 lines at a time, over 60-90 minutes, finds between 70% and 90% of defects.
Above 400 lines, the ability to find them drops.
Above 450 lines per hour, the density of defects found is below average in 87% of cases.
Detection collapses after 90 continuous minutes.

That ceiling is physiological and has not moved in seventeen years. When you push more volume through than it can take, this happens:

Faros AI measured exactly that. Telemetry from 22,000 developers in more than 4,000 teams, over two years. Moving from low to high AI adoption:

This goes up	And so does this
+33.7% task throughput per dev	+242.7% incidents per PR
+66% epics per dev	+57.9% monthly incidents
+16.2% merged PRs per dev	+54% bugs per dev
	+31.3% PRs merged with no review at all
	+441.5% median time in review

Look at the pair in the first row. It is not that there are more incidents because there is more code: each PR is more dangerous than before. You do not fix that by producing faster.

And the 31.3% of PRs merged without review comes from there: reviewers cannot keep up.

Putting another agent on review doesn’t close the loop

The obvious answer is to delegate review to an agent. There is a study of 278,790 review conversations across three hundred GitHub projects that measures what happens:

Agent suggestions are adopted 16.6% of the time. Human ones, 56.5%.
More than half of the ones not adopted were simply wrong.
When they are adopted, they increase code complexity and size more than human ones.
Between 85% and 87% of reviews started by an agent end after the first comment, with no discussion.

An automated reviewer with an adoption rate three times worse than a human’s does not take work off your plate: it adds noise someone has to filter.

The two success stories, read at the source

The two cases everyone cites to prove the factory works say something else when you go to the original post.

Stripe published in February that more than a thousand pull requests merged each week are produced entirely by its agents, the “minions”. That is the figure that gets passed around. What does not get passed around is in the same post, and I quote it because it changes the whole meaning: “Over a thousand pull requests merged each week at Stripe are completely minion-produced, and while they’re human-reviewed, they contain no human-written code.”

In other words: the company that has built the factory best has not removed the verification bottleneck. It has fed it. Now it has a thousand more PRs a week to review.

Spotify reports more than 1,500 PRs merged to production by its agent Honk, saving 60-90% of the time. The work they list is this: language modernization (Java records), upgrades with breaking changes in Scio pipelines, UI component migrations, YAML and JSON file updates.

They are migrations. Mechanical, repetitive work, with an objective correctness criterion that can be verified by compiling and by the tests that already existed.

That does not refute anything above. It confirms it: agents perform spectacularly where verification is cheap and automatic. And Spotify itself, in a later post titled “Coding Is No Longer the Constraint”, admits they now have 76% more PRs to review and that the model performs worse on fragmented codebases. They have measured it.

There is an underlying problem with the three figures that get passed around: Stripe, Spotify and OpenAI’s million lines. All three publish the numerator and none publishes the denominator. How many PRs were actually reviewed, how many were reverted, what incidents they caused. And counting PRs is discredited as a measure of value: in the Copilot productivity study, net lines of code correlate with perceived productivity at ρ ≈ 0.09. Zero.

None of the three figures needs to be debunked; you just have to ask for the other half.

What I have found myself

Everything above is other people’s data. This is mine, from the last two months building an agent pipeline on a real project. No names, but the figures and the sentences are real (translated from Spanish).

August, a Tuesday. An agent closes its task like this: “service done. 7677 tests · 28506 assertions · 0 failures”. It also brings the diagram of the whole chain, from the HTTP request to the database, and three justified design decisions. A flawless deliverable.

No test covered the new behavior. And the tests that did pass did so because the agent itself had added a mock of the collaborator it had just introduced. In other words: the mock it added to make things compile was exactly what kept the light green. It could not be reached through the interface either, because the front end was not sending the field yet.

A one-line question caught it: “did you test it? From the browser too?”. It went to check and answered this:

“I ran the suite and it’s 7677 green, but that proves nothing about what I just wrote. The 5 tests pass because I added the resolver mock to them, not because they check anything. It’s a textbook false green.”

Two weeks later, another one. An agent delivers the test that protects the order of two calls inside a service, and delivers it with this claim: “if someone swaps the lines in this service, this test fails”. Full suite green, over nine thousand tests, zero failures.

I did not believe it, so I swapped the lines on purpose. The test passed anyway: seven out of seven green with the order reversed. The reason is a subtlety of the mocking library: the ordered() modifier only orders calls within the same mock, and across different mocks you need globally()->ordered(). A later sweep found another identical case in the same repository.

An order test that does not fail when you reverse the order is worse than having no test, because you assume a rule is protected when it is not.

The pattern, which was not the one I expected

Looking for these cases I expected to find the agent cheating: modifying the test to pass, reverting the work of whoever was going to judge it. There is not a single one. In none of the incidents I have been able to reconstruct did the agent know it was giving a false result.

What there is is more boring and a lot more dangerous: the agent believes its own signal. Mocks holding up the green. Assertions that cannot fail because they check a field that does not exist. A helper that did console.log and return inside the body of the it(), so the runner counted as passed tests that did not execute a single line. A ... | head && echo "OK" where the && was chained to head and not to the type checker, so it printed OK with real type errors.

And in the five cases I have reviewed, what exposed it was always the same thing: someone who broke the code on purpose to see if the test noticed, or who asked “did you really test it?”.

Not once did the suite catch it.

And yet, it works

The only large, multi-company controlled trial says it does work. Cui, Demirer, Jaffe, Musolff, Peng and Salz published three field experiments at Microsoft, Accenture and a Fortune 100 company in Management Science in 2025, with 4,867 developers: +26.08% completed tasks. Peer-reviewed. Three hundred times more subjects than the METR study everyone cites against it.

Google measured +21% in its own internal trial. And in July Microsoft published the results of rolling out command-line agents to tens of thousands of engineers: adopters merged around 24% more PRs, with sustained adoption and not a novelty effect. The authors limit their own claim with an honesty I appreciate: “a merged PR is not the same as the value it delivers”.

The study that goes against what I advocate

I have been advocating specification-driven development for a while. In October, Brenn Hill published a paper at ESEM 2026 that tests five hypotheses derived from SDD vendors’ claims on 88,052 pull requests across 119 repositories, with defect tracing and within-author fixed effects. None of them held. PRs with specifications do not have fewer defects (−0.6 points, p = 0.133, over 17,375 matched pairs). They do not reduce rework either: within the same author it comes out at +1.2 points (p = 0.001), and in the matched pairs at +0.5 (p = 0.146). And specification quality has no effect on rework (p = 0.860).

His conclusion, verbatim: “Specification artifacts proxy for task complexity, not quality improvement.”

One clarification. Without adjusting for complexity, PRs with specifications come out 1.4 points higher in defects within the same author, and the author himself warns that this should not be read as specifications causing defects.

That said, here are the two limitations the author himself declares, because they are the ones that matter. The first:

“If the benefit comes from the thinking rather than the document, our measure is too coarse to detect it.”

To show how coarse it is: almost half of the PRs it counts as specified (11,348 of 24,298) are counted that way because the description links an issue with a fixes #456. That is not spec-first; it is paperwork detectable in a piece of metadata.

The second:

“A genuinely agentic workflow – a specification authored as agent input and implemented end to end by the agent – is barely represented here, and we cannot rule out that it would behave differently.”

But they do not change the state of things either: today there is no study showing that specification-driven development reduces defects. There is this one, which finds no effect, and two reasons (from the author himself) to think it measures something else.

And I buy the distinction he closes with completely:

“SDD vendors conflate directing the AI (telling it what to build) with ensuring quality (fewer defects, less rework). Specifications may be effective at direction, but an agent faithfully executing a spec will reproduce every gap, ambiguity, and wrong assumption in that spec. Direction is not quality.”

That is exactly what happened to SDC in 1978 with the Los Angeles police system.

The signal matrix

If the green light is not enough, what do you use?

Every signal sacrifices something. Three properties:

Scalable: can you run it a thousand times at no cost?
Truthful: does it measure what you actually care about?
Tamper-proof: can the agent cheat it?

Signal	Scalable	Truthful	Tamper-proof
Tests	✅	❌	❌
Types and linter	✅	❌	✅
Accessibility tree	✅	➖	✅
Screenshot	❌	✅	✅
Another AI judging the work	✅	➖	❌
Real user feedback	❌	✅	✅

No row has all three.

Tests scale perfectly, but passing is not being right, and on top of that the agent can touch them. Types and the linter are not truthful either (correctly typed code can be badly designed), but they are tamper-proof: you cannot talk the compiler round.

The accessibility tree is the one almost nobody uses and everyone should. It is the structure of the page as text, and Playwright already exposes it as a snapshot assertion. It scales, it cannot be tampered with, and it is half truthful: it tells you which elements are there and what they are called, but not whether they rendered correctly.

There is something that gets repeated a lot and needs correcting. People say the accessibility tree is ten or a hundred times cheaper in tokens than a screenshot. It doesn’t add up. With Anthropic’s formula for image tokens, width times height divided by 750, a viewport screenshot comes to about 1,600 tokens; an ARIA snapshot of a typical page runs between 500 and 4,000. The advantage of the accessibility tree is not the cost: it is deterministic text, which you can compare with a diff and write assertions against. The determinism argument holds. The price argument does not.

The screenshot is the opposite: it is the only one that sees whether the animation ran, whether the chart rendered, whether the notice says what it was supposed to say. And it does not scale.

Another AI judging the work, with the criteria you give it, deserves its own warning, because there is an outdated cliché. A Berkeley study with twenty-one judges and about 541,000 judgments measured verbosity bias below 0.011 across all twenty-one models. The idea that judges reward long answers no longer holds. What does fail is something else: raw agreement overestimates real discrimination by 34 to 41 points. High reproducibility does not imply validity. A panel of judges can be wrong consistently and with great confidence.

How to apply it per layer

On the back end: tests to scale, plus types and linter to cover the tampering gap. And if you can, randomized tests, so the agent does not overfit to the ones it sees.
On the front end: the accessibility tree as the default signal, because it is cheap and deterministic. Screenshots only where the structure tells you nothing: animations, charts, notices.
On top of everything: someone checking whether the design holds up. You do not automate that.

And one that is not in the table because it is not a signal, but the way to check that your signal exists: break the code on purpose and see if the test notices. Swap the two lines whose order it claims to protect. Change the value it claims to validate. If it stays green, you do not have a signal, you have a decoration. It is the only thing that caught the five cases I described above, and it costs thirty seconds per test.

A practical warning about the second point: the accessibility tree includes elements that are off screen, and there is an open bug in Playwright about this. If you do not filter them, your agent writes tests against buttons the user cannot even see.

And about property-based tests, which are being sold as the solution: one study concludes that the properties models generate are often trivial or wrong, and another measures that property-based and example-based tests each find 68.75% of bugs, but combined they reach 81.25%. Together they find more than either one alone.

If you want the detail of how to build the scaffolding around these signals, I covered it in harness engineering and loop engineering (in Spanish); when to split work across several agents, in graph engineering (in Spanish); and what to put in the agent’s window so the signal reaches it, in context engineering (in Spanish).

What is sold as “the factory” is these four layers together.

The dark factory doesn’t work

The strongest argument against the dark factory comes from physical manufacturing.

It is the most favorable domain possible for full automation. The parts are identical. Tolerances are measured. The complete specification fits in a CAD drawing. And verification is objective metrology: the part measures what it has to measure or it does not.

FANUC has run its dark plant since 2001. Twenty-five years. Robots building robots, about fifty units per twenty-four-hour shift, up to thirty days in a row without supervision.

According to the International Federation of Robotics, the global average density is 132 robots per 10,000 manufacturing employees. 1.3%. And that is after doubling between 2014 and 2024.

The plants sold as “humanless” are not either. In Xiaomi’s plant in Changping there are people in a control room, maintenance technicians and software teams; what has been removed is manual line operations, not people. The industry is heading toward plants with fewer people inside, not empty ones.

Twenty-five years, the most favorable domain there is, and the dark factory is still a statistical anecdote. Software has none of those four advantages.

And there is one more argument, the deepest of all. Jack Reeves wrote in 1992 that source code is the design: in software, manufacturing is done by the compiler and the linker, and it costs practically nothing. If that is true, the factory metaphor is wrong at the root. Automating manufacturing saves nothing, because manufacturing was already free. Everything humans do in software is design.

Which is, word for word, what Brooks wrote in 1986:

“The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is as difficult as establishing the detailed technical requirements… No other part of the work so cripples the resulting system if done wrong.”

And what Parnas told him in 1985:

“Automatic programming always has been a euphemism for programming with a higher-level language than was presently available to the programmer.”

What to do with all this

Everyone is selling you the factory. Almost nobody tells you that the work moves, from writing to verifying.

Five concrete things before you build the pipeline:

Decide the signal before the tool. Scalable, truthful, tamper-proof: pick two, deliberately, and be very clear about which one you are missing. That is the architecture decision; the rest is plumbing.
Check the signal before you trust it. Deliberately break what the test claims to protect. If it does not notice, fix it before you put an agent to produce against that test.
Put a maximum size on the change. Not for elegance: because cheating scales with lines and human detection drops off after 400 lines.
Take advantage of migrations being the easy case. There verification is objective and automatic, and that is where Spotify gets its 60-90%. Start there, not with the feature with an open design.
Do not switch off the only truthful signal you have. Dark factories do not fail because of the models, but because they have removed the only thing capable of seeing that the design does not hold up.

P.S.: if you have built a pipeline like this and you have the denominator Stripe, Spotify and OpenAI forget to publish (how much was really reviewed, how much was reverted, what incidents came out of it), I would really like to see it, even if the number is ugly. And if you have a test that claims to protect something, try it: break it on purpose and tell me on X (@lm_martinbar) whether it noticed.