Three +1s and a prayer

How software quality worked before agents, and why I changed my view on traditional code review.

For most of my career I thought code review was one of the things that separated a serious engineering team from a mediocre one. Somebody writes code, opens a pull request, somebody else reads it, questions a few decisions and eventually approves. That second pair of eyes looked like a reasonable guarantee of quality, and over the years we quietly promoted the mechanism into doctrine. A pull request with no review felt irresponsible. A pull request approved by three engineers felt safe. My opinion has changed over the last year, mostly from working with agents and watching how the production of software is shifting underneath us, and what I now think is that we spent a very long time confusing a reasonable solution given our limitations with a solution that is good in itself. Human code review is not a fundamental property of software engineering. It is a historical approximation to a problem we did not know how to solve any better.

I started thinking about this seriously after a fairly stupid argument on X. The argument itself does not matter much, but the reaction does: suggesting that an agent can review code better than a human is still read by a lot of people as an attack on the profession. There is an almost emotional defense of certain practices because they are part of our professional identity. If for twenty years you were told that writing good code, reviewing it carefully and arguing about architecture with other engineers was precisely the thing that made you valuable, it is understandable that it feels uncomfortable to imagine a machine doing a meaningful part of that work more systematically. But something being part of our identity does not make it technically optimal. Technology has spent centuries removing activities that looked inseparable from a trade, and not because the people performing them were incompetent. Better infrastructure appears and the activity stops making economic or technical sense.

Software engineers have been, for decades, a layer between intention and execution. Somebody wants a machine to do something, the machine needs extremely precise instructions, and so we take a human idea, formalize it, and turn it into structures, conditions and finally code. A good part of our value came from being the middlemen of that translation. Now there is another intermediary, one that can take an ambiguous instruction, write an implementation, run the tests, read the failures and fix itself for hours without getting tired. It still fails, partly because it was trained on our own historical mistakes, and it still needs a lot of context. The direction is hard to ignore anyway. The intermediary is being intermediated, which should feel familiar to us, because doing exactly that to other industries has been the profession’s main export for fifty years.

We told ourselves for a long time that code review existed to guarantee the code was correct, and it never did that. Somebody opens a pull request of two and a half thousand lines, looks at the diff for twenty or thirty minutes, leaves a few comments and writes LGTM. From that moment on we feel the code has been validated. The only thing we actually know is that another human looked at it for a limited amount of time with a limited amount of context. We do not know how many architectural invariants they held in their head, whether they know every path the change touches, or whether they were mostly trying to finish before the next meeting started. Two people looking at the same change do not turn that inspection into exhaustive verification either. It means two people made two human approximations of the same problem.

There is also a fairly uncomfortable piece of evidence that manual review was never the safety mechanism we liked to imagine, which is that it never stopped the bugs, the vulnerabilities or the CVEs. Before I started working with agents it was normal on most projects to turn up one or two serious problems of that kind every so often and take that as a sign the review process had done its job reasonably well. Now, when we point agents at whole repositories and have them run the same analysis repeatedly, audit dependencies and follow error paths, dozens of alarms come back for things that had been sitting there for months or years: violated invariants, missing validation, authorization mistakes, edge cases nobody had considered. All of that coexisted perfectly well with pull requests that humans reviewed, approved and merged. I do not think AI created a new epidemic of vulnerabilities, although I would not rule out that some share of it is freshly generated code going in faster than anyone can read it. Mostly it is older than that. If traditional code review had really been a strong barrier against these problems, the arrival of agents should not be continuously digging up historical defects in codebases that already went through years of it. The software did not suddenly get worse. We finally started looking at it deeply enough to see what the human process had been letting through, and the backlog was already there.

Another thing that strikes me when I read old arguments about code review is that we never actually agreed on what problem it solves. It gets defended because it catches bugs; when somebody points out that plenty of bugs reach production anyway, the answer becomes that its real value is spreading knowledge. If it slows the team down too much, the value shifts again and turns into ownership. If the comments are mostly about style, then at least it is teaching the less experienced developers. Every one of those functions can be worth something, and it is still suspicious that the same ceremony gets justified by a different reason each time it fails at the previous one. My guess is that we have spent years using a single mechanism to solve several unrelated problems. Sharing knowledge and training engineers both matter, and not letting one person be the only one holding the context of a system is still good practice. What is not clear at all (to me) is why manually reading every diff has to be the way we get any of those things.

Traditional review is not especially logical or reproducible either. Every reviewer carries experience, preferences, fatigue and biases accumulated over years. An engineer can reject today an abstraction they would have accepted six months ago. One prefers a reduce() because it expresses the intent better; the person sitting next to them prefers a for because it is more explicit. One believes a particular generalization anticipates real future needs, the other calls it premature complexity. These discussions can be genuinely useful, and they also expose something uncomfortable about the process, which is that a significant part of what we call quality depends on whichever brain happened to receive the pull request that day.

Teams tried to reduce this by inventing rules. We wrote coding guidelines and conventions, and we built linters, formatters and CI pipelines. Every one of those tools is, in a sense, a defeat for human code review, because it takes an argument that used to happen between people and converts it into infrastructure. Nobody wants a reviewer spending time on spaces, quotes or import order anymore. A machine resolves those the same way every time. We also do not want to depend on somebody remembering whether a vulnerable dependency slipped in or whether a test stopped passing. The more mature the engineering, the more decisions we pull out of human conversation and turn into verifiable properties of the system.

That trend matters more than we usually admit. If the same comment keeps appearing across different pull requests, that is not a good review process, it is an automation nobody has built yet. If every three weeks somebody has to explain that a particular layer must not talk to the database directly, the organization does not really have that rule. It has a person who remembers it. If an architectural decision evaporates when the engineer who made it leaves the company, it was never part of the system to begin with. A good engineering decision should outlive whoever made it: first as documentation, then as a test, and wherever possible as a constraint no implementation is able to violate.

There is also a limitation far more basic than our biases, which is how much context a human brain can actually hold and consciously check. A three thousand line pull request can contain dozens of architectural decisions, invariants and distinct error paths. Reviewing it for real, rather than scrolling the diff until something catches your eye, could take days. You would have to check every rule in the guidelines, follow every side effect and read every test, and do it with the same discipline on the last line as on the first. No team works like that, because it would be economically absurd and cognitively exhausting. What we call a thorough code review is usually human sampling: we recognize familiar patterns, we stop where something looks dangerous and we trust that nothing important fell outside the sample. An agent operates under a completely different cognitive budget. It can walk every rule, every line and every relevant path one after another without starting to skip checks because it has been staring at the same diff for two hours. The advantage is not necessarily that it reasons better than we do about any single isolated problem. It is that it can sustain a level of review discipline that is practically impossible for a person to maintain.

Agents push this further than linters ever could, because for the first time a part of the judgment that resisted being written as a static rule becomes automatable. An agent can read the documentation, compare earlier implementations in the repository, run the benchmarks and repeat the whole loop. It can receive the same context again and again. It can revisit a decision without getting bored. It can go through thousands of lines with a patience no reasonable human would spend on every pull request. That does not make today’s models infallible, and they clearly are not. It means we are starting to have infrastructure capable of converting a growing share of informal judgment into a systematic process of search and verification.

When I worked at Rakuten, one of the things that impressed me most at the beginning was their code review culture. There were many teams working with completely different technologies, but the code was visible across all of them and anyone could take part in a review. An Objective-C developer could comment on a pull request written in Ruby without belonging to the team that maintained the project. At the time I thought it was a fantastic idea. The more eyes on the code, I assumed, the higher the odds of catching problems and the better the decisions that ended up in production. It was exactly the sort of system you picture when you imagine a mature engineering organization.

With time I started to see something different. A large share of those reviews did not uncover important errors or question deep architectural decisions. They were observations like you are using let here when it should be const, those little bastard smells and details we would not even expect a human to look at today. That does not make the observations wrong. It means the existence of many reviewers did not imply that anybody was building a complete model of the change and actually verifying its behavior. A developer could review code written in a language they did not work in precisely because a good part of what they were finding sat on the surface.

Then there was the daily reality of the process. Slack filled up with messages asking for review my PR or please give me a +1, I want to merge, and the operational goal quietly became getting enough eyes on the change to unblock it. A mechanism designed to improve quality also produced enormous fights over tiny things. I remember arguments escalating over trivial implementation choices, and messages in Slack or over email where somebody ended up saying the code was pure shit, that they could not work with retards, that they were done. Code review had not removed human subjectivity from engineering. In some cases it had given it a formal interface to express itself through.

There is one story from that period, a genuinely funny one, that paradoxically does justice to a real virtue of traditional code review. The only engineer who owned an iOS application left the company, and his replacement went two or three months without opening a single pull request. When the day finally came, the pull request had around four hundred and fifty thousand modified lines, in just a handful of commits. You can picture the diffs. It went around several Slack channels within minutes and people spent most of the day laughing at it, because without the required +1 nothing could be unblocked for merge. The new engineer had decided the codebase he inherited was not organized the way he liked it, converted spaces into tabs, restyled whole files, and added enough on top of that to touch practically every line of the application. A change that size was impossible to review seriously under the traditional model, since nobody was going to read hundreds of thousands of lines one by one and work out the consequences. The VP of Engineering at the time rejected it. That is a fair example of review working as a social containment mechanism, because you did not need to understand every line to see that something was badly wrong with the whole idea. It is also where the limit shows. The system could stop an obvious atrocity, and it was never built to verify a modification of that size.

There was one rule that gave the whole thing away: if nobody left any comments on a pull request, you could merge it after a day. That was bonkers. Silence became a form of approval, which meant the most reliable way to ship anything was to write a pull request so boring that nobody answered it. That is a very clean illustration of the difference between review and verification. The system could prove the code had been available to be reviewed. It could not prove that anyone had understood it, checked the relevant invariants, or spent any meaningful attention on it at all. The ritual was complete, and nothing about the software followed from that.

What that experience eventually taught me, and it took me years to put into words, is that increasing the number of humans who can look at a change does not solve the fundamental problem. You can have ten potential reviewers and still not have an exhaustive review. You can collect three +1 and none of those people investigated the architecture. You can have hundreds of cultural rules and most of them live only in the memory of whoever happens to be around. Even an extremely strong review culture still depends on a variable nobody can standardize, which is how much time, context, attention and energy each person decided to spend on that specific change on that specific afternoon. And if code review still feels useful to you today because you keep catching things in your colleagues’ pull requests, my honest read is that your agent rules are not good enough yet. Most likely they are incomplete.

Underneath all of it is that we confused the opportunity to review with review having happened, and the confusion gets worse as volume grows. When the number of changes goes up, the process degrades in a completely predictable way: people skim, leave a couple of comments, and somebody approves because the team needs to keep working. That is not negligence. It is what happens when you try to scale a process whose basic unit is still human attention. I have approved changes like that, not many and not recently, but enough to know that the pressure to unblock somebody is a real force and that it does not feel like carelessness while it is happening. The more code we produce and the more complex our systems get, the wider the distance between having given somebody the chance to look at a change and having shown that the change actually holds the properties we expect from it.

There is another human limitation we usually skip over when we talk about review, which is that nobody actually knows everything they are reviewing. I have been writing JavaScript for more than twenty years, along with a few other languages, and I do not remember all of its methods, let alone all the APIs added along the way. I still look up what sort() does to numbers every single time, the way you go back upstairs to check whether you turned the stove off. I have worked with CSS for years and I still forget properties, specific behaviors and relatively recent features until I need them again. This is not a personal failing or a sign of inexperience. It is the unavoidable consequence of working inside technical ecosystems that grow much faster than our capacity to memorize them. A modern language, its standard library, its dependencies and the tooling around it add up to an amount of information no reasonable person keeps permanently loaded.

The problem gets worse in real projects, because almost none of them live inside a single language. A backend in Go sits next to a frontend in TypeScript, infrastructure declared in Terraform and the usual sediment of shell scripts and SQL. A single change can touch several of those layers at once. Expecting a reviewer to know all of them deeply is not realistic. Somebody can be an excellent Go engineer and not remember that a particular syscall already does correctly what the author implemented by hand. They can see perfectly well that the code works and still not know that Fchownat, to pick one example, avoids reinventing an operation the system already provides. How many engineers hold in their head every function in Go, every relevant Linux syscall and every recent change in CSS, plus the company’s internal libraries? Probably none, and the few who come close will still have enormous regions of incomplete knowledge.

Human review depends, then, not only on what the reviewer knows, but on what they happen to remember at that moment. That distinction turns out to matter a lot. Somebody can have used an API five years ago, understand the concept perfectly well, and simply not recall it while reading a pull request. If nothing in the code looks suspicious, there is no particular reason for them to go open the documentation and look for an alternative. The review approves an implementation that is perfectly functional and unnecessarily complicated, because none of the people involved remembered that the ecosystem already had a better answer.

Agents have a structural advantage here, because their knowledge does not have to be limited to whatever fits in a human memory at one time. An agent connected to the right tools can consult the documentation for the exact version the project uses, read the source of a library, and then verify whether the alternative it found actually works. It does not need to remember that a function exists in order to benefit from it. It needs to be able to discover it. That converts a knowledge problem into a retrieval problem, and machines are particularly good at doing retrieval repeatedly and at scale.

The gap widens with time. An engineer may have learned JavaScript in 2004, Go in 2015 and Rust in 2022, and none of those languages stayed frozen at the moment they learned it. Every year brings new APIs and idioms while others quietly go obsolete. Keeping deep knowledge of a single ecosystem already takes continuous effort; keeping it across the five or six ecosystems a modern system spans is essentially impossible. This is why even the best engineers work with the documentation open on a second screen. The difference is that a human runs those searches when they suspect they should. An agent can be instructed to run them always.

That introduces an asymmetry between the two kinds of reviewer that I find more interesting than the usual comparison. The human tends to ask do I see anything wrong here?. An agent can be given a far more ambitious challenge: for every non trivial operation, check whether the language, the standard library, the operating system or some dependency already provides an equivalent primitive, and compare both solutions. Doing that by hand across three thousand lines would be absurd. Doing it automatically is just part of the computational cost of the review. The human uses intuition to decide where it is worth investigating; the machine can investigate in places where there is no visible signal of a problem yet.

None of this means the model should trust whatever it thinks it remembers. That would be committing exactly the error we just criticized in humans, and it is the failure mode I see most often in practice. The right system does not ask the model whether it recalls that an API exists. It gives the model access to the documentation, to the source, and to the version the project actually uses, and it demands evidence before any suggestion turns into a change. The advantage is not a bigger pile of memorized knowledge. It is the ability to combine reasoning with systematic retrieval and repeat that loop as many times as necessary.

Put all these limits together and the problem with human code review starts to look deeper than it first appears. We have limited memory, limited attention, limited time, and incomplete knowledge of ecosystems that keep moving. Then we put a person in front of thousands of lines written in several languages and expect them to find not only what is broken but also what works and could be implemented far better. We call that process review, when what we are really asking is for a human brain to perform a search over a knowledge space much larger than it can contain. Agents do not make that space disappear, but for the first time we can build reviewers whose ability to explore it is not bounded by what one person managed to learn and retain over a career.

This is where the argument about code review starts to feel too small to me. The interesting question is not whether GPT or Claude or any current agent reviews better than a senior engineer. That comparison will probably be irrelevant in a few years. The question that matters is what happens when writing software stops being mainly a creative process driven by intuition and becomes, progressively, a search problem over an enormous space of possible implementations. We cannot write ten thousand versions of a function, run them against real workloads, measure CPU and memory and latency under load, and keep the one that best satisfies our constraints. The human cost would be ridiculous. A machine does not have that limitation.

In that scenario, a lot of discussions we currently consider engineering simply stop existing. If two engineers argue about which algorithm is faster, a machine can implement both and measure them. If there are twenty reasonable strategies, it can try twenty. If an architecture promises less memory at the cost of more latency, it can measure exactly how much and tell you whether it stays inside the limits you defined. If we want to minimize p99 while keeping memory under a certain figure and guaranteeing specific invariants, we are no longer asking which implementation a reviewer likes best. We are defining an objective function and a set of constraints, and once that is written down the discussion stops being philosophical and becomes an optimization problem.

Talking about a mathematically perfect solution obviously requires defining what perfect means, and in most systems there is no implementation that dominates all the others. One alternative uses less memory and is slightly slower, another maximizes throughput at the cost of binary size, a third cuts latency and burns more energy. Perfection shows up once we formalize the objective. If we state what must be minimized, what can be sacrificed and which limits are non negotiable, the space for argument starts to shrink. We do not need the machine to have an opinion about which implementation is more elegant. We need it to explore the space of solutions and show which one best satisfies what we asked for.

I suspect this transition eventually reaches the idea of a bug itself. I wrote a while back that the last bug on Earth will not really be a programming error but a business case nobody documented properly. The better our systems get at synthesizing implementations, measuring them and correcting deviations automatically, the higher the point where errors can still appear. If the program implements the specification exactly and the specification is wrong, the problem is no longer in the code. It is in what we decided correct meant.

That changes the job considerably. For decades we spent an enormous share of our time deciding how to build something. In a world of increasingly good synthesis and verification, the work moves toward deciding what it means for something to be built well. Which properties it has to satisfy, which states are valid, which trade-offs we are willing to accept. What should happen when two requirements contradict each other, which in my experience is the requirement nobody writes down and the one that ends up causing the incident. The better the machine gets at implementing, the more our capacity to specify becomes the bottleneck.

Human review occupies part of that space today because we are still using people as a compensation mechanism for incomplete specifications and insufficient tooling. A reviewer is not only hunting for bugs, they are trying to guess intent. They ask whether this was really what the product wanted, or whether the implementation faithfully represents a business rule that may never have been written down anywhere. Those questions exist precisely because so much of our process remains informal. As specifications get more complete and our systems get better at automatically verifying that an implementation satisfies them, the useful space for a reviewer’s opinion gets smaller.

Maybe in a few years it will look strange that entire teams spent hours every week reading diffs by hand, the way it now looks strange to picture somebody walking a city at dusk lighting streetlamps one at a time. Not because those teams were doing something absurd. It was the best available approximation. We wrote imperfect code from imperfect requirements and used other imperfect humans to try to catch some fraction of the errors before production. It worked reasonably well, and it was always an approximation.

The natural evolution of engineering consists precisely of removing approximations when we can replace them with something stronger. We stopped trusting that somebody would remember the format and built formatters. We stopped relying only on examples and added property testing. Scanners took over the job of remembering which dependency had a known problem. Code review has no special claim to be exempt from that. If we can turn an opinion into a rule, we should. If we can turn a rule into an automatic check, better. If we can explore thousands of alternatives and measure which one best satisfies our goals, there is not much sense in sitting two people in front of a screen to argue about which one they believe might work better.

Which is why I no longer picture the future of code review as a process where an agent helps a human read a pull request. That looks like an intermediate phase. The end state is probably a system where the notion of a pull request loses most of its importance. The human defines intent, constraints and invariants; the system produces candidates, discards the ones that fail the required properties and keeps searching until it finds an implementation that is good enough or demonstrates that the constraints are incompatible. Review stops being a conversation about code and becomes evidence that certain properties hold.

At that point it is hard to see what value there would be in saying that you would have preferred a for instead of a reduce(), or that a particular abstraction feels more elegant. Those are concerns born in a world where humans have to read, understand and maintain every line directly for years. If we reach a moment where no human needs to modify that implementation by hand because the system can regenerate it, verify it and optimize it whenever the requirements change, a good number of our current ideas about readability and maintainability will need revisiting too. Not because they stop mattering overnight, but because they were designed around our own cognitive limits and we never really separated the two.

The goal was never to produce code that other programmers enjoy reading, and it was certainly never to accumulate approvals on GitHub. The goal was to build correct systems. For a long time we used humans reviewing the work of other humans because that was the best approximation we had, and now we are starting to build tools that can turn it into something much closer to search, measurement and verification. The further we go in that direction, the less room is left for opinions about implementation, and the more weight falls on the part I cannot see any way to delegate, which is somebody sitting down and deciding what the machine is actually supposed to do. We have historically been quite bad at that part.