The gate that passed by never running

We build tools that make claims about themselves. A sound meter that states its own uncertainty. A trainer that tells you it verified every position with an engine. Software that says what it cannot do as plainly as what it can. All of that rests on checks, and a check is a claim too: it says the thing it guards is fine.

Last week four of ours were lying. Not wrong about what they looked at. Silent about what they never looked at.

The first was the plainest. Our lint script had been broken since the framework removed the command it called. Every run failed instantly with a complaint about a missing directory, and every run exited in a way that nobody noticed. It had been reporting nothing for months. When we finally pointed it at a working linter, it found six real errors in the first pass, including one in the very tool we had built to catch problems in our writing.

The second was ours and newer, which makes it worse. We wrote a check that reads our published copy and blocks a deploy if the writing drifts from the house voice. It passed. The site still had em dashes on it. The check had been aimed at the file where most of our prose lives, and prose written directly into page components was outside the aim. The rule was correct. The scope was one file, and it matched one file, and it reported that as coverage.

The third came from a sibling repo. Their check for the same problem listed the file types it should scan. Two of those types were not among the types the scanner could read. They matched nothing. A configuration that looks thorough invites less suspicion than a rule that looks thorough, and it produces the same green light.

The fourth was the most interesting, and it happened while its author was reading a warning he had written about it minutes earlier. His build reports which guarantees a run has proven. That report was the union of everything the passing checks claimed. Split one check into two, have one of them fail, and the other one covers for it. A run whose entire writing check was missing still announced that the guarantee was proven, three lines under the word FAILED.

These look like four bugs. They are one, wearing different clothes. A check that did not happen reported as a check that passed. That is the thing to be afraid of, because a check that fails is a good day. A check that is absent leaves a green light on top of nothing, and a green light is the reason nobody looks.

There is a fifth shape, and it is quieter. Two checks can each correctly decline to look at the same thing. In one of our repos a scope rule skipped a directory on the grounds that a second check covered it, and that second check had filed the copy in question under a key it treats as a comment. Both were behaving correctly. The screen a user reads went unchecked by either. Nothing was broken. Two mechanisms each assumed the other had it.

The pattern in how we found them is the part worth keeping. One turned up because someone deleted a symlink to see what would break. One because a probe was written for an unrelated reason. One because we tested an encoding we assumed was safe, and the assumption was correct while the conclusion was wrong: the check was fine, and the step before it was deleting the evidence. Not one was found by reading the code, and we had all read the code carefully several times.

So we have stopped predicting how a probe will come out before running it, and we have started demanding that every new rule be shown to fail. Reintroduce the exact defect. Watch the count go from zero to one. Put it back. A rule that has never been observed failing is not evidence of anything, it is a decoration that makes people stop checking.

We wrote that sentence down, and then two of us violated it within the hour, in new code, in the file where the sentence lives. Writing a rule down does not make you subject to it. The only part of this that does not forget is the test.