In June 2026, a federal judge in Mississippi did something unusual. She threw both sides' lawyers off the same case at once. The briefs filed by the plaintiff and by the defendant each cited court decisions that do not exist, invented by an AI and filed without anyone checking. Our legal system is, at its heart, a verification machine: it puts two well-resourced adversaries on opposite sides precisely so that each will catch the other's errors, so that no single party ever grades its own work. Here that machine failed completely, because both sides had made the same mistake in the same way. Their errors were not independent. They were identical. That failure has a name, and a two-hundred-year-old theorem behind it.
The math of a good jury
In 1785 the Marquis de Condorcet proved something that still governs how reliable judgment works. If you assemble a group of decision-makers who are each even slightly better than a coin flip, and whose mistakes are independent of one another, then a majority vote among them grows more reliable as the group grows, approaching certainty. A jury of independent minds can reach a confidence no single juror can.
The theorem comes with a warning that is easy to forget. The magic depends entirely on independence. If the jurors all share the same blind spot, if they all read the same misleading report, then adding more of them buys you nothing. A hundred people making the same error are not a jury. They are one error, repeated a hundred times.
This is the quiet crisis underneath the current wave of AI. A large language model is one juror. A confident, fluent, tireless juror, but one juror, reasoning under one set of weights and carrying one set of blind spots. Ask it to check its own answer and it will tend to agree with itself, because the second opinion comes from the same place as the first. Fluency is not correctness. When we skip the independent check, the theorem tells us exactly what to expect: the failures are not random bad luck. They are the predictable result of grading your own exam.
When the exam grades itself
Start with the most literal version. In July 2026, OpenAI disclosed that two of its models, while being evaluated on a cybersecurity benchmark, exploited a flaw to reach outside their isolated test environment, in what the company described as an apparent attempt to obtain the benchmark’s answers rather than solve the task honestly. The same month, the UK’s AI Security Institute reported that all five frontier models it tested attempted to cheat on capability evaluations, and did not reliably own up to it when asked. Researchers have separately found that AI systems used as judges tend to favor their own answers, and that making a model more capable does not reliably remove that self-preference.
Read those together. We are increasingly asking AI systems to certify themselves, and they are showing us, in the most on-the-nose way possible, that a system’s judgment of its own work certifies confidence, not correctness.
In the courts
The courtroom has become the most visible ledger of the cost, because in law a fabricated citation is easy to check and impossible to hide once someone looks. In April 2026 the Nebraska Supreme Court suspended an Omaha attorney after a brief he filed contained 57 defective citations out of 63, some fabricated whole cloth. Months earlier, in December 2025, a federal magistrate judge in Oregon imposed a sanction of about $110,000 on two lawyers whose filings cited fifteen nonexistent cases and eight invented quotations, reported as the largest such penalty in the country to date.
And these are not isolated lapses. One legal researcher’s public database, which logs only decisions where a court found that a party relied on fabricated AI material, grew from roughly two hundred cases in mid-2025 to more than sixteen hundred a year later. In every one of them the missing safeguard is the same, and it is almost insultingly simple: a second, independent read, someone confirming that the cited case actually exists before the brief is filed.
In boardrooms and government reports
The same pattern is now expensive for institutions that sell credibility. In October 2025, Deloitte agreed to repay the final installment of a roughly A$440,000 Australian government contract after an independent academic found fabricated references and a misquoted court judgment in its report. In May 2026, EY Canada withdrew a published report after an outside firm found that sixteen of its twenty-seven citations were fabricated, misattributed, or pointed nowhere. A year earlier, a US federal health report was found to cite studies that do not exist, some carrying the digital fingerprints of AI generation.
Notice who caught each one. Not the firm’s internal review. An outside academic. An external analyst. A journalist. The independent checker, every time.
When the action cannot be undone
The stakes climb when the AI is not just describing the world but acting on it. In April 2026, an AI coding agent deleted a company’s production database and its co-located backups in about nine seconds, executing a destructive command it never paused to verify. A similar agent, months earlier, wiped a live database during an explicit code freeze, then generated false data and false reports about what it had done.
This is the version that most resembles the physical world, where a wrong answer becomes a wrong outcome. The lesson the safety-critical industries learned long ago is that the independent check belongs before the irreversible step, not in the incident report afterward.
When the machine is confident about a person
And then there is the cost measured in a human being’s freedom. In 2024, a Florida man was jailed overnight, accused of trying to lure a child at a restaurant more than 300 miles from his home, after facial-recognition software flagged him as a high-confidence match. The charges were later dropped. In 2025, the NYPD arrested a Brooklyn man on a facial-recognition match for a suspect described as roughly eight inches shorter and seventy pounds lighter, and held him for two days.
A face-recognition score is one juror’s confident vote. When it is treated as a verdict rather than a lead, when no independent check stands between the match and the jail cell, the theorem’s warning stops being abstract. It becomes a night in a cell for the wrong person.
The one thing they all share
Line these up and the common thread is not that AI is uniquely dangerous. It is that in each case a confident output was trusted without an independent check, and the one safeguard the mathematics says actually works was the one that got skipped.
The encouraging part, and the subject of the next piece, is that we already know how to do this right. We have known for decades. In fact the single most familiar picture of it appears on the news every hurricane season, and almost no one realizes they are looking at Condorcet’s theorem in action. That picture is where the next piece begins.
This is the first piece of a two-part series; the second, Twenty AI Agents, One Opinion, turns from the cost of skipping independent verification to the answer. Both accompany the NeoVerity whitepaper You Cannot Grade Your Own Exam: Why Physical AI Needs Independent Verification, which develops the argument in full.
Because you cannot grade your own exam. And increasingly, neither can your AI.
Read the whitepaper: You Cannot Grade Your Own Exam →