Why do the winners’ stories mislead you?

Picture a flight line in 1943. Bombers come home from over Europe, engines ticking as they cool, and the crews walk the aluminum with chalk, circling every hole. Wings. Tail. The long flanks of the fuselage. Do this across a whole squadron and a map appears: the damage clusters in the same places, plane after plane. The conclusion writes itself. Put the armor where the bullets go.

A statistician named Abraham Wald, working in a Manhattan office for the Statistical Research Group, looked at that same map and read it the other way around. The planes on the flight line were the ones that lived. Every hole chalked on a wing was a wound a bomber could take and still carry ten men home. So the map of holes was really a map of the survivable places. The parts left clean told the darker story by their silence: a plane hit in the engine or the cockpit tended to stay in a field in France, so it never reached the flight line to be chalked. Armor the clean spots.

(One footnote, since you have likely seen the picture. The famous diagram, a bomber peppered with red dots, came from a web designer around 2005, plotted with invented data on the wrong kind of aircraft. Wald himself left equations, in a memo so obscure the Navy reprinted it in 1980 to rescue it from a filing cabinet. The idea outlived its own paperwork, which is a small survivorship story of its own.)

Sit with the shape of the mistake, because it is the whole guide. The data in front of you was filtered before you saw it, and the filter removed exactly the cases that would have changed your mind. The engineers had every returning plane to study and zero of the planes that went down. Their sample was pre-sorted by the outcome they were trying to understand. So the map looked complete and pointed the wrong way, and the more planes they measured, the more confident and the more wrong the map became.

The move that saves you is small and specific. When a pattern comes from a set of visible winners, ask one question: who is missing from this picture, and why. The answer names your blind spot. The failed startups that folded before anyone blogged about them. The customers who left so quietly they filed no review. The funds that closed and dropped off the performance table. Each one is a plane that went down, and each absence is bending the pattern you can see.

Here is the flight line, drawn from above. The hollow marks are the hits on the planes that made it back. Flip the control to bring in the ones that never did, and watch where their damage lands.

The fatal hits fall precisely where the survivors were clean. That is the tell, every time: the smooth, unblemished region of your data is smooth because the counterexamples were removed from the room before you walked in. The place to armor was always the metal that came home clean. It simply refused to appear in the evidence, and appeared instead as confidence.

Now leave the war and walk into a sales pitch. A company hands you its case studies, its testimonials, its five-star wall. Every one is a returning bomber. They are real, they flew, and they are also the exact planes selected for being able to fly. The customers whose orders arrived late, whose accounts froze, whose refunds crawled, those crews are the silence around the display, and the display was assembled to leave them out.

The clearest paper trail runs through multi-level marketing, because the law forced the numbers into daylight. A September 2024 FTC staff report read the income-disclosure statements that seventy of these companies publish. Those statements headline the handful of top earners, the winners on the wall. The FTC put the rest of the room in numbers: most participants made $1,000 or less per year, and in at least seventeen of the seventy, most participants didn’t make any money at all. The recruit meets the survivors on stage. The people who went down are a footnote in a disclosure, if they appear anywhere.

This is the reflex worth building. Whenever a product shows you its winners, the useful evidence is the shape of who went missing, and the only way to see it is to go find them on purpose. That hunt is a whole discipline. A Nolemy record is one run of it: instead of the testimony a company curated, it goes looking for the customers who went quiet, and reads the promise against what they lived.