Abraham Wald and the Missing Bullet Holes
This incident in history struck me particularly as a remarkable story of questioning the assumptions.
Abraham Wald was a Hungarian mathematician. Since he was a Jew and faced discrimination from Austrian government, he immigrated to the United States once Nazis invaded Austria and began persecuting Jews. In the US, he became a member of the Statistical Research Group (SRG) while working at Columbia University, a classified program comprising of brilliant American statisticians. This group used to take up statistical assignments from the US government during World War II.
The US Government approached SRG with a problem. They needed to further reinforce and armor their bomber planes to prevent casualties from enemy firing. To do that, they had identified the places on the returning planes which were covered with bullet holes (highlighted in dark in the below image). They approached SRG with the compiled data of bullets distribution on returning planes from enemy territory and asked to optimize the distribution of armor on the susceptible places (wings and fuselage) such that the weight of the plane remains optimum for flying and maneuvering.
At least that’s the problem they thought they needed a solution for.
The above illustration on the right is that of a surviving bomber plane which got hit in the darkened portion with bullets. The majority of the bullet holes were found in the wings and fuselage and very few in the engine and cockpit of the plane.
Wald, after analyzing all the samples of returned bombers, concluded counterintuitively that the cockpit and engine needed the most reinforcement. The armor shouldn’t go where the most bullet holes are, it should go where they aren’t, concluded Wald.
Being a mathematician, Wald was trained to question the assumptions and analyze the sample data presented to him.
Wald’s insight was simply to ask: where are the missing holes? The ones that would have been all over the engine casing, if the damage had been spread equally all over the plane? Wald was pretty sure he knew. The missing bullet holes were on the missing planes. The reason planes were coming back with fewer hits to the engine is that planes that got hit in the engine weren’t coming back. Whereas the large number of planes returning to base with a thoroughly Swiss-cheesed fuselage is pretty strong evidence that hits to the fuselage can (and therefore should) be tolerated. If you go to the recovery room at the hospital, you’ll see a lot more people with bullet holes in their legs than people with bullet holes in their chests. But that’s not because people don’t get shot in the chest; it’s because the people who get shot in the chest don’t recover.
The normal chain of thought and reaction would’ve been to further reinforce and strengthen the portion which got hit the maximum number of times, the wings and fuselage. However, by doing so, the decision would’ve been biased by the samples, the bomber aircraft’s returning with the bullet holes were the ones which survived, the ones which got hit in the engine and cockpit never returned. This is called Survivorship bias.
Of course, I’ve oversimplified his work for the sake of explanation, he came up with a whole range of complicated mathematical equations to explain likelihood of aircraft’s survivability on being hit at different places in the aircraft [4].
Ironically, Wald and his wife died in an airplane crash.
References:
Comments
Post a Comment