In the course of reviewing the history of SRG, I was reminded of some ingenious work by Wald that has never seen the light of day. Arrangements have now been made for its publication, although the form and place are yet undecided. Wald wrote a series of memoranda on estimating the vulnerability of various parts of an airplane from data showing the number of hits on the respective parts of planes returning from combat. The vulnerability of a part (engine, aileron, pilot, stabilizer, elevator, etc.) is defined as the probability that a hit on that part will result in destruction of the plane (fire, explosion, loss of power, loss of control, etc.). The military was inclined to provide protection for those parts that on returning planes showed the most hits. Wald assumed, on good evidence, that hits in combat were uniformly distributed over the planes. It follows that hits on the more vulnerable parts were less likely to be found on returning planes than hits on the less vulnerable parts, since planes receiving hits on the more vulnerable parts were less likely to return to provide data. From these premises, he devised methods for estimating the vulnerability of various parts.
But clearly the most prominent statistical insight from the SRG was the idea of sequential analysis, which Wallis calls "one of the most powerful and seminal statistical ideas of the past third of a century." In his 1980 article, he reproduces a long letter that he wrote in 1950 on the subject. Doing quality control testing on potential new kinds of ordnance required firing thousands of rounds. Apparently, a general observed to Wallis that if someone "wise and experienced" was on hand, that person could tell within a few thousand or even a few hundred rounds if the new ordnance was either much worse or much better than hoped. The general asked if there was some mechanical rule that could be devised for when the testing could be ended earlier than the full sample. Wallis noodled around with this idea, and expressed it this way in his 1950 letter:
The fact that a test designed for its optimum properties with a sample of predetermined size could be still better if that sample size were made variable naturally suggested that it might pay to design a test in order to capitalize on this sequential feature; that is, it might pay to use a test which would not be as efficient as the classical tests if a sample of exactly N were to be taken, but which would more than offset this disadvantage by providing a good chance of terminating early when used sequentially.
Wallis remembers a series of conversations with Milton Friedman on the subject, after Friedman joined the SRG in 1943. They made some progress in thinking about tradeoffs between sample size and statistical power and what is learned along the way. But they also ended up feeling that the discovery was potentially important to the war effort and that they weren't well-equipped to solve it expeditiously. Wallis remembers a momentous walk:
We finally decided to bring in someone more expert in mathematical statistics than we. This decision was made after rather careful consideration. I recall talking it over with Milton walking down Morningside Drive from the office to our apartment. He said that it was not unlikely, in his opinion, that the idea would prove a bigger one than either of us would hit on again in a lifetime. We also discussed our prospects for being able to work it out ourselves. Milton was pretty confident of our (his?) ability to squeeze the juice out of the idea, but I had doubts and felt that it might go beyond our (my!) depth mathematically. We also discussed the fact that if we gave the idea away, we could never expect much credit, and would have to take our chances on receiving any at all. We definitely decided that even if the credit situation turned out in a way that disappointed us, there would be nothing to do about it,They ended up getting permission to talk with Abraham Wald on the subject, which wasn't easy, because Wald's time was "too valuable to be wasted."
At this first meeting Wald was not enthusiastic and was completely noncommital. ... The next day Wald phoned that he had thought some about our idea and was prepared to admit that there was sense in it. That is, he admitted that our idea was logical and worth investigating. He added, however, that he thought nothing would come of it; his hunch was that tests of a sequential nature might exist but would be found less powerful than existing tests. On the second day, however, he phoned that he had found that such tests do exist and are more powerful, and furthermore he could tell us how to make them.