Tuesday, July 24, 2018

Early Examples of Randomization in the Social Sciences

Randomization is one of the most persuasive techniques for determining cause and effect. Half of a certain group get a treatment; half don't. Compare. If the groups were truly chosen at random, and the treatment was truly the only difference between them, and the differences in outcomes are meaningful and the size of the samples are also large enough for drawing statistically meaningful conclusions, then the differences can tell you something about causes.

Economists have surged into randomized experimental work in the last few decades. From the 1970s, up into the 1990s, such work was often focused on social policies, with experiments on different kinds of health insurance, job training and job search, changes in welfare rules, early childhood education, and others. More recently, such work has become very prominent in the development economics literature, as well as on a variety of focused economics topics like how incentive pay affects work or how charitable contributions could be increased. Running experiments is now part of the common tool-kit (for a small taste, see here, here, here, and the three-paper symposium in the Fall 2017 issue of the Journal of Economic Perspectives on "From Experiments to Economic Policy").

Thus, economists and other social scientists may find it useful to keep some historical examples of randomization near at hand. Julian C. Jamison provides a trove of such examples in "The Entry of Randomized Assignment into the Social Sciences"  (World Bank Policy Research Working Paper 8062, May 2017).

Jamison has a run-through of the classic examples of randomization over the centuries, which were often in a medical context. For example, he quotes the correspondence between poet/writers Petrarch and  Boccaccio in 1364, in which Petrarch wrote:
"I solemnly affirm and believe, if a hundred or a thousand men of the same age, same temperament and habits, together with the same surroundings, were attacked at the same time by the same disease, that if one half followed the prescriptions of the doctors of the variety of those practicing at the present day, and that the other half took no medicine but relied on Nature’s instincts, I have no doubt as to which half would escape." 
Or Jan Baptist van Helmont, a doctor writing in the first half of the 1600s, proposed that the two "cures" of the day--bloodletting vs. induced vomiting/defecation--be tested in this way:
"Let us take out of the Hospitals, out of the Camps, or from elsewhere, 200 or 500 poor People, that have Fevers, Pleurisies, etc. Let us divide them in halfes, let us cast lots, that one half of them may fall to my share, and the other to yours; I will cure them without bloodletting… we shall see how many Funerals both of us shall have."
There are cases from the 1600s, 1700s, and and 1800s of randomization as a way of testing for the effectiveness of  treatments for scurvy, or smallpox, salt-based homeopathic treatments. Perhaps one of the best-known experiments was done by Louis Pasteur in 1881 to test his vaccine for sheep anthrax. Jamison writes:
"He was attempting to publicly prove that he had developed an animal anthrax vaccine (which may not have been his to begin with), so he asked for 60 sheep and split them into three groups: 10 would be left entirely alone; 25 would be given his vaccine and then exposed to a deadly strain of the disease; and 25 would be untreated but also exposed to the virus .... [A]ll of the exposed but untreated sheep died, while all of the vaccinated sheep survived healthily."
There are LOTS of examples. But in Jamison's telling, the earliest example of randomization in an experiment within a subject conventionally thought of as economics was actually done by two non-economists working on game theory, psychologist Richard Atkinson and polymath Patrick Suppes, in work published in 1957:
"Atkinson and Suppes (1957), also not economists by training, analyzed different learning models in two-person zero-sum games, and they explicitly “randomly assigned” pairs of subjects into one of three different treatment groups. This is the earliest instance of random assignment in experimental economics, for purposes of comparing treatments, that has been found to date."
As to broader social experiments about the effects of policy interventions, the first one goes back to the 1940s:
"The first clearly and individually randomized social experiment was the Cambridge-Somerville youth study. This was devised by Richard Clarke Cabot, a physician and pioneer in advancing the field of social work. Running from 1942-45, the study randomized approximately 500 young boys who were at risk for delinquency into either a control group or a treatment group, the latter receiving counseling, medical treatment, and tutoring. Results (Powers and Witmer 1951) were highly disappointing, with no differences reported." 
Back in high school, we had to design and carry out our own experiment in a psychology class. I wrote up the same message (a request for some saccharine and meaningless information) on two sets of postcards. One of the sets of postcards was typed; the other was handwritten. I chose the first 60 households in the local phone directory, and sent the postcards out at random. My working hypothesis was that the typewritten notes would get a higher response (perhaps because they would look more "professional," but actually the handwritten notes got a much higher response (probably because they reeked of high school student). Even at the time, it felt like a silly little experiment to me. But the result felt powerful, nonetheless.