However, the threat posed by the virus has made randomized trials extremely difficult to conduct. While researchers randomly assigned macaque monkeys to receive either a vaccine or a placebo, and then exposed them to the virus (those with vaccine-produced antibodies were less likely to be infected), it would be unethical, given the fact that doing this experiment on people at risk of serious illness and death. Instead, thousands of people participate in vaccination trials, randomly giving them a vaccine or placebo, and waiting months for a small percentage of them to become infected in their normal life course. At this point they will find out which group they were in. This process is much less efficient and provides less detailed information about the circumstances and biology of any infection than an experiment in which participants are closely monitored in a controlled environment.
This makes cases like those of the Dynasty, where a high percentage of people were infected with the virus under fairly uniform conditions while others did not, potentially very useful. Tests during an outbreak that infected around 700 people on the Diamond Princess cruise ship in February showed that many infections never produced any discernible symptoms. "We're trying to learn with limited evidence," says Emily Oster, an economist at Brown University. What are the specific details of an incident that we can learn from? I think there is a lot of value there. "
Such digging can help generate hypotheses. The classic example is the study by John Snow, an English doctor, of a cholera outbreak in London in 1854. Mapping out cases of illness and interviewing residents – "It's like the original contact tracing," Oster says – began Snow I suspect that a pump that gave many sick people their water was the culprit. However, Snow had also tested his water-based cholera theory by obtaining maps of households (randomly) supplied with different water sources by one of two competing water companies and identifying which of them had cholera deaths . When it was found that one company's death rate was much higher than that of the other, it was clear that water was the cause. (The pollution of the sewage was to blame.)
Such “natural experiments”, in which an event or factor randomized participants into test groups and control groups, were particularly difficult to find during the pandemic. The urgent need to stop the virus from spreading has led policy makers to change many variables at once – school and business closings and reopenings, mask regulations. This makes it difficult to separate their effects. For example, to find out if school closings reduced infection rates in the community at the start of the outbreak, you can look at demographically similar areas where schools closed in either mid-March or early April and compare their infection rates in early May. "But the hardest hit places might be the ones that pulled the trigger earlier," says Joseph Doyle, an economist at the M.I.T. Sloan School of Management – which may suggest that school closings lead to high infection rates when in fact an expected surge in infections did lead to schools closes. Randomization would mean finding those who were closed for reasons "unrelated to community health," says Doyle. For example, several schools in Tennessee were hit by tornadoes in March and closed early while neighboring schools stayed open. A comparison of infection rates in the community weeks later could approach a randomized study – if the storms did not significantly affect other local interactions.