Yeah, you know me. Mercifully leaving that antiquated music reference in the dust, we turn to an interesting article about the problems of randomized control trials to assess if masks prevent the spread of COVID (boldface mine):

One way to avoid these confounds could be a randomized controlled trial (RCT) that divides people into otherwise equivalent groups that either do or do not receive an intended treatment. Even more robust evidence comes from meta-analyses, which are statistical summaries that combine the results from all available RCTs. Meta-analyses give greater weight to the better studies, such as ones with consistently administered protocols and larger samples. But even these sometimes come with major limitations.

Cochrane has published two major meta-analyses of RCTs evaluating field trials of face masks involving the general public. Both meta-analyses have been widely misinterpreted as showing that face masks don’t work. What they really show is that the RCTs asked questions that they could not answer. Cochrane’s leadership recognized these limits in an editorial accompanying the first meta-analysis, recommending that policymakers rely on other sources of evidence.

…What if it is so difficult to conduct scientifically sound randomized trials of mask wearing that even the best studies reveal little? Such studies can confuse people who want to know how effective face masks are, while emboldening people who are already completely convinced that face masks are ineffective — and are looking for grounds to sow doubt about them.

…The designs of most clinical trials are too weak to answer the question that they pose — namely, whether an intervention succeeded. The Gates Foundation article notes that only an estimated 5% of RCTs for Covid-19 drugs were designed to yield statistically meaningful results. Such ”uninformative research” wastes precious time and money, while incurring the incalculable opportunity costs of diverting resources from research that might advance public health and increasing the risk of chance “positive” results that lead down dead ends.

RCTs have value only when researchers can be sure that the treatment is administered as intended. With an RCT for a drug, that means knowing, for example, whether providers’ biases affected who got the drug, whether patients’ habits affected how they took it, and whether control group participants somehow got it on their own. Without that knowledge, an RCT produces noise, and meta-analyses produce piles of noise.

With behavioral interventions like wearing masks, it may be impossible to produce anything but noise without vastly more ambitious studies than have been conducted to date.

Those last two paragraphs are critical. RCTs are essentially the application of traditional molecular biology to medicine–and this is not intended to denigrate molecular biologists. In molecular biology, people attempt to use very simple systems to control for all potential confounding variables and then the one condition that is varied can account for the effect. Often these studies don’t even occur in actual organisms (e.g., enzyme kinetics), and the effect being measured is quite obvious (something is produced versus nothing changes). This has been highly successful for understanding the basic mechanisms of biology, and will continue to be important. But adapting this approach to the real world, which is messy and variable, can be difficult, if not impossible.

As the authors note, even with something as straightforward* as taking a medication, there are a whole bunch of confounding effects that can lead you astray. With a behavioral intervention that is the equivalent of continuously dosing yourself–you don’t just put on a mask for five minutes, then take it off, and call it a day–in a very non-standard environment (not only will people’s adherence differ, but their exposures will too), the confounding variables mean the comparison is virtually impossible. That, however, doesn’t mean we should abandon all hope:

Instead of fatally flawed RCTs, we may need to rely on two sources identified in the November 2020 Cochrane editorial. One source is field trials in which behavior can be observed and controlled. Such trials have found that requiring high-quality, well-fitted masks in hospitals reduces disease transmission. That evidence gives reason to hope that face masks will benefit ordinary people wearing imperfect, imperfectly fitted masks, under everyday circumstances.

The second alternative source is studies of factors that might affect mask efficacy. For example, how well do various kinds of mask block virus-sized particles in laboratory tests that simulate inhaling and exhaling? How well can people put on various masks, with various kinds of instruction? When do people wear masks in various real-life settings (stores, restaurants, buses, planes, airport terminals)? Are people wearing them to protect themselves or others? How is mask wearing affected by what other people are saying, and doing?

Lots of scientific disciplines have to cope with the inability to conduct RCTs or use the molecular biology approach, including multiple disciplines within biology. The Discourse™, rather than parroting one-liners about RCTs, must realize the limits of various kinds of experimental approaches.

*People who conduct clinical drug trials are probably laughing at the word straightforward…