Roughly a century ago, modern medicine got off to a roaring start with the steady introduction of population-wide solutions for illnesses. Louis Pasteur’s discovery in 1870 of the bacterial origin of many illnesses laid a foundation; handwashing cut deaths from operations and childbirth; antibiotics cured many infections; vaccines wiped out polio, smallpox and chicken pox. Starting in the 1920s, insulin significantly prolonged some lives.
Then the momentum towards medical omnipotence slowed. As researchers moved to harder and harder problems, the treatments they discovered became less effective, temporary or even risky. In retrospect, poisoning bacteria is relatively simple compared with the challenge of treating chronic conditions ranging from multiple sclerosis to Parkinson’s to migraines to psoriasis.
Humans are vastly complex and intertwined biological systems, and it turns out that for these conditions, individuals experience treatments differently. Researchers call this the “heterogeneous treatment effect,” or HTE.
Among researchers, this variety of response is common knowledge. “The development of medical interventions that work ubiquitously (or under most circumstances) for the majority of common chronic conditions is exceptionally difficult and all too often has proven fruitless.” Lillie 2011
But medicine’s century-old reputation for omnipotence lives on in the minds of patients and, to some degree, physicians. HTE gets little ink from journalists.
Parallel group random control trials are perceived to be the gold standard of drug research, yet statisticians argue that this research mode has a number of flaws that minimize or obscure HTE.
Researchers and physicians often erroneously generalize from the results, argued Richard L. Kravitz, NAihua Duan and Joel Braslow in a 2004 paper.
First, the “average treatment effects” have been the “primary focus on clinical studies in recent decades.” The “average” is what researchers look for, what the FDA approves, what drug companies sell, and what a doctor’s prescription is based on.
But averages are irrelevant for individuals. Buried within the headlines about drug trial success, “modest average effects may reflect a mixture of substantial benefits for some, little benefit for many, and harm for a few.”
The second problem with most drug trials is that they’re conducted on narrow cohorts. Heterogeneity “may be dramatically underestimated” because “by convenience, randomized control trial are characterized by narrow inclusion criteria and recruitment.” In fact, “nonrepresentativeness is probably the rule rather than the exception.”
Unfortunately, “the pharmaceutical industry currently has little direct incentive to collect data on risk, responsiveness, and vulnerability that would better inform individual treatment decisions.” (In fact, prescriptions based on “average” results create far larger markets.)
Finally, though the generalizability of large parallel group RTCs is relatively weak, their findings are often distilled into treatment guidelines, and these can too easily “creep” into being rigid practice standards.