There’s been a lot dialogue and debate in regards to the cuts to Medicaid eligibility that Congress simply handed and, specifically, what they might imply for present Medicaid recipients. A key piece of proof on this debate has been outcomes from the Oregon Well being Insurance coverage Experiment (OHIE), a randomized trial, which I helped lead, analyzing the impression of protecting low-income uninsured adults with Medicaid for one to 2 years. Whereas it’s at all times gratifying to see one’s work utilized in coverage deliberations, it’s irritating when the outcomes are misinterpreted.
An necessary sticking level is the interpretation of so-called “null outcomes” — estimates of Medicaid’s impression that we can’t statistically distinguish from no impact. Within the case of the OHIE, we discovered no proof of statistically important impacts of Medicaid protection on mortality, or on a number of measures of bodily well being, comparable to hypertension, excessive ldl cholesterol, or diabetes.
Sadly, individuals are making a standard mistake: They’re misinterpreting the shortage of proof of impacts as proof of no impression.
For instance, two economists not too long ago wrote a letter to the Wall Road Journal noting that:
“The perfect proof on the well being results of the Medicaid enlargement comes from the Oregon Well being Insurance coverage Experiment. The OHIE is a randomized managed trial, or RCT — the gold customary for such analysis. … The OHIE discovered no enhancements in mortality or another bodily well being final result from increasing Medicaid.”
Null outcomes will be extraordinarily invaluable. They’ll make us query what we predict we all know and spur innovation. When making evidence-informed choices, understanding what doesn’t work is simply as crucial as understanding what works.
Nevertheless, deciphering null outcomes appropriately is crucial. The outcomes from the OHIE indicated no statistically important impression of Medicaid on a number of bodily well being measures or on mortality.
However we can’t say we discovered proof that Medicaid has no impact on these outcomes. The distinction between no proof of impression and proof of no impression might seem to be wordplay, however when virtually 12 million individuals are susceptible to shedding medical health insurance, understanding this distinction is essential.
We have to look past a simplistic abstract of whether or not or not there may be proof of a statistically important impact of Medicaid to think about the magnitude of the estimated impact and the quantity of uncertainty round it. Each analysis end result comes with a vary of believable values round it (a confidence interval) that represents statistical uncertainty in regards to the true impact. If this vary contains zero, we are able to’t rule out no impact. However we can also’t rule out any of the opposite values inside the believable vary.
Take into account a number of the well being outcomes within the OHIE for which there was no proof of a statistically important impression of Medicaid. A few of these “null outcomes” had been sufficiently statistically exact to be informative.
One instance of an informative null end result was the research’s findings for hypertension. My co-authors and I discovered no impression of Medicaid protection in decreasing hypertension, and the outcomes had been sufficiently exact to rule out a lot bigger estimates of Medicaid’s skill to cut back hypertension that had been present in earlier, quasi-experimental research.
In different phrases, even the utmost doable profit in our vary of believable values was smaller than what earlier analysis had discovered. So, from this “null end result” on hypertension, we realized that the impact of Medicaid on decreasing hypertension could also be smaller than what was beforehand thought. (Once more, although, it’s not proof that Medicaid has no impact on hypertension.) That’s a helpful addition to the dialogue.
Nevertheless, the null outcomes of the impression of Medicaid on charges of uncontrolled diabetes (i.e., excessive charges of glycated hemoglobin) and on mortality weren’t informative. This stems from a mix of the comparatively small pattern dimension of the Oregon experiment (solely about 10,000 people gained Medicaid protection) and the (luckily) low charges of diabetes (about 5%) and mortality (lower than 1%) within the research inhabitants. The end result was a excessive diploma of uncertainty.
For diabetes, the vary of believable impacts for Medicaid included zero, but additionally included the enhancements one would possibly anticipate given the estimates from the quantity Medicaid elevated use of diabetes medicine and the estimates from the scientific literature on what such a rise in medicine would predict for enhancements in glycated hemoglobin ranges.
So we couldn’t rule out both no impact on diabetes or the chance that Medicaid had the very impact we’d have anticipated primarily based on its impression on diabetes medicine. I’d name any such null end result uninformative.
The research’s mortality outcomes had been likewise uninformative. They had been unable to rule out the chance that Medicaid decreased or elevated mortality by a considerable quantity. A subsequent, a lot bigger, randomized managed trial by which virtually 4 million individuals had been inspired to enroll in medical health insurance discovered that medical health insurance has a statistically important impression on decreasing mortality amongst 45- to 64-year-olds. The authors of that research explicitly famous the outcomes had been completely in keeping with the findings from the OHIE, as a result of wide selection of believable mortality results we had estimated.
Randomized evaluations can present a number of the most compelling proof on program impacts, because the authors of the Wall Road Journal letter identified. However the applicable use of that proof to tell coverage debates requires understanding that having a null end result doesn’t essentially imply a program has no impression. Researchers and policymakers alike have an obligation to signify and use proof, together with null outcomes, responsibly.
Amy Finkelstein is a professor of economics at MIT and the co-Scientific Director of J-PAL North America.
