For a quarter of a century, US health policy has been influenced by the findings of the largest social science experiment since Noah’s ark – the RAND Health Insurance Experiment. The experiment randomly assigned thousands of families to insurance plans with different levels of cost sharing, and then followed them for up to five years. The results proved for the first time that there is a demand curve for health services – that higher cost sharing leads individuals to use fewer health services, and further that (except for some lower income patients) the lower use of services had no negative health impacts. The results live with us today – thinking about all kinds of cost sharing: deductibles, copayments, coinsurance, health savings accounts, finds essential support in the RAND findings.
Locally, MIT Prof. and Connector Board member Jonathan Gruber is a big RAND fan. Last year, he wrote a report for Kaiser (click here) calling RAND “the gold standard” in evaluating the extent to which higher coinsurance reduces use of medical services and results in patient harm:
“Perhaps the most striking conclusion from the HIE is that while higher coinsurance rates lead to lower levels of both effective and ineffective medical utilization, they do not have an adverse impact on health outcomes for the average person.”
But, what if RAND is WRONG?
That’s the disturbing conclusion from a new paper in the October 2007 issue of the Journal of Health Politics, Policy and Law, but Dr. John Nyman of the University of Minnesota (click here). Nyman went to original sources, looked under the hood, and found a few surprises:
“Of the various responses to cost sharing that were observed in the participants of the RAND HIE, by far the strongest and most dramatic was in the relative number of RAND participants who voluntarily dropped out of the study over the course of the experiment. Of the 1,294 adult participants who were randomly assigned to the free plan, 5 participants (0.4 percent) left the experiment voluntarily during the observation period, while of the 2,664 who were assigned to any of the cost-sharing plans, 179 participants (6.7 percent) voluntarily left the experiment. This represented a greater than sixteenfold increase in the percentage of dropouts, a difference that was highly significant and a magnitude of response that was nowhere else duplicated in the experiment.
“What explains this? The explanation that makes the most sense is that the dropouts were participants who had just been diagnosed with an illness that would require a costly hospital procedure. … If they dropped out, their coverage would automatically revert to their original insurance policies, which were likely to cover major medical expenses (such as hospitalizations) with no copayments … As a result of dropping out, these participants’ inpatient stays (and associated health care spending) did not register in the experiment, and it appeared as if participants in the cost-sharing group had a lower rate of inpatient use. … the cost-sharing participants who remained exhibited a lower rate of inpatient use than free FFS participants, not because they were responding to the higher coinsurance rate by forgoing frivolous hospital care but instead because they did not need as much hospital care, since many of those who became ill and needed hospital care had already dropped out of the experiment before their hospitalization occurred. …
“To my knowledge, RAND researchers have yet to present a benign explanation for this large and statistically significant voluntary attrition rate differential.”
Sorry – the Journal is not available for free – but send me an email (mcdonough@hcfama.org) and I’ll send you a copy of this article. I have no idea if Nyman is correct, but his findings deserve a significant airing. Too many folks are getting crammed with cost sharing – and if the RAND findings are suspect, we all have a right to know.
John McDonough
I have always had a question about the RAND study. I believe the study set limits on the amount for which any of the participants could be liable — the total anyone could be liable for was $1,000 out-of–pocket expenses, or $4,000 in 2005 dollars. While this is not insignificant, it pales before the costs that some people could be faced with in today’s market with today’s prices and today’s deductibles. If there was a maximum out-of-pocket liablity, could this not also have affected the results?
it pales before the costs that some people could be faced with in today’s market with today’s prices and today’s deductibles
I completely agree.
I have a different take on this. Most people with insurance plans in Massachusetts do NOT have high deductible plans or plans requiring high levels of cost sharing for visits to the doctor. Although in RAND they were capped for out-of-pocket expenses the cost sharing day-to- day was significantly more than what many people experience today. The “average” co-payment for a doctor’s visit is $15 and that pales in comparison to a 25% or 50% cost sharing arrangment. In addition, people typically had insurance plans that only covered hospitalizations and the like….certainly not doctor’s visits and pharmaceuticals.
Pingback: Healthcare Economist · Health Wonk Review
Pingback: Health Affairs Blog
Health care has substantially changed snce the time of the RAND study.
(1) Catastrophic case management, disease management, physician practice profiling, 24/7 nurse telephone triage, and other elements of managed care are included not just in HMO plans but also in most PPO plans and in some of the high-deductible consumer-directed health plans.
(2) Most HMO plans now have patient copayments – with varying copayments for specialists versus primary care physicians, prescription pharmacy benefit, and so forth.
(3) I recall that a further analysis of the RAND study that appeared in Medical Care did demonstrate that low-income and chronically ill patents did have less utilization of appropriate care under high cost sharing.
The consumer directed health plan-HSA advocates need current data to bolster their claims.
I have not seen a single comparison of a high deductible health plan with an HSA to an HMO (as would happen in a large employer providing their employees a choice among those two options). All of the comparisons that have been shown to me are high deductible health plans with HSAs compared to PPOs. My employer has over 85% enrollment in HMO plans (the rest are in PPO plans). A high deductible health plan with a Medical Savings Account was offered for three years several years ago and less than 1% of employees opted for that choice.
Until I see current data, I am skeptical.
Pingback: A Healthy Blog » “Is RAND WRONG” Gets Noticed…
Nyman raises the question of whether the HIE could be wrong in it’s conclusion that health is not affected by the addition of co-insurance payment requirements to reduce moral hazard in insured individuals consumption of healthcare. His assertion is that since 179 individuals dropped out of the 3+ year study from the group with co-insurance and only 5 dropped out of the free-care group; a sixteen-fold difference, that the members of the study that dropped out did so because they were in poor health and needed to reestablish their previous insurance coverage that would not of had co-insurance in that timeframe 30 years ago.
This suggestion of a cause-effect relationship created from a statistical phenomenon, while factually based, is problematic at best. There are several other potential causes that may have created the effect of dropping out of the study. It is hard to imagine all of the determinants in a study some 3 decades ago in quite a different healthcare environment, but at least two seem plausible. First, the nature of the cognitive decision process that lies at the heart of the cost-sharing incentive. When the individual is at home or work and begins to think about seeing a healthcare provider for a symptomatic or asymptomatic perceived problem, they are encouraged by the incentive to balance the cost with the need. An issue with the method used in co-insurance incentives is the uncertainty of the amount of the payment required as it is a percentage of the cost of treatment. This uncertainty creates anxiety in the individual. Some individuals tolerate anxiety better than others. It is possible that some portion of the 179 individuals did not wish to continue with this anxiety even though the real money involved was negligible as the subjects in the study were virtually made whole by the payments made to them for being included in the study. Second, the state of the reimbursement process was complicated by the addition of a co-insurance calculation and payment process not just for the subject, but for the provider; in fact, both were burdened additionally by the process that was unfamiliar to them and must have seemed cumbersome at the time. It would be easy to extrapolate that some part of the 179 subjects that dropped out did not appreciate the extra bureaucracy created by the paperwork and did not see that it was worth being in the study since they would not benefit enough to put up with it.
Would Occam find that the subjects were all ill as the most likely cause of dropping out? Perhaps the psychological distress theory is more likely. Was there an adverse selection in reverse that was caused by illness in the 179 subjects? Nyman proposes a logical argument based on a statistical fact, but is that enough to discount the Health Insurance Experiment?
David -
What was really being studied, albeit unintentionally, was health care outcomes in a free plan vs. health care outcomes in a high-cost plan which you could leave to go back to your original insurance if it got too expensive in the high-cost plan. There is missing data on the health care outcomes for those who left the high-cost plan. There is an excellent reason to suspect the missing data doesn’t have the same probability distribution as the data that is present for the high-cost plan people, namely, why would it get too expensive to remain in the plan if there wasn’t some adverse health outcome occurring? And why wouldn’t it often get too expensive to remain in the plan if some really serious adverse health outcome was occurring?
This is a classic missing data problem in statistics, but unfortunately most of the foundational work in missing data hadn’t been done at the time of the RAND study, so they may well have not realized their results were going to be biased by not following up on all the dropouts. We wouldn’t do it that way today.
It seems unlikely that even in the primitive days of the seventies, researchers at RAND would simply ignore the fact that a disproportionately large percentage of on of their test groups dropped out of the study. Has there been no questioning of this over the last three decades? If this study really is the only basis for the popular assertion that high co-pays do not affect health outcomes, why has it escaped critical scrutiny until now?
Pingback: A Healthy Blog » The RAND Chronicles: Newhouse Responds
Pingback: Healthcare Economist · The truth about the RAND HIE