Shocking facts about primary health care in India, and their implications

|

This page in:

There’s nothing quite like a cold shower of shocking statistics to get you thinking. A paper that came out in Health Affairs today, written by my colleague Jishnu Das and his collaborators, is just such a cold shower.

Fake patients
Das and his colleagues spent 150 hours training each of 22 Indians to be credible fake patients. These actors were then sent into the consulting rooms of 305 medical providers – some in rural Madhya Pradesh (MP), others in urban Delhi – to allow the study team to assess the quality of care that the providers were delivering.

A lot of thought went into just what conditions the fake patients should pretend to have. The team wanted the conditions to be common, and to be ones that had established medical protocols with government-provided treatment checklists. The fake patients shouldn’t be subjected to invasive exams, and they needed to be able to be able to credibly describe invisible symptoms.

The conditions the team chose were unstable angina, asthma, and dysentery in a fictitious child that had been left at home. The fake patients were trained to consistently and credibly portray the physical, emotional, and psychosocial aspects of the condition, and were told what answers to provide to questions the provider might plausibly ask. They were coached in how to avoid an invasive exam, and in what to remember from the encounter. Fake patients retained any medicines from the consultation, and were debriefed within two hours of the encounter.

In case you’re a little skeptical (I must admit I was early on in this research), consider these two facts. First, in follow-up visits, no provider in MP voiced any suspicion about fake patients; in Delhi, private providers did spot some fake patients, but spotted less than 1% of them. Second, providers who stuck closest to the checklist were more likely to get the “correct” diagnosis; had the fake patients been unconvincing, the study team would have found the opposite.

As a way of getting at quality of care, fake patients have advantages over other methods, such as observations of provider-patient interactions, exit interviews, etc. There’s no observation bias with fake patients. Fake patients also allow for standardization of casetype, severity, and non-health characteristics: this allows case detection rates to be estimated, and allows for valid quality comparisons across providers because patients don’t choose providers on the basis of their symptoms and severity.

Das and his colleagues had to think, of course, about where to send their fake patients. In MP, they gathered data on providers and the population’s use of them, and came up with a sample of 226 primary care providers that was representative of the primary facilities used by rural households in the state. That meant a sample that included mostly private providers – with and without formal training – and some public clinics. In Delhi, the sample of 64 providers was simply a convenience sample, not necessarily representative of the facilities used by the Delhi population.

What the fake patients encountered
In just one third of fake patient interactions in both MP and Delhi did the provider ask all the essential questions and do all the essential exams. This didn’t vary much across the three conditions. And in only one third of cases in MP did the provider give a diagnosis. Shockingly, only 12% of the diagnoses the MP providers offered were completely correct; another 41% got a partially correct diagnosis. Providers in Delhi did better, but managed only a 22% fully-correct diagnosis rate. Unsurprisingly, the rate at which providers prescribed the right treatment was highly unspectacular: 30% in MP and 46% in Delhi.

What the study team uncovered next was even more shocking. While unqualified providers in both MP and Delhi asked fewer of the recommended questions and did fewer of the recommended exams, they were no less likely to prescribe the correct treatment. Moreover, while providers in better-equipped facilities in MP asked rather more questions and did rather more tests, they were also no more likely to prescribe the right treatment. Interestingly, private providers were significantly more likely to ask the right questions and do the right exams. However, they were not more likely to prescribe the right treatment; in Delhi, in fact, they were significantly less likely to do so.

Implications
It’s pretty staggering that – at least for these conditions – the rural residents of MP and Delhi face 70% and 55% chances respectively of being prescribed the wrong treatment.

It’s also pretty staggering that hiring qualified staff doesn’t appear to increase this probability. Das and colleagues suggest that part of the issue might be the variation in the quality of instruction in Indian medical training institutions. So there may be some institutions from which a qualification does make a difference. But given the paper’s results, the effect of such institutions must be rather small. The fact that providers working in better equipped facilities don’t have a higher probability of prescribing the right treatment is also alarming.

The results on the public-private differences are pretty interesting. In rural MH both sectors do equally – and very – badly in terms of ensuring the patient gets the right treatment. In Delhi by contrast going to the private sector halves the odds of getting the correct treatment, even though it raises the number of recommended questions the provider asks. It is the latter quality indicator that Das et al are presumably referring to in their conclusion when they say “we observed better quality care in the private sector”. That’s a bit misleading – it’s surely the correctness of the treatment prescribed that matters at the end of the day, not the number of questions asked. This work isn’t exactly a great advertisement for India’s private sector, or for the view that financial incentives will improve quality. But it’s not exactly a great advertisement for the public sector either. It also begs the question: Why does the private sector in Delhi do worse on the correctness-of-treatment indicator while the private and public sectors in rural MH do just as badly as one another?

A cold shower that invigorates but befuddles 
It would be unwise to generalize too much from this one study, but at the same time it would be unwise to assume that these results apply to just unstable angina, asthma, and childhood dysentery, and to just Madhya Pradesh and Delhi. It seems more likely that these results will be replicable elsewhere in India and using other conditions. It’s also likely the results aren’t India-specific.

If so, the paper suggests that the developing world may well face a huge challenge in terms of the quality of care at primary level – much bigger than we probably thought. And while this paper isn’t an impact evaluation of any program or policy, the results don’t exactly inspire confidence in the usual policy knobs we reach for when thinking about improving quality. It doesn’t look like more training and better equipment will solve the problem. Nor does it look like the quality deficit will be reduced simply by building up one side of the public-private divide and scaling back the other. The paper also makes it clear that simply giving everyone free access to health providers (à la Universal Health Coverage) isn’t necessarily going to do much to improve population health.

Some “cold showers” leave you invigorated and with a clear sense of where to head next. This one’s a bit different – awake yes, but a clear sense of where to go next, no.
 

Topics
Countries

Authors

Adam Wagstaff

Research Manager, Development Research Group, World Bank

Prashanth
December 07, 2012

Just went through the paper's abstract. The quality of care provided in most PHCs in India is certainly of questionable quality. Indeed, the average time per consult at OPs I have visited is often around a minute or less. That and the resulting poor quality of care is nothing new (cf. recent studies using clinical vignettes by Rao et. al. "Which doctor for rural India?"). In fact, if one sees the kind of records maintained (or not), the diagnoses are often - "headache", "breathing problem" and such. A complex interplay among various factors including the poor HR practices in terms of financial and non-f incentives, working atmosphere, near-nil private sector regulation and many such factors have a role. While I appreciate the approach towards the research and the rigor, the difficulty is in now asking how this can be addressed. Especially considering that the most important (though unfortunately long-term) solution lies in medical education reform and private sector regulation.

Muraleedharan
December 07, 2012

This is in response to the observations made by Adam Wagstaff on the quality of care in primary health care in India. In my view, this is a mis-directed research and total waste of money and energy. You really dont need any research on this; no body disputes this issue and it is an acknowledged issue Besides, such studies with "fake patients" also poses some ethical concerns, which the researchers are not sensitive enough. As revealed by their own survey, patients get just about a minute or a little more than a minute from physicians for diagnosis, and these fake patients take a away a portion of even this less than a minute consultation time available for genuine patients. undoubtedly this is an unethical research methodology and therefore unscientific. I wish to know Wagstaff's response to this issue. I have read this paper.

thanks

Muraleedharan

Adam Wagstaff
December 07, 2012

Thanks, Prashanth. I agree with you that some of these things, such as the short consultations, we knew about already. But this study breaks new ground in at least two areas, and I’d urge you to read more than the abstract of the paper! First, as Das has discussed elsewhere in his work, vignettes capture what a provider knows, while what’s he and his coauthors are trying to capture here is what a provider does. By sitting in on consultations, they already found there’s a big gap between the two. But sitting in on consultations may produce misleading data because the provider knows they’re being watched. One area Das et al break new ground is moving to standardized patients which are considered a better way of getting at what providers actually do. Second, Das et al. show the correlates of quality. They find that quality isn’t higher among qualified providers, nor is it higher among public providers in MH. In fact, in both MH and Delhi it’s the private providers who ask more of the right questions and do more of the right tests. That’s why I say that this study is a bit of a cold shower: it’s hard to argue based on this study that medical education and private sector regulation are the right answers.

Adam Wagstaff
December 07, 2012

Thanks, Muraleedharan. You’re right, of course, that studies like this cost money, and it’s good to ask what we learn from the money that’s been spent. As I argued in my reply to Prashanth, this study really did generate important new knowledge. Was it worth the cost? I guess that depends in part on what impact it is, and on that we’ll have to be patient. On the ethics issue, I think I’d be very concerned if providers were operating at full capacity and taking a long time over a consultation. In that case, fake patients would actually depriving a real patient of care, or making them wait a long time for it. But it looks like these providers aren’t seeing that many patients a day, and any extra wait for the real patient is just the length of the fake patient’s consultation, which as you say is just a minute or two.

Kabir Sheikh
December 11, 2012

The paper has its uses in enlightening a minority of global actors about something that many of India's people, policymakers and researchers know and experience intimately. It typifies the chasm between formal academic research and common knowledge. Bridging that is necessary, so the paper is useful. But the more important conversations are the ones that have moved beyond acknowledging poor quality of practices, to exploring how to holistically strengthen the governance of health care practice. The authors' statement of policy recommendations - including more research, better education, performance incentives - is non-specific, innocent of the intricate political economies of health research, medical education and performance incentives in India, and hence unfortunately of limited real-world utility. The ethics of the method are worth scrutinising. 'Ethics' is mentioned once in the paper - as a limitation.

Muraleedharan
December 11, 2012

The primary aim of the study was to prove that the overall quality of care is low. Does this require the methodology adopted by this paper? How about having a qualitative interviews with the providers in these settings and analysing their view points, before attempting to assess the views and experience of the patients in these settings? I am sure the physicians in PHCs in India (once you have made sure that you are not a journalist waiting to expose them in the public media) will be open enough to share their views and assessment of what they do and quality of care they provide and how to improve the quality of care they provide.
I continue to feel that the overall ethical basis of this study is weak and "objectionable"...
muraleedharan

Dinesh
January 02, 2013

take home messages from this study has to be clearly articulated. To me it seems quality of medical education is a major concern and with mushrooming growth of private medical colleges with hardly any clinical expousre, this is going to worsen.Second important concern is ( non) governance as doctors dont show up at the facilities and the paramedics and other run the show. Lack of any opportunities in continuing medical education worth the name at any level is also a serious challenge.

As far as methodlogy is concerned "mystry clients"have been used in the past in QOC studies especially for RH programmes in many countries and I dont see any problem.

R Manohar
February 28, 2013

This is a very interesting study. Whether the methodology is ethically perfect or not, the aim is clear but the conclusions are slightly skewed. As Adam has highlighted, this doesn't cover any one sector in glory.
Rather, it exposes lack of credibility across the board nationally and regionally. Unless physicians themselves are willing to be accountable and harbingers of better medical care supported by honest and brave leadership from the medical council, the trends are unfortunately not going to change. One has to give credit to millions of providers who do provide excellent and affordable care across the nation in spite of the odds, who were not captured in this study. Thank you for highlighting this and the opportunity for a cold shower Adam!

urgent care near me
April 23, 2013

It is very clear that healthcare systems are found under a worst condition now days; as a result people are not able to get better and sufficient health care facilities from various health care organization. Especially in countries like India and in other Asian countries we have encountered with a worst primary health care system. So in order to repair our primary health care system we should avoid the above circumstances.