Syndicate content


Pitfalls of Patient Satisfaction Surveys and How to Avoid Them

David Evans's picture

A child has a fever. Her father rushes to his community’s clinic, his daughter in his arms. He waits. A nurse asks him questions and examines his child. She gives him advice and perhaps a prescription to get filled at a pharmacy. He leaves.

How do we measure the quality of care that this father and his daughter received? There are many ingredients: Was the clinic open? Was a nurse present? Was the patient attended to swiftly? Did the nurse know what she was talking about? Did she have access to needed equipment and supplies?

Both health systems and researchers have made efforts to measure the quality of each of these ingredients, with a range of tools. Interviewers pose hypothetical situations to doctors and nurses to test their knowledge. Inspectors examine the cleanliness and organization of the facility, or they make surprise visits to measure health worker attendance. Actors posing as patients test both the knowledge and the effort of health workers.

But – you might say – that all seems quite costly (it is) and complicated (it is). Why not just ask the patients about their experience? Enter the “patient satisfaction survey,” which goes back at least to the 1980s in a clearly recognizable form. (I’m sure someone has been asking about patient satisfaction in some form for as long as there have been medical providers.) Patient satisfaction surveys have pros and cons. On the pro side, health care is a service, and a better delivered service should result in higher patient satisfaction. If this is true, then patient satisfaction could be a useful summary measure, capturing an array of elements of the service – were you treated with respect? did you have to wait too long? On the con side, patients may not be able to gauge key elements of the service (is the health professional giving good advice?), or they may value services that are not medically recommended (just give me a shot, nurse!).

Two recently published studies in Nigeria provide evidence that both gives pause to our use of patient satisfaction surveys and points to better ways forward. Here is what we’ve learned:

How to attract and motivate passionate public service providers

David Evans's picture

In Gaile Parkin's novel Baking Cakes in Kigali, two women living in Kigali, Rwanda – Angel and Sophie – argue over the salary paid to a development worker: "Perhaps these big organisations needed to pay big salaries if they wanted to attract the right kind of people; but Sophie had said that they were the wrong kind of people if they would not do the work for less. Ultimately they had concluded that the desire to make the world a better place was not something that belonged in a person's pocket. No, it belonged in a person's heart."
It's not a leap to believe – like Angel and Sophie – that teachers should want to help students learn, health workers who want help people heal, and other workers in service delivery should want to deliver that service. But how do you attract and motivate those passionate public servants? Here is some recent research that sheds light on the topic.

What’s the latest in development economics research? A round-up of 140+ papers from NEUDC 2017

David Evans's picture

Did you miss this year’s Northeast Universities Development Consortium conference, or NEUDC? I did, unfortunately!

NEUDC is a large development economics conference, with more than 160 papers on the program, so it’s a nice way to get a sense of new research in the field.
Thankfully, since NEUDC posts submitted papers, I was able to mostly catch up. I went through 147 of the papers and summarized them below, by topic. If a paper you loved or presented isn’t in the rundown, feel free to add a brief summary in the comments. (Why 147 instead of 160? I skipped a few macro papers and the papers that weren’t posted.)

These links should take you to your topic of interest: Agriculture, cash transfers and asset transfers, credit and insurance, crime, conflict, violence, and war, culture, norms, and corruption, education, elections and political economy, firms, governance, bureaucracy, and social capital, health (including WASH), jobs (including public works), marriage, methodology, migration, mobile phones and mobile money, poverty, inequality, and shocks, psychology, taxes, and traffic.

How hard are they working?

Markus Goldstein's picture
I was at a conference a couple of years ago and a senior colleague, one who I deeply respect, summarized the conversation as: “our labor data are crap.”   I think he meant that we have a general problem when looking at labor productivity (for agriculture in this case) both in terms of the heroic recall of days and tasks we are asking survey respondents for, but also we aren’t doing a good job of measuring effort. 

Weekly links February 17: Don’t give up on your research ideas but do give up on unwarranted policy recommendations

David Evans's picture
  • Chris Blattman provides an incentive to delay giving up on that great research idea you’ve been peddling for years in this story from the EconTalk podcast: For years, he pitched random African factory owners the idea of an RCT of factory employment. “They’d usually look at me kind of funny. They wouldn’t leap at the possibility. I was just this person they met on a plane.” One day it worked, and six weeks later he was randomizing applicants.

Why Do Harmful Norms Persist? Female Genital Cutting in Burkina Faso: Guest post by Lindsey Novak

This is the fifteenth in our series of job market posts this year. 
For better or for worse, social norms have profound influence on many of the decisions we make—from political to personal. These norms can be particularly influential when it comes to making decisions surrounding child rearing, including the decision parents make to participate in the practice of female genital cutting (FGC). Parents living in communities that practice FGC—located primarily in parts of Africa, the Middle East, and Asia—decide whether or not their daughter will undergo FGC based on social pressure and the perceived costs and benefits of adhering to or deviating from the social norm.
The practice has no known medical benefits, and it is associated with a wide range of health complications, both physical and psychological. Women who undergo FGC are more than twice as likely to experience birthing complications (Jones et al., 1999), and are 25 percent more likely to contract sexually transmitted diseases (Wagner, 2014). In addition, women who have undergone FGC are more likely to experience depression, anxiety, and post-traumatic stress disorder (Dorkenoo, 1999;  Behrendt & Moritz, 2005). These health complications make working in and outside of the household more difficult.

In Sub-Saharan African, as malaria rolls back, human capital moves forward: Guest post by Maria Kuecken

This is the eleventh in our job market paper series this year. 

Malaria is preventable and treatable – but it is still deadly. In 2015, there were 214 million cases of malaria and an estimated 438,000 deaths. Nearly nine in ten cases occur in Sub-Saharan African, and the direct and indirect costs of this burden are high.

Traveling with ease, carrying disease? Using mobile phone data to reduce malaria: Guest post by Sveta Milusheva

This is the eighth in our series of job market posts this year
The Global Fund has disbursed nearly $28.4 billion in the last decade to reduce the disease burden from malaria, TB and HIV (Global Fund 2016). However, travelers can reverse the progress from campaigns that have decreased infectious disease prevalence (Cohen 2012 et al, Lu et al 2014), or can rapidly spread emerging diseases such as Ebola and Zika (Tam et al 2016, Bogoch et al 2016). While policymakers have largely targeted environmental drivers of malaria, this research provides evidence that human movement can play an important role in spreading disease in areas where incidence has been reduced.  Given that migration has numerous economic and social benefits, policymakers face important trade-offs in designing policies to reduce travel-linked malaria cases.  This paper provides a useful framework for identifying high-risk populations in order to reduce malaria incidence with minimal interference to movement patterns.

Cash transfers and health: It matters when you measure, and it matters how many health care workers are around to provide services

David Evans's picture

A whirlwind, surely incomplete tour of cash transfer impacts on health
Your run-of-the-mill conditional cash transfer (CCT) program has significant impacts on health-seeking behavior. Specifically, there are conditions (or co-responsibilities, if you prefer) that children get to school and/or that they get vaccinated or have some wellness visits. While the school enrollment effects are well established, the effects on both health seeking behavior and on health outcomes have been much more mixed. CCTs have led to better child nutritional status and improved child cognitive development in Nicaragua, better nutritional outcomes for a subset of children in Colombia, and had no impacts for child health in studies on Brazil and Honduras. CCTs conditioned only on school enrollment did not lower HIV infections among adolescent girls in South Africa; and in Indonesia CCTs increased health visits but did not translate into measurably improved health. Unconditional cash transfer programs have also had mixed results on health, with better mental health and food consumption in Kenya, better anthropometric outcomes for girls (not boys) in South Africa, no average impacts (although some for the poorest quarter) on child outcomes in Ecuador, and no average impacts on maternal health care utilization in Zambia (albeit yes effects for women with better access to such services).

Is giving birth at home a luxury good?

Berk Ozler's picture

The December 31, 2015 issue of the New England Journal of Medicine published an article by Snowden et al. that compared outcomes for births planned at a hospital vs. at home or at a freestanding birth center. I’ll discuss the findings and identification in a little bit (you can see the NYT article by Pam Belluck here). But, I actually want to discuss the characteristics of women who plan their births at a hospital vs. elsewhere.