Syndicate content

Do informed citizens hold governments accountable? It depends...

Stuti Khemani's picture

We are increasingly—and more openly than ever—grappling with what to do about the problems of politics and government accountability. Much emphasis and faith seem to be placed on the role of information and transparency. Using information interventions to enable civil society to hold their governments accountable seems so eminently sensible that it’s become an end in and of itself, an “already known” and ticked box. Is it?

Community radio in Africa, for example, has been supported as a particularly powerful vehicle to inform and enable the poorest citizens; yet surprisingly, we have no real evidence about its impact on government accountability for public services. That is, until now.

My colleague Phil Keefer and I find the impact of radio access does not operate through greater government accountability. To examine the issue, we gathered new and unique data from Benin, a country which serves as a “natural laboratory” in village access to community radio. It also has relatively greater political freedoms and competitive political processes than several others in the African region, and therefore a plausible setting in which to undertake such work. The data show that government inputs into public education, the only source of education services available to the majority of Benin’s citizens, are no different in villages with greater access to community radio. This is particularly surprising because we collected our data at a time following large increases in public education spending, so that if radio influenced people’s ability to demand better public services, there would be scope to capture its impact on availability of government-provided inputs to public schools. Instead, radio programming persuades households to invest more private resources into the education of their children, leading to higher literacy rates in villages with greater radio access.

This result from Benin is similar to what I learnt in working with Pratham, an influential civil society organization in India, and the ALJ Poverty Action Lab. We found that information and advocacy campaigns had no impact on community monitoring and the performance of village public schools; yet surprisingly, it had a large impact on community activity outside the public school system. Village youth volunteered to hold private remedial classes, parents enrolled their children to class capacity, and the children made dramatic improvements in learning.

Such findings of private citizen participation are, of course, inspiring, but they leave open a billion-dollar question: how do we make public resources less wasteful and more effective in delivering development?

Radio’s lack of impact on government accountability in Benin contrasts with demonstrated impact of mass media in the U.S. and in India. Phil and I argue that this is because the political and economic context within which media operates in Benin, unlike those in the U.S. and India, neither encourages media to provide "accountability" information nor facilitates citizen mobilization in response to that information. Investments in building capacity of media alone may not be sufficient to achieve greater accountability. Instead, to make public spending more effective, we may need to invest more in specifically-designed information that address the underlying political incentives for poor government performance.

The lack of impact of our grassroots mobilization interventions at the village-level in India can be contrasted with the results of other work with more success in changing the behavior of public agents. In ongoing work reviewing the varied evidence on information, transparency, and civil society action, Shanta Devarajan, Mike Walton and I are attempting to provide some arguments—hypotheses really—about what successful interventions depend on. One of our emerging conclusions is that accountability failures of lower-level bureaucrats and providers are more likely to be addressed when higher-level politicians or powerful state actors endorse the information, signaling to civil society that their action on its basis will be taken seriously.

But what information enables citizens to hold higher-level politicians accountable, for perhaps the larger resource allocation decisions taken at those levels? There are far fewer instances of such information interventions that have been credibly evaluated. We don’t really know, for example, whether institutions for budget transparency work in the political contexts of many of our client countries. We don’t know whether other types of information interventions suggested by theories of political accountability—such as those designed to spur yardstick competition across political jurisdictions on the basis of performance improvements—can overcome clientelist politics. Some civil society organizations are working with researchers to begin to fill this gap. Yet there is scope to do much more, especially in the context of institutions like ours that work in larger policy-making arenas.

Can we be at these front-lines to generate credible knowledge on what works, when, why, and how to scale it up? Can we be more experimental and learning-oriented in our operations and less sanguine that we already have the answers?


 

Comments

Submitted by Ron on
I think one question is in particular whether those with the potential to influence (spending) decision-making or those who are able to apply for funds (e.g. when tenders are made public) receive information in a digestible format at the right time. It may well be that villagers are informed about public financing initiatives, but if they can't translate this knowledge into concrete action that would influence the decision on how much money they (or their public services) receive. In other words: If the radio would inform local citizens that they may directly apply for specific funds and they wouldn't react to that this would be surprising. But the mere fact that the radio is available doesn't seem to be a sufficient condition for a change in behaviour.

Ron, Thank you for your comment. Schools in Benin do have institutions (at least de jure) to enable collective action--all schools in our reasonably large sample (of 210 schools) have a PTA, some of the roles of which would be consistent with voicing demands to government authorities. Yet, we find no evidence that greater access to a specific type of media, described as carrying public interest programming, encourages these existing institutions to take greater action. There is no difference in PTA activity in villages with access to fewer or more radio. That is, no change in public behavior, even as there is significant evidence of change in private behavior. And, as described in the blog, this difference in impact on public versus private behavior similarly emerged in other work with far more concerted grassroots mobilization efforts using information and advocacy. There seems to scope for a lot more evidence-based learning on how to design information interventions, based on a better understanding of constraints to public action for government accountability, so that these constraints are effectively addressed. Thanks again for engaging in this discussion.

Submitted by Janmejay on
Your findings support an insight that was also shared at a recent InterMedia seminar at the Bank, where they highlighted the role of information intermediaries or "info-mediaries". These can be anything from NGOs, village politicians, teachers, or your parent - but they are the key actors that translate available information into civic action and demands for better government accountability. What would be interesting to study as a follow-up to your findings, therefore, would be whether or not such info-mediaries exist in the Benin context and what kind of role they played. It is often these actors that convert passive information into "actionable" awareness.

Janmejay, Thank you for your comment. As indicated in my reply to the previous comment, such "info-mediaries" indeed exist in the Benin context--every school in our sample has a PTA, which can take specific actions to improve the effectiveness of existing resources in the school, and to voice demand for more resources. We gathered data on PTA activities, so we were able to examine whether these were different across villages with fewer or more radio access. We found no difference. In the other study I referenced in the blog--the work with Pratham and JPAL--the aim was to activiate and energize the India-specific "info-mediary"-the Village Education Committees. Once again, there was no impact on actions of the VECs, even as there was action by other volunteers in the village that bypassed the VEC and the public school system. Our result contrasts with other work by Priyanka Pandey et al which did find some impact on VEC activities. The burden of resolving why these results are aso different remains. What Shanta, Mike and I are exploring in our ongoing work is whether the extent to which the information campaign is seen as carrying the endorsement of higher-tier state authorities, and signals their expectations from lower-tier providers, is a factor crucial to succeess. Thanks again for engaging in this discussion.

Submitted by Bill Walker on
Stuti, I think your response to Janmejay (hi again Janmejay!) begs further questions, because naturaly the mere existence of PTAs does mean they are effective. Nor that they are necessarily and regularly going to be intermdiaries? Do members know their role is; are they motivated to express voice and take action, and if so, whose voice, and in whose interests is action being taken? (eg is it directed towards accountability, and if so whose, and which?) Also how well are PTAs equipped to take action? The same applies to VECs. How often a) do both (PTA's and VECs) have the knowledge they need and b) are they representative of the village, especially those most needing education and c) on what basis do they prioritise? While VEC's and/or PTAs may sometimes provide a way forward (and may need to be part of the answer in the longer term), they may not necessarily provide the best one, especially initially. It seems to me that examining the issue of 'power' in its various dimensions and manifestations, in each context, may be important to understand and explain what is happening. I look forward to more updates on this and other research which throws further light on the topic.

Dear Dr. Khemani, Thank for a very revealing post. I am so glad to learn that researchers such as yourselves are spending your time examining this very critical issue very carefully and building up the evidence base for what does not work; what works and the transmission path through which it does. I wanted to take this opportunity to share with you some of my perspectives on this issue as well and to solicit your reactions to them: 1. I wonder if the Central Limit Theorem does not apply to political processes so that what comes to fore are the central tendencies and not the tails. And perhaps the only way to counter this effect is to ensure that separate political processes are set up for each area of importance (such as Brazil's Health Assemblies and Councils) so that the averaging process now applies within a narrower domain. 2. As you point out in your Post, it is indeed possible for address failures at lower levels even within the current system. But given the complex ways in governments actually work in Federal structures like India there are many ways in which "accountability" finds expression. A recent report (www.bit.ly/TAGUP) takes the view (see Chapter 9) that transparency and crowd-sourced feedback has to become an integral part of government's own information systems and the default rather than the exception. I feel that these instruments will gradually reduce information-latency in the system and make it possible for resource allocation to happen in a more "accountable" fashion. For example this information could feed into the NDC and into Planning Commission led allocation processes. Eventually these channels could have great power as well even if they don't immediately impact electoral outcomes. Sincerely, Nachiket Mor

Dear Dr. Mor, Thank you for these very thoughtful remarks, and for engaging this discussion. If I might react first to your point 2): the availability of credible channels of information is, of course, a pre-condition for using these for accountability gains. Your efforts towards institutionalizing these channels within government in India are truly admirable. Thank you for sharing the recent report on this high profile initiative in India to harness ICT for more effective government functioning and greater accountability. I wonder whether this initiative might also champion the cause of filling the knowledge gaps we’re discussing here? Let me continue with education as an example: the key accountability failure in public education services that seems to have been identified for India is weak teacher incentives, reflected most visibly in high rates of teacher absenteeism. Community residents are likely to be the repositories of information about the presence or absence, or misbehavior of teachers in schools. If channels for crowd-sourced feedback of such information were created, would citizens respond in "sufficient" numbers? Would this enable state action to strengthen teacher incentives? Would this improve teacher performance and learning outcomes in public schools? My guess is the answer would be “it depends”—on the political economy context of different states, and perhaps even different districts within states; and on the design of accompanying interventions, perhaps tailored to different political economies, to encourage citizens to use such channels to exact accountability. Would there be scope within this initiative for a learning program on such questions, to evaluate actual impact on accountability? (I do have reactions on your point 1 as well, but I wonder if you would bear with me in taking this offline, so this very important discussion you have initiated on further institutionalizing and facilitating the “Right to Information” may remain in the limelight.)

Submitted by Joel Turkewitz on
Thanks for this very useful blog post. I am eager to see the work that you and your team are doing. If you have not already done so, I would suggest that you look at a great piece of work, Lily Tsai's Accountability without Democracy - an examination of the provision of public services at the local level in China. Professor Tsai reports on the extensive field work that she did in the early 2000s that indicates the limited impact democratic reforms have had on public service provision, and the very large impact of inclusive social groupings (temple societies, lineage groupings - which engage whole communities as well as government officials) have on the provision of services (as well as how non-inclusive groups - like churches - can generate increased private provision of public services). The work you describe is fundamental if we are going to think constructively about accountability.

Thanks very much for your comments and interest in this work.I am indeed aware of the research by Lily Tsai (I was also a discussant for her presentation in one of her visits to the Bank). It's always useful, however, to have another perspective on how to interpret it. Thanks! One of the big messages I took away from it was the great variation in public services across local jurisdictions even in a context of very centralized and tightly controlled institutions. Endogenous and historical variation across local jurisdictions in local institutions seem to account for a lot of disparity in experience with public services. In the accountability "interventions" of the Bank's governance work it would be useful to examine both what we can learn from such endogenous institutions, and how the local variation might be used to facilitate greater demonstration effects (spillovers, yardstick competition)of "well performing" areas on the "lagging" ones.

Marshall McLuhan suggested that radio is a hot media and so has different effects. He saw radio as a technology that best centralizes that enables totalitarianism. So, the notion of radio as a mechanism for citizens to hold government accountable is an interesting counterpoint. Perhaps the differences in effects among countries is reflected by the extent to which government has control over radio vs. community radio vs. strong interests like opposition party control over radio vs. the aggregate effect of all of the other media. The recent events in Tunisia and Egypt seems to show that social media is a more effective mechanism for citizen efficacy and influence than radio or television. For example, the regimes resorted to traditional media.

Submitted by Ritva Reinikka on
Thanks very much for sharing the findings of your interesting paper on Benin and those from your work in India. In the Africa region we are discussing demand side governance a lot these days. You are finding that information alone will not create incentives to perform better and improve service delivery. One would need enforcement or at least a threat of enforcement. You mention that, "one of our emerging conclusions is that accountability failures of lower-level bureaucrats and providers are more likely to be addressed when higher-level politicians or powerful state actors endorse the information, signaling to civil society that their action on its basis will be taken seriously." In my Uganda experience in the 1990s this threat was definitely there as finance and other central government officials took the findings of leakage from the first public expenditure tracking survey in education seriously and initially indicted numerous district officials (which made headline news). While I have no research results on this aspect specifically, it likely sent a strong signal that the higher levels cared about what happened on the ground. And having the evidence out in the open made sure that the officials knew that the citizens knew ... and that the citizens knew that the officials knew that the citizens knew....

Ritva, Many thanks.It's very useful to get your perspective on why the Uganda experience was such a success. You make the point of credible enforcement, and "common knowledge", but the other striking feature your comments allude to is the power of your large sample, quantitative evidence on leakage in persuading the government to take action for greater accountability. It seems there is enormous untapped potential of using such large sample statistical methodology, and impact evaluations, in the governance work in the Bank. Doing more such governance interventions IE work in the HD sector has the advantage of measuring impact on actual development results--improved health and education outcomes. Is such governance IE work happening in HD? Is there scope for such work to use political economy insights to inform intervention design to overcome accountability failures?

Submitted by Ritva Reinikka on
I fully agree with you about the “large sample” approach. After the initial success in Uganda in the late-1990s, Tanzania implemented a PETS, too. If I recall correctly, KPMG was hired to do it. But only in 3 districts. Subsequently, the Ministry dismissed the findings saying that all other districts were different! Even in Uganda the systematic (quantitative) nature of the evidence made a difference. You cannot sweep it under the carpet as you can do with an anecdote. There has been an increased emphasis on providing space for strengthening accountability channels (and through this improve governance) within standard education projects in Africa. The usual approach for this, so far, has been through school based management-type projects, some of which are being rigorously evaluated. A good example for this is the Ghana School Management Committee (SMC) intervention in which SMCs will receive information on school performance and their rights and responsibilities and training on how to act on that information. The project has an RCT design and the baseline survey is currently ongoing. In Tanzania, another SMC project is under design, the primary objective of which is to promote more effective use of school grants. The details of project and impact evaluation are still being finalized and there is scope to build in explicit political economy questions. An impact evaluation in Uganda is aiming to address slightly different governance questions. This is related to a public-private partnership program in the education sector under which the government will finance the education of eligible students in participating private schools. The impact evaluation aims to examine the impact of public funding on school level governance. Baseline data collection for this project is ongoing. It is worth noting that there interest in exploring issues around decentralization in education and health – how to make the newly decentralized systems work effectively. We look forward to expanding this agenda with you and your DECRG colleagues!

Submitted by Mwendalubi Maumbi on
Thank you for the blog. Very timely for what I am doing - strengthening the link between civil society organisations and community radio stations in Zambia. I have worked with various community radio stations in Zambia and I have observed that the problem is simple yet complicated. While many civil society organizations and even government departments recognize the impact community radio stations could have on their delivery, they do not understand the costs involved in production and as such they either have meager budgets or non at all for program production but have high expectations of information flow. I've experienced situations where organisations only seem eager to pay for programs at the end of their fiscal year to at least show to their donors that they had a media component in their activities and such programs are studio based without the participation of the community. On the other hand, media organizations harbor the myth that civil society organizations are all out to just make money and therefore seldom include them in their programming unless airtime it paid for because anyway, they couldn't produce the programs for free even if they wanted because of cost restraints. The truth is that information alone is inadequate to yield much results. Beyond information there ought to be structures where community members can access services or make follow ups and that is a great challenge as many areas are rural and lack structural support. Could you send me your email address? I've a report that I could share, thanks.

Submitted by Mazuba Mwiinga on
I am not such an expert in the social and economic paragymns at stake. But my area of interest in this discussion is on community radio's role in making the government accountable. From Zambia's context, though some of the setbacks highlighted above fit in even here; on the other hand, we have seen a greater change in the way these community radios here have been operating. The new radio trend of Phone-in programs have tramendously shifted community radios'concetration on just speaking to the community they serve and become a useful advocacy link between the government and the community in suh a way that the governement has become jittery because the community has been given a voice on thier problems, challenging government leaders on various issues that it has failed to honor. These radio programs on almost all the various community radios in the country, have taken the government to task. In this avenue the government have come to realise how powerful these community radios are, and have started trying to surpress them thorugh draconian laws and threats of closure. Much as they are a challenge, these programs have helped the community find their feet on evaluating government's performance, though they need alot of dedication, prowess and abit more of courage for the producers to host them successfully.

Submitted by carl on
Having many community radio stations does not equate to freedom of press and information. There can be a thousand community radio stations, but for as long as a country does not have freedom of the press and and information, there is very little that community radio producers can do to influence their leaders. In this case it is clear the community radio stations are not free to report on issues that demand for accountability from their leaders

Submitted by Mazuba Mwiinga on
Its true Carl that the mushrooming of so many community radio stations does not equate to a free press. But to a larger extent it has opened new avenues to a public that had no voice whatsoever on making governments accountable. Before, people relied on the national broadcaster to show and report on rural poverty levels, under development etc; which never happened because the national broadcaster was and still is under immense government control. Today, community radios are closer to the people. Even when there is no guarantee of a free media, people still speak out their mind on government failures. Radio reporters and producers become more and more commited to be accountable to the people and not to the powers that may be. These community radios have a young fearless generation that want to implement what they learnt in journalism schools; thats why the government even finds it so challenged that despite the threats it breathes out, information critical of government still find itself on air. So many governemnt scandals have been exposed which was almost a taboo before.

Submitted by Bonaventure Fandohan on
Accountability in the public service still has a long way to go. The findings about Benin’s community radio are interesting but not surprising. Community radio is relatively new concept in Benin and programs targeting accountability in public service delivery still lag behind. The notion of public good in Benin is not exactly the same as seen in many developed or emerging countries. Another aspect and not less important is the fact that Benin is well regarded in term of democracy in West Africa. However, this country has long to go when it comes to accountability and this is not exclusive to public education. Community radio is a powerful tool that can be used to make positive changes in developing countries. Better programs design, improved skills and consistent effort from NGO and multilateral agencies like World Bank will help along the way.

Submitted by MG Chandrakanth on
while delivering a lecture to our participants in WTO issues, I learnt from a participant that any new institution in general leads to increased corruption. Thus institutional innovations in India are adding to increased rent seeking and leakage of developmental programs and are not reaching the poor to the extent desired. For instance the Agricultural Produce Market Committees (APMCs) are to ensure fair marketing of farmers' produce through middlemen. When farmers bring their produce to the APMC, the wholesaler to whom the produce is brought for sale should not charge any commission from the farmer. But instead should collect the charge of 1.5 percent prescribed from the buyer. Instead the wholesaler collects the same from the farmer and also from the buyer and pockets the same right in the eyes of the Market officials. In addition, since the wholesaler has to pay the commission/tax to the APMC, he underreports the volume of sale. For instance if he handles 100 quintals of a produce, he will report 50 quintals or even less for which he will pay the APMC tax and will not report the remaining 50quintals, for which the APMC or the Government will not get the tax revenue. This is done in connivance with APMC officials who also get a share of this kitty. While these are not transparent, one can easily get this information through participant observation. So E governance is the only panacea. Every truck which enters the APMC yard should automatically be camera clicked by remote camera and weighed as the truck passes through electronic weigh bridge. This provision should be made in all APMCs. At least the total weight of each truck can be compared with the total volume of transactions of all wholesalers / commission agents at the end of the day or the week or the month, in order to find the degree of compliance.

Thank you very much for the very interesting post and paper. In many ways, the finding that information by itself does not lead to more government accountability is not very surprising. In our work at the International Budget Partnership, we have often found that governments publishing more information on public spending does not necessarily lead to governments becoming more accountable. The ‘missing link’ lies in organized citizen action to use available information in order to pressure governments to provide better services, or to better respond to citizens’ needs and demands. We have documented a number of successful cases of civil society organizations using budget information to affect government policies (you can find some of them here). Passive ‘info-mediaries’ such as community radio stations cannot substitute for more active ones where organized groups analyze information and utilize it to put pressure on governments. This, in turn, requires capacity within civil society to gather and analyze budget information, and the existence of participation channels through which these groups can make their voice heard and influence policy-making. The Uganda Debt Network, for example, used to disseminate the results of its budget monitoring activities in the education sector through community radio stations, raising debate about the quality of government spending, which in turn led to government revising its procurement guidelines for classroom construction (though this case might not fall within the ‘large sample’ research you talk about above, it illustrates the broader point I want to make). This relates to a question I have on your paper. Could it be that access to community radio led to more private rather than government spending on education because of the content of radio programs? If education programs talk about the need for families to invest in education that’s one thing, if they talk about what is happening with government policies and what can be done to improve them, it might spur a different kind of reaction.

Thanks for your comments. What we do know from our surveys is that community radio in Benin carry substantially more education and health programming than other radio. However, we don't know the detailed nature of the content of this programming. We hypothesize in the paper, based on our "reduced form" results, that either the detailed programming content indeed focuses on that which influences private behavior; or, at least, that's the programming that households pay attention to and act upon (even if other content exists). Yet, the question was a testable one to begin with-- the arguments of several proponents of community radio, and donor support to capacity building for community radio, are predicated on the assumption that community radio as an institution enables collective action and therefore would be expected to carry the programming that facilitates such action. To identify which types of programming content would lead to public action for public accountability is, of course, an important question (and one highlighted in the blog). I would argue that to answer it, credibly, and for replicability and scalability, we would need to use rigorous empirical methodology. Examples of such research from Brazil suggest that disclosure of audit information on radio airwaves around municipal elections improves mayoral accountability. Would such programming content have similar effects in the political economy context of Benin or other parts of Africa? What programming content could be designed to be tailored to the opportunities that exist in local political contexts, to "nudge" public action following the momentum of change that may exist in different contexts? I think there is scope for such a research program in collaboration with civil society organizations, and with donor support. It just needs open mindedness to learn through experimental "doing"...

Add new comment