Where Are We Driving this Truck?

The Center for International Media Assistance (CIMA) is one of our valued partners in the work on communication for governance and accountability. Very relevant to our own work on media development, CIMA just published a report on "Monitoring and Evaluation of Media Assistance Projects." Author Andy Mosher, formerly of the Washington Post, interviewed Monitoring and Evaluation (M&E) practioners in major US donor and implementation agencies to find out what is being done - and what is being done successfully - to assess the impact of media development projects. Representative of his question is a quote from one of his interviewees: "Where are we driving this truck?" According to what I read in the report and what I heard at its launch this week in Washington, I'm not sure we even know how to start the truck.

Mosher summarizes his findings by listing the tools that are necessary to evaluate media assistance in a meaningful manner: baseline data, logframes, tracking activities and participants, focus groups, content analysis, sharing data with clients, balancing quantitative and qualitative data, keeping things measurable, and hiring outside evaluators. His recommendations include: more "aggressive" funding, a shared but adaptable approach to M&E methods, sharing non-competitive information (so that the next organization in the field won't make the same mistakes you made), and bookkeeping of the costs of M&E.

The respondents to this report mainly pointed out this: 1) M&E is important. 2) M&E needs more funding. 3) Approaches to M&E need to be flexible and adaptable. To my ears this sounded a little like: 1) Donors want to see that their money is doing something. 2) We have no money to do M&E. 3) And we don't know how to systematically do M&E anyway.

The latter, of course, is my own interpretation of the sense of generality that I found in the report as well as in its presentation. We now know - as we did before - that it's important to monitor the outcomes of media assistance and that this costs money. What we don't know is how to measure change.

Now, we have never claimed that it's easy to measure the impact of media development. Looking at aggregate measures of, for instance, press freedom or democracy will not necessarily help because it's almost impossible to establish a clear causal relationship between those scores and media assistance. But if not indicators, what then? Surveys about attitudes and knowledge might help, provided there is baseline data to compare with. Logframes are certainly very helpful for figuring out what an outcome actually is supposed to be and how it might be measured. Anecdotes are always fun to tell to the like-minded who know how important media development is (and whom you always meet at launches such as the one of this report), but donors might find them hard to swallow.

Guess what: I have no solution either. But discussions such as this confirm my belief that donors need to accept the relevance of a free and independent media as a fundamental part of democracy and need to accept that media often has indirect effects that just may be impossible to measure reliably. Faith can move mountains - lets see if it can steer this truck.

Photo credit: Flickr user kgbbristol