Call for Feedback: How-To Note on a Framework for Evaluating the Impact of ICT Programs


This page in:

The review process for this How-to Note has ended. The paper has been downloaded 46 times and we received 2 comments.

We are grateful to the many reviewers for their valuable comments. The author will carefully review and consider all comments when finalizing the note. The final version of the How-To Note will be published on the Open Development Technology Alliance website and announced in the World Bank blog forum


The Open Development Technology Alliance (ODTA), in collaboration with the World Bank's OPCS' Governance and Anticorruption Team (GAC) and the Social Development Network (SDV), is holding a consultation period to invite feedback on four short How-To reports. These draft papers explore the role information and communication technologies (ICTs) can play to enhance governance, strengthen social accountability mechanism and ultimately improve development outcomes.

You are invited to download and review the how-to note, "Valuing Information: A Framework for Evaluating the Impact of ICT Programs," and submit your feedback in the comments section below.

Valuing Information: A Framework for Evaluating the Impact of ICT Programs

Download the report »

About the note: This paper aims to unpack the link between ICT and development and provide operational teams with a framework on how to evaluate the impact of ICT programs. It applies Amartya Sen’s “capability approach” to the study of ICTs and thus places people’s well-being, rather than technology itself at the center of the analysis. Investment in ICT hardware, software and training is of little value unless individuals can make meaningful use of ICTs, resulting in sustainable improvements in people’s human well-being


To learn more about this How-To series, click here.


Join the Conversation

Julian Bass
June 28, 2012

Many thanks for your how-to notes on evaluating the impact of ICT programmes. This addresses an important and interesting area.
The use of the capability approach in evaluating ICT4D interventions has been explored in (Zheng, 2009). While the Capability Approach is helpful in understanding the conversion factors between commodities and functionings, it says relatively little about the social context. Our work has used Institutional Theory to explore regulative, normative and socio-cultural aspects of ICT4D projects (Bass, Nicholson and Subrahmanian, to appear) which is a development of an earlier paper (Bass, Nicholson and Subrahmanian, 2011)
However, an import earlier source in this area is (Heeks, 2002) who proposes a design-reality gap approach for ICT4D interventions. This approach identified several dimensions for exploring technical as well as other success factors. This approach has been applied more recently in (Bass and Heeks, 2011).

Julian Bass, & Richard Heeks. (2011). Changing Computing Curricula in African Universities: Evaluating Progress and Challenges via Design-Reality Gap Analysis. Electronic Journal of Information Systems in Developing Countries, 48(5), 1–39.
J. M. Bass, B. Nicholson and E. Subrahmanian, “A Framework using Institutional Analysis and the Capability Approach in ICT4D,” Information Technologies and International Development, to appear.
J. M. Bass, B. Nicholson and E. Subramanian, “Institutional Analysis, the Capability Approach and ICT4D” IFIP 11th Int. Conf. on the Social Implications of Computers in Developing Countries, Kathmandu, May 2011, pp. 209-224.
Richard Heeks. (2002). Information Systems and Developing Countries: Failure, Success and Local Improvisations. The Information Society, 18, 101–112.
Zheng, Y. (2009) Different Spaces for e-Development: What Can We Learn from the Capabilities Approach. Information Technology for Development, 15 (2), 66-82.

Bincy Ninan-Moses
July 12, 2012

The Open Technology Institute (OTI) finds that Gigler’s application of the capabilities framework is an effective and meaningful way to evaluate ICT projects beyond technical and large-scale economic development. However, the paper does not describe the way a program could use the framework to plan an evaluation in concrete terms; the case studies provided only show general applicability. Specific examples of how an agency could tailor the framework for evaluating a given program along with sample indicators or indices would be helpful, since the ideas presented in the paper are mostly conceptual. For example, how does one measure the construct of “ICT Capacity Building” in practical terms? Do programs need to include all of the constructs presented in the framework or can they select a subset of ideas? Practical guidance for how to use the framework is an important next step to enable programs to start using the framework on the ground.

As the external evaluator and documentor for Broadband Technology Opportunities Program (BTOP) projects in Philadelphia and Detroit, Open Technology Institute (OTI) has developed an approach similar to Sen’s “capabilities framework” in designing our program evaluations.

Please read our blog post written in response to this paper for more comments and recommendations:…

Open Technology Institute
New America Foundation
Washington D.C.