Using mobile phones in data collection: Some questions to consider

Recent posts on the EduTech blog have explored some of the general opportunities, issues and challenges that are common to many efforts to use mobile phones as part of data collection efforts and have identified some of the key lessons as a result of projects which have used mobile phones to collect data in the education sector in Uganda.

Even where there is common agreement on the potential utility of deploying mobile phones as part of a particular data collection effort, as well as a consensus understanding about relevant challenges that may complicate such an effort, decision makers may still be unsure about how to start their related planning efforts – or how best to change course once such efforts are underway.

In many instances, an intriguing proposal by a vendor of a particular product or service may help instigate initial considerations to use mobile phones as part of data collection efforts; news reports and information sharing between key practitioner groups may as well. Whatever catalyzes consideration of the use of mobile phones as aids in data collection efforts – in some cases it may simply be a general dissatisfaction with the status quo – here are some general questions that may be worth asking:

What are the high level goals of the data collection effort being considered?
If a sponsoring group does not have a clear idea of the goals and objectives of a particular data collection effort, it won’t matter what technology is chosen to help make it happen. If you are pointed at the wrong target, introducing a new technology can help get you there faster. While the opportunities to collect data in new ways using mobile devices – more quickly, more cheaply, through processes that are ‘innovative’ – can be very enticing, it is important to remember that whatever technologies are chosen are just means to an end, not an end in themselves. Before deciding on a particular technology ‘solution’, it is best first to have a clear understanding of the ‘problem’.

To what extent can mobile data collection efforts be handled ‘in-house’ and to what extent will they need to be outsourced to others?
Initial mobile data collection efforts may often be planned and implemented to a greater or lesser extent by groups external to existing related data collection processes. The reasons for this can be quite understandable: The technical knowledge and competence in the use of new mobile technologies for such purposes may be considered (rightly or wrongly) outside the competence of many ‘traditional actors’. This is especially the case when such efforts are in their early or pilot stages. As such efforts become more widespread, costly and strategic, however, care should be taken to ensure that, at a minimum, sufficient competency exists within the sponsoring organization so that staff can plan, direct and evaluate the efficacy of such efforts, even if mobile data collection efforts themselves are largely implemented by third parties.

Is there sufficient local capacity to plan, implement and sustain mobile data collection efforts?
Whoever is responsible for the planning and implementation of mobile data collection efforts in their initial stages and iterations, over time it may be important that such actions are increasingly led and implemented by local groups, whether such groups come from the public or private sectors, from civil society or academia. Indeed, the sustainability of such efforts over time may well depend on the development and existence of supporting ecosystems of local actors and expertise.

How will new systems to collect data using mobile devices integrate with existing legacy information systems and processes?
Care should be taken to ensure that the results of mobile data collection efforts can be absorbed into existing information management systems. Where such efforts and systems are incompatible with each other, the operation of essentially parallel systems and processes may be expedient in the short run, but costly and inefficient over the longer term. The introduction of mobile data collection efforts may contribute to exposing deficiencies in existing information systems and act as a catalyst for the upgrading of legacy systems. Efforts to better integrate the tools and processes that characterize and enable mobile data collection activities, as well as the data that are generated as a result of such activities, are often best considered as part of larger, more holistic planning processes related to the collection, sharing, analysis, and storage of related data more broadly. 

Who are the key stakeholders and partners who will need to be engaged during the course of this mobile data collection effort – and what are the key components of this engagement?
Efforts to collect data through the use of mobile phones may require that new partnerships with other groups – some of which may be ‘non-traditional partners’ – be established and strengthened. At the same time, the nature of partnerships and interactions with existing stakeholder groups may change as well. Sponsoring groups would do well to map out the universe of key stakeholder groups, attempt to analyze and predict the potential impact of mobile data collection efforts on such groups, and plan accordingly.

How can data security be assured and data privacy be protected when utilizing mobile devices?
Collecting, sharing, and storing data in digital formats brings with it a whole set of new challenges and opportunities that are in many ways far beyond those which characterize paper-based survey efforts. Data security issues may well be more acute, and the potential consequences of inattention more immediate (and potentially profound). Where data reside on connected devices, such data may be insecure in ways that (e.g.) boxes of completed paper questionnaires are not. This is true whether or not devices such as mobile phones are used as tools in this process, but some aspects of the nature and characteristics of the use of mobile phones for such purposes are worth specific consideration. Because of the potential to link individual data points with both geographic location (as a result of GPS) and individual people (given that data may, for example, be attributed to specific phones at specific times of day), the potential implications related to privacy may well warrant special attention. In addition, the sponsoring organization would do well to ensure that it retains (for example) usage rights (if not full ownership) of the data collected and to consider, at each stage of the data collection and sharing process: Who has the rights the data collected, and what might they do (and not do) with them?

There are certainly many more questions that can, and should, be asked. But this short list might provide a good place to start.

The answers may change over time and according to circumstance. Indeed, it is expected that the answers to these sorts of questions will often change, given the variety of policy and research objectives that mobile data collection may assist, local contexts and constraints, and the speed at which the underlying technologies, end user devices, related governing legal frameworks and social norms may evolve in the coming years.

Four related observations

Together with this initial set of questions, groups contemplating utilizing mobile phones as part of their data collection efforts may do well to consider the following short additional observations:

Piloting and iteration can be critical components for success
A ‘big bang’ approach, where mobile devices are deployed quickly at scale as critical tools in large scale data collection efforts, may be bold – but foolhardy. The German military strategist Helmuth von Moltke is meant to have observed that “no battle plans survives contact with the enemy”. Veterans of early large-scale deployments of mobile devices as part of data collection efforts often say something similar. Piloting and testing various technology tools in local environments can be invaluable in deciding which course to take – as well as which corrections to make to initial plans. They can also be critical components in user-centered design processes which may help better target data collection efforts to the needs and usage contexts of various important stakeholder groups critical to mobile data collection efforts, from those surveyed to the enumerators to various support personnel to the eventual analyzers and consumers of the data collected.

Mobile phones can not only be useful in the collection of data – but can aid in their broad dissemination as well
While much of the focus on the potential use of mobile phones in large scale survey efforts relates to their utility in data collection, such devices can also enable the wider and quicker dissemination of the results of such data collection efforts – as well as the data themselves. The broad availability of mobile phones across societies can help to amplify messages from some groups whose collective voices may not have been widely heard in the past. It is important to note that mobile phones can not only be useful tools to collect data more quickly, but to share them more quickly and widely as well. Indeed, mobile phones are often considered key tools as part of citizen engagement and transparency efforts in a number of places around the world. Many data collection efforts suffer as a result of an inherent misalignment of incentives between those who supply data and those who benefit from the collection and use of data. Where mobile phones are used, the tools of data collection are at the same time potential vectors for the dissemination of such data (in individual and aggregate form), the analyses of such data and the decisions made as a result of such analyses. This means that data can not only flow more easily ‘up’ into the system (as is typically the case in large scale data collection efforts), but back down and across the system as well.

Costs associated with mobile data collection efforts may change over time – in both directions
Budgeting for the costs of mobile data collection efforts can be quite challenging where such efforts are new to a particular context or environment. As more experience is gained, such costing exercises may become easier and more predictable in some instances. At the same time, however, changes in technologies, and the related changes in the business models that both enable and are enabled by such technologies, can result in certain individual cost components varying widely over time. It is a consensus opinion that most technology and device costs drop over time. However, the costs of complying with related guidelines and regulations may move in the opposite direction as technologies become more settled and as lawmakers and regulators ‘catch up’. Where data collection efforts are locked into specific vendors, the potential for prices to rise can be considerable. One example is the potential impact of reliance on bulk SMS rates for the transmission of data by one mobile operator. Where such rates rise – as has been in the case in some countries where anti-spam measures have led to precipitous increases in the wholesale SMS rates -- the business model of mobile data collection efforts may need to be quickly and seriously reconsidered.

Certain groups may be threatened by mobile data collection efforts
One challenge to many groups, especially government agencies, which sponsor and implement large scale data collection efforts is that they may have developed significant internal competencies and expertise in the implementation of survey efforts the ‘old fashioned way’. Indeed, institutional structures and bureaucratic processes may have developed which, while (perhaps) useful to the way things were done in the past, may be impediments to doing something different in the future. While it may be useful to the system and process as a whole that, for example, fewer people need to be involved in transporting and transcribing paper-based questionnaires, as well as in processing related paperwork, the livelihoods of individual people and viability of existing institutional structures may be called into question as a result of some of the increased efficiencies that can potentially be realized as a result of mobile data collection efforts. Where the potential for disruption to existing processes, implementing organizations and stakeholder groups is high, care should be taken to anticipate and mitigate potential opposition. In addition, where the dissemination of data and related information and analyses as a result of the use of mobile phones results in greater transparency around a particular topic or subject, certain groups may organize themselves to impede or inhibit related activities. Such things are not unique to the use of mobile phones in data collection efforts, of course, nor to the introduction of new technology-enabled processes more broadly – but they can profoundly impact the potential success of such efforts.

 

This is the third of three related blog posts looking at the use of mobile phones in data collection efforts:

• Using mobile phones in data collection: Opportunities, issues and challenges
• Using mobile phones to collect data in the education sector in Uganda
• Using mobile phones in data collection: Some questions to consider

 
Note: The image used at the top of this blog post of the science library of Upper Lusatia in Görlitz, Germany ("as you move through the data, new questions might present themselves") is © Ralf Roletschek / roletschek.de. It comes via Wikimedia Commons and is used according to the terms of Creative Commons Attribution 3.0 Unported license. (It is also made available for re-use under the copyleft  Free Art License).