Syndicate content

Can our parents collect reliable and timely price data?

Nada Hamadeh's picture
Also available in: 中文 | Español | Français | العربية

During the past few years, interest in high-frequency price data has grown steadily.  Recent major economic events - including the food crisis and the energy price surge – have increased the need for timely high-frequency data, openly available to all users.  Standard survey methods lag behind in meeting this demand, due to the high cost of collecting detailed sub-national data, the time delay usually associated with publishing the results, and the limitations to publishing detailed data. For example, although national consumer price indices (CPIs) are published on a monthly basis in most countries, national statistical offices do not release the underlying price data.

 
Crowd sourced price data
* This map was produced by Staff of the World Bank. The boundaries, colors, denominations and any other information shown on this map do not imply, on the part of The World Bank Group, any judgment on the legal status of any territory, or any endorsement or acceptance of such boundaries
 

This led us to search for alternative ways to collect and present price data. We realized one important fact: our parents, neighbors, friends and the rest of the crowd can collect price data! However, would this price data be reliable and timely? We set out to examine the feasibility of this approach.

Our innovative pilot study of crowd-sourced price data collection through mobile phones has combined the need for high-frequency data, recent developments in the ICT sector, and the power of the crowd. Unlike the CPIs, which employ professional price collectors to collect price data, our method involves employing “non-professional” price collectors using their mobile phones as a way to collect the prices of food items in eight pilot countries - Brazil, Bangladesh, India, Indonesia, Kenya, Nigeria, Pakistan and the Philippines. Prices were collected for 30 basic commodities - such as rice, meat, vegetables, and sugar - from around 2,500 supermarkets in 270 survey locations, by a crowd of 7,000 non-professionals.

A private company - JANA (www.jana.com) - helped conduct the pilot study, recruit non-professional price collectors, and collect the data. Data were verified and validated, and those providing price data were rewarded with phone airtime credits though the backend billing systems of mobile operator partners.

What did we learn? The results from the pilot tell us that, yes, the crowd can collect reliable and timely prices – but you need to provide incentives and implement good verification and validation processes. The resulting data are comparable over time and space, and timeliness is pretty good – the time lag is only about a month. And, importantly, the resulting data are open to all users.

We’ve been analyzing the data for a few weeks now, and put together an interactive dashboard to help you explore the dataset on your own. Or, if you prefer, download the entire dataset at data.worldbank.org/data-catalog/crowd-sourced-price-collection and use your own tools. Let us know what you find, and whether you would deem it useful if we went beyond the pilot.

Comments

Submitted by Thomas Eng on

This was a very interesting article. I just wonder how these price indices could be used in a global context. For instance I'm interesting to find a way to compare prices on similar projects country-wise by using an index. An index cannot tell all but without such an index we don't have any data information to explain why the same project has different level of cost in X than in Y country.

Hi Thomas, thank you for your comment, and sorry for the delay in getting back to you which was, unfortunately, caused by technical problems on the blog site. Your comment is very valid, and we have actually calculated a few price indices, or price level indices (PLIs), as we call them. Please have a look on our paper, published at the NTTS (New Techniques and Technologies for Statistics) conference, at http://www.cros-portal.eu/sites/default/files//NTTS2013fullPaper_82-v2.pdf.

Submitted by Vilas MAndlekar on

This is an excellent innovative process formulated by the World Bank using technology. Kudos to the team ! A great example of getting reasonably good results at low cost. A good solution need not be an enemy of the best !

We thought about the same problem (delays in data collection, etc.) and developed a rough prototype to collect data in a Hackathon organized by GEN (Global Editors Network). The result was a collection method using Google forms, though in the two days of the Hackathon we already got like 80+ entries. Our approach was a form (not fit for final data publication). In the event of strange numbers (e.g. mistakes or much too high or too low prices) we would either use an average, use a scatterplot and/or correlate the numbers collected versus more reliable averages.

We called this application "I pay this, you pay that" - the prototype is online here: http://playground.dw.de.s3.amazonaws.com/iptypt/index.html

We would love to hear from you, maybe there is a way to solve this issue.

Hi Mirko, thank you for your comment, and sorry for the delay in getting back to you which was, unfortunately, caused by technical problems on the blog site. Thank you for sharing your project, this sounds very interesting indeed.

Is is possible to have access to this "open data" without having to download and istall a 3rd party software ( Tableau) as the link you have provided at the bottom of the blog is not active. We would like to see how these prices compare with those we collect here at FAO.
Suffyan

Hi Suffyan, thank you for your comment, and sorry for the delay in getting back to you which was, unfortunately, caused by technical problems on the blog site. Yes, you can download the dataset from the World Bank data catalog at http://data.worldbank.org/data-catalog/crowd-sourced-price-collection. Please let us know if this does not work!

Submitted by Elton on

Hi, i'm 4th year student at the University of Nairobi. I like the work you've done to address this need. I also independently converged on crowd-sourcing as the solution to price-asymmetry problem in Kenya and already I find your work to quite insightful.

In contrast I plan to use intrinsic motivators as incentives unlike your overtly extrinsic approach as I believe this is more sustainable and thereafter expose this information through an ideally an open API.

If possible i'd like to discuss this further especially on the platform and technologies used to carry out your study.

Hi Elton, thank you for your comment, and sorry for the delay in getting back to you which was, unfortunately, caused by technical problems on the blog site. You can read a bit more about our approach on our paper, published at the NTTS (New Techniques and Technologies for Statistics) conference, at http://www.cros-portal.eu/sites/default/files//NTTS2013fullPaper_82-v2.pdf. We’re looking forward to your feedback!

Submitted by Samuel Chege on

I'm currently working on a school project seeking to utilize crowdsourced data to furnish consumers with pricing information for basic commodities at various retail outlets. What I'd like to know is, what are the quantities of the various items in your study? For example, for Nairobi, Kenya, the average price for Maize flour for the month of December 2012 is about USD 1.5. What is the quantity of the product?

Hi Samuel, thank you for your comment, and sorry for the delay in getting back to you which was, unfortunately, caused by technical problems on the blog site. Yes, the quantity information is available on the dashboard above. If you select a product, for example “Apple”, and mouse over the price curve, a pop-up window appears, giving you metadata, such as number of observations collected, standard deviation, and the quantity. Hope this helps!

Add new comment