Making indicators from diverse sources comparable

Individuals studying development are often interested in using data to make comparisons at various levels. By using data to make these comparisons, researchers can develop valuable insights that inform policymaking aimed at improving development outcomes at all levels.

Researchers and analysts may want to make comparisons at:

• A global level: At the global level, researchers and analysts often compare indicator values across different countries to understand how a particular development dimension varies worldwide. This can be done over time to see trends and changes or at a specific point in time to get a snapshot of the current state. For example, one might compare the literacy rates of Country X with other countries in the world or with countries in a specific region to find patterns, disparities, and areas needing improvement.
• A country level: Within a country, comparisons can be made over time to track progress in various development indicators. For instance, comparing healthcare access in Country X from ten years ago to the present can reveal improvements or highlight ongoing challenges.
• A subnational level: At a more granular level, comparisons can be made between different regions within a country. This is crucial for identifying regional disparities and ensuring equitable development. For example, comparing the economic development of Region A with Region B within Country X can help pinpoint areas that require more attention and resources.
   

Not all data are comparable

Not all data are comparable, nor can they be compared at every level. However, to derive valid statistical inferences and provide appropriate policy recommendations, it is crucial to ensure that the data being compared are measured consistently across different contexts and over time, i.e., you are comparing apples with apples and oranges with oranges. This does not imply that methods should remain unchanged; rather, there is often a trade-off between comparability and quality. Methods evolve continually, and adhering to outdated methodologies solely for consistency may lead to deteriorating estimates over time. Therefore, adjustments to methods might be necessary for improved measurement, but it is important to recognize that such changes can result in reduced comparability with past estimates.
 

Examples of different levels of comparability:

• National Poverty Numbers: Poverty numbers vary in country X with country Y, due to different measurement methods. However, within a country, consistent use of the same poverty line and collection methods allows data comparability over time.
• Gross National Income (GNI): Estimates of a country's GNI produced under the same System of National Accounts Framework (SNA 1994, 2008, 2025) should be comparable over time for both intra-country and inter-country comparisons. However, GNI estimates produced under different versions of the SNA may not be directly comparable. Even when countries use the same framework, varying update and revision cycles by national statistical agencies can affect comparability.
• Labor Force Data: Labor force data collected in one country may not be directly comparable to another due to differences in methods and definitions. However, the ILO harmonizes or models this data to produce estimates that allow for country comparisons over time.

National statistical offices producing local estimates for comparing progress over time, need to ensure that methodology and measurement is consistent over time. But what about producers of global databases, who don’t have control over the data collection process within countries?

Global databases of indicators, such as the World Development Indicators (WDI), play a crucial role in providing data that is comparable. The WDI database provides standardized data, compiled using set criteria, that can be used for valid comparisons across countries and over time. Users of these databases care about being able to compare estimates of indicators such as national accounts, poverty rates, and labor market indicators both within a country over time and with other countries.
 

What does it take to produce a globally comparable database?

Consistent data collection is essential for making valid and meaningful comparisons. It requires collaboration to produce comparable data. Databases like the WDI rely on international organizations, governments, and other stakeholders to standardize data collection and harmonize data. Consistent measurement methods and standardized definitions are crucial for ensuring reliable data at global, national, or regional levels.

Considering the various methods countries may use to measure the same thing, along with differing data collection quality, how can we develop a globally comparable database for accurate global comparisons?

To ensure comparability of data across countries, strategies include incorporating only data that adheres to international collection standards, that aligns with global frameworks, or that has been harmonized to be comparable. 

While these strategies cannot fully resolve issues or limitations related to data collection, such as the insufficient measurement of income in certain household surveys in some countries or the lack of participation by all countries in international programs to produce the PPPs, global databases like the World Development Indicators (WDI) are important. They provide standardized data in an accessible format, facilitating comparisons across countries and over time. By adhering to international standards, aligning with global frameworks, and harmonizing national data, these databases make it possible for researchers and policymakers to make informed decisions based on reliable and consistent data. This helps in identifying trends, assessing the effectiveness of policies, and distributing resources where they are needed most, ultimately contributing to improved development outcomes at all levels.
 

This blog benefited from discussions with, Daniel Mahler and Christoph Lakner. Visualizations were created by Daniel Boller.