Illuminating insights: harnessing NASA's Black Marble with R and Python packages

|

This page in:


Satellite imagery of nighttime lights has become a widely used data source across various sectors. Nighttime lights function as a proxy for local economic activity and facilitate monitoring urbanization trends, energy consumption patterns, power outages, impacts of disasters and subsequent recovery efforts, conflict events such as the ongoing conflict on Gaza, among various other applications. The use of nighttime lights typically involves tracking changes in artificial lights generated by human activities—which requires removing extraneous lights. For instance, moonlight can cause light reflectance from snow, smoke, and dust; atmospheric conditions can alter observed lights; and clouds can hide lights altogether.
 

Figure 1. Nighttime lights before and after the start of conflict in Gaza. Source: NASA Black Marble VNP46A4.


Noteworthy among the sources of nighttime lights data is NASA's Black Marble, which employs sophisticated algorithms to correct for extraneous lights. (For a comprehensive comparison between NASA Black Marble and alternative sources of nighttime lights, we suggest this article.)
 

Figure 2. Global nighttime lights. Source: NASA Earth Observatory using Suomi NPP VIIRS data from Miguel Román et al., NASA GSFC.
Figure 2. Global nighttime lights. Source: NASA Earth Observatory using Suomi NPP VIIRS data from Miguel Román et al., NASA GSFC.


Black Marble data can be obtained from NASA's LAADS DAAC, allowing users to download daily, monthly, and annual nighttime lights data spanning from 2012 to the present. However, converting raw Black Marble data into an analysis-ready format involves several steps, including downloading and collating individual tiles, converting them into a georeferenced file format (GeoTIFF), and removing low-quality pixels flagged by Black Marble, such as those affected by cloud cover. Many workflows then require aggregating nighttime lights data into larger spatial units—such as tracking trends in average nighttime lights within cities or districts over time.

Google Earth Engine (GEE) presents an alternative and widely used approach for accessing publicly available satellite data. GEE stores petabytes of satellite imagery and facilitates analysis of satellite imagery on Google Cloud. As of this writing, the daily Black Marble nighttime lights data is available on GEE, but monthly or annual aggregates are not available.

To streamline the process of accessing Black Marble data, the World Bank’s Development Impact Department (DIME) and Data Lab have developed R and Python packages: blackmarbler for R and blackmarblepy for Python. These packages offer functions that retrieve data directly from NASA LAADS DAAC, taking care of processing steps to convert the data into an analysis-ready format. Designed for workflows where GEE may not be suitable, these packages are particularly useful for querying monthly or annual nighttime light aggregates or accessing data directly from the NASA Black Marble archive.

Both packages offer two primary functions: bm_raster, which supports the downloading of nighttime light rasters tailored for a specified region of interest, and bm_extract which facilitates the generation of composites (e.g., sum, mean) of nighttime lights for designated regions of interest and within specified date ranges.

The following code excerpts exemplify the application of the packages, providing a demonstration of downloading  annual nighttime lights rasters for Lebanon and constructing a dataset of annual nighttime lights for each of Lebanon's primary administrative divisions. The code snippets are presented in both R and Python.

R:


Python:


The outcomes derived from the functions (i.e., ntl_r and ntl_df) are readily applicable for creating maps and plotting trends of nighttime lights.  The figures below show the distribution of nighttime lights throughout Lebanon, revealing a pronounced decrease in nighttime luminosity observed since the onset of Lebanon's economic crisis in 2019.
 

istribution of nighttime lights throughout Lebanon, revealing a pronounced decrease in nighttime luminosity observed since the onset of Lebanon's economic crisis in 2019.
Figure 3. Changes in Nighttime Lights Radiance Across Lebanon. Source: NASA Black Marble VNP46A4.



Nighttime lights data has become a valuable source for tracking economic trends and development indicators. By harnessing the capabilities of the new Black Marble R and Python packages along with specialized analysis, the World Bank Data Lab continues to assist operations in Gaza, Syria, Myanmar, and Lebanon to leverage nighttime lights data. In practice, the specific packages have been employed in supplement to analysis and conventional methods, contributing to the World Bank's efforts in understanding economic trends, assessing development impact, and making informed policy decisions. 

 



This work received funding from the ieConnect for Impact program which is a collaboration between the World Bank’s DIME group and the Transport Global Practice. The ieConnect program has been funded with UK aid from the UK government. 

Countries

Authors

Robert Marty

Research Analyst, Development Impact Evaluation (DIME), World Bank

Gabriel Stefanini Vicente

Data Scientist, Development Data Group (DECDG), World Bank

Join the Conversation

Tiger
February 23, 2024

Jankari