16 July 2015.
Working as an ODI Fellow in the statistics bureau for Liberia, the Liberia Institute of Statistics and Geo-Information Services (LISGIS), I experienced at first hand the challenges of producing and obtaining statistics in a developing country environment, from logistical, technical and political aspects. It is widely agreed that some data is better than no data, and that generally data is as good as it can be, given the context, but this is true only to the extent that the country context is understood. Liberia provides an instructive example of an extreme case where data collection, and resultant indicators, may suffer as a result of constraints, whether logistical, financial, cultural, educational or otherwise. I briefly discuss a very limited number of these below.
Background in Liberia
A 14-year civil war, caused in part by the marginalisation of a large portion of the Liberian population from political power, ended in 2003, when a peace agreement was negotiated and signed, and a transitional government was put into place. The United Nations Mission in Liberia (UNMIL) was established to ensure the peace process was implemented and law and order maintained throughout the country. Liberia had by then suffered an estimated 270,000 deaths and the destruction of vital institutions and infrastructure, and the country’s economy had come to a halt. In 2005, Madam Ellen Johnson Sirleaf was democratically elected as the first female president of Africa.
Transformation strategies
The country’s first poverty reduction strategy, named Lift Liberia, was designed and implemented by Madam Sirleaf’s government to raise Liberia from post-conflict emergency reconstruction and position it for future growth. Madam Sirleaf was re-elected in 2011; during her second term as president, a new long-term vision for Liberia’s socioeconomic development was articulated: Liberia Rising 2030. This national vision sees Liberia’s economy transformed to middle-income status by 2030. In 2012 the government outlined a five-year development strategy, the Agenda for Transformation (AfT), whose goals and objectives represent the first steps towards achieving the goals of Liberia Rising.
Statistics and data in Liberia
Most of the data from surveys and censuses taken before 1989, prior to the civil war, were lost during the conflict. After the war, capacity constraints limited the collection of socioeconomic and geo-information data. In response, the government re-established LISGIS as an autonomous agency responsible for producing statistics and spatial data for Liberia. In recent years, LISGIS has been working to implement the National Strategy for the Development of Statistics (NSDS), approved in 2008, which aims to establish a robust national statistics system through rebuilding statistical capacity and strengthening coordination across data collection agencies.
Since 2008 LISGIS has achieved many objectives set out by NSDS, including the completion of major data collection activities. Among these are the Core Welfare Indicators Questionnaire (2007, 2010), Liberia Demographic and Health Survey (2007), censuses (National Population and Housing Census 2008, Human Resource for Health Census 2009, and School Census 2007, 2009 and 2011), Labour Force Survey (2010), National Accounts Survey (2008 and 2012) and Agricultural Crop Survey (2008, 2009, 2010, 2011). These recent efforts have helped to narrow the statistical gap that resulted from the conflict.
Despite the significant narrowing of data gaps and statistical capacity, data in Liberia still suffer from substantial limitations due to physical and logistical constraints imposed by the country’s geography and lack of infrastructure, and due to low capacity, both of which affect quality of data collected.
Monitoring progress of the Agenda for Transformation
Five key pillars are at the heart of AfT, one of which is economic transformation. The economic transformation plan identifies and focuses on improvements in seven key areas: private sector development, macroeconomic issues, infrastructure, agriculture and food security, forestry, mineral development, and management and capacity development needs.
Initially, around 220 indicators were proposed in order to monitor progress of AfT, including a significant number aiming to measure economic transformation.
Besides this original suggestion including superfluous and inefficient indicators, there is a clear lack of understanding of the country’s statistical limitations that results from weak bilateral relations between LISGIS and designers of the monitoring and evaluation for AfT. Over a period of almost two years, the number of indicators was significantly refined and reduced to approximately 50. Indicators come from a variety of data sources, from LISGIS and elsewhere. Many data sources are not vetted by the official statistics bureau, LISGIS, due to a lack of coordination, funding and capacity.
Challenges with data and lessons from the field
Challenges in collecting accurate, robust data are ubiquitous; in developing countries that have weak statistical systems, low capacity levels and vast, remote and hard-to-reach areas, the challenges are even more pronounced.
A significant amount of my time at LISGIS has been spent on the 2014-2015 Household Income and Expenditure Survey (HIES). The HIES is a multi-topic household survey which collects information on household consumption and income in key areas such as health, education, employment, and food security. The primary objectives of the HIES are to address major shortcomings in gross domestic product (GDP) and consumer price index (CPI) estimates for Liberia, and to produce detailed poverty analysis and robust and nationally representative agriculture statistics. Collecting data for the HIES presented a number of challenges.
Impacts of seasonality and Ebola virus disease on data collection and analysis
Liberia has two extreme seasons: a dry season, from approximately November to April, and a rainy season, from May to October; induced by the rain, the harvest season for staple crops runs from July until December. In order to capture effects of seasonality on key indicators, data was scheduled to take place over a 12-month period. The spread of the Ebola virus disease rapidly accelerated in August 2014, and our teams were pulled out of the field for their civil and health safety and in accordance with government recommendations and countrywide travel restrictions.
As a result, only six months of data collection were completed, instead of the planned 12 months. Although the sample was only about half the target sample size, it was designed to be nationally representative on a quarterly basis, and data can be used to produce estimates with a fair amount of precision. However, estimates may suffer from bias attributed to lack of seasonality. In particular, the harvest season and major festive period (including predominant Liberian holidays and Christmas) fell outside of the data collection period. This resulted in the following:
Consumer Price Index: A new consumer basket and weights will not represent real annual consumption patterns in Liberia; however, using a six-month dataset is far superior to previous CPI methodology, which was based on four neighbouring countries and outdated weights. This six-month data will be used in the interim until a 12-month survey can be conducted, and it should be heavily caveated.
National Accounts: Compiling the household component of National Accounts using six months of data is far from ideal; furthermore, National Accounts data in constant prices should also be used with caution due to the suboptimal deflator available. Updates will be made as soon as a 12-month survey is completed.
Poverty: Poverty measurement is highly sensitive to the effects of seasonality, and therefore poverty indices will not be released based on the six-months data.
Physical terrain
In terms of surface area, the majority of Liberia is remote and hard to reach, yet it must be visited to ensure that data are nationally representative and inclusive of such vulnerable remote populations. HIES field teams have often walked long stretches, easily up to 12 hours, in order to reach a cluster selected in the sample; they have taken motorbikes, crossed rivers by canoe or by raft (a piece of wood), pulling themselves across using a rope tied between two trees across the river; they have also walked across many logs over water, or across makeshift bridges of logs as narrow in diameter as 10cm. Enumerators need to be physically fit and highly motivated to complete the work, and a high level of monitoring by the head office is required to ensure that work is conducted according to plan. In reality, delays in obtaining funding, and overstretched project staff in headquarters, mean that monitoring is infrequent.
Further challenges with National Accounts estimates
Liberia’s current GDP estimates are The sources of information for estimating different components of GDP include the financial and non-financial companies, the government, non-profit organisations, and households. Information from the first three sources is generally available in some form; however, significant deficiencies exist in the source data. National Establishment Censuses (2007, 2012) and National Accounts surveys (2008, 2012) have been conducted to provide source data for the business component of National Accounts. Refusals and incomplete interviews are common, particularly when questions are revenue related, despite the training and data collection expertise of LISGIS personnel. Businesses highly distrust LISGIS because they fear information will be passed on to government revenue authorities. Furthermore, information on the activities of non-profit organisations is unreliable due to weak capacity and enforcement, and data related to households’ activities and behaviour are mostly non-existent. For the first time, detailed household data from the HIES 2014 will be available to prepare the household component of National Accounts.