Overview of the Malaria Surveillance Assessment Toolkit

Malaria Policy Advisory Committee Meeting

3—4 December 2020, Geneva, Switzerland

Background document for Session 1

This document was prepared as a pre-read for the meeting of the Malaria Policy Advisory Committee and is not an official document of the

World Health Organization.

WHO/UHC/GMP/MPAC/2020.06

Overview of the Malaria Surveillance

Assessment Toolkit

The Global technical strategy for malaria 2016–2030, published by the World Health Organization

(WHO) in 2015, emphasizes surveillance as a core intervention for accelerating progress towards

malaria elimination across endemic settings. Malaria surveillance, monitoring & evaluation: a

reference manual published by WHO in 2018, provides guidance on the principles and requirements

for a strong malaria surveillance system. However, there is a lack of coordination and standardization

of tools to monitor the quality of malaria surveillance and to understand its strengths and weaknesses.

What is a malaria surveillance assessment?

A malaria surveillance assessment is a systematic approach to measuring the performance of malaria

surveillance systems (i.e., their quality), and identifying and evaluating the determinants of that

performance. Malaria surveillance assessment results can be used to provide actionable and

prioritized recommendations on how to strengthen the surveillance system for malaria control and

elimination. A malaria surveillance assessment can be undertaken at any time. However, to ensure

that its findings can inform future activities, it is recommended that an assessment be implemented

as part of key national malaria control programme (NMCP) planning milestones, such as during malaria

programme reviews (MPRs) and National Strategic Plan (NSP) development.

What are the gaps in malaria surveillance assessments?

To date, malaria surveillance assessments have been implemented in multiple countries, using a

variety of tools to assess systems. The shared goal of these assessments has been to enable NMCPs to

improve their performance towards achieving control and elimination goals. However, past

approaches and tools have not been standardized across assessments, making it difficult to compare

results between countries, between regions within a country, or over time in any select geographical

region.

To address this issue, WHO has developed a standardized Malaria Surveillance Assessment Toolkit

to align and adapt available tools into a single set of best practices, and to provide guidance for

conducting comparable and replicable malaria surveillance assessments across multiple countries

and partners.

What is the Malaria Surveillance Assessment Toolkit?

The Toolkit consists of multiple tools, including question banks, an implementation protocol template,

a final report template, etc. A complete list of tools and corresponding links is given in Table 1. These

tools can be used throughout the implementation of an assessment – from initiation of the project, to

data collection, analysis and output generation, and prioritization and dissemination of results.

There are multiple potential users of the Toolkit (WHO, donors, implementing partners, and NMCPs).

Each potential user has different goals for their assessment and for the use of the Toolkit in general.

Overview of the Malaria Surveillance Assessment Toolkit | 2

As such, this Toolkit has been designed to reflect a multiplicity of purposes through the following

characteristics:

• Adaptability: A malaria surveillance assessment conducted using the Toolkit can address a

range of possibilities by customizing the assessment scope, defined as the framework and

surveillance strategies

to be covered by the assessment. The data collection tools within the

Toolkit can be tailored accordingly by filtering the content of each tool to include only relevant

aspects.

• Standardization: To enable findings to be comparable across countries and between

assessments within a country over time (e.g., for longitudinal assessments), any malaria

surveillance assessment conducted using the Toolkit will include a minimum set of priority

indicators and generate consistent expected outputs.

TABLE 1.

Malaria Surveillance Assessment Toolkit content by implementation phase

IMPLEMENTATION

PHASE

TOOL*

TOOL DESCRIPTION

Phase 1 – Country-

specific project

initiation

Indicator Table

A compilation of key objectives, sub-objectives and indicators that can

be used to quantify and/or qualify gaps in surveillance

Protocol Outline

Template and guidance for a protocol for implementing a malaria

surveillance assessment using this Toolkit

Phase 2 – Data

collection*

Desk Review

Guide

Template and guidance for conducting a literature review supported

by key informant interviews to compile, summarize, analyse and

organize what is known about malaria surveillance

Data Quality

Assessment

Guide

Templates and guidance for gathering, analysing and presenting data

that will be assessed for data quality

Question Banks

Question banks for data collection at service delivery and

subnational/intermediary levels

Phase 3 – Data

analysis and output

development

Analysis Tools

Excel tools and code (in statistical software, e.g., STATA) that can be

adapted for data analysis of all data collected during a surveillance

assessment

Phase 4 –

Prioritization of

recommendations

and dissemination

Report Outline

Suggested outline for presenting the final results from the

assessment, including templates to organize, visualize and interpret

results

Assessment

Evaluation Plan

Protocol for evaluating the quality of the surveillance assessment

implementation itself, including an expenditure tracking template and

guidance

Assessment

Implementation

Log

A log for tracking surveillance assessments that have been

implemented using the Toolkit

*Materials that can be used for each implementation phase will need to be prepared and reviewed during

earlier phases/time periods in order to avoid delays.

The primary focus of the Toolkit is case surveillance in public, private and community sectors in both high-burden and low-burden

settings. However, the Toolkit can also be applied to assess additional malaria surveillance strategies using priority indicators from

Objectives 2–4. These include entomological surveillance, commodities tracking, and intervention monitoring and evaluation. The goal of

an assessment of these strategies is to understand what information is being collected and how, and if it is being integrated and used

along with case surveillance data. Currently, the Toolkit does not explicitly enable the assessment of drug resistance monitoring systems

(i.e., therapeutic efficacy studies) or early warning detection systems. Other available tools may need to be leveraged for comprehensive

assessment of these additional strategies.

Overview of the Malaria Surveillance Assessment Toolkit | 3

Framework

The Toolkit builds on the PRISM (Performance of Routine Information System Management) model by having a

framework based on four objectives that a surveillance assessment can address (Fig. 1).

FIG. 1.

Malaria Surveillance Assessment Toolkit objectives

In the Indicator Table, each objective is built out into sub-objectives for which qualitative and quantitative

indicators are provided. These indicators further detail the components or factors that make up malaria

surveillance performance or determinants, and can be measured by one or more of the data collection tools

within the Toolkit. A subset of indicators have been flagged as ‘priority indicators’, representing the minimum

set of metrics to be included in any assessment conducted using the Toolkit.

Methodology

Implementation of a malaria surveillance assessment supported by this Toolkit can include various

activities, including desk-level data gathering and analysis, as well as primary data collection, e.g., a

health facility level survey. The assessment scope will determine which of the nine tools are required,

e.g., whether a survey at the service delivery level is needed.

➔ For example, an NMCP may only be interested in conducting a quarterly desk-level analysis of

surveillance data, in which case it will select indicators and associated questions related to the sub-

objective “measuring data quality” of Objective 1 “performance diagnosis”. Data collection may

be restricted to desk-level analysis of retrospectively compiled data and audits at the health facility

level. Such an assessment may be implemented once every quarter, alongside routine supervision.

Moreover, depending on the assessment goals, there are three general approaches to conducting a

surveillance assessment using the Toolkit; tools can be selected and adapted accordingly:

RAPID

TAILORED

COMPREHENSIVE

Scope

Only priority indicators from all

four objectives for only case

surveillance

Selected indicators of all four

objectives and selected

surveillance strategies

All indicators of all four

objectives and all surveillance

strategies

Methods

Primarily limited to desk review

only

Desk review and surveys at

different levels of the health

systems (i.e., central,

subnational, a sample of

facilities and communities)

Desk review and surveys at

different levels of the health

systems (i.e., central,

subnational, a sample of

facilities and communities)

Overview of the Malaria Surveillance Assessment Toolkit | 4

RAPID

TAILORED

COMPREHENSIVE

Access

Tool

can be downloaded off-

the-shelf

Data collection tools are

customized then downloaded

Data collection tools can be

downloaded off-the-shelf

Estimated

resource

requirement

Low: up to 4 months depending

on context

Medium/High: up to 12 months

depending on context

High: up to 12 months

depending on context

Suggested

frequency

Once every year, such as during

annual programme reviews

Every 3–5 years to establish a baseline or to assess the system

comprehensively

Expected outputs

To facilitate comparability between assessments over time and across geographies, a set of results

expected from all assessments conducted using the Toolkit has been defined. These results include a

cascade diagram of the representativeness of surveillance data, a dashboard of charts and tables for

all data quality indicators, and a score card that quantitatively summarizes findings from priority

indicators from Objectives 2–4. These outputs provide a high-level understanding of or first glance at

the context, infrastructure, process, and technical and behavioural aspects that may be driving the

surveillance system’s poor performance.

The in-depth findings from the malaria surveillance assessment can be presented using the Report

Outline, which includes a summary of the methods, a more in-depth description of the assessment

results, and recommendations for surveillance strengthening actions based on key findings.

Upon completion of an assessment, recommendations should be developed based on the assessment

results and prioritized in consultation with the NMCP and other stakeholders based on their impact

and feasibility for strengthening the surveillance system.

Limitations

• It may be required to align the assessment with other assessment activities that are

ongoing/planned in the country in order to ensure that outputs are not duplicated. This may

require significant stakeholder coordination and cause delays in implementation. However,

individual assessments may be driven by specific donor commitments, and therefore some

activities and outputs from parallel assessments may be duplicated.

• The Toolkit provides indicators that are useful for measuring and understanding expected

operational research questions; however, this list of indicators is not exhaustive and all aspects of

a country’s context may not be captured. Additional indicators may need to be added for specific

contexts.

Next steps

The following will be addressed in the next version of the Toolkit:

• Content relevant for elimination settings is incomplete in the current version. This will be

developed and incorporated into the Toolkit and tools within.

• Additional indicators (per partner feedback) will be included in the Surveillance Assessment

Indicator Table to ensure that all aspects of surveillance are assessed as per the scope of this

Toolkit. This will prompt updates to the data collection tools, e.g., question banks and

questionnaires.

For ease of use, content for a rapid assessment of priority indicators has been compiled in to one workbook

Overview of the Malaria Surveillance Assessment Toolkit | 5

• Priority indicators that should be assessed across all country settings will be refined.

• Data quality assessment (DQA) tools within DHIS2 (e.g., DQA app.) that are currently being refined

will be included once finalized.

• Data analysis code will be developed for data gathered through health care worker interviews at

service delivery levels (as per data analysis of data gathered through one or two country

implementations that have yet to be completed).

• The user interface and navigation of the tools within the Toolkit will be improved. A web-based

interface will be developed.

• Training materials will be developed for easier access/use of the Toolkit.

Additional links

Malaria Surveillance

Assessment Toolkit

version 1.0

https://clintonhealth.box.com/s/cfz464rir5oitqd2qusz2c432u871h1p

Request for Proposals for

the development of a

web-app for the Malaria

Surveillance Assessment

Toolkit

https://www.ungm.org/Public/Notice/116331