Many campaigns work with a measurement partner for both monitoring and evaluation (often it is useful to work with the same partner engaged in the formative research phase).

Measuring behavior change well is a complex and time-consuming challenge. It is recommended that you work with a vetted market research agency, a team of academic researchers, or another experienced measurement partner for both monitoring and evaluation (note, the same partner can support both monitoring and evaluation).

Ideally your campaign team can also work with a third-party media monitoring partner, which will provide important, accurate indicators from mass media. While many of these indicators can be obtained directly from television stations, radio stations, and other outlets, this is not recommended (campaigns in many regions have found significant inaccuracies in outlet-reported indicators). Working with a media monitoring partner can ensure more precise monitoring, and better outcomes as a result.

Finally, in addition to expert partners, remember to engage as many stakeholders as possible in planning your monitoring and evaluation work. Giving stakeholders a voice and a role in monitoring and evaluation will build trust, ensure all voices are heard, surface useful tactics for measuring success, and complement your overall campaign.

There are numerous ways of monitoring and evaluating campaigns. Methods can be:

• Qualitative (e.g. case studies, interviews, focus groups, survey questionnaires)
• Quantitative (e.g. statistics or trends that indicate a change over time)

You will need to establish both monitoring indicators, which track your activities during implementation, as well as evaluation indicators, which track the success of your campaign overall.

Your monitoring indicators will seek to track when, where, and through what channel was each campaign tactic or element was disseminated.

The process of establishing monitoring indicators starts with the channels you identified in your Media Plan (Phase 5). Create a list all of the channels you will be using in your campaign, and identify how you will measure the reach and impressions of each one.

The Common Media Channel Monitoring Indicators table will help you understand the most common ways different media channels’ reaches are measured.

To identify your evaluation indicators, revisit the objectives that you developed in Phase 1 of this toolkit. Your evaluation will be based on these objectives, as well as on your overall problem statement, or the larger goal you’re seeking to achieve.

Evaluation indicators will include:

• Monitoring Indicators which you have already established; these measurements of whether your activity is being implemented as planned are an important part of your overall evaluation
• Outcome Indicators which measure progress on your objectives. These are more short-term indicators, and include results of specific tactics, such as “percent of target population reached by an ad”
• Impact Indicators measure what happens when your objectives are achieved, in order to measure progress on solving the problem. These are long-term indicators, such as “number of new cases of HIV at the regional level”

Your media channels will also guide you in creating outcome indicators, as you should have at least a few outcome indicators for each of your media channels (make sure you have enough indicators to represent both the extent of campaign implementation and the number of individuals reached).

To establish impact indicators, use the following table to break down your objectives (download template):

Once you have selected indicators, consider how you will go about measuring them.

Your monitoring indicators will largely be tracked through the quantitative methods listed in the Common Media Monitoring Channels table. Note that, while many media channels can be tracked through pre-established tools, you will need to develop tracking tools specifically for your interpersonal communications workers to track their engagements (you may work with your measurement partner on this task). When developing tracking tools, remember to include a plan to train interpersonal communications staff on properly using these tools and tracking their engagements—this training is critical to ensure each worker is capturing data correctly.

For outcome and impact indicators, you will likely require a mix of quantitative and qualitative data, gathered through multiple data collection methods.

In choosing specific measurement methods, you’ll also need to take into consideration your budget, time constraints, purpose of the evaluation, audience characteristics, health issue-related factors, and donor requirements.

Consider not only how data will be collected, but how often—from continuously or daily, to monthly or quarterly (or even annually). This factor is also determined by budget, donor, and other requirements; your monitoring and evaluation plan should clearly list both the how and the how often of data collection.  

Your monitoring & evaluation plan is not complete until it includes your plans for detailing how you will present your results.

This plan should include monitoring reports for checking in on campaign progress throughout the campaign. Work with measurement partners and analysis teams to establish a standard reporting framework and timeline. These interim reports will allow you to adjust or tweak the campaign to increase effectiveness—just be sure to keep track of any changes you make, so you can include this in your campaign evaluation!

One way to create ongoing reports is by using an indicator scorecard, which records preliminary data in a standardized way— Template: Data Collection

Finally, you should plan to share the results of your final evaluation. This report should include an analysis of the impact, as well as an analysis of how well the strategy was implemented. Consider the following:

• What is the best format to present monitoring and evaluation findings?
• Does the data need to be analyzed, or is it most useful to report straight data?
• Who will read the report? Internal or external dissemination?
• What recommendations can you make based on monitoring and evaluation findings?

A Sample Evaluation Report Outline is provided here.

Best Practices in Monitoring & Evaluation

• Establish a solid plan early. Determine program objectives and milestones/metrics before you begin the campaign; that way, you can start tracking early, allowing for continual refinement and improving your campaign’s overall efficacy
• Use mixed methods for data gathering. Track both qualitative and quantitative variables, use a variety of sources and methods, and gather evidence from both internal and external sources; this will help ensure you gather a complete picture of the data
• Use a representative sample. You most likely can’t measure every individual in your target audience, so you’ll be tracking behavior change amongst a sample of individuals. Your measurement partner can help you choose from the different sampling methods available to assemble the most valid sample—known as a “representative sample”—in a way that is most appropriate for your campaign
• Use standardized metrics. Your measurements will be most useful if they can be compared with others; especially if you want to do a comparative analysis of multiple campaigns, you must be sure to use standardized metrics across regions

Checking in on your progress

Monitoring and evaluation is obviously important for accountability, and is often required when donor funding is used to support a program or campaign. But the value of this measurement work is far-reaching—it builds trust with all of your stakeholders, it empowers implementers to adjust the campaign mid-stream for greater success, and it’s important for the larger public health community to understand your work and build on your lessons learned. By disseminating the results of your monitoring and evaluation, you can help advocate for additional resources and policies, and help improve long-term public health outcomes.