We are making part of the first module of the course available for free so that you can gauge whether the content level is right for you. The discussion forum and quizzes are not available.

   

Introduction to M&E

Welcome to Module 1. Across health programs, there continues to be high demand for organizations to demonstrate the effect and impact of their interventions.

Program implementers want to know what works and doesn't work. Ministries of Health want to be strategic in what they support. Donors want to know you are spending their money well.

In response, there has been greater demand for monitoring and evaluation (M&E).

This week, we will begin by defining M&E, their distinct purposes, the circumstances in which each is used, and how they work together to improve programs. We will also discuss key M&E activities in a typical program cycle and the roles these activities play in enhancing a program's success.

Note: You can turn on closed captions by clicking the CC button on the video player navigation bar.

If you have trouble watching the lectures due to a slow Internet connection, try downloading the lecture or the transcript listed below each video. Right click the link and select "Save Link as" to download.

Learning Objectives

After completing this module, you will be able to:

  • Define monitoring and evaluation;
  • Distinguish between monitoring and evaluation;
  • Explain why M&E is important; and
  • Explain how key M&E activities fit into a typical program cycle.

Reading: Program vs Projects (1 minute)

Though we primarily focus on M&E at the program level in this course, many of the concepts also apply to projects. How do projects differ from programs?

Programs are typically on-going, compared to projects which typically have a clear end point. Programs typically require a greater number of people and resources compared to projects. In fact, most programs tend to have multiple projects, so the program is often the bigger context for the project.

Video: What Is M&E (5 minutes)

In the lecture below, Dr. Kawakyu gives an overview of what monitoring and evaluation is and describes how it can be beneficial to programs and projects.

Downloads: Video (29 MB), Audio (2 MB), Transcript

Video: M&E Skills (17 minutes)

In these three videos, experts discuss what skills they feel are important for working in the field of M&E. As you view these videos, reflect on your skills and experience. Do you feel you have some of these skills already? Which ones might you need to work to improve or gain?

First, you will hear from Mr. Shreshth Mawandia, the program director with the Botswana chapter of I-TECH, part of University of Washington Department of Global Health.

Downloads: Video (35 MB), Audio (2 MB), Transcript

Next, you will hear from Marya Plotkin, a Senior Technical Advisor for strategic information at FHI360.

Downloads: Video (20 MB), Audio (2 MB), Transcript

Finally, you will hear from Mark Shepard, the Director for Strategic Information at I-TECH at the University of Washington.

Downloads: Video (106 MB), Audio (6 MB), Transcript

Video: An Introduction to Monitoring (8 minutes)

In this lecture, Dr. Kawakyu will discuss how monitoring aids programs in achieving its objectives.

Downloads: Video (22 MB), Audio (4 MB), Transcript

Video: HIV Tests in Mozambique (9 minutes)

In this video, Mr. Jordan McOwen, with CDC in Mozambique, describes how monitoring HIV testing in Mozambique allowed them to increase the number of people getting tested for HIV.

Downloads: Video (32 MB), Audio (4 MB), Transcript

Video: An Introduction to Evaluation (7 minutes)

In this lecture, you will learn about how evaluation is different from monitoring and the purpose evaluation serves in programs.

Downloads: Video (21 MB), Audio (3 MB), Transcript

Reading: Monitoring vs. Evaluation (2 minutes)

The table below shows the difference between monitoring and evaluation.

 
  Monitoring Evaluation
Purpose Driven by need to oversee a program’s operation Driven by need to document program effectiveness and report to donors, other stakeholders
Timing Data are collected on an ongoing, regular basis Data can be collected on an ongoing basis, but are typically collected and/or analyzed periodically, at key points before, during or after the program 
Scope Tracks program progress, especially in terms of inputs/resources, activities, and outputs Assesses program progress toward goal, objectives, and outcomes and assesses if changes are due to the program
Data sources Data sources are typically readily available (e.g., existing databases, program documents) Typically requires data collection
Common participants People directly involved in implementing the program (e.g., program staff, M&E staff, and program users) Program stakeholders (e.g., program staff, M&E staff, donors, community members) and external evaluators
Common deliverables Regular reports and updates to stakeholders (e.g., program users, management, donors) Presentation to stakeholders; formal written report with evidence of improvements made, lessons learned, and recommendations for future

Activity: Monitoring or Evaluation? (10 minutes)

For each question, determine if it is best answered by monitoring or evaluation, and why.

Video: ME and the Project/Program Cycle (9 minutes)

This lecture covers the main phases of a project or program cycle and how M&E fits into each phase.

Downloads: Video (56 MB), Audio (4 MB), Transcript

Video: Disconnected vs Connected Process (7 minutes)

In these two videos Mr. Jordan McOwen tells two stories. One is about a program that used a ‘disconnected M&E planning process’, or one that did not connect the program reality to the needs of the priority population, and the other one details a ‘connected process’. As you watch the video, reflect on the differences between these two processes. What made the second one successful as compared to the first?

Downloads: Video (22 MB), Audio (2 MB), Transcript

Downloads: Video (22 MB), Audio (2 MB), Transcript

Reading: Hand Hygiene Case Study (3 minutes)

The case study below will be referred to throughout the course. This information is also available in the handouts in upcoming modules.

Background: You are the district health officer for Awani district. There are 8 health facilities in Awani district, all with existing hand hygiene policies and guidelines. The National Health Strategic Plan has just been updated and a key priority focus is to decrease hospital acquired infections. You are responsible for planning and overseeing implementation and ME for this work.

Health care-associated infections (HCAI) are infections that occur in a client while receiving health care services for another condition. HCAIs affect hundreds of millions of people globally, causing preventable illnesses, death, and contributing to the economic burden of clients and the health care system. Hand hygiene among health care workers is a critical strategy in preventing HCAIs. There are both system-level and individual, behavior-level reasons for why health care workers do not always follow proper hand hygiene, for example, health care workers often are unaware of, have not received training in, and do not have the resources to follow evidence-based hand hygiene guidelines.1

A baseline evaluation was recently conducted at the 8 facilities in Awani district to assess health care workers’ hand washing-related knowledge and practices, and the facilities’ hand washing-related infrastructure gaps. Key findings included: only 15% of health care workers were able to demonstrate knowledge of hand hygiene guidelines and only 10% of all observed health care workers washed their hands correctly. The evaluation also found that while there were adequate numbers of well-placed, functioning sinks at all facilities, 100% of facilities reported hand soap shortages.

Problem Statement: Poor hand hygiene infrastructure and practices at health facilities in Awani district are contributing to health care associated infections, causing preventable illnesses, death, and contributing to the economic burden of clients and the health care system.

Program Goal: Decrease health care associated infections in 8 facilities in Awani district.

Program Objectives:

  • Increase the percentage of health care workers in target facilities who comply with hand hygiene guidelines from 10% to 75% by the end of Year 3.
  • Increase the percentage of health care facilities with adequate hand hygiene infrastructure from 25% to 100% by the end of Year 1.

1. World Health Organization. (2009). A guide to the implementation of the WHO multimodal hand hygiene improvement strategy.

Discussion (20 minutes)

In this module's discussion forum, please introduce yourself to your group and describe your experience with M&E in relationship to the project/program cycle.

Go to Discussion

*Please note that there is no access to the discussion in this sample course.

Reading: Module Summary (5 minutes)

In this module, you learned that M&E is a process of collecting, analyzing, applying, and disseminating data to assess progress toward program goals and objectives. 

Monitoring is the ongoing, routine collection and analysis of data about a program’s activities in order to measure program progress. It is concerned with answering the fundamental question, “what are we doing?” and usually focuses on the processes that occur during implementation. It allows you to: 

  • Track activities and performance over time, such as what activities are being carried out, to whom, where, when, and how frequently
  • Identify if activities are being implemented as planned and on time
  • Make informed, timely decisions based on data, including improving programs by identifying which aspects are working well and which require adjustments 

Evaluation, on the other hand, is the systematic process of collecting, analyzing, and using data about a program’s design, implementation, and results to determine its effectiveness. Though it works jointly with monitoring to assess and improve programs, there are many ways in which the two processes are distinct from one another. Evaluations are commonly used to:

  • Understand why activities have (or have not) been implemented as planned
  • Explain whether the program had an effect    
  • Determine the extent to which measured or observed changes can be attributed to the intervention
  • Describe if the intervention had any unintended consequences
  • Assess whether a program was cost-effective

Data sources differ between monitoring and evaluation. Monitoring data commonly come from readily available, routine data sources. Because evaluation requires measuring changes at the population or beneficiary level, you often need to collect additional data from the intervention’s priority group. 

Who conducts monitoring and evaluation activities also can differ between the two. Monitoring is primarily done by people directly involved in implementing the program, such as program staff, M&E officers, data clerks, and program users. Evaluations are usually conducted by more experienced, senior M&E staff and often includes external evaluators, advisors, collaborators and/or donor input. 

The timing of data collection can also be different between monitoring and evaluation. Monitoring data are gathered and reviewed on a regular, continuous basis in order to correct course as needed. Data for evaluation are often only collected at specific points in time before, during or after the project or program.

There are three major phases in a typical project and program cycle: the initial assessment, planning, and implementation. An initial assessment is conducted to determine whether a project is needed, and, if so, to inform the project’s planning efforts. The main process used in this phase is a needs assessment. The planning phase is directly informed by the findings from the needs assessment and often includes the development of a logic model and M&E plan and conducting a baseline evaluation. The implementation, monitoring, and evaluation phase includes monitoring, and process, outcome, and impact evaluations as well as dissemination activities.

Quiz (15 minutes)

Go to Quiz

*Please note that in this sample course, the quiz is not available.

Optional Resources

M&E Overview

Basic Terminology and Frameworks for Monitoring and Evaluation
A book that explains commonly used terms and frameworks by UNAIDS.

M&E Fundamentals
A short eLearning course on the basics of M&E by USAID.

Monitoring and Evaluation Guides
A list of M&E resources and their pros and cons.

Needs Assessment Resources

Community Health Assessments & Health Improvement Plans
Resources for conducting assessments by the US Centers for Disease Control and Prevention.

Community Health Assessment Toolkit
A step-by-step approach for conducting community health assessments from the Association for Community Health Improvement.

Needs Assessment
A chapter on needs assessment from a guidance manual for capacity building of local NGOs.