For organisations about to begin their Measurement, Evaluation and Learning (MEL) journey, there are four steps you should take.
Step 1: decide the scope and purpose of your MEL project
The first step is to agree within your organisation what you are actually trying to achieve with MEL. This will determine the best approach to take. There can be multiple objectives for your project, but it’s important to be clear on what they are so you can plan carefully to satisfy them.
For example, is the goal to celebrate achievements, or is this an internal exercise to understand how an initiative or policy has worked and what could be improved for next time?
Other questions to consider are related to the methodology and intended outputs. Do you want to run a large quantitative trial, or are you looking to run interviews to begin a library of case studies that you can add to progressively? Do you have a baseline to assess progress against, or is this project intended to establish that?
It’s also a good idea to have a budget in mind at the outset. This allows you to work with your impact management partner to design a plan that uses this efficiently. It would be a shame to use all your budget on data collection then not have any left for analysis or to communicate what you find out.
Step 2: define your intended impact and theory of change
Defining the impact you are aiming for and how you will achieve it is critical in working out what and how to evaluate. Your theory of change is an articulation of the impact you aim to achieve and how you will achieve it. This is the hypothesis that your evaluation will test.
It’s crucial to define your intended impact precisely so you know what information to gather to see if you are achieving it. A great framework to help you think about this is the Impact Management Project’s Five Dimensions of Impact.
Organisations sometimes feel they can skip the theory of change step as they know implicitly what they are aiming for and how the intended change comes about.
This doesn’t need to be a huge piece of work if you feel you already know it, but getting it down either as a narrative or in a diagram is invaluable, first to ensure agreement between different internal stakeholders, then to systematically identify outputs, outcomes and impacts you can measure and evaluate.
Step 3: create your indicator framework
Your indicator framework includes details on precisely what you will measure to demonstrate your impact.
For this step, you need to consider what you would expect to change if you achieve the impact you are aiming for. Most likely, you are looking to increase or reduce something, which you would define precisely in the previous step using the Five Dimensions of Impact.
It is helpful to think through the outputs, outcomes and impacts from your theory of change here. The indicators you use should be decided based on a combination of the feasibility of collecting data on them and their meaningfulness as an indicator of you achieving the change you are aiming for.
Step 4: prepare your MEL methodology
Once you are clear on what data you want to collect, the final step is creating your research methodology. Your impact evaluation partner can help you to design this to enable you to evaluate progress against the indicators you have chosen while using your resources efficiently.
The methodology includes a plan for how to gather information – surveys or interviews, for example – and the tools to do it: surveys questions, interview templates, and a breakdown of target populations for your study, i.e. who you want to speak to or gather information from.
Your methodology should also include details on how you will evaluate the information you collect to test if your hypothesis was correct. This may involve statistical analysis, or have greater emphasis on qualitative evaluation, depending on the questions you are trying to answer.
Importantly, the methodology should include an estimation of the resources to conduct the evaluation so you can plan appropriately. Areas to cover include how much time it would take for members of staff and the costs for using an external partners, such as for data collection.
What next?
Once you have completed your evaluation, you will be equipped with lots of information about your project that a variety of stakeholders will be interested in. Communicating your impact can take a variety of different formats depending on your objectives, audiences and channels.
With impact evaluation, we like to think beyond the bread-and-butter ‘impact report’ to the more holistic ‘impact reporting’. This may include the standard hard-copy or PDF document that many will be experienced with.
But thinking about the reporting process in a more modular, strategic fashion, such as how different pieces of content can be taken from the report and used on different channels for different audiences, allows you to get more value from your project.
Some common formats to use in impact reporting, whatever the channel, include case studies, infographics and visuals, photos and key learnings.
- Case studies are useful in picking apart individual examples of your work and describing how a problem was overcome and the effect on the target group. A case study can include quantitative data alongside qualitative, aiming to give the reader a zoomed-in view of how your intervention works at a large scale.
- Infographics, visuals and charts help to take dry data and present it in a way that presents a point of view or provides an answer to a question. Done well, with graphics expertise, these can be standalone pieces of content, rather than relying on lots of explanatory text, and can work well on social media and presentations.
- ‘A picture paints a thousand words’, as the saying goes. High quality photography showing the human or environmental impact of your work goes a long way to bringing sometimes dry copy to life. This could be as simple as portraits of the people who have benefitted from your work, or before and after shots for environmental protection projects.
- MEL is not just about celebrating your successes, but also taking note of what you have learned and sharing this with others. This allows those working on similar issues to learn from your experiences. The key learnings section can focus on answering the ‘what could be done better next time?” and “what do you wish you’d known at the start?”. This can be one of the most rewarding parts of the MEL exercise, and get you fired up for your next project.
