Planning for evaluation at the outset of a research project is important, but how do you select the best evaluation strategy for different moments in its lifecycle?
“Research evaluation is specific to your project. There is no one size fits all framework,” Research Impact Facilitator Catherine McElhone said. “It’s about thinking about what you are trying to do.”
Access the training - Fundamentals of Research Impact Module 4: Evaluating and monitoring impact
Selecting the right approach
Catherine highlights the importance of being able to identify appropriate evaluation strategies and suggests the following questions to think about when planning evaluation strategies:
- What would you like to learn/know at this stage of the project?
- Who should be involved in the evaluation and how?
- What evaluation approaches would work in this context?
- What resources do you need to make your evaluation happen?
Planning for how and when to conduct evaluation needs to be considered at the very start of a research project.
“Doing it retrospectively is hard, because the project is finished. If you plan evaluation and carry it out say midway during the project, you can adapt and change.”
When you understand what you are measuring and evaluating, then you can find the appropriate metrics.
Other considerations to think about when planning evaluation include:
- motivations, learning and accountability?
- new ideas
- ask ‘So what?’ Why do people care?
- monitor changes in behaviour
- document both short and longer-term changes.
Learning from UTS researchers
Catherine referred to the work of Professor Meera Agar’s team in the UTS Faculty of Health. Meera’s studies into delirium has produced groundbreaking discoveries that indicate the ineffectiveness of medication.
Meera’s theory of change and evaluation plan included:
- improved patient care
- improved capacity/education of health workers
- reduction in treatment costs
- reduction in onward effects
- adoption of research into clinical guidelines
- endorsement by professional bodies.
“If you don’t have a clear picture of what you are reaching for, it all starts to fall apart,” Catherine warned.
“When you understand what you are measuring and evaluating, then you can find the appropriate metrics. You’re building up these different pieces that enable you to tell the story of the impact,” she said.
Looking at UTS Rapido Vietnam and its water monitoring solution for aquaculture in Vietnam to improve water quality, Catherine showed how the theory of change and evaluation plan included:
- improved access to clean water and information
- employment and community ownership
- reduced water waste
- improved aquaculture yields