One of the best moments of my recent trip to Bhutan was the pleasure of spending time and having fantastic conversations with Jim Rugh, co-author of the book ‘RealWorld Evaluation’.
The RealWorld Evaluation approach was developed specifically to address the need for practical strategies on how to ensure the highest level of methodological rigour, consistent with the circumstances under which the evaluation has to be conducted. Sage first published the book in 2006 and the 2nd edition in 2012.
The essence of the approach lies in the fact that programme plans and implementation realities are most times starkly different and subject to dynamic environments and scenarios. It is a well-known fact that evaluators face innumerable challenges. All too often, evaluations are an ‘afterthought’ of programmes, not designed until the end. Baseline data or comparison groups are not established. The time and resources available are limited. Evaluation is seen as a tool to ‘prove’, not to ‘improve’, and clients have prior expectations for what they want evaluation findings to say. Further, many stakeholders do not understand evaluation, distrust the process, or even see it as a threat or as being ‘judged’.
As a result of these kinds of constraints, many of the basic principles of rigorous impact evaluation design (comparable pre-test – post-test design, control group, adequate instrument development, and testing, random sample selection, control for researcher bias, thorough documentation of the evaluation methodology, etc.) are often sacrificed.
RealWorld Evaluation is an integrated approach to ensure acceptable standards of methodological rigour while operating under a real-world budget, time, data, and political constraints. It promotes seven core steps, each with checklists for identifying constraints and determining how to address them.
4th Wheel Social Impact will soon be hosting a one-day workshop on the approach. We are excited to spread this approach among personnel involved in designing, implementing, monitoring, documenting, and evaluating social programmes in India.
I am humbled to have been trained by Jim Rugh to conduct this workshop. I am now officially an FOJ (Friend of Jim), who is determined to share the learning and benefits of this approach.
Thank you, Jim. (For everything)!
This blog is written by Sharon Weir, Co-Founder of 4th Wheel Social Impact.
