Quality Assurance of Evaluation Office Reports
The quality of evaluation reports are systematically assessed by a set of quality rules that cover: the evidence-base of evaluation report findings and recommendations; conciseness of executive summary; appropriateness of evaluation methods and limitations; coverage of terms of reference; quality of recommendations; and how the evaluation dealt with differences in opinion.
Quality assurance processes apply at inception, draft report and final report stages. The collation of assessment ratings for the quality of evaluation reports at the various stages allows the Evaluation Office to monitor the value added by its internal review processes.
The template for the assessment of the quality of evaluation reports is available here.
Who evaluates the evaluators?
Independent external assessments of the performance of UN Environment’s evaluation function are an essential unpinning to its accountability and have played a significant role in enhancing the credibility of the Evaluation Office both within and outside the organisation.
In 2012 The Evaluation Office was the subject of a Professional Peer Review undertaken by the United Nations Evaluation Group / Organisation for Economic Co-operation and Development-Development Assistance Committee and the findings were very positive.
Each year the Global Environment Facility Independent Evaluation Office assesses the performance of Global Environment Facility Agencies against a number of performance measures that cover important aspects of the performance of the Evaluation Office (UN Environment), bench-marked against the performance of the World Bank’s Independent Evaluation Group and United Nations Development Programme Evaluation Office, including the quality of Global Environment Facility project evaluations. This is reported in the Global Environment Facility Annual Performance Report. Based on a 2015 review, 100% of UN Environment evaluation reports of Global Environment Facility projects were assessed as being of ‘Moderately Satisfactory’ or better for quality. (see graph below for historical data)
Peer Review of the Evaluation Office
Each biennium the Evaluation Office of the Office of Internal Oversight Services produces evaluation scorecards (2010-2011 , 2012-2013 ) for all UN Secretariat evaluation functions. In 2017 UN Environment’s Evaluation Office was found to be the most productive secretariat evaluation function in terms of evaluation report delivery, producing 52 reports in the biennium assessed compared to a Secretariat average of 12 reports per function.
In an assessment by the Multilateral Organisation Performance Assessment Network (MOPAN), the quality of independent evaluations conducted by UN Environment’s Evaluation Office was rated as good to very good, with appropriate quality assurance systems operating effectively.