Ministry of Transport - Evaluation framework

What we did:

The Ministry of Transport engaged Allen + Clarke to produce an evaluation framework for its proposed changes to the road user charges (RUC) system.  The framework included a description of the main changes to the RUC system, an analysis of what the changes were expected to achieve, a set of high level evaluation questions and measures, and advice on feasible and robust methods of evaluation and reporting.  It also included a step by step plan to assist the Ministry to implement the framework.

How we did it:

Allen + Clarke used a Realist Evaluation approach to develop the framework.  This meant that we were able to not only identify the expected outcomes of the RUC changes, but also the 'mechanism' which explained how the outcomes were expected to be achieved (i.e how the system works).

We began by reviewing the main policy changes to the RUC system as well as a number of other relevant documents and research.  We them undertook extensive consultation with Ministry representatives and other relevant government departments to determine their information needs from the evaluation.  We also talked to transport industry representatives, who would be affected by the RUC changes, about what they felt the evaluation should focus on.

The project team then used this data to define overarching evaluation issues and specific evaluation questions, and developed specific information sources, indicators and methods for collecting and interpreting the information.  There was a high level of ongoing interaction between Allen + Clarke and the Ministry throughout the project, to ensure that the framework was feasible and met the Ministry's needs.

Benefit for the client:

Using Allen + Clarke provided the Ministry with specialist evaluation expertise that was not available internally, and offered a fresh perspective on the new road user charges system.  The resultant framework has provided the Ministry with a concise and user-friendly guideline from which to conduct an evaluation of the high profile changes to the system.