Evaluation

There are numerous reasons for undertaking evaluations including to review innovations, gain evidence, demonstrate success or challenges as part of a learning process, assess value for money and to be accountable to key stakeholders such as donors and, especially, to the affected population.  An evaluation looks at the overall changes which can be attributed to a sanitation programme and examines the outcomes achieved, the relevance, efficiency, sustainability and wider impact on people’s lives. It can produce recommendations to improve the programme (including capacity strengthening if needed) and capture learning to inform future policy and practice. It is an important aspect of Accountability. Sharing and using evaluation findings encourages transparency and Learning in the sector.

There are different types of evaluations depending on the objectives. Some evaluations are carried out at, or after, the end of the programme (or mid-way in longer-term programmes) and aim to provide accountability and influence future policy and practice. Real-time evaluations are carried out during the programme, are interactive and involve multiple stakeholders; the evaluator acts as a facilitator to generate an overview of the programme and provide immediate feedback so that issues can be addressed during the response. All types of evaluation can be external and independent or conducted by an agency with the support of an external evaluator or by staff members.

It may be appropriate to do joint evaluations in collaboration with other programme staff, partners and other organisations (e.g. within the WASH Cluster) to minimise duplication of resources. Some evaluations have a strong focus on accountability to the affected population, empowering them to play a key role in carrying out and contributing to the process in order to strengthen ownership of the programme and ensure that they are in a position to make use of the findings.

Key evaluation criteria include:

Relevance: asks whether the programme is doing the right things, e.g. is the sanitation programme meeting the needs according to the context? Does the programme target the right people in terms of geographical areas as well as vulnerabilities to WASH-related health risks?

Effectiveness: analyses whether the programme has achieved its objectives and intended results and examines the factors influencing the achievement of those objectives, e.g. has the sanitation programme achieved its objectives of providing accessible and inclusive toilets and safe faecal sludge management services? To what extent can these changes be attributed to the programme? If intended results did not occur – why not?

Efficiency: measures both quantitative and qualitative outputs in relation to the inputs, e.g. how efficient is the installed treatment technology at reducing pathogens or the organic load in faecal sludge? Does this method make the best use of the resources available? Were there alternative options to improve sanitation conditions?

Impact: examines whether there were significant or lasting changes resulting from the programme and whether they were intended or unintended, positive or negative, e.g. has the goal of the programme been achieved? Have there been any changes in public health? Has the programme made a real difference to the affected population?

Sustainability: evaluates the extent to which the net benefits of the intervention will continue or are likely to continue, e.g. have people been supported to continue using, maintaining and repairing the sanitation facilities? What behaviours have changed as a result of the intervention and how likely are these changes to last? Has local capacity strengthened?

Coherence: considers how well the intervention fits with existing country plans and local priorities (e.g. does the programme align with Government policies) as well as with programmes and interventions of other response agencies?

Participation: examines the level of active community engagement and inclusive involvement of all segments of the affected population in planning, management and decision making of the programme in order to achieve appropriate ownership over the outcomes. It includes aspects such as people’s motivation, capacities to engage and opportunities they are given to participate.

Existing National Standards, Sphere Standards, the Core Humanitarian Standard and the Code of Conduct can be used as references to assess the quality of the programme in conjunction with the programme objectives and indicators.