| |
Contributions from
the Column
Studies and reports
Not enough drugs
Measuring aid impacts
Caused by ill-conceived political decisions
Preventive strikes ‘yes’, unilateral action ‘no’
Half the world’s workers
living below poverty line
01/2005 |
|
[ Evaluation ]
Measuring aid impacts
Evaluation experts have been debating for a long time the best method for determining the impacts of development aid on a particular target group or even on a society as a whole. Evaluations usually focus on the outputs of a project and on their immediate benefits to the target group.
Susanne Neubert from the German Development Institute (GDI) contends, however, that this is no longer enough, the more that development cooperation takes place as programme or budget support rather than in the form of individual projects. It is true that cross-project analyses and final evaluations by the German Federal Ministry for Economic Cooperation and Development and the implementation organisations also look for long-term impacts. But according to Neubert, this occurs neither systematically nor according to standard methods. In a paper for the GDI, Neubert proposes a new method for impact analysis.
According to Neubert, impact evaluators must first accept that it is not possible to produce quantifiable results in impact analyses. There are far too few comparable cases and, on the other hand, the effect of other variables is far too great, to be able to measure the impact of development interventions in mathematically precise terms. Impact analyses must therefore be based on qualitative methods.
Moreover, she says, the current practice of determining impacts on the basis of standard hypothetical impact chains has to be changed. This method prejudices the result, because it limits the evaluators’ focus at the outset to the impacts assumed ex ante. Unexpected or undesired impacts are thus ignored. “However, awareness of these surprising impacts is fundamentally important for the potential use of analysis results, error adjustment and the formulation of best practices,” writes Neubert.
Usually, impact analyses start with looking at the development intervention. Neubert, an agricultural economist, advocates a different method. She proposes, firstly, to investigate the situation in the programme or project region (e.g. in relation to poverty) and to identify trends. The second stage in her method is to produce a list of the development measures of various agencies in the project area and to arrange the measures according to their relevance for selected criteria (e.g. income, school attendance or access to water) and according to target groups.
In the decisive third stage, living conditions and development trends in the project area are correlated with the development measures. In Neubert’s view, participatory procedures should be used wherever possible here. The population should be involved as widely as possible, for example, in workshops to which various interest groups are invited. Individual voices which could distort the evaluation would thus be put into perspective.
At the end of the process, the evaluators assess the influence of the projects according to the chosen criteria, and award points. A matrix is thus created which, according to Neubert, shows directly the impact of the individual programmes and projects on specific criteria such as the income situation or school attendance levels as well as on the overall living conditions of a population. At the same time, the joint impact which all projects concerned have on individual spheres of life can be determined. In Neubert’s view, this method would allow it to evaluate Germany’s overall commitment to development in a partner country.
Tillmann Elliesen
Website:
http://www.die-gdi.de
|
