1.1 Terms of Reference
1.2 Background to the study
1.3 Development of the study
This study was carried out as a part-time activity over two and a half years, in two phases. The Terms of Reference for Phase 1 were "To compile information, data and reports on a representative selection of soil conservation projects with a view to subsequent evaluation of the effectiveness of the projects in achieving their objectives". The final report of Phase 1 was submitted in January 1987, and reported 121 projects as possibly suitable for further study, of which about half were expected to yield sufficient documentation.
The Terms of Reference for Phase 2 were "On the basis of information gathered during Phase 1, analyse a number of selected soil conservation projects with the objective of identifying those factors which have led to success and failure, and prepare and submit a report covering the findings".
In recent years there has been much analysis and evaluation of aid and technical assistance programmes, spurred by the worry that many programmes and projects appear to be less effective than the expectations at the time of planning. Several major studies have confirmed this, and show that projects in the agricultural sector are particularly weak. For example:
- A World Bank analysis of 994 projects over ten years 1973-1983 showed a failure rate of 14 percent in all sectors, but 26 percent in agriculture.- The Bank analysis of 212 agricultural projects 1979-1983 shows an average of 75 percent successful, with wide geographical variation, from Asia 85 percent to 45 percent in agricultural projects in East Africa (World Bank 1984).
- In an FAO study of 310 projects over 5 years to 1986, 63 percent were assessed as good or satisfactory in the definition of objectives, and 52 percent as good or satisfactory (FAO 1987).
The second point which comes through strongly in many studies is that the reasons for disappointing success rates have now been identified, and the question is how soon agencies can learn from these results and adapt their procedures. Two contrasting changes are worth mentioning.
FAO rated 50 percent of projects as good or satisfactory in the period 1980-1984, improving to 73 percent in 1985-1986. However the World Bank in 1985 said "Failure rates in agriculture continue to rise. Their proportion tripled over 1979-84, from 12 percent to 37 percent, the average for the period being 27 percent. The 1986 figure is 40 percent" (World Bank 1985a).
In his excellent survey of success stories in agricultural development in Africa, Harrison (1987) sums up the position by saying "It is not the purpose of this book to pretend that the battle is well on the way to being won. That is far from the case. By and large the breakthroughs are surrounded and vastly outnumbered by failures. On present trends, disaster will carry the day. But it need not be that way. Our success stories are like seeds. If they are sown widely enough, they can take over the field".
The brief was to study projects with a component of soil conservation and try to identify the reasons for success and failure. However in most projects, soil conservation is one of many components, and frequently a minor one. It was therefore necessary, in order to answer the question about soil conservation projects, to collect and study the literature concerned with broader projects, often covering the whole agricultural sector, and sometimes beyond. These wider studies are useful because although looking at wider issues, e.g. the whole question of aid (Cassen 1986), or soil conservation policy (Morgan and Rickson 1989), the conclusions can also be applied to soil conservation projects.
During the preparation of the FAO "African Agriculture: the next 25 years" study, (FAO 1986) there was a preliminary study of projects in Africa with a soil conservation component. Fourteen projects were subjected to a quantitative assessment, on a five-point scale, of thirteen parameters considered likely to have a bearing on the success of the project. It was decided to use a similar approach for the present study, but using many more parameters divided into:
Before project: concerned with identification, appraisal and design;During project: concerned with implementation and monitoring;
After project: concerned with evaluation and sustainability.
An example of the evaluation sheets is shown in Appendix 3, and the results are discussed in Appendix 1.
The first plan was to make a detailed analysis of a few representative projects, but it soon became clear that projects vary in so many different ways that a small sample cannot be representative. So it was decided to use as large a sample as possible.
At the end of Phase 1, 41 projects were identified as suitable for assessment with sufficient documentation available, and a further 80 projects to be included if further documentation could be obtained. By the end of Phase 2 information had been collected on 133 projects (list in Appendix 4).
It was surprising that there were so many projects which could not be evaluated because there was insufficient documentation available. Eventually about 35 projects were assessed (listed in Appendixes 1 and 2).
During the study it became apparent that evaluation and assessment of assistance projects have become very topical in recent years. In addition to self-analysis by many development agencies, both large and small, there have been several major conferences devoted to the subject, and a shelf-full of books and reports. The distillation of information and ideas from these sources, listed in Appendix 5, has therefore been a major activity throughout the study.
The projects studied covered a wide range of types of assistance, so the terms donor, lender and aid agency have been used, as if they were interchangeable. Similarly the other side may be called the borrower, recipient, host or local institution.
The purpose of the study is to identify factors affecting the success or failure of projects with a significant soil conservation component, but attempts have been made to avoid recrimination for the mistakes of fifteen or twenty years ago because these are mostly recognized and admitted. Neither was it a duty to debate the strengths and weaknesses of different agencies, or the political economy of aid in general. It would be artificial to ignore all the peripheral factors affecting projects (such as the place of NGOs or the use of Food for Work) so some of these are briefly covered in Chapter 6 with references for readers wishing to study them in more detail.
