6.1 Developments in Rice Genomics
6.2 Intellectual Property and Private Sector Linkages Policies
6.3. The Sustainability of The Natural Resource Base
6.4 Impact Assessment
Over the past seven years, IRRI has made a commendable entry into the science of biotechnology, particularly in applications of marker technology and the use of genetic transformation as adjuncts to breeding. The present plans for the continuation of this work, both as described in, and transcending, the MTP and as presented to the EPMR, are evolving alongside developments elsewhere and, in the main, involve extensions of the present strategy by building on results and experience gained to date.
The Panel wishes to raise another issue that could affect IRRI's approach to biotechnology, i.e., the emergence of the new science of genomics. Over the past few years, the first effort to obtain the complete DNA sequence of a model plant has gained momentum and the 100 million or so DNA base pairs that make up the arabidopsis genome will be available to both public and private company researchers by the year 2003 or earlier. This will mean that the sequence of all 25,000 genes of one plant will be known, although only a handful will have been assigned functions. Alongside this public international initiative both public and private organizations, but particularly the latter, have begun to assemble databases of expressed sequence tags (ESTs), which are fragments of individual genes expressed in different plant tissues. Private databases already contain some 200,000 ESTs from maize, for example. This information will be used with the arabidopsis sequence and new 'chip' technology to assign functions to the maize genes as the information becomes available in the model plant. 'Expression chips', already being applied in human genome research, will allow researchers to screen through tens of thousands of genes simultaneously to find those associated with particular plant characteristics or responses to, for example, environmental stress or disease attack. It is not possible to patent an anonymous gene sequence, but it is possible to patent a gene for which a function and application can be defined.
Moreover, the focus of plant genomics has now moved on to rice itself. Of all the major cereal crop species, rice has the smallest genome at about 400 million DNA base pairs. Also, it has become clear over the past three or four years that the genes and their organization in the different grass genomes have remained remarkably conserved, even though they have diverged during 60 million years of evolution. An initiative, led by the Japanese group at Tsukuba, to sequence a japonica rice will begin this year. It is likely that the US and Europe will join this initiative, which may eventually cost US$ 200 million and take less than ten years to complete, because the results will be able to be extrapolated to maize and wheat. With its present plans, IRRI will have no part in this and will be on a par with the hundreds of laboratories in both developed and developing countries with capability in rice molecular biology. The advantages of those organizations with large EST databases are obvious.
When we know the DNA sequence of rice, we will know all of the genes of rice but still not their function. As the present patenting laws stand, there is likely to be an international functional genetics 'gold rush', a window of perhaps ten or twenty years, after which most of the significant genes with application for crop plant modification and improvement will have been discovered.
IRRI does have the required expertise and no doubt will use genomics and large-fragment transformation to approach those genes for which it already has a research lead. IRRI could still decide to collaborate in the international effort directly by scaling up its DNA sequencing capacity. On the other hand, as a collaborator for the mainstream sequences and other laboratories focusing on the functional genetics of rice, IRRI does have a great deal to offer, particularly for laboratories in temperate countries where large-scale rice field trials are not possible or practical. Certainly IRRI should use the comparative advantage of its existing reservoir of knowledge and expertise in all aspects of rice agronomy and breeding to attract collaborators.
The Panel suggests that IRRI could go further and be proactive in assembling tools that will ensure that IRRI is the preferred collaborator for functional genetics research. Several ideas have already been formulated. The establishment of a deletion or transposon-based "gene-machine" is outlined in the CE review in Chapter 4. 'Core collections' developed with GRC's germplasm, an initiative already underway at IRRI, are likely to become vital resources for researchers seeking useful natural allelic variants of genes identified in the sequence of the japonica rice, Nipponbare. The Panel is sure that, if the question is addressed collectively by IRRI scientists, other proposals will emerge that will promote IRRI's advantages. The Panel suggests that incorporation of these ideas into IRRI's research plans will be vital if IRRI is not to become marginalized in this exciting new era of rice science.
IRRI's present position regarding IP is clear. The Policy on IP Rights (PIPR), agreed upon by the Board in October 1994, clearly states that (i) information, inventions, and biological materials will be made freely available, (ii) material held in the germplasm collections and any genes derived from them will be made freely available, and (iii) all breeding materials, including hybrid parents, will be made freely available. The Policy on Partnerships with the Private Sector (PPPS) is equally clear. These policies are designed, in line with the CGIAR's present position, to ensure continued free access by IRRI's partner NARS to the very best germplasm, advanced lines, and technology.
The Panel argues below that, in the light of the changing international attitude toward IP rights, IRRI should re-examine both the PIPR and the PPPS. Indeed, some members of the Panel perceive dangers in continuing to follow the PIPR. The Panel also notes that two CGIAR panels have been convened to consider (i) IP issues (the CGIAR Proprietary Science and Technology Panel) and (ii) industry relationships (the CGIAR Biotechnology Panel), which are both due to report by the end of May 1998. The reports are likely to be most instructive in this respect.
Elsewhere, for example in the US, Europe, and Australia, public sector research organizations are today, in contrast with ten years ago, actively seeking IP protection on their discoveries. Today, these organizations attempt to retain their IP in-house, although, of course, they provide licences as are appropriate, usually depending on the source of funds with which the original research was undertaken. There are several reasons for their new approach: (i) to ensure more rapid transfer of the IP to collaborating industry; (ii) to ensure that the IP is used in the widest and most appropriate manner; (iii) to build up a portfolio which may be used as collateral when the organization needs to employ other already patented technology; and (iv) last but not least, to attempt to build up a future income stream which will recover costs and supplement dwindling public support. Of these reasons, only (iii) may be relevant to IRRI and the NARS today, but, in the Panel's view, IRRI will soon need to examine all the reasons given above for considering IP.
Below we highlight scenarios in four areas of IRRI's science, identified in a meeting between IRRI staff and the EPMR in September 1997, where IP considerations may be relevant. The Panel hopes that these examples will be discussed in a wider context by IRRI, the NARS, and other interested bodies.
The need to protect IRRI's germplasm. DNA manipulation technology is now in expert hands in both the multinational breeding industry and developing country breeding organizations. Moreover, we are now entering into a new phase where the interests of the multinationals are no longer exclusively centred on the temperate crops. Indeed, at least one such company is targeting rice itself and is investing heavily in rice biotechnology research and rice-related IP. The significance of these developments is emphasized by the fact that the private sector now outspends the public sector worldwide in biotechnology R&D. Moreover, the increasing interest in genomics and the assembly of massive EST databases, taken together with the recent decision by the Japanese and others to completely sequence rice and make the data publicly available, is likely only to increase industrial capability and interest in the Asian rice market. The spectre of a top IRRI line being modified in a profitable and politically or ethically unacceptable way to produce a superior product is very real. Para. 4 of the PIPR attempts to prevent this, but IRRI does not produce registered 'varieties', only 'advanced lines' for the recipients to do with as they will. An IR72 derivative finished as a variety by a third party and converted to a protected herbicide-resistant variety for dry season planting, for which IRRI may be criticized by anti-GMO NGOs but receive hone of the profits, is an ugly thought indeed. Trademarking the 'IR' prefix, as in "IR-Phil", "IR-India", etc., which IRRI plans to explore with CORRA, may control the use of the name but probably not the germplasm itself.
Genes isolated from materials held in trust in IRRI's genebank. Today, the situation is clear. IRRI has placed most of the collected germplasm under the auspices of FAO in accordance with the Convention on Biological Diversity and the Agreement between IRRI and FAO. Para. 2 of the PIPR states that IRRI will supply germplasm under a Materials Transfer Agreement (MTA) which is designed to ensure the continued free availability of the germplasm and any genes derived directly it.
If the clause in the FAO agreement, "...nor shall (IRRI) seek any IP rights over that germplasm or related information", does preclude, for all time, IP being taken on genes isolated from the germplasm, then IRRI and the NARS are at risk of being by-passed by potential research collaborators wishing to isolate genes directly or to 'mine' potentially useful alleles of genes identified from the DNA sequence information, because few will be willing to invest in work they cannot eventually protect.
On the other hand, assuming that it is legally possible, why should IRRI and the NARS not isolate and patent key genes from the materials IRRI holds in trust? Indeed, in other circumstances, and in the absence of any specific instructions to the contrary, trustees of others' property are obliged to manage, rather than simply hold, that property. IRRI might, for example, argue that the objective of such a patent is to ensure that a gene is used freely by rice partners but that the same clients benefit by control of the discovery for use in developed nation rice industries and, indeed, for use in other temperate crop species. Xa21, the leaf blight resistance gene discovered by IRRI in an Oryza longistaminata accession originally collected from Mali but held in the Genebank and now isolated and patented by a US university, is a case in point. When the situation arises again, should IRRI and the NARS block the research? Probably not. Should IRRI and the NARS collaborate with the ARI and take out a joint patent to ensure free delivery to the NARS breeders, and possibly an income stream from its use in industrialized nations' agriculture? Probably so. Such arrangements may, in fact, become the preferred route in the immediate future, because the partner is likely to be in a better position to licence and protect the IP.
Novel generic technologies. IRRI's present position is that it will place new inventions in the public domain. However, IRRI has now entered the field of DNA technology where discoveries are often generic. Increasingly, IRRI will wish to use technologies which have been patented by others. Increasingly, as commercial interest in the Asian rice market grows, IRRI may be required to honor those patents. Why, therefore, should not IRRI or the NARS attempt to recover the costs of their own discoveries , or offset the costs of using others' discoveries, by licensing applications in organisms other than rice? A move to protect such discoveries might again also more easily ensure their continued delivery to rice research partners.
Partnerships with industry. It is likely that partnerships with private and public sector organizations will become more common as more and more technology is already held under patent. Moreover, IRRI's present policy of buying IP outright, in a consortium with NARS under "one-off arm's-length" agreements, as with the first Bt constructs to produce insect resistance, may become untenable as the prices rise or where the source organizations are potential competitors with IRRI in rice breeding.
These developments probably mean that IRRI's future must include an even closer relationship with industry in order to have access to the best technologies and the best genetic materials, in order to maintain a supply of the very best germplasm and rice production technology to farmers. There are many reasons why industry will want to collaborate with IRRI, e.g., the expertise and experience of IRRI's scientists, IRRI's research facilities, particularly the field capability, IRRI's established international networks, and, of course, IRRI's international standing and reputation. Above all these is, however, access to IRRI's advanced germplasm. At present, this remains unregistered, unprotected, and freely available to all, NARS and industry alike.
Conclusions
Among the CGIAR-mandated crops, the case of rice in Asia is to some extent special and IRRI should have a clear understanding of the nature and timing of any possible changes in the laws concerning varietal protection (note that in Europe regulations are being put in place to ensure that even farmer-saved seed is liable to plant variety rights (PVR)) in its partner countries. Future laws relating to PVR will have a clear bearing on IRRI's future policies.
It is clear to the Panel that means of approaching these difficulties and opportunities must be identified soon. The issues already recognised by IRRI, i.e., conflicts of interest, arrangements involving exclusivity, public liability, endorsement of commercial products, equitable treatment of private companies, and equal access to research results by all, remain pertinent. The proposed mechanisms for dealing with new relationships individually, involving NARS wherever possible, and using an internal review process all appear apposite. However, the Panel believes that situations will soon arise where the only means of continuing to supply the best rice to NARS will require significant modifications to the present policies if IRRI is not to be marginalized while its research, its technology, and its germplasm are exploited by others to underpin the commercialization of the Asian rice industry.
The Panel recognizes, in the light of the situation elsewhere in the CGIAR System, the legal situation, and the current attitudes in private research organizations and other international public organizations, a need for urgency in re-examining IRRI's IP policy.
The Panel recommends that IRRI, the donors, and the NARS convene to address potential problems related to IRRI's current private sector and intellectual property policies, after the reports of the CGIAR Panels on Biotechnology and Proprietary Science are published.
6.3.1 Soils Characterization
6.3.2 Soils and Yield Decline
6.3.3 Water
6.3.4 The Climate and Atmosphere
The advances in the biological sciences promise great increases in the ability of the rice crop to use the physical resources of the environment in generating yield and quality. The core NRM research on how the crop interacts with soils, water, and the atmosphere must therefore be maintained, and it is important to consider whether this resource base can be improved in terms of its ability to support the rice crop, or if it is fully sustainable.
The first requirement of NRM research is to identify the size and quality of the resources. IRRI is moving into studies on land use, especially in the major river deltas and the associated watersheds. It is important that this be done in a fully quantitative manner, to define the land and soil in relation to their exploitation for rice farming, and for mixed land use in which rice cultivation has a part. From the resulting databases it will be possible to predict yields under rice cultivation, extrapolate from site-specific experiments, and build up information on the likely sustainability of different land uses in different areas.
A particular issue is the loss of land from rice farming to urban, industrial, and commercial uses. Sometimes it is converted to other agricultural uses, such as high-value vegetables, but this appears to happen very largely on the rainfed uplands. We have not been given any data on the loss of high-quality irrigated land to rice cultivation, but such flat fields are attractive to developers, and we suggest that IRRI should ensure that it collects all the available statistics on this, as part of its Ecoregional studies. The ability to deploy crop modelling in predicting the best use of land is important, as it allows all the interacting factors in the environment to be included in the assessment. It is discussed at greater length below.
Soil conditions have been brought into prominence by the term "yield decline'. This subject was considered at length by the 4th IRRI EPMR, in 1992, which concluded that there were unexplained declines in yields of rice, especially at the highest levels of intensive cultivation, and that this was a critical problem for the future. This suggested that such high yields from intensive culture were not sustainable, and the hypotheses to explain this alarming possibility ranged over new diseases, loss of 'soil health' or soil structure, and changes in the microbial cycles in the soil. The Recommendation 3.1 included the statement that"... IRRI lead a major research effort, enlisting the best talents available in the world, to seek solutions for this complex of problems - a task that may take a decade or longer to complete".
The Panel has looked carefully at this subject. There is no doubt that the information available to the 4th EPMR indicated a serious problem, with declining yields on at least 3 of IRRI's flagship long-term field experiments (Cassmann et al. 1995)1, for which a careful study of soil and crop properties provided no explanation. There was also anecdotal evidence from farmers that they were using more fertilizers to achieve the same yields as before (Cassmann and Pingali 1995)2. However, there was insufficient evidence to reach any firm conclusions. The observations of yield decline were most convincing on the long-term experiment on the IRRI Research Farm, and least clear on rice-wheat experiments in India.
1 Cassmann K.G., De Datta S.K., Olk D.C., Alcantara J., Samson M., Descalsota J. and Dizon M. (1995) Yield decline and the nitrogen economy of long-term experiments on continuous irrigated rice systems in the tropics. Advances in Soil Science, pp. 181-222.2 Cassmann K.G. and Pingali P.L. (1995) Extrapolating trends from long-term experiments to farmers' fields: the case of irrigated rice systems in Asia. In: Agricultural sustainability in economic, environmental and statistical terms (eds. V. Barnett, R. Payne, & R. Steiner) pp. 64-84. London: Wiley and Sons.
More information from the last five years is now available. Firstly, no evidence has been found that intensive farming yields are declining in general, so long as inputs are sufficient. Whereas the accuracy of the observation is clearly less in farmers' fields than on a long-term experiment, it seems unlikely that a widespread decline of the magnitude seen in the IRRI long-term experiment on the IRRI farm, of 33% over 25 years, could have gone unnoticed. Experiments were established by IRRI in 1993 in about 200 farmers' fields to obtain more accurate information about the use of inputs and the productivity that these maintained, but it is too early to draw conclusions from these yet. No evidence has been found that any single pest or disease is responsible. However, it has been found that the decline in yield from the previously high levels can often be reversed if the nitrogen supply is increased, and at the same time is managed more carefully in accordance with the increasing demand of the growing crop.
This suggests that the cause lies in a steadily decreasing rate of supply of natural nitrogen from the soil (Cassmann et al. 1995). With hindsight, this possibility could have been tested much more conclusively if the long-term experiments had been planned with more and larger rates of nitrogen and better timing of applications, and this needs to be borne in mind in the planning of the new generation of long-term experiments, which it is now essential to lay down. The more detailed studies made on the long-term continuous cropping experiment at IRRI suggest that there is a continuing decline in the annual rate of supply of natural nitrogen from the soil of some 60 kg/ha over a period of 22 years. There were also changes in the increase in grain yield per unit amount of nitrogen, but it is not clear whether this was due to soil changes or to changes in the agronomic treatments. The soil scientists at IRRI have told the Panel that these changes in the natural soil nitrogen relationships, and possible changes in the efficiency of uptake of N by the rice plants, are the best current hypothesis for the cause of yield decline, and the Panel concurs in this view. If this occurs in farmers' fields, it could have been masked by the steady increase in the rates of nitrogen used over this period.
The decline in the rate of supply of natural nitrogen within long-term trials is not new. The usual cause is that the type of land use has changed when the experiment was laid down, and the soil nitrogen and carbon dynamics slowly adjust to the new situation - in temperate soils, this may take up to a century. This may be called a decline in the quality of the soil, but it is perfectly natural, and is compensated by supplying more fertilizer in appropriate ways. It can be interpreted as a reduction in 'factor productivity', but this economic concept has little relation to the biophysical processes. This general process in anaerobic soils appears to be different from that in drained soils, where it usually is linked to a simple decrease in soil organic matter. In this case, the total C and N levels remain constant, or even increase where high N fertilizer rates are applied.
It does, however, emphasize the point that increasing levels of nitrogen will be used for rice as the yield frontier expands, and that land use is likely to change more as new farming systems develop. Nitrogen is more difficult to manage in anaerobic soils than in aerobic ones, and the efficiency of use of nitrogen under wetland rice conditions is substantially lower than that of nitrogen for high-yielding cereal crops in aerobic temperate soils. The causes are probably known in general (e.g., hydrolysis of urea and loss of ammonia as the pH shifts with the bicarbonate level; denitrification; the formation of resistant nitrogen/carbon polymers under anaerobic conditions), but the integrated understanding of the carbon-nitrogen dynamics in flooded soils, as expressed in mathematical models of the microbial processes, is far behind that of the drained agricultural soils. This situation needs to be remedied urgently as the intensification of rice cultivation continues. Apart from the nitrogen supply, this impinges upon the phosphorus dynamics of the soil, the organic matter level and the soil structure, and the behavior with agrochemicals. The Panel acknowledges the progress that IRRI has made in defining the factors required for yield sustainability in experiments and on the farm. It believes that IRRI should increase its efforts to improve the basic understanding of flooded soils and of the root systems of plants growing in them, thereby meeting the recommendation of the 4th EPMR, by improving the focus, strategic planning, and resources invested in current programmes.
The Panel recommends that a strategic programme on soil carbon and nitrogen dynamics in flooded soils should be designed and mounted in collaboration with appropriate centres of excellence.
The water supply is fundamental to rice, as a wetland plant. The supply of water for rice is frequently stated to be declining, but the Panel is not sure what this means. The primary demand for industry and domestic use is obviously growing rapidly in the developing Asian economies, but a great deal of this water is recycled back into rivers or seeps down to the groundwater. If the quality of this water is maintained, the net loss is small, and this needs to be taken into account, rather than quoting the gross abstraction rate of water. The use that actually loses most water is clearly agriculture, because evapotranspired water cannot be used again. We suggest that IRRI should ensure that complete statistics on such real use of water are gathered within the Systemwide Initiative on Water Management.
Whatever the result of this, the supply of water will shrink rather than grow, and water economy is a very important aim of IRRI's AAPA and SWS Divisions. As stated in Chapter 4, the Panel is impressed with the work on water management in Vietnam, and such studies should be expanded to other countries. This also suggests added support for direct seeding, which makes great savings in irrigation water. The Panel believes that IRRI should look for innovative solutions in this topic, in an attempt to bring the transpiration ratio for rice closer to those of dryland crops.
It is well known that the 1990s are likely to be the warmest decade on record in global terms, and the Intergovernmental Panel on Climate Change has cautiously indicated that this change is probably due to anthropogenic causes. IRRI and others have produced several crop models on rice growth, whose predictions agree reasonably with each other. These suggest that the predicted increase in temperature over the next few decades will on average increase rice yields in the subtropical zone but decrease them in the tropical zone, where most poor rice farmers live. Crop modelling is becoming an essential integrating discipline in soils, water, and atmospheric research. IRRI needs to watch the situation carefully in relation to its plant breeding work, and the Panel encourages it to ensure that it remains part of the very active global crop-modelling community so as to retain the capability to model the response of rice to these and other changes.
As opposed to these still uncertain climatic predictions, the carbon dioxide in the atmosphere will almost certainly continue to increase; the current level is already about 70 ppm above that in pre-industrial times. The consequences for rice cultivation seem almost wholly benign. . Work on field crops other than rice has shown a direct increase in rate of growth because of more rapid photosynthesis (at least in C3 plants), and a general increase in the ratio between net photosynthesis and loss of water by transpiration. This should therefore assist the general upward movement in yield levels caused by plant breeding and better agronomy and help to mitigate any shortage of water. This change must be borne in mind in the crop modelling studies.
6.4.1 Introduction
6.4.2 Evolution and Current Focus
6.4.3 Methodology
6.4.4 Designing Ex-ante Impact Assessment Methodology - Problems and Limitations
6.4.5 Limitations of Applying the Model to Research
6.4.6 Conclusion
IRRI is a major research organization having numerous research programmes competing for resources and it has to allocate resources to each of these after their careful appraisal including an assessment of their likely impact. The given goals must be achieved in an economically efficient manner, and for public sector research organizations like IRRI, this means both ensuring that resources are used so as to minimize the cost of producing a given research output (cost efficiency) and ensuring that the research outputs are of maximum value to the organization's clientele (allocative efficiency).
Thirty-five years after its setting up, it is important for IRRI to build a system that attempts to assess its total contribution, along with that of its individual programmes, in relation to the costs incurred. More specifically, there is a need to develop more transparent priority-setting mechanisms through development of proper methodologies for impact assessment of its research programmes.
IRRI has been concerned with impact assessment since its early days. There are two main types of impact assessment studies. The first group assesses ex-post the impact of new technologies on increase in output, on cost of production and prices of rice, and the extent to which the goal of improving food security and welfare of rice producers and consumers has been achieved.
Secondly, ex-ante impact studies are needed to help planning, through prioritization of objectives and allocation of research resources across programmes and projects. These studies attempt to assess the likely future impact of current or planned research activities on new technologies, on likely increases in yield and output, and on efficiency in the organization of rice production and the management of rural resources.
IRRI scientists have for a long time worked to develop appropriate methodologies for undertaking ex-post and ex-ante research assessment at IRRI, keeping in view the main characteristic of IRRI, namely, its focus on single commodity research. These methodologies are briefly discussed below.
Ex-post studies. The methodology developed by IRRI scientists for undertaking ex-post impact assessment studies has changed over time. Initially, these studies used an economic surplus model that tried to define the market price effects of new technology on demand and supply and on income distribution. The methodology now being used in impact studies is based on process and outcome evaluation through in-depth household surveys. Since 1986, IRRI in collaboration with NARS has been engaged in generating household-level data on economic activities for process and outcome evaluation. Several major studies have been published based on this methodology, the latest being that by David and Otsuka (19943) which contains the results of a Differential Impact Study (DIS) Project jointly conducted by IRRI and NARS colleagues in China, India, Bangladesh, Nepal, Thailand, Indonesia, and The Philippines, on the basis of agreed research methodology.
3 David, C.C. and K. Otsuka (eds.), 1994, Modern Rice Technology and Income Distribution in Asia, Lyn Rienner Publishers, Boulder and London.
In 1994, IRRI launched a project to resurvey these villages with the same methodology to generate two point data for further assessment of impact of new technology at the household level. In addition, IRRI, as the participant in the CGIAR Impact Assessment and Evaluation Group (IAEG), is assessing ex-post the impact of mechanical technologies. Also in 1996, IRRI held an international workshop on ex-post impact of rice research, in collaboration with the Thailand Development Research Institute. The proceedings are being published as a book 'Impact of Rice Science and Production'.
Ex-ante studies. IRRI scientists, in collaboration with the Rockefeller Foundation and Yale University, have developed a methodology for prioritization of research problem areas by analyzing perceptions of knowledgeable rice farmers, extension workers, and researchers. Relevant data on yield gaps and yield losses due to various constraints are being generated in collaboration with NARS social scientists, with a view to prioritizing research problems and determining the need for further research on rice. IRRI has been regularly holding bilateral research planning workshops and dialogues in which findings of the ex ante evaluation studies are used. Over the last two years, IRRI and NARS have jointly organized India-IRRI, Bangladesh-IRRI, China-IRRI and Pakistan-IRRI research planning dialogues. In each, research managers and policymakers were invited to review the achievements of past collaboration and develop future research programmes. For future ex-ante impact assessment studies, it is proposed to include assessment of the capacity of NARS to undertake research in the priority areas, delineation of the comparative strengths of NARS and IRRI and identification of areas for research collaboration, and assessment of the training needs of NARS.
IRRI scientists in collaboration with other scholars 4 have been engaged in refining the methodology for impact assessment. Much of the methodology is derived from project evaluation in terms of benefits and costs. However, it has been modified to take into account numerous peculiarities of research output. Six steps are involved in this evaluation. These are: (i) specification of the project in terms of objectives, techniques, inputs, costs, and time; (ii) estimation of the benefit stream associated with the project over its life span; (iii) use of ex-post evidence of research impact for ex-ante priority setting; (iv) using subjective probability estimates of likely benefits and costs; (v) using modifiers to revise the estimates; and (vi) discounting the annual streams of benefits and costs by using an appropriate discount rate, or alternatively finding the internal rate of return (IRR) that makes the net value of annual streams of net benefits (benefits minus costs) equal to zero, and calculating the benefit-cost ratio or comparing the IRR with the prevailing rate of interest.
4 See Evanson, R.E; R.W. Herdt and M. Hossain, (eds) 1996, Rice Research in Asia: Progress and Priorities, IRRI, CAB International, Wallingford.
There are serious problems of a generic nature, with respect to each of the above points when using this method to assess the impact of scientific research. Firstly, it is difficult to estimate with sufficient precision the time span to fruition of research and the duration of the flow of benefits from it. Secondly, for research projects, there are serious problems in measurement of both benefits and costs. Unlike many other projects, such as building a bridge, estimates of benefits are intangible and uncertain and cannot by definition be made on the basis of past ex-post experience. This is also because technologies do not stay constant themselves, but evolve over time, leading to qualitatively different outcomes, so that benefits and costs of earlier project may not be relevant. This is an obvious limitation of use of data from ex-post studies. However, the reliability of ex-post information improves as the unit of operation that is the research programme area becomes bigger, and the more aggregated it is, the more reliable is ex-post evidence. For some broad purposes such as comparing rice research priorities for broad ecoregions or with other research programme areas, ex-post evidence for research impacts from statistical studies may be the best guide. Ex-post studies clearly show that irrigated rice research programmes (in the aggregate) have had high impacts and few, if any, upland rice research programmes have shown impacts. There is every likelihood that unless better technologies (germplasm) for upland rice become available, the experience reflected in the ex-post evidence will continue.
The third problem is the need to clearly define and delineate research programme areas as basic units for priority setting. These areas should be consistent with currently perceived research problems. They should have clear and non-conflicting research objectives; have adequate economic and statistical data; an ecosystem dimension; and a technique (germplasm) or research tool dimension.
Fourthly, the research output is not at all certain. In many cases, it is possible to treat this as uncertainty, and some form of uncertainty can be treated as random elements and expected values can be based on past experience. But for most research projects, some form of subjective estimation and informed judgment about the benefits from the project is required. Moreover, for ex-ante assessment of new research projects, using past evidence is not generally possible because of the novelty of research.
Fifthly, because of the reasons discussed above, resort might have to be made to subjective estimates of benefits given by specialists. This method is called subjective probability estimates. The scientists working in the area are the best possible judges of, say, germplasm potential and they are in the best position to observe its performance in the field. There are four procedures to obtaining these probability estimates: (i) ranking techniques, (ii) scoring or scaling techniques, (iii) single point probability estimates, and (iv) subjective probability distribution.
Finally, even the subjective estimates suggested above may have to be supplemented through appropriate adjustment factors to take care of some important equity considerations such as: sustainability, unfavorable ecosystem properties, equity, weak NARS programmes, and environmental effects.
IRRI scientists are aware of these limitations and have been trying to improve the accuracy of estimates of benefits of research activities by quantifying yield gaps and yield losses due to various biotic and abiotic constraints and the probability of research success. Although initial estimates of benefits were based on studying perceptions of rice scientists, recent studies aim at generating the information by interviewing knowledgeable farmers and extension workers using rapid rural appraisal methods. An IRRI pathologist has been collaborating with NARS pest scientists to develop a database on pest losses, to further refine these estimates.
The subjective probability estimates for specific research problem areas and techniques are being built up by interviewing researchers from NARS and IRRI, and also scientists in advanced research institutions in developed countries engaged in biotechnology research. A questionnaire was administered to scientists participating in the Rockefeller Foundation's rice biotechnology meetings held in Bali, Indonesia in 1995 and in Malacca, Malaysia in 1997 to solicit this information. The above steps are in the right direction to improve the quality of ex-ante impact studies.
IRRI scientists are quite strong in undertaking ex-post impact assessment research studies. Most of the work done at IRRI is universally acknowledged as high quality in terms of methodology, analysis of data, and its interpretation. A beginning has been made in conceptualizing ex-ante impact assessment and a modified cost-benefit evaluation methodology has been developed. It is a correct strategy to involve NARS as collaborators in this type of research. However, though this has helped IRRI in its prioritization for research resource allocation, IRRI or the NARS have been able to provide only limited guidance to national governments for their planning and prioritization of research programmes. Further, there are three important gaps in the ex-ante research methodology which IRRI should address. These are the need: (i) to test the validity of ex-ante assessment of work projects by undertaking ex-post evaluation; (ii) to identify 'flag posts' representing intermediate targets in terms of achievements and expenditures during project implementation; and (iii) to identify impact pathways that are likely to operate in converting research outputs into desired outcomes and impact.
Ex-ante impact assessment research is extremely important, and the Panel suggests that IRRI should give high priority to this subject, including the refinement of existing methodology with a view to reducing subjectivity in assessments for priority setting, and identifying impact pathways.
