9.4 Some issues in planning trial work

Contents - Previous - Next

Before looking at the possible trial designs, five important points that should be considered when planning trials:

• The Need to Specify Trial Objectives. Clearly defining trial objectives is perhaps the single most important activity in planning a trial. The trial objectives are statements explaining what the trial is supposed to achieve. Decisions regarding the trial design (i.e., the format, that is RMRI, RMFI, or FMFI, the treatments to be included, the data that must be collected, etc.) depend entirely upon the objectives of the trial. For this reason, before starting any trial activity, the researchers and other interested/involved parties must agree on the exact objectives of the trial, and these must be written down, Only after that can the trial be designed and implemented. Beginning a trial without a clear idea of its purpose and exactly what questions need to be answered can result in a lot of wasted effort.
• The Need to Choose an Appropriate Format to Fit Trial Objectives. Once the trial objectives have been defined clearly, the trial design can be formulated. To decide whether a trial should be RMRI, RMFI, or FMFI, researchers will need to consider the type of information required from the trial. If only technical data are required -- such as stand establishment, or grain yield -- the trial probably can be implemented as RMRI. However, if much farmer assessment is required, or if the farmer's own resources need to be involved, then the trial will need to be implemented as either RMFI or FMFI. The choice between RMFI and FMFI will depend largely on the complexity of the trial design, the number of treatments and replications per location, and the type of data required. The more complex the trial and the more technical data are required, the more researcher management will be needed.
• The Need to Limit Research Topics to Fit Research Resources. Researcher-managed trials (i.e., RMRI or RMFI) require a good deal of attention from senior researchers. Aside from being present during critical implementation stages, researchers must make regular field visits to each trial site throughout the season. In addition, much time is required to: plan the trials; collect the necessary inputs; and arrange for tillage, planting, weeding, spraying, and other activities, Trials also require training and supervision of the technicians and often involve a lot of discussions with farmers, Where research teams share resources -- for example, vehicles or technicians that perform work for agronomists, economists, and animal scientists at the same time -- care must also be taken to ensure that individual researchers do not monopolize the resources to the detriment of other disciplines (see Section 6.2),

There are always more problems than researchers have time or resources to address. However, because of all the above requirements for researcher-managed trials, researchers must be very careful in planning their work load for the season. Team members should work together to set research priorities and carefully consider the amount of time and resources that will be required for each planned activity, Taking on too much work will result either in some activities being dropped during the season or some work being lost due to insufficient attention, and the quality of the research generally will be reduced. Taking on less work than is possible will result in a waste of resources, Careful planning and experience will be the best tools for developing optimum work loads.

In contrast to researcher-managed trials, FMFI trials require much less of the researchers' time during the season. Under this format, researchers are involved in planning the trials with the farmers, perhaps in collecting and supplying the necessary external inputs for the trials, and in monitoring the progress of the trials. But this is much less work per trial than is required for day-to-day management of RMFI trials. To reap the full benefit from FMFI trials, it is necessary for researchers and farmers to have frequent discussions, which can be handled efficiently through regular group meetings (see Section 9.8.6). Because farmers are providing the day-to-day management of the trials, researchers can visit the trials at their convenience, rather than having to visit at specific critical periods, This also helps researchers to spread out their work load and improve efficiency,

• Careful Consideration of Experimental versus Non-Experimental Variables. The design stage produces potentially appropriate sets of improved practices for testing at the farm level, The following procedure is recommended for making decisions about the experimental and non-experimental variables in RMRI trials and, when necessary, in RMFI trials that constitute part of the testing stage:

• The experimental variables should consist of practices for which farmers, management is flexible or of new practices that ex ante evaluation suggests would increase productivity. Flexibility in management is increased when there are under-utilized resources? whereas increasing productivity of variables is vital to breaking constraints (Section 6.4.2).
• The feasible range of treatments for such variables is set by the flexibility that exists. For example, some flexibility could be introduced by assuming that the policy/ support system could change. Then, for example, it could be assumed that an institutional source of credit could be made available to supplement the cash flow of the farm business. The above remarks suggest that development of improved practices usually should consider the existing or definitely expected policy/support system. Therefore, results from cause-effect type research will be more relevant if researchers include in the levels of their experimental variables those that farmers might actually be able to implement. If all levels of input required are too high for the farmers to adopt, then the research may have relatively little relevance without the aid of special support programmes for farmers. This applies not only to external inputs like improved seed or fertilizer, but also to internal inputs such as household labour availability.
• Non-experimental variables should be set at levels representative of local farm practice and/or should be implemented by farmers themselves. The failure to take into account this interaction between farmers' practices and recommendations is a neglected and crucial reason for poor adoption. For example, seed beds are often better in RMRI variety trials undertaken by breeders. Varietal testing under such conditions can provide very different results from what would occur if the seed bed preparation more nearly approximated that generally used by farmers.
• Therefore, it is important to evaluate ex ante whether the levels of the non experimental variables are likely to influence the relationships being examined between the experimental variables, Special justification should be made if the levels of the non-experimental variables differ significantly from what the farmer is likely to be able to achieve, For example, in an RMRI tillage trial, the decision might be made to keep the treatments as weed-free as possible so that they do not complicate an analysis of the differences between the tillage treatments. It is important to recognize, however, that farmers may not be able to create a weed-free environment. Therefore, it may be appropriate to record measurements of the time required in each treatment to keep the plot weed-free. Such information could be useful in later evaluation of the appropriate tillage treatment(s) to use in trials involving farmers. In this connection, such information could be given to farmers to help in their assessment of which treatments might be appropriate.

• Inclusion of Control Plots. It is important to remember that researchers and farmers need to compare treatments with something. Thus, in all types of trials, some form of control, which 'copies? the farmer's usual levels of the experimental variables, is generally used for comparison. It is amazing how often control plots are omitted (see Box 9.4).

Many farmers might feel confident that they can evaluate new options on the basis of their experience, even when a control or reference is not physically nearby for visual comparison. This is, in fact, the situation with testing new prototype equipment (see Section 10.4.1 ). This approach is generally not advisable for evaluating most cropping and livestock management options. Two problems that can unknowingly arise are:

• Bias in the thinking of the evaluators may not be recognized,
• The evaluators (i.e., the farmers implementing the trial, visiting farmers, and researchers) may not ascertain correctly all the circumstances that surround a particular response,

Both of these can lead readily to evaluations that are better than or worse than what would be the case if controls were included correctly.

To achieve systematic and critical assessments in FSD programmes in the farmer's own design of management strategies, it is strongly advised to conduct test options using controls and a systematic unbiased treatment of each, This latter implies use of randomization, even in the farmer's own testing.

• Plot Design. Because of a number of different constraints, FSD researchers and participating farmers often have adopted the use of field-long and management-friendly plots. These plots are likely to cut across considerable amounts of variability in the field (e.g., soil, hydrology, etc.). In this case, the plot design tactic is to uniformly incorporate field variability into every plot instead of the conventional tactic of carving out portions of land that are homogeneous for making comparisons. Long, management-friendly plots first found their way into on-farm work out of necessity. Using these plots was the only way to interest many farmers and also proved the best way to incorporate farm operations in the trial in a realistic manner. It is now becoming apparent that long management plots also can provide statistical analyses equally as powerful as conventional, small, homogeneous plots (see Box 9.5).
• Careful Attention to Recording Appropriate Data. Although the following discussion concentrates on trial designs, it is important to remember that implementation requires that good records and data are collected either in field books or on specially designed data collection sheets.

This data usually will be largely technically oriented in RMRI and, to a lesser extent, in RMFI trials, There are many methods for collecting such data (i.e., both input and output data) corresponding to the array of enterprises, operations, and subject area concerns, etc., that are found in farming systems around the world. Sometimes data collection procedures that would be appropriate for local FSD work can be found in local extension bulletins or research reports, Training manuals produced by the international agricultural research institutes and by NGOs and training institutions in the country where the FSD team(s) is located also can often be useful, This is because methods often need to be adjusted to fit particular situations and farming systems. For example, FSD teams helped produced guidelines for data collection in Botswana. These are available in Worman et al. 11992: pp. 149-202], Included are procedures for trial site selection, calculations of seeding and fertilizer rates, soil characterization, and so forth. Two excerpts of the Botswana work, covering measurements of plant density and grain yields, are included as Appendix A3 in this manual, They have been included because they treat the topic of measurements in mixed cropping that generally is not covered elsewhere, while at the same time illustrating the need to make adjustments in the data collection procedure to fit the environment in which the research is being undertaken.

An important component of this data collection -- and, unfortunately, it is something that often is omitted -- is recording of data that describes the environment under which the trial was undertaken, These site descripters include data on soil type, weather (i.e., temperature and rainfall), unusual features, etc., which help in ascertaining the potential for extrapolation to other areas.

However, it is important particularly in RMFI and FMFI trials, that such data be supplemented by socio-economic information. If this is not done, and often it is not done satisfactorily, the trials are really simply technically oriented, multilocational-type trials and the major objective behind involving the farmer -- that is assessment of relevancy under practical farming conditions -- is lost. With reference to socioeconomic type data, attitudinal or more qualitative-type data can be very useful, Because of costs, however, care must be taken in deciding what objective or quantitative types of data need to be collected (see Section 6.5.1 ). Usually such types of data relate to physical data (i.e.' usually technical data already collected); labour and equipment requirements by type and operation; and price data relating to inputs, outputs, and wages [Worman et al, 1992: pp. 187-193, 200-201]. Collection of such date can be particularly time consuming. Therefore, sometimes it is possible to use standard labour coefficients for some operations that don't change between operations and only collect labour data on operations that are likely to be affected by the operation. In other cases, it is appropriate to collect socio-economic data on only a subset of all trial sites.

• Decision Concerning Evaluation Procedure. In order to ensure an appropriate experimental design and data collection procedure, the evaluation procedure should be decided on before the trial is implemented.

BOX 9.4: FARMERS CAN BE RELUCTANT TO HAVE CONTROL PLOTS

The use of controls or check plots is important to researchers in providing standards against which experimental treatments can be compared. A recent survey of farmers in Kansas, USA, concerning their attitudes about on-farm research [Freyenberger et al, 19941, which incidentally were very positive, indicated that farmers tended not to be very concerned about controls, perhaps because of familiarity with their own farm and the fact that they felt it was necessary only to convince themselves of the value of the results. Only 36% of the farmers implementing their own on-farm research had controls likely to be acceptable to researchers, with those farmers particularly interested in sustainable agriculture practices being the least supportive of this strategy (i.e., only 28%). In fact, 25% of all the farmers and 35% of the 'sustainable agriculture' farmers used only a before- and-after comparison.

The implication of these findings is that compromises will be needed on both sides, if effective, collaborative, working relationships are going to develop between farmers and FSD researchers and if farmers are to be able to provide an effective means for other (i.e., visiting) farmers to compare the proposed change with what is currently done. This latter point is particularly important, because the survey also revealed that farmer-to-farmer communication and interaction were very important sources of information on possible changes.

BOX 9.5 LONG NARROW PLOTS MAKE GOOD PRACTICAL AND STATISTICAL SENSE

Farmers participating in an on-farm research programme in the state of Iowa, USA (i.e., Practical Farmers of lowa, PFI) opted for narrow plots that ran the entire length of the field. Usually, the width of these plots was about two widths of the widest piece of equipment used, Farmers found that using this plot design enabled them to make side-by-side comparisons of a number of treatments and even to randomize treatments much more easily than they could using small 'garden plots, typical of most research work.

These management-friendly plots not only improved the quality of implementation of on-farm trials, they also boosted their statistical power [Thompson and Thompson, 1990], Statistical power, as shown by low percent coefficients of variation (CVs), was found to be actually greater for on-farm trials with long plots that cut across large amounts of field variability than for shorter on-farm plots or for small station-plots.

For measurements on corn, soybean, and wheat yields in a range of experiments, CVs ranged from:

• 0.7-5,9% for plots 6m wide by 365m in length in which the entire plot was planted and harvested by the farmer.
• 4.5-15.2% for 12m by 38m plots managed by researchers but conducted on farms,
• 8-15% for small station-plots that were planted and harvested by hired help,

A related study showed that costs and information returns for farmer managed research was superior to those of station-based research in these subject areas.

This implies that FSD personnel should not conclude that on-farm work, though relevant, is always technically inferior to station managed research,

9.5 Trial designs appropriate to crop research

A number of possible trial designs can be used in FSD work, especially for those relating to crop research. For convenience in presentation, they can be divided into two major groups as follows:

• Those that involve replication of treatments within fields. These are most suitable for RMRI trials, whereas the simpler designs can be used in RMFI trials.
• Those that usually have no replication of treatments within fields. These very simple designs are used in FMFI trials.

The following sections briefly outline the different types of trial designs.

9.5.1 Trial Designs Involving Replication within Fields

Some experiments require accurate comparisons of several quite specific treatments. For example, a tillage trial might involve comparisons of several different tillage options, or a fertilizer trial might involve examining the effects of several different levels of fertilizer application, of both nitrogen and phosphate fertilizer. For the comparisons to be meaningful, the treatments must be applied in a particular way and at very specific times on each plot. To be sure of detecting important differences between the treatments, it is usually necessary for this type of trial to be replicated both within a farm and across several farms and often across years as well. Because of the need for precise treatment applications and replications, these types of trials generally require researcher management either RMRI or RMFI. For these types of trials, the use of simple, formal, trial designs is quite appropriate:

• Completely Randomized Design (CRD). The CRD is one of the simplest trial designs. When using the CRD with a field trial, the number of treatments and the number of replications per treatment are decided first. Then the appropriate number of plots are drawn, and the treatments are assigned randomly to all plots. This usually is done with the help of a random number table.

This type of design is appropriate only for experiments where plots cannot be grouped meaningfully. An example would be where all plots are on the same soil type, the soil is of equal depth in all plots, and the field has very little or no slope. The CRD is the simplest design in terms of the analysis of the data, especially if some treatments are lost. Less information is lost with missing data in the CRD than with other designs. The disadvantage of the CRD is that it is not very precise, because all of the variation between experimental units is included in the experimental error term,

• The Randomized Complete Block Design (RCBD). The RCBD is used where experimental units -- that is, plots in field trials -- can be grouped meaningfully. For example, plots higher up a slope may be grouped together, and plots lower down the slope may be grouped together. Or, if soil depth varies from one end of an experimental area to the other, plots on the end with shallower soil may be grouped together, and plots on the end with deeper soils may be grouped together, These 'groups' generally are referred to as 'blocks', In the RCBD, because each block contains a complete set of treatments, these blocks also can be called replications or 'reps' for short, The reason for grouping the treatments this way is so that differences observed between treatments are more likely to be due to the treatments themselves, rather than the differences between experimental units,

The RCBD is one of the most commonly used designs for field trials. This is because differences exist in most fields, and using 'blocks' helps to improve the precision of the experiment, It is also relatively simple to analyze, so long as one can avoid missing data points. It should be noted, however, that when using the RCBD, the researcher must have some idea of what field differences he or she is blocking against and lay out the trial accordingly.

• Split Plot Designs. A split plot refers to a trial plot that receives a certain treatment and then is 'split' or divided into sub-sections, to which different levels of another treatment are applied. For example, suppose an experiment is designed to examine the effect of irrigation on sorghum grain yields. Three treatments are included: no irrigation, moderate irrigation, and heavy irrigation, Then suppose the researchers also wish to examine the effects of irrigation, with and without nitrogen fertilizer. Each of the irrigation treatment plots is 'split'. One half of each plot receives the nitrogen treatment, and the other half does not. In each replication, then, there will be one non-irrigated plot, one plot that receives moderate irrigation, and one with heavy irrigation. Additionally half of the area that receives no irrigation will receive nitrogen; the other half will not. Half of the plot that receives moderate irrigation will receive the nitrogen treatment, and the other half will not, Half of the plot that receives heavy irrigation will also receive the nitrogen treatment, and the other half will not. The irrigation treatments have been 'split' for different levels of nitrogen.

Split plot designs are used commonly when:

• Large experimental areas are required for one treatment but not for others (e.g., irrigation treatments on different crop varieties).
• Researchers wish to include an additional factor in an experiment to increase the scope of the experiment (e.g., adding a fertilizer treatment to a trial involving several tillage treatments).
• Larger differences are expected from one factor than the others (e.g., irrigation treatments on different sorghum varieties).

A split plot experiment is often a convenient approach for implementing research that involves the comparison of different levels of two or more factors. However, the designs and the analytic procedures are slightly more complicated than the CRD or RCBD.

In RMRI type trials, any of the above designs can be used satisfactorily in on-farm trials. However, in the case of RMFI trials, it is probably best to restrict trial designs to an RCBD or CRD, to restrict replications to two or a maximum of three per farm, and to restrict treatments to a maximum of four or five. Remember, the more complicated the trial is, the more difficult it will be to obtain 'qualitative' farmer assessment of the different treatments. The idea of seeking farmer assessment as early as possible in the research process, as argued earlier (Section 9,2), can be important to avoid wasting research resources and time on developing technologies that will never be, or are unlikely to be, acceptable to farmers.

9.5.2 Trial Designs Usually Not Replicated within Fields

In these types of trials, replication takes place across fields. The following three types of FMFI trials are used commonly in FSD work:

• Paired Comparisons. Using this approach, farmers implement a planned comparison in their field on previously marked plots. This approach generally is used for simple comparisons, for example, with and without fertilizer, row planting versus broadcast planting, or single ploughing versus double ploughing. However, it also can be used to test an 'improved' system against the farmer's 'traditional' system, for example, a system involving double ploughing, row planting, and phosphate fertilizer versus the traditional broadcast planting, single ploughing, unfertilized system. This format generally requires that the same crop and variety be used for all paired comparisons, with the same seeding rate across trials and farms, and the same planting date for each plot in a pair. Then, aside from treatment variables, the management of the comparison follows the farmer's usual practices.

The number of treatments in each set of comparisons also can be expanded where it is necessary and appropriate. For example, the approach could be used for crop variety trials, where researchers wish to determine the best variety of a crop to recommend within a region. In this case, the comparison might include three or four new varieties, to be compared against the farmer's own.

The paired comparison format has been used extensively in FSD work for FMFI trials. It is appropriate for farmers because:

• It is fairly simple to understand and implement.
• It generally does not require much of the farmer's time or management, because it is a small trial.
• The results are fairly clear and easy for farmers to interpret -- whether or not there has been any benefit from the 'improved' treatment.

The format is also easy for researchers to apply over a large number of fields. Once the sites have been selected and marked prior to the start of the season, farmers can usually implement the treatments themselves, with researchers checking the progress in the regular group meetings and/or visiting the fields at their convenience (Box 9.6).

As indicated, these types of trials usually are best replicated across a number of farms, rather than being replicated within a farm, However, this observation is based on experiences in low-income countries with limited-resource farmers. Limited farmer resources in terms of labour, traction, land, etc., often make it difficult for a farmer to handle more than one replication adequately, especially because plot sizes used tend to be larger than would generally be the case in RMRI trials. Nevertheless in high-income countries replication within a field becomes a more realistic possibility, although as will be seen later (see Section 10,4,3), there are advantages to involving more farmers and testing under as wide a range of environments as possible, Therefore, replication within farms should not be undertaken at the expense of involving more farmers,

• Field Testing New Equipment Prototypes, One of the simplest trial formats can be used when testing new pieces of farm machinery. In this case, the equipment simply is used by several farmers within a region, No plot size is specified, and no replications are required within a farm, The farmers simply are provided with the equipment, shown how to use it, and -- as long as they have an interest -- are requested to put it to use. Researchers then assess the performance and effectiveness of the equipment by:

• Observing its operation in the field.
• Holding informal discussions with the farmers who use it,
• Conducting a formal survey of the farmers who use it.

A formal survey may be used to obtain quantifiable data on farmers' opinions, the problems they encounter in using the equipment, their suggestions for improvements, and their judgments as to whether or not the equipment could benefit their farm operations (Box 9.7),

This approach is particularly suited to testing equipment that has been used in other areas and that researchers wish to introduce into the target area. In such cases, the design of the equipment is known to be effective, and the primary purpose for the tests is to determine whether the equipment is appropriate for farmers' systems and resources and the environment in the target area.

• Superimposed Trials. As indicated earlier (see Section 9.2), superimposed trials can be used to test relatively simple innovations under farmer management. In these trials, a treatment is superimposed in a plot or plots on a farmer's field, often after the farmer has started the initial crop husbandry work, usually after planting. The comparison in this case is generally between the farmers own system and that system plus an additional treatment, for example, an extra weeding or a top dressing of nitrogen, etc. The treatment plot yields are compared with yields from an equal area of the field adjacent to the treatment plot. There may be more than one comparison per field, and comparisons usually are made in a number of fields.

Unlike the other types of trials, discussed earlier, which are implemented according to a plan drawn up at an earlier date, superimposed trials can be pre-planned or unplanned in the sense of responding to a problem that has arisen and for which a satisfactory solution is sought. Examples of the flexibility of superimposed trials are given in Box 9.8 .

BOX 9.6: FARMER GROUPS ARE SUITABLE FOR MONITORING FMFI TRIALS

Farmer groups (see Section 9.8.6) have many good characteristics, one being to provide a forum for making the use of researchers' time more efficient, particularly with reference to FMFI trials that they need to visit only irregularly. In one area of Botswana, when a set of this type of trials was managed through group meetings, farmers about 100 comparisons were properly implemented in both the 198788 and 1988-89 seasons.

BOX 9.7: TESTING EQUIPMENT REQUIRES A DIFFERENT APPROACH

A new type of donkey collar was introduced in one area of Botswana. Originally, it was designed and manufactured in Kenya where it was in general use. It Botswana, it was tested by several farmers. Both informal discussion and survey data showed that it was an improvement over the equipment farmers had been using up to that time. After that study, local production of the collar was started and collars were offered as part of a development extension programme.

BOX 9.8: SUPERIMPOSED TRIALS PROVIDE FLEXIBILITY FOR RESEARCHERS

Two examples from Southern Africa illustrate the flexibility of superimposed trials:

• Pre-Planned. Experiment station trials had determined earlier the ideal plant populations for sorghum and millet. However, in FSD work, it was found that few farmers achieved the ideal plant population, Therefore, the cooperation of a few farmers who already had standing crops was obtained, in order to investigate the practicality of gap filling -- through transplanting -- and thinning, so that farmers could more closely approach the optimum plant population.
• Unplanned. A farmer with a plot infected with many weeds was not keen on investing a lot of labour in weeding. In order to demonstrate the benefits of weeding on the eventual yield of the crop, it was suggested that he weed only part of the plot.

Advantages of superimposed trials include the following:

• Often they can be undertaken at a lower cost than RMRI or RMFI trials.
• A potential exists for responding to research opportunities that arise during the agricultural year.
• They can provide an easy way to demonstrate new practices or technologies to farmers.

Thus, there is considerable potential for superimposed trials to respond to unanticipated opportunities and as tools in convincing farmers to change their strategies.

Contents - Previous - Next