After the presentations, the participants divided into regional groups to continue the analysis exercise based on the process established. A fourth group was established to devise a process to arrive at a final set of weights for the different data sources in each country. A second task was to apply the methods developed to a specific country case to try to arrive at country level estimates.
Using the guidelines developed by the convergence groups, participants in the regional groups revised the documents thoroughly noting procedures, input materials, the rationale for using or not using inputs and recommendations for applying the convergence method. Participants tried to derive a best estimate for the state of forest as of 2000 and change from 1990 to 2000.
a) Reviewed existing information including four country inventory reports, TM imagery for the region and for Bolivia, 1990 FRA model estimates of change and 1990 FRA report for cover.
b) Prepared spreadsheet assessing the quality of the data sources. Three of the four country inventories were completed national inventories covering the country from the late 1970s to the mid 1990s. They were deemed the most relevant information.
c) A simple linear regression was applied to the three (time, cover) pairs in order to estimate the average annual change from 1975 to 1995 (approximately 120 000 hectares per year). This estimate was then applied to the country cover (state) estimate in 1995 to project the country cover estimate in 2000.
The estimate was substantially below the change estimates provided by FRA 1990. The country data was deemed to be more reliable. The estimates were a little (but not much) below the estimates from two individual TM scenes for Bolivia.
It was decided to use 6 classes to determine relevance/quality of data inputs:
0 = not relevant,
1 = poor up to
5 = excellent.
A final weight of 100% was given to the estimate based solely on country data.
Relevance assessment is done at two levels: relevance of individual data sets (for estimating change) as well as relevance of combinations of data sets (for trend). Some data sets may be paired better than others due to consistency of methods. Information is needed about the extrapolation procedure used to project state to 2000 in order to assess the quality of the change estimate.
Since estimating state in 2000 will always require a projection of some earlier state estimate, all cases will include change assessments as a precursor to final estimate of state as of 2000.
A general discussion of procedures for projecting to year 2000 is needed – linear extrapolation to start with, then adjusting the 2000 end point up or down based on auxiliary information.
Separation of quality assessment and weighting of state estimate versus change estimate, e.g. very consistent series of estimates over time that are biased, so the change estimate is good while state is biased.
Expert opinion data should be used if there exists no other class of data. It is important to recognise that expert opinion is always incorporated in the data evaluation and weighting phase.
There will always be remote sensing regional estimates and AVHRR cover estimates; might have remote sensing samples from the country; will have 1990 FRA model (i.e. always disregard FRA 1990/1995 model estimate). It was proposed to use the mean regional remote sensing estimate as the “worst case” scenario.
Objective: To derive in a transparent way final figures (estimates) of forest cover and forest cover change. Both estimates should carry a reliability measure. The following steps were proposed:
Step 1. The group recommended that the FRA team arrive (among other descriptive measures) at reliability class and change/cover estimate using each data source from each country. FRA should also determine estimated weights, but these are kept confidential. The three columns shown in the following table are those that are used in the overall weighting procedure.
Table 5. Overall weighting procedure.
Data Source |
Reliability |
ParameterEstimate |
Est.Weight |
Country data: A B C Remote sensing sample Regional estimate, 1980-1990-2000 Country estimate, 1980-1990-2000 Model estimate Other data sources |
X X X X X X X |
X X X X X X X |
X X X X X X X |
Step 2. Data (without FAO weights) is sent to countries for comments. Define deadline well before step 3. Countries may comment on the data set prepared by FAO.
Step 3. A regional workshop by region is suggested utilising experts from the region to determine final weights for each data source from each country (weights to determine final estimate of cover and change for each country), chaired by an FAO staff member (regional correspondent). The workshop would use a Delphi process to arrive at final weights and final overall estimates of cover and change by country.
Step 4. Suggested final step: sum the final estimates by country and compare to the total estimates by region determined through the remote sensing sample as a control - the sum across countries from the Delphi process should be within the confidence limit of the remote sensing sample total by region.
a. FRA staff present reliability classes and change/cover estimates for each data source by country, explaining how they determined the class and giving all necessary background information.
b. Experts discuss the reliability classes and parameter estimates and perhaps adjust them based on their expert opinion. The experts may identify additional information or data sources.
c. Experts assign preliminary weights.
d. FAO staff reveals FAO weights.
e. Continued plenary discussion to review the set of weights by expert for each data source.
f. Experts assign final weights; FAO staff possibly modifies FAO weights.
g. Weights are averaged for each data source and averages are multiplied by the estimate from each data source to determine a final single estimate for each country.
h. Overall reliability is quantified by the sum of the average weighting units multiplied by the final reliability class for the data source. The result will then be on the same scale as the reliability class for the data source.
This process is considered to be open, interactive and participatory in character. It ensures as far as possible that all experts have a common level of knowledge of the particular country under discussion.
It is still necessary to define whether the process should cover all data sources (models, expert opinion) or only the ones on hand to the FRA staff (country data, remote sensing samples).
It has not been determined what part of the entire process to publish and what to only keep in internal files.
The above process should be done for the state estimates and for the change estimates. It is not clear whether or how to combine the two.
A test run to make a country estimate for Cameroon using the framework assessment table was the main focus of the group’s activity. The country information was reviewed and two possible new sources of information were examined. The table was completed for several cases discussed in previous sessions and several new scenarios developed. There was insufficient time to calculate the needed numbers and averages using the data from FRA. Landstat scenes for the scenarios, but evaluations and weights for each case are given.
The new sources of information generally proved of little value, but one, a very local study, indicated the need to make provision for special studies that might add useful information for an expert analysis but not of value for making actual calculations. A “special study” row was added to the table to indicate this (see attachment).
It was also found that the meaning of the various evaluation columns in the table was not clear and the group constantly had to reread and ponder their meaning.
Since there was not strong country data, work concentrated on how to use the FRA satellite plot data. Use of one scene (1501) or scene 1501 partially averaged with a similar wet zone image (1502) was considered. The group also discussed just using the regional averages for the wet, moist and dry ecozones of all of Africa in proportion to their occurrence in Cameroon. The existing sample scenes were then examined for their appropriateness to the Cameroon situation. Ecozone, socio-economic and distance were considered. The scenario that was proposed is to use scene 1501 (80% of its values) plus 20% of scene 1502 to represent the wet ecozone of Cameroon (nominally 20% of the country). Scenes 1512 and 1503 (see map of group) would be used in equal proportions for the moist zone (70% of Cameroon) and scene 1466 only for the dry zone (10%). It was felt this was correct but not overly strong and that the country data was of value. So for this scenario the final estimate would be 20% of the value of the country data and 80% of the value of the satellite-based estimate (see Appendix 5).
A scenario where new imagery can be acquired was explored. It was determined that at least seven new images would be available near enough to be considered as candidates to contribute to the Cameroon total. These were placed on the map in a reasonable pattern. The details of which scenes would be used and what weight to apply to each image was not determined but would follow a similar logic to that described above. It was felt this scenario would give good results and, indeed, if it were available would be the only numbers used.
Scene representativeness was also discussed as a possible useful concept. Before a scene was used to represent a country it would be examined to see if it was indeed representative of the country (using expert regional knowledge and examining any images within the country and ecozone – these need not be FAO plot imagery).
The exercise of completing the table proved feasible. The evaluation of the various criteria was unclear and needs further discussion. Quite complex logic and considerations were used to determine which images to use and the weight for each. Regional expertise or guidance would be needed and local knowledge would also be useful. It would be difficult or at least too lengthy to document all the reasoning. Some guidelines and brief discussion would be the most that could be expected. The group also speculated that in areas of low-reliability data there would be only one or two dominant choices to consider as sources of information. The final weighing of source numbers may be quite easy and not require a complicated analysis. The value of additional images was considered very high.
Remote sensing estimates of forest change in southeast Asia in many cases overestimates the loss figures, as plantation areas will not show up in the satellite images until approximately 10 years after planting. It is also difficult to separate home garden plantations from actual forests in remote sensing images.
For Vietnam the data set choice was easy as the group considered both state and change data to be recent and of high quality. The figures were therefore accepted and simply extrapolated to year 2000. However, there were some concerns about the change figures, as these were the sum of deforestation of natural forest on the one hand and plantation figures on the other hand. The figures were added to each other, but they might include errors of unknown size.
For Indonesia the procedure was much more difficult. The data sources were of low utility for state and change. However, there was hope that a World Bank report could be used as a source of data for state and the FRA remote sensing scenes as a data source for the change estimate. Therefore, the results of the World Bank report should be compared with the FRA 2000 remote sensing results for the seven locations for 2-3 points in time per location. One conclusion is that in the Indonesia case there may be more sources of information available in the form of reports and satellite images. One suggestion is to contact CIFOR.
There was also some confusion about the 1990 modelling approach as forest change figures were about "natural forest" while the state figures were about all "forest" areas.
This refers mainly to the Indonesia data compiling process.
Several data sources were available but the input of regional experts was needed to determine the utility of the different sources. The results of the reviewing process would have been substantially different without the regional expertise as the available reports initially seemed comprehensive and with a number of detailed components relevant to the FRA 2000 work. More specifically this meant that:
Ø The National Forestry Inventory (NFI) report was evaluated as of low utility, because the methods applied in the inventory were insufficient and generated unacceptably biased results.
Ø The Regional Physical Planning Programme for Transmigration (RePProt) report was considered useful to some extent for state figures, but it was old (1985) and a single inventory which made it impossible to estimate change.
Ø The Scotland Fraser Report (1999) used a combination of the NFI and RePProt for change estimation. This report concluded that the results from this comparison were incorrect and misleading as it arrived at positive change estimates.
Ø A confidential document provided interesting figures regarding change but we could not consider this source of information due to its current restricted state.
Ø Numerous other analyses have been conducted, providing a range of results that makes us question the methods and different official sources of information (Sundferlin & Resusardmo, 1997).
