Global |
AF |
AU |
AS |
EU |
NA |
SA |
|
Sample size |
312 |
69 |
13 |
68 |
43 |
82 |
37 |
Classes 1-4 |
76.92 |
68.12 |
92.31 |
82.35 |
72.09 |
79.27 |
78.38 |
Standard error |
0.24 |
0.57 |
0.77 |
0.47 |
0.69 |
0.45 |
0.69 |
Forest/nonforest |
85.58 |
76.47 |
92.31 |
85.29 |
86.05 |
86.59 |
97.30 |
Standard error |
0.20 |
0.51 |
0.77 |
0.44 |
0.54 |
0.37 |
0.26 |
Notwithstanding the factor of limited sample sizes, accuracy estimates at the continental level clearly show that land cover homogeneity is an important factor in determining map accuracy. For forest and nonforest land cover classification, the FRA2000 map is most accurate in South America, largely due to the continuous, unbroken tropical moist forest in Amazon Basin; and it is least accurate in Africa, as Congo Basin rain forest is smaller than that of South America, and the large areas of miombo woodland in southern Africa are spatially and temporally highly variable. Reasonable accuracy results in the northern hemisphere perhaps reflect the fact that land cover is relatively established and knowledge of land cover in the regions can be easily obtained. For Australia and tropical Asia, the high accuracy may be the result of the small sample size (13 sample sites). Of course, regional land cover within the continents, and associated map accuracy, are highly variable spatially. An ideal and more complete assessment would be to obtain sufficient sample sizes to capture the regional variations.
China and the U.S. are two special cases (Tables 5 and 6) in the validation because of the availability of wall-to-wall land cover mapping with 30-meter Landsat data in the two countries. Perceptively, the all-pixel comparisons should be statistically more rigorous than sampling approaches for the same mapping areas. However, any gain in statistical precision may be discounted by the facts that the fine-resolution land cover maps have their own mapping errors4, and that the two countries probably represent the best case scenarios in the mapping with ample ancillary information and researcher familiarities. On the other hand, none of the three reference data sets were developed for the mapping effort, resulting in a mismatch with the FAO legend, despite the use of "cross-walk" to make these reference data sets as comparable as possible with the FAO legend. The legend mismatches should be included in consideration of overall quality of the FRA2000 map.
Figure 3 is a RGB (red, green, blue) color composite image for the conterminous U.S., with the FRA2000 forest canopy map displayed as red and MRLC-derived forest canopy map displayed as green and blue. The MRLC forest canopy map was derived by recoding the three forested classes (see Appendix 1 for MRLC classes) to forest and the rest of the classes to nonforest, and then aggregating 30-meter pixels to corresponding 1 km pixels. Areas of agreement between the two maps are shown in shades of gray (from black to white depending on canopy density value estimated). Areas of disagreement are shown in colors of red (canopy density estimates of the FRA2000 map are greater than that of the MRLC map) and cyan (vise versa). The eastern U.S. woody wetlands (which should have been included in the MRLC map as forested land cover) and the western chaparral (classified by MRLC as shrubland, but considered forested by the U.S. Forest Service) accounted for most of the FRA2000 overestimation (red shades). Conversely, the FRA2000 underestimation (cyan) occurs primarily in three situations: 1) the Great Basin (Nevada and Utah) dry woodland and shrubland, 2) patchy or riparian woodland in the Midwest agriculture region, and 3) southeastern U.S. fragmented forest lands. While the U.S. MRLC is a special case, the differences in Figure 3 probably represent typical errors or variations in the FRA2000 global forest canopy density map.
Table 5. Area estimates (rounded to thousand km2) and rate of forest cover for China and the conterminous U.S. FAO estimates are derived from 1-km AVHRR data, CAS and MRLC area estimates are derived from 30-meter Landsat data.
Forest area estimates (x 1,000 km2) |
China |
Conterminous U.S. |
||
FAO |
CAS |
FAO |
MRLC |
|
Total mapped |
9,474 |
9,496 |
7,943 |
8,080 |
Total forest |
1,770 |
1,766 |
2,591 |
2,480 |
% forest cover |
18.68 |
18.60 |
32.62 |
30.70 |
Table 6. Estimates comparing the FAO map to reference maps of China (CAS) and the conterminous U.S. (MRLC). Overall agreement for forest/nonforest classification is 86.12 and 83.01 percent, respectively for China and the U.S.
Forest/non forest classification |
China |
USA |
||
Producer’s |
User’s |
Producer’s |
User’s |
|
Total forest |
62.89 |
62.72 |
75.36 |
73.11 |
Total nonforest |
91.44 |
91.50 |
86.69 |
87.99 |
Certainly, the ultimate judgement of a map’s overall quality lies with its usefulness and effectiveness in meeting requirements of a variety of applications. User feedback can be an opportunity to examine potential areas of map error, whereas sensitivity of applications using the map as input may be a more objective indicator of the map’s accuracy. However, these holistic approaches are long term and difficult to measure in absolute numbers; traditional accuracy assessment is still necessary to provide a more immediate assessment to both producer and user. In this study, the use of existing reference data sets for accuracy assessment may not be the ideal approach, but it was a practical decision to obtain an accuracy estimation given time and budget constraints.
4Accuracy assessment for either map product is not complete at this time.