The session focused on the in situ data necessary to ensure that satellite remotely sensed data are properly calibrated and validated. Four presentations were made and this was followed by a general discussion.
Summary of Presentations
In situ Observations for Calibration and Validation: a CEOS Overview
The CEOS defines calibration as the process of quantitatively defining the system response to known, controlled signal inputs, and validation as the process of assessing by independent means the quality of the data provided. Only with both elements of calibration and validation present can higher level products be successfully generated from satellite sensor data, and changes in these products identified as either a change in the instrument or a change in the environment. By the year 2000, atmospheric, land and ocean observations will be possible from around 60 earth observation satellites, which in turn will lead to hundreds of derived products. The calibration of this range of satellite data and products is a major task. The CEOS WGCV was established to provide a forum for international dialogue on calibration and validation issues, to enhance coordination, to promote international cooperation and to focus activities in the field. Today, calibration and validation activities are sensor, rather than product driven, but there is an increasing need for product validation and this has to be based on in situ observations. A further problem is that environmental factors can generate anomalies, even when sensors remain stable, e.g., volcanic gas. Unfortunately, calibration and validation requirements for different parameters are not usually complementary. Furthermore, calibration and validation sites are often mutually exclusive. Calibration calls for spatial and temporal homogeneity and stability, yet validation is often associated with the need for variable sites.
The CEOS WGCV has two tasks: sensor-specific calibration and validation of geophysical parameters and other derived products. Most effort to date has been placed on sensor-specific Calibration/Validation activities, but there are proposals for cross calibration activities and the identification of common test sites.
Advanced Earth Observation Satellite (ADEOS) Calibration
The ADEOS calibration programme will cover all the main 8 instruments on board the satellite. The complete range of activities will take time with some value added products being available after 6 months. The presentation also showed some of the first images of the ADEOS satellite from the Advanced Visible Near-Infrared Radiometer (AVNIR) instrument.
The presentation focused on Ocean Color Temperature Scanner (OCTS). A range of techniques is proposed which involve in situ and airborne data sets to be collected. Some of these involve collaboration such as that for the OCTS instrument. Here, test sites in California (and offshore) and Japanese waters are being used. Vicarious calibration techniques will be used for OTCS validation.
Validated OCTS products for the sea around Japan are expected after 6 months with a global product available after 12 months.
The Earth Observing System (EOS) Validation Activities
International coordination is providing a focused international satellite monitoring programme with a common science agenda (e.g., Integrated Global Observing Strategy (IGOS), IGBP, CEOS). A coordination activity needs to be developed associated with the validation of the derived global data products from such a programme. There are common validation needs for different sensors, platforms and agencies. The higher order data products developed by these sensors are intended for use by the international science community and this in itself will provide a level of international validation that will benefit from an appropriate level of coordination. These science users will need data products with known and stated accuracy. It is clear that there are insufficient resources for any one programme to provide validation of global data fields. In addition, there is a limited pool of scientists in any one country to perform validation measurements, especially those that require high technology instrumentation. A large number of in situ measurements are already being taken by national and international agencies or organizations with little overall coordination or data management. There is the possibility to augment some of these existing networks to assist in satellite product validation. These reasons and others combine to make international coordination essential for global satellite product validation.
Science users currently need consistent fields of geophysical parameters to drive and validate regional and global models. The demand is for higher order satellite products beyond satellite radiances using community algorithms based on peer reviewed journal articles, tailored for model input (time and space scales). These input data need to be well calibrated which is a prerequisite for product validation. Similarly quality assurance, quality control and systematic product evaluation needs to be performed as part of the product chain prior to product validation. Product validation needs to be undertaken through independent measurement of geophysical parameters at a spatial scale appropriate for the sensing system to characterize the product accuracy, consistency and uncertainties. Validation is needed with spatial and temporal representation over the range of environmental conditions encountered by the product. The level of validation should be tailored to satisfy intended use of the data. In the context of the global observing systems, the satellite validation should be driven by their prioritized science objectives. The existing in situ networks can provide a good basis for developing the satellite product validation.
NASA's EOS provides a good case study on product validation issues and demonstrates the overlap between the data needed for satellite product validation and in situ data needed to meet the climate objectives of the three global observing systems. EOS involves multiple platforms, multiple instruments, multiple science objectives and multiple products and as such provides a microcosm of the validation issues for the broader international satellite user community.
In the EOS programme, product validation is a contractual obligation of the instrument teams that are supported to undertake scientific research using the data they generate. It is recognized that an important step in the validation is performed by the final data users. EOS has established a Validation Science Office responsible for coordinating instruments and platforms. Validation planning is currently underway; instrument teams are generating coordinated validation plans, workshops have been held and the utility of existing national monitoring networks are being evaluated. There is an urgent need to internationalize the EOS validation coordination for which there is currently no workable mechanism. The EOS Validation Program is being designed based on discipline and product. The validation plan for the Moderate Resolution Imaging Spectroradiometer (MODIS) has three components: field campaigns, ground validation test sites and airborne measurements. The land validation community has adopted and adapted the TOPC GHOST to meet EOS test site validation needs.
The following recommended actions assume that the satellite data products have been prioritized for global change research according to their significance, uncertainty and measurement feasibility. This prioritization is appropriate for joint activities of TOPC and CEOS. There is a need to establish a validation coordination mechanism to develop a coordinated validation plan(s) to match the coordinated satellite observation programme. This coordination mechanism needs to include a combination of satellite and in situ communities. CEOS is the obvious forum for this coordination but the existing WGCV would need to be reconstituted to meet this enhanced commitment.
There is a need to determine the appropriate level of validation needed for the products. This needs to be determined by the science community, e.g., through the IGBP and WCRP. Similarly, timelines for product data availability are needed. This is clearly within the remit of CEOS. It is important to reduce the current list of satellite derived products to a smaller number of high priority products and use these initially to provide a pathfinding activity for international validation coordination. This task should be undertaken by the global observing systems. There is also a need to determine the synergies and efficiencies between existing individual instrument product validation programmes and plans, and engage instruments currently in design-phase in the validation coordination process. This task is more suited to the WGCV. It will be important to involve the in situ monitoring communities in this design to ensure that the plans are able to be implemented.
There is a need to evaluate the utility of existing in situ monitoring networks to provide the necessary validation data needed by the global observing systems and to work up a short-list of candidate sites and perform an in-depth assessment of capacity. It is also essential to scope the predicted costs and state clearly benefits of the proposed coordinated validation activities. It will then be necessary to seek commitment from agencies to implement validation programmes and to proceed to implement pilot activities to test the measurement and data protocols and demonstrate the benefit of validation coordination.
Finally, it will be important to ensure close involvement of data producers and algorithm developers in this process, to encourage feedback from product validation programmes to algorithm refinement and eventual data reprocessing.
Oceanic calibration/validation issues; computation of sea surface temperature and sea surface salinity
A global sea surface temperature (SST) analysis is computed at the US National Centre for Environmental Prediction (NCEP). The analysis provides weekly data from November 1981 to the present on a one-degree grid using optimal interpolation (OI). The analysis uses in situ and satellite SSTs plus SSTs simulated by sea-ice cover. Because the number of in situ observations is relatively small compared to the number of satellite observations, the satellite data dominate the analysis. However, the in situ data are critically important in correcting the satellite data for biases before these data are used in the OI. The complete analysis is designed to emphasis the relative strength of each type of date to produce high quality global SST fields. Comparisons with independent data show that in situ data are needed for bias correction on five- to ten-degree spatial scales. However, there is no clear preference for the type of the in situ data and either ship or buoy data are sufficient.
An analysis of sea surface salinity (SSS) was carried out for the tropical Pacific. This analysis was needed to help eliminate sea level errors in the NCEP Pacific ocean model that were caused by neglect of the interannual signal in salinity. Because salinity data are so sparse, the analysis can only be done using the SSS ship track data. However, these data do not provide a complete field in the tropical Pacific. The surface forcing fields of SSS [evaporation minus precipitation (E-P)] were used to fill in the SSS data. This was done by computing empirical orthogonal functions (EOFs) from the E-P anomalies. The first six spatial EOFs were then selected and used as basis functions for the SSS data. This technique was applied to the SSS data to yield full monthly anomaly fields of SSS for the tropical Pacific from 1979-1995. The SSS fields will be evaluated by comparison with other salinity data and will be tested using the NCEP ocean model. The international variability of SSS in the western Pacific is important and it is critical to continue the SSS observation in this region.
Plenary Discussion
The question of whether calibration and validation should be sensor or product driven was discussed. In general, calibration is sensor driven and should primarily be the responsibility of the space agency concerned. Validation is more complex, because of the need for long-term sensor validation as well as product validation. It was accepted that the data users should be involved in or lead discussions on validation. A further issue is the role of assimilation. In meteorology, it is common to use four dimensional data assimilation techniques involving integration of raw calibrated satellite and in situ data in models and providing an output, which is effectively a high-level, validated product. It was accepted that in the future the global observing system programmes may use assimilation techniques more widely, but in any case will require multi-sensor products. These trends will require a greater involvement of the data users.
An important issue was the need for information on the calibration and validation procedures being provided as part of the product and thus providing an 'audit' trial of processing. This is seen by some groups as essential for understanding the quality of the data being used. It also relates to the problem of calibration of equipment. This issue is often underestimated, but is essential for some global products to be accurately developed.
There was a discussion of elimination of systematic bias. For example, a dawn, morning, or afternoon orbit may introduce a systematic bias that needs to be understood, particularly if multi-system products are being generated. One example is SST products, where satellites observe the skin temperature, which has greater daily variability than the bulk temperature, which is the in situ observation.
It was agreed that much more could be done with respect to improved collaboration. Satellite calibration and validation can make much better use of the existing in situ data measurement programmes. This requires better knowledge of ongoing activities to provide a basis for improved cooperation. It could even be possible to add extra data collection procedures to existing in situ programmes to gain greater overall benefit. One issue is the need and realism of having common test sites for calibration. If possible, a 'common core' of measurements should be identified which may then be added to four specific campaigns. Are protocols for data collection and handling possible? The meeting welcomed the CEOS proposal to address this issue.
In the debate on collaboration the role of CEOS was discussed. It was agreed that the WGCV should consider broadening its role. One aim would be to ensure a better and more coordinated input from the affiliates. A second aim could be to provide better information on future calibration and validation activities. A third aim would be to act as a forum for setting out the individual agency priorities as a basis for more effective integrated planning of calibration and validation activities. There is also the possibility of CEOS providing a framework for specific multi-agency validation activities. It was also pointed out that calibration/validation data handling could be an issue for the CEOS Working Group on Information Systems and Services (WGISS) which has responsibility for this subject.
Overall, the group felt that calibration and validation should be seen as an integral part of the mission specification. The WGCV's activities have tended so far to concentrate on calibration activities and there should be more consideration of the validation of satellite geophysical data. This might involve restructuring of the WGCV.
Recommendations
Recommendation C1. Recognizing that all the global observing systems will soon be affiliates of CEOS, the global observing systems should coordinate their participation in CEOS though the GOSSP to optimize input to the Members for their planning of validation.
Recommendation C2. The GOSSP should develop a scheme for identifying priorities for validation issues. Recommendation C3. CEOS should consider enhancing the activities of the WGCV to address international co-ordination and collaboration of satellite product calibration and validation. For validation this should take into account priorities identified by the global observing systems.
Recommendation C4. On the basis of the priorities established above the WGCV should develop a pilot project to address issues such as measurement protocols, test regimes and data management in relation to calibration and validation.
Recommendation C5. CEOS members should maintain electronic bulletin boards providing access to up-to-date consensus calibration information for their sensors and CEOS members should provide access to them.
Recommendation C6. The meeting endorsed the recommendation from WGCV to the CEOS plenary concerning cross-calibration activities and establishment of test sites.
