SDS Ocean Peate

Introduction
The Office of Earth Science (OES) of the National Aeronautics and Space Administration (NASA), the National Oceanic and Atmospheric Administration’s (NOAA) National Environmental Satellite, Data, and Information Service (NESDIS) Integrated Program Office (IPO), have agreed to jointly implement a mission called the NPOESS Preparatory Project (NPP). NPP has the objectives listed below.

1.Demonstrate and validate:
a.A global imaging radiometer and a suite of two sounding instruments, associated algorithms, and data processing
b.An ozone mapping and profiling instrument, associated algorithms, and data processing
c.A NPP Command, Control and Communications Segment (C3S), an Interface Data Processing Segment (IDPS), an Archive and Distribution Segment (ADS), and a Science Data Segment (SDS).

2.Provide continuity of systematic, global, calibrated, validated and geo-located Earth science imaging radiometry, sounding observations, and ozone mapping and profiling observations for NASA Earth Science research.

This document specifies the requirements and concept of operations for the NPP Ocean PEATE (Product Evaluation and Test Element) that will be managed and developed by NASA. As defined in NASA’s NPP Level 1 Requirements (see Appendix 1), the role of the SDS is principally directed to assessment and verification of NPP product quality, where those products are identified as Raw Data Records, Sensor Data Records, and Environmental Data Records (RDRs, SDRs, and EDRs, respectively). For the Ocean products, these will be generated exclusively using data from the Visible and Infrared Imager/Radiometer suite (VIIRS). In executing its responsibilities for assessment and verification, the SDS functions will be performed principally by a distributed and interoperable architecture of Climate Analysis Research Systems (CARS).

Acquire RDRs, SDRs, and Ocean EDRs from the SDS
Assess the quality of the NPP Ocean EDRs for accomplishing NASA’s climate research requirements
Provide suggested algorithm improvements to the IDPS via the Project Science Working Group (PSWG)
Process selected data subsets in support of Evaluation and Validation activities
Support field programs and other research activities of the Ocean Science Community.


Interfaces

The NPP Ocean PEATE will support interfaces with the following facilities and organizations:

2.1SDS Data Delivery Depository (SD3) Component

The Ocean PEATE will acquire VIIRS RDRs, SDRs and Ocean EDRs from the SD3. Prior to the NPP launch, these will be either simulated data sets for algorithm testing, or test data sets from instrument or spacecraft tests. Following launch, these will be data routinely transmitted from the IDPS. All RDRs will be acquired as they are made available at the SDS interface, while selected SDRs and EDRs will be acquired on request.

2.2SDS Integration and Test System (I&TS)

The Ocean PEATE will interact with the I&TS for processing code deliveries and updates. The I&TS will provide the latest version of the operational SDR and Ocean EDR processing code to the Ocean PEATE. The PEATE will provide proposed code and algorithm updates to the I&TS.

2.3NASA NPP Characterization Support Team (NCST)

The Ocean PEATE will interact with the NCST in support of VIIRS calibration. The NCST will provide calibration LUTS for evaluation, which may utilize more recent calibration measurements than the operational LUTS provided by the IDPS. The Ocean PEATE will provide the results of calibration evaluations to the NCST.

2.4NASA VIIRS Ocean Science Team (NVOST)

The Ocean PEATE will be co-located with the NVOST leader and will be the primary facility for analyses and evaluations performed by the NVOST. The NVOST will provide deliveries of science code, proposed algorithm improvements and evaluation processing requests to the PEATE, and will serve as the primary liason between the NASA Ocean researcher community and the Ocean PEATE. The PEATE will provide operational processing code, standard SDRs and EDRs acquired from SD3, and SDRs and EDRs generated from evaluation processing, to the NVOST.

2.5Ancillary Data Providers

The Ocean PEATE will acquire ancillary data products needed for EDR processing from various providers. The current ancillary data sets consist of TOMS and TOVS ozone data and NCEP meteorological data (wind speed, atmospheric pressure and humidity). The sources and providers of ozone and meteorological data will be reviewed and revised as needed.

Requirements

The NPP Ocean PEATE will meet the following requirements.

3.1Acquire RDRs, SDRs, and Ocean EDRs from the SDS

The Ocean PEATE will require access to the Visible and Infrared Imager/Radiometer suite (VIIRS) RDRs, SDRs, and Ocean EDRs. The Ocean EDRs are Ocean Color (40.7.6) and Sea Surface Temperature (SST) 40.2.4). (The SST requirements are TBD pending a decision by NASA Headquarters.)

The Ocean PEATE will ingest the full stream of VIIRS RDRs as they are made available at the SDS interface.

The Ocean PEATE will ingest selected subsets of the SDRs and EDRs as specified by the NVOST, for assessment and validation.

The Ocean PEATE will provide storage for VIIRS RDS, SDRs and EDRs as follows:
One full-mission set of RDRs
Approximately 10% of SDRS
Two full-mission sets of Ocean EDRs.
All prelaunch test data sets.

3.2Assess the quality of the NPP Ocean EDRs for accomplishing NASA’s climate research needs.

The Ocean PEATE will support the NVOST in performing the following assessments of the Ocean EDRs:

Ground truth validation using match-ups with in situ radiometry and chlorophyll data sets

Cross-comparisons with concurrent satellite data sets (e.g., MODIS Aqua) (if possible)

Comparisons with the climatological data sets from past missions (e.g., SeaWiFS)

Assessments of internal consistency (e.g., analysis of interannual repeatability in clear-water and deep-water regions; assessments of short-term stability).

Assessments of the effectiveness of flagging and masking algorithms (e.g., clouds, glint, stray light, zenith angle limits, turbidity).

Prelaunch assessment of algorithm functionality and implementation using simulated TOA radiances in the SDR format.

In order to fully evaluate the quality of the EDRs, the Ocean PEATE will also support the NVOST in performing the following assessment of the SDRs:

Assessment of the long-term radiometric stability of the VIIRS Ocean bands

Assessments of the instrumental corrections (e.g., temperature, RVS, polarization.)

Analysis of prelaunch test results and instrument characterization data sets.

3.3Provide suggested algorithm improvements to the IDPS via the PSWG.

The Ocean PEATE will provide algorithm and software updates to the SDS I&TS, when algorithm improvements are recommended by the NVOST for the Ocean EDRs.

3.4Process selected data subsets in support of Evaluation, Validation and Algorithm improvement activities

The Ocean PEATE will provide the capability to process any desired set of VIIRS data, up to and including the full mission data set, from RDRs to Ocean EDRs. This is required in order to support the NVOST in developing and assessing algorithm improvements for the Ocean EDRs.

The Ocean PEATE will acquire the operational SDR and Ocean EDR processing software from the SDS I&TS.

The Ocean PEATE will provide the capability to process RDRs to SDRs using the operational software. This will include the capability to accept alternative LUTs, either provided by the NCST or generated locally by the NVOST.

The Ocean PEATE will provide the capability to process SDRs to Ocean EDRs using the operational Ocean algorithms, software and LUTS.

The Ocean PEATE will provide the capability to process SDRs to Ocean EDRs using the operational Ocean algorithms and software, and alternative LUTS generated locally by the NVOST. These may include, for example, vicarious gains, flag thresholds, optical properties, or aerosol models.

The Ocean PEATE will provide the capability to process SDRs to Ocean EDRs using alternative algorithms, software and LUTS provided by the NVOST, to evaluate potential algorithm improvements and additional products.

The Ocean PEATE will support cataloging, searching, ordering and distributing of all internally processed data products.




3.5Support field programs and other research activities of the Ocean Science Community. (Note: this activity will make use of NPP/VIIRS data but is not an NPP requirement nor is the funding requested from the NPP Project for this activity.)

The Ocean CARS will support the field programs of the Ocean Science Community by providing satellite overpass predictions and near real time image support.

The Ocean CARS will provide access to local data archives by the Ocean Science Community to support the assessment of alternative algorithms and non-standard data products.
Operations Concept

The Ocean PEATE operations concept is built on the highly successful Ocean Discipline Processing System (ODPS), originally developed for SeaWiFS and enhanced for MODIS. Although operational processing of standard data products is not a PEATE function, the requirements given in Section 3 map perfectly into the ODPS approach. The basic elements of this are:

An integrated, cohesive team supporting all activities, including systems, software, data processing, calibration, validation, and distribution;

Substantial reprocessing capacity, to support rapid, multiple evaluations of potential algorithm improvements;

Rapid turnaround on critical test data sets, during prelaunch testing, initial on-orbit checkout and commissioning, and significant operational changes;

Management of a local archive of in situ ocean data for vicarious calibration and validation, in the SeaWiFS Biological Archive and Storage System (SeaBASS);

Maintenance and development of a highly capable data analysis and display package, the SeaWiFS Data Analysis System (SeaDAS), utilized for in-house analysis and freely available to the public; and

Close cooperation with the scientific community in algorithm and product evaluation and in situ data collection.

The following sections summarize the activities to be performed in support of VIIRS EDR assessment at the Ocean PEATE, followed by a summary of facilities to be provided.

4.1 VIIRS Ocean EDR Assessment

Assessment of a satellite-based ocean color sensor, such as VIIRS, should be both global and regional in scope. It commences well before launch, with the analysis of instrument test data sets and the verification of algorithms, software and procedures using simulated data. It must be a continuing process throughout the life of a satellite mission, particularly if the data products are to be used in a long-term, multi-sensor time series. For VIIRS, continuous validation is necessary for the assessment of the accuracy of the calibration across platforms.

4.1.1VIIRS Data Acquisition and Management

The complete set of VIIRS RDRs will be acquired as made available from the SD3. The timeliness of these data is critical for support of prelaunch test data evaluation, initial postlaunch instrument checkout, and field programs (see below). Selected SDRs and EDRs will be acquired as needed for calibration and data product assessment. The fraction of data to be acquired is still TBD. The facilities for storing and managing the acquired data are described in Section 4.2.

4.1.2Prelaunch and Early Mission Testing and Verification

The Ocean PEATE will support several critical activities during the prelaunch phase of NPP. These include: verification of the science algorithm and operational processing code; evaluation of test versions of the EDRs, produced by the IDPS from simulated data and transmitted to the SD3; and quick-response processing of instrument test data sets, as soon as they are acquired from the SD3.

The processing code will be acquired and installed in the ODPS to support verification and testing. The initial versions are expected to be the science algorithm code, which will be obtained by the NVOST and provided to the PEATE. As operational software becomes available on the I&TS, it will be acquired directly by the PEATE. The prelaunch versions of the software will be tested and verified using simulated data sets, either SDRs or RDRs. In order to fully exercise the logic of the Ocean EDR algorithms, high-fidelity simulated data with realistic radiances over water and clouds and geographically correct land pixels will be required.

These same data sets, along with sample EDRs produced by IDPS, will also be used to test the EDR evaluation procedures to be used during the mission. The types of evaluations to be performed are described in other sections, below.

Finally, these capabilities in the PEATE will be used to acquire and process the prelaunch instrument test data sets for analysis by the NVOST. The test data sets will be acquired from the SD3 as they become available, immediately processed as needed (i.e., RDR to SDR) and made available to the NVOST. This quick turnaround will be absolutely critical during instrument testing, to allow feedback from the NVOST to the VIIRS test team while the tests are still in progress. By analyzing test results during the actual testing, the NVOST will determine whether tests need to be rerun, either in original or modified form, to achieve the required instrument characterization result.

4.1.3 Evaluation Processing

To evaluate the efficacy of changes to the VIIRS SDR processing and the cross-calibration with MODIS/Aqua, we will develop the capability to process VIIRS RDRs, in the ODPS to produce level-2 water-leaving radiance and chlorophyll EDRs. We will use this capability to process limited amounts of VIIRS data for in situ validation, and larger amounts of VIIRS data for global, long time-series analyses. This will enable evaluation of multiple versions of the VIIRS mission data, similar to that which was performed for all SeaWiFS and MODIS reprocessings..

4.1.4 Global and Regional Comparative Time Series Analyses

We will perform global and regional comparative time-series analysis between VIIRS and MODIS/Aqua standard, archived level-3 products, and repeat the entire analysis following any VIIRS reprocessings. For periods beyond the life of the Aqua mission, we will perform clear-water and deep-water mission trend analyses. This work will establish the interannual consistency of the VIIRS radiometry.

4.1.5 In situ Matchup Comparison

The ODPS has already developed an automated system for validation of MODIS, SeaWiFS, and other sensors against the SeaBASS in situ radiance, chlorophyll and aerosol optical thickness (AOT) holdings. We will perform the in situ validation of the standard VIIRS Ocean EDRs, extending the analysis as additional VIIRS data or in situ data becomes available, and regenerating the match-up comparison following any VIIRS reprocessing. The methodology for performing the validation of VIIRS is consistent with that which we have developed and successfully applied to SeaWiFS and MODIS. Using a consistent methodology across missions and platforms will aid in developing a consistent long-term, multimission, ocean color data set.

4.1.6 Scientific Community Involvement

The SeaWiFS and SIMBIOS Projects maintained an outstanding record of involvement by the scientific community. From the first major SeaWiFS Project review in 1992, through the fourth reprocessing (Feldman and Patt, 2003), all SeaWiFS activity was conducted so as to maximize opportunities for scientific input, evaluation, and commentary by the community. This has included continual posting of the latest Project plans, analysis results, and evaluation products on the Web, and holding workshops to facilitate the exchange of ideas. The SIMBIOS Project maintained this philosophy from its inception in 1996 through 2003, with regular team meetings and wide dissemination of data and results. This approach was initiated for the MODIS Ocean team in 2004 with the selection of the new MODIS Science team.

We propose to follow this approach for our VIIRS Ocean calibration and validation activity. We will hold workshops at NASA GSFC, which will be open to all interested members of the community. These will follow the same general format as previously used for the SeaWiFS workshops and the SIMBIOS and MODIS team meetings.

We will maintain a public Web site for all of our VIIRS activities. This will include results and proceedings of the workshops; descriptions of proposed algorithm changes and calibration methodologies; results of data analyses, processing tests and product evaluations; and sample products for outside evaluation.

4.1.7 Near Real-Time Field Support

For calibration and validation purposes, in situ measurements should be made as close as possible to the sensor overflight time. The SIMBIOS Project offered overflight prediction services for MODIS (Terra and Aqua), as well as SeaWiFS and other sensors. We will continue this service and implement the capability to provide near real-time images from VIIRS level-1 and level-2 data, as is currently being done with SeaWiFS and MODIS data. These images provide useful information in oceanographic cruise planning, both prior to and during cruises. The ability to perform near-real-time image support with VIIRS will be limited by the turn-around time between observation and availability of the VIIRS RDRs.
4.1.8 SeaBASS Maintainance

Validation of remotely-sensed ocean color data is essential for determining how well the satellite-derived values reflect true conditions. This is best achieved through a comparison of coincidentally measured satellite and in situ observations, collected under the widest possible range of conditions. A comprehensive, spatially and temporally diverse in situ data set with measurements covering a wide range of oceanographic conditions is essential for such an effort. The SeaBASS database developed by the SeaWiFS and SIMBIOS Projects was designed with this goal in mind (Hooker et al., 1994; Werdell and Bailey, 2002). The in situ data archived in SeaBASS and used for validation purposes were collected using a consistent protocols (Mueller et al., 2002) and processed using a well-defined quality control procedures, as recommended by Hooker et al. (2001). We will continue the maintenance of the SeaBASS archive, soliciting voluntary contributions from the ocean color community at large to ensure the widest possible geographic and oceanographic coverage.

4.1.9 SeaDAS VIIRS Processing

We will integrate, within the current SeaDAS infrastructure, the capability to process VIIRS data to EDRs. There has been a need in the user community to be able to process data for the purposes of algorithm testing, regional and high latitude processing. Integrating VIIRS processing into SeaDAS would fill this need, without requiring a massive investment in systems and resources. It would also allow the users the flexibility to download RDR files, which can then be processed to any level, and reprocessed as soon as new calibration or software updates become available. As the MODIS/Aqua mission nears the end of its lifespan, researchers will have to make the transition to working with VIIRS data. Most of these users are already using SeaDAS and IDL, so the integration of VIIRS processing into SeaDAS would make this transition much easier for the ocean color community. Furthermore, both existing and new ocean color researches will benefit from the continuation of the well-established user support provided by the SeaDAS group.


4.2 Systems and Facilities

The systems and facilities of Code 970.2 Ocean Color Group provide the optimal environment to perform the assessment of NPP VIIRS Ocean data. By using an existing infrastructure developed specifically for this type of work, NASA will receive the maximum benefit for a minimum of cost. The group has significant network, processing, storage and software resources, in addition to its demonstrated expertise in characterizing a wide range of ocean color instruments. Using MODIS and SeaBASS data as a reference to validate the VIIRS data will require access to a large amount of data on a regular basis. Adding functionality to SeaDAS to support VIIRS data requires direct access to the developers and maintainers of SeaDAS. The facility is co-located with the ODPS, the SeaBASS archive, and the SeaDAS development team. In addition to access to data and staff, the group has access to the ODPS computing resources. The ODPS is a high-capacity, production-proven system that has been providing high-quality science products to the ocean color community daily for nearly 7 years. Data, software, and processing capabilities from that effort will be applied to a VIIRS EDR assessment effort.
4.2.1 Data Acquisition and Distribution

The systems are currently sized to acquire/process/archive and distribute SeaWiFS and MODIS data (Terra and Aqua) on a daily basis through both access the the near real-time data flow via the EDOS/NOAA bent pipe data feed and a data subscription with the Goddard DAAC. The data is sent to our dedicated file server via Goddard's high performance Science and Engineering Network (SEN). Data travels from Building 32 to Building 28 over the campus ATM backbone at 622 megabits per second, giving ample throughput to acquire the large data sets generated by MODIS. The file server receives data through its gigabit Ethernet interface, preventing bottlenecks at the destination. The connection to the SEN provides access to Internet2, a 2 gigabit per second network that can be used for collaboration and data transfer to other Internet2-connected sites. Available bandwidth for MODIS data transfers is maximized by routing all other network activity through the GSFC CNE network. The facility's servers are connected locally via a private gigabit Ethernet network, minimizing file transfer time within the project. Analyst workstations communicate via Fast Ethernet. Mechanisms for sharing data with the scientific community are already in place, requiring no additional expenditures or set-up time. The facility includes dedicated web, FTP, and mail servers. The long-established "ocean-color" mailing list is hosted there and has become a significant source for distributing ocean color information to researchers around the world.

4.2.2 Development

Software development for calibration and validation in a production environment requires a wide range of tools to be successful and easily maintained. The facility already has a rich programming environment in place, supporting development on IRIX, Linux, and Solaris. The environment includes all the necessary compilers, libraries, debuggers, and version control software. Additional development tools include IDL, Purify, GUI builders such as UIM/X and BX Pro, and SGI's ProDev development package including CaseVision and WorkShop. Maintenance contracts are in place for the commercial packages. Our group has the infrastructure, knowledge, and experience to support code development for VIIRS calibration and validation activities. Furthermore, the modular design of the ODPS system allows developers to easily add steps to the processing mix without affecting the control functions of the system.

4.2.3 Physical Facilities

The GSFC code 970.2 team has a well-supported facility located in Building 28 of Goddard's Greenbelt campus. The servers reside in a 900 square foot raised floor computer area. All the critical infrastructure required to operate a large computer facility is already in place. The room is supported by dual air conditioning systems, and has 54 KVA of UPS power to keep the servers operational 24x7. Environmental monitoring is active 24x7 as well. All hardware and software maintenance contracts are in place and kept up to date. Data security is maintained via automated backups to the backup server. The network equipment, critical to a distributed system design, is fully functional. The facility has a NASA-compliant IT security infrastructure, ensuring the safety of the data and the Project's computing resources. The systems are supported by staff located on site.