SDS LAnd Peate

Introduction

The Office of Earth Science (OES) of the National Aeronautics and Space Administration (NASA), the National Oceanic and Atmospheric Administration’s (NOAA) National Environmental Satellite, Data, and Information Service (NESDIS) Integrated Program Office (IPO), have agreed to jointly implement a mission called the NPOESS Preparatory Project (NPP). NPP has the objectives listed below.

1.Demonstrate and validate:
a.A global imaging radiometer and a suite of two sounding instruments, associated algorithms, and data processing
b.An ozone mapping and profiling instrument, associated algorithms, and data processing
c.A NPP Command, Control and Communications Segment (C3S), an Interface Data Processing Segment (IDPS), an Archive and Distribution Segment (ADS), and a Science Data Segment (SDS).

2.Provide continuity of systematic, global, calibrated, validated and geo-located Earth science imaging radiometry, sounding observations, and ozone mapping and profiling observations for NASA Earth Science research.

This document specifies the requirements and concept of operations for the NPP Land PEATE (Product Evaluation and Test Element) that will be managed and developed by NASA. As defined in NASA’s NPP Level 1 Requirements (see Appendix 1), the role of the SDS is principally directed to the assessment and verification of NPP product quality, where those products are identified as Raw Data Records, Sensor Data Records, and Environmental Data Records (RDRs, SDRs, and EDRs, respectively). In executing its responsibilities for assessment and verification, the following SDS functions will be performed:

Acquire RDRs, SDRs, and VIIRS Land EDRs from the SDS
Assess the quality of the VIIRS Land EDRs for accomplishing NASA’s climate research requirements
Provide suggested algorithm improvements to the IDPS via the Project Science Working Group (PSWG)
Process selected data subsets … in support of Calibration/Validation activities
Interfaces

The NPP Land PEATE will support interfaces with the following facilities and organizations:

2.1SDS Data Delivery Depository (SD3) Component

The Land PEATE will acquire all VIIRS RDRs, and selected SDRs and Land EDRs from the SD3. All RDRs will be acquired as they are made available at the SDS interface, while selected SDRs and EDRs will be acquired on request.

2.2NASA NPP Characterization Support Team (NCST)

The Land PEATE will interact with the NCST in support of VIIRS calibration including running large-scale tests of improved calibration software in the Land PEATE. The NCST will provide calibration Look-Up Tables (LUTs) for evaluation, which may utilize more recent calibration measurements than the operational LUTS provided by the IDPS. The Land PEATE will provide the results of calibration evaluations to the NCST.

2.3VIIRS Land Science Team (VLST)

The NPP Land PEATE will be co-located with members of the VLST and will be the primary facility for analyses and evaluations performed by the VLST. The VLST will provide proposed algorithm improvements and evaluation processing requests to the PEATE and serve as the primary liaison between the NASA Land research community and the NPP Land PEATE. The PEATE will provide standard SDRs and EDRs acquired from SD3, and SDRs, EDRs and Level 3 land products generated to support product evaluation and validation activities to the VLST.

2.4Ancillary Data Providers

The Land PEATE will acquire ancillary data products needed for EDR processing from various providers. The ancillary products required by VIIRS Land EDRs running in the IDPS will be documented in the Environmental Data Records Interdependency Report (D36385). The Land PEATE will also acquire additional ancillary data products identified by the VLST as required for testing improved algorithms for the generation of EDRs.
Requirements

The NPP Land PEATE will meet the following requirements.

3.1Acquire RDRs, SDRs, and Land EDRs from the SDS

The Land PEATE shall receive the all Visible and Infrared Imager/Radiometer suite (VIIRS) RDRs and SDRs and the following Land EDRs:
Albedo (Surface)
Land Surface Temperature
Vegetation Index
Snow Cover and Depth
Surface Type
Active Fires
Ice Surface Temperature

The Land PEATE shall ingest the all VIIRS RDRs as they are made available at the SD3 interface, nominally within 1 day but always within 16 days of availability.

The Land PEATE shall ingest selected subsets of the SDRs and EDRs as specified by the VLST, for assessment and validation.

The Land PEATE shall provide storage for VIIRS RDRs, SDRs and EDRs as follows:
One full-mission set of RDRs (i.e. one year of RDRs for each year of the mission).
120 data-days of SDRs
Two full-mission sets (i.e. two 1 year sets of EDRs, an improved and a baseline set, for each year of the mission) of Land EDRs.
An additional 25% margin in storage capacity will be provided as requested by the NPP System Engineer upon receipt of additional funding from the NPP Project.

3.2Assess the quality of the NPP Land EDRs for accomplishing NASA’s climate research needs.

The Land PEATE shall support the NVLST in performing the following assessments of the Land EDRs:

a.Ground truth validation using satellite observations acquired over land validation sites and in-situ data acquired by instrumentation at the sites.
b.Cross-comparisons with concurrent satellite data sets (e.g., MODIS)
c.Comparisons with the long-term data sets from past missions (e.g., AVHRR)
d.Assessments of product in terms of internal consistency and expected trends over time (time series analysis of seasonal variability and outlier identification).
e.Assessments of the effectiveness of masking algorithms (e.g., clouds, land/water boundaries)
f.Pre-launch assessment of algorithm functionality and implementation using simulated TOA radiances in the SDR format.
g.Assessments of the impact of algorithm changes on products downstream in the processing chain.
h.Analysis of pre-launch test results and instrument characterization data sets.


3.3. Assess the Long-Term quality of the NPP Land EDRs for accomplishing NASA’s climate research needs.

The Land PEATE shall perform the following assessment of SDRs including:
Assessment of the long-term radiometric stability of the VIIRS Land bands
Assessments of the instrument corrections (e.g., temperature, RVS, polarization.)

Ongoing assessment of geolocation accuracy and adjustments to geolocation LUTs to improve earth-location accuracy will be performed by the geolocation team funded by the NPP Project Science Office. While not included in the Land PEATE effort, accurate geolocation is required by most land products and is vital to product quality assessment activities especially those that involve time series analyses over test sites.

3.3Provide suggested algorithm improvements to the IDPS via the PSWG.

The Land PEATE shall support the delivery of improved algorithms from the VLST to the IDPS via the PSWG. This activity includes the following:

a. Receiving an improved algorithm or software which implements it from a member of the VLST and integrating it into the MODAPS processing system. This step includes configuration management and unit and chain testing to verify that the algorithm works properly for a limited set of test cases.

b. Conducting larger scale science tests which incorporate the proposed improvement within existing production streams to produce 8 or more data days worth of products.

c. Packaging the improved software, its documentation and test results into deliveries to the NPP Project’s Integration and Test facility in a TBD agreed upon format and contents. The I&TF staff who are funded by the NPP Project will re-work the delivery into a format that can run in the IDPS and compare results of production runs in the I&TF with products generated in the PEATE. PEATE staff will be available to answer questions that the I&TF staff may have with regard to the software delivery.


3.4Process selected data subsets in support of Calibration/Validation and Algorithm improvement activities

The Land PEATE shall provide the capability to process any desired set of VIIRS data, up to and including the full mission data set, from RDRs to Land EDRs and the Level 3 gridded products used to evaluate quality of the EDRs.

3.5 Process RDRs to SDRS using standard 0 to Level 1 software provided by the IPO

The Land PEATE shall provide the capability to process RDRs to SDRs using the standard Level 0 to Level 1 software provided by the IPO. This will include the capability to accept alternative LUTs provided by the NVCST.

3.6 Process SDRs to Land EDRs using standard Land algorithms, software and LUTS provided by the IPO.

The Land PEATE shall provide the capability to process SDRs to Land EDRs using the standard Land algorithms, software and LUTS provided by the IPO.

3.7 Process SDRs to Land EDRs using the standard Land algorithms and software, and alternative LUTS generated locally by the VLST.

The Land PEATE shall provide the capability to process SDRs to Land EDRs using the standard Land algorithms and software, and alternative LUTS generated locally by the VLST.

3.8 Process SDRs to Land EDRs using alternative algorithms, software and LUTS provided by the VLST

The Land PEATE shall provide the capability to process SDRs to Land EDRs using alternative algorithms, software and LUTS provided by the VLST, to evaluate potential algorithm improvements and additional products.

3,9 Support Cataloging, search, browse, ordering, and distribution

The Land PEATE shall support cataloging, searching, ordering and distributing of all internally processed data products.

3.10 Additional Requirements

The Land PEATE shall support the following general requirements:

Shall be capable of unattended operations on at least one shift per day
Shall retrieve data from the SD3 server at an average rate of 1 day per day but have the capacity to retrieve data at rates of up to 5 data days per day such that no data products are lost from the 16-day rolling buffer on the SD3.
Shall have the processing resources to be able to complete a science test that involves generating 8 data-days of global products within 3 working days provided the science software runs without error.



Operations Concept

The Land PEATE operations concept is built on the MODIS Adaptive Processing System (MODAPS) originally developed to support MODIS processing and the integrated Land Team/ Science Data Support Team approach for software integration and testing and product quality assessment. The basic elements of this approach are:

An integrated team supporting all activities, including systems, software, data processing, product quality assessment, calibration, validation, and distribution;
Substantial processing capacity, to support rapid, multiple evaluations of potential algorithm improvements;
Close cooperation with the scientific community in algorithm and product evaluation and in product validation.

The following sections summarize the activities to be performed in support of VIIRS EDR assessment at the Land PEATE, followed by a summary of facilities to be provided.

4.1 VIIRS Land EDR Assessment

Assessment of a satellite-based sensor, such as VIIRS, should be both global in scope and should be a continuing process throughout the life of a satellite mission, particularly if the data products are to be used in a long-term, multi-sensor time series. For VIIRS, continuous validation is necessary for the assessment of the accuracy of the calibration across platforms.

4.1.1VIIRS Data Acquisition and Management

The complete set of VIIRS RDRs will be acquired as made available from the SD3. Selected SDRs and EDRs will also be acquired as needed for calibration and data product assessment. The fraction of data to be acquired is still TBD. The facilities for storing and managing the acquired data are described in Section 4.2.

4.1.2Pre-launch and Early Mission Testing and Verification

The Land PEATE will support several critical activities during the pre-launch phase of NPP. These include: verification of the science algorithm and operational processing code; evaluation of test versions of the EDRs, produced by the IDPS from simulated data and transmitted to the SD3; and quick-response processing of instrument test data sets, as soon as they are acquired from the SD3.

The processing code will be acquired and installed in the MODAPS to support verification and testing. The initial versions are expected to be the science algorithm code, which will be obtained by the NVLST and provided to the PEATE. As operational software becomes available in the I&TS, it will be acquired directly by the PEATE. The pre-launch versions of the software will be tested and verified using simulated data sets, either SDRs or RDRs. In order to fully exercise the logic of the land EDR algorithms, high-fidelity simulated data with realistic radiances over land and clouds and geographically correct land pixels will be required.

These same data sets, along with sample EDRs produced by IDPS, will also be used to test EDR evaluation procedures to be used during the mission. The types of evaluations to be performed are described in other sections, below.

Finally, these capabilities in the PEATE will be used to acquire and process the pre-launch instrument test data sets for analysis by the NVLST. The test data sets will be acquired from the SD3 as they become available, immediately processed as needed (i.e., RDR to SDR) and made available to the NVLST. This quick turnaround will be absolutely critical during instrument testing, to allow feedback from the NVLST to the VIIRS test team while the tests are still in progress. By analyzing test results during the actual testing, the NVLST will determine whether tests need to be re-run, either in original or modified form, to achieve the required instrument characterization result.

4.1.2 Evaluation Processing

To evaluate the efficacy of changes to the VIIRS SDR processing and the cross-calibration with MODIS/Aqua, we will develop the capability to process VIIRS RDRs in the MODAPS to produce land EDRs and Level 3 gridded products. We will use this capability to process limited amounts of VIIRS data for validation activities, and larger amounts of VIIRS data for global, long time-series analyses. This will enable evaluation of multiple versions of the VIIRS mission data, similar to that which was performed for MODIS science testing and the generation of MODIS experimental products.

4.1.3 Global Comparative Time Series Analyses

We will perform global comparative time-series analysis between VIIRS and MODIS standard, archived level-3 products, and repeat the entire analysis following any reprocessing campaign. For periods beyond the life of the MODIS mission, we will perform comparisons with data collected from land validation sites and other fine to moderate-resolution land sensors.


4.1.5 Scientific Community Involvement

The MODIS land team has held a series of successful workshops that focused on algorithms, quality assessment and validation results for suites of related land products (e.g. albedo, vegetation indices, land cover conversion, net primary productivity). We propose to follow this approach to engage the community in the evaluation and improvement of VIIRS land EDRs by holding workshops at NASA GSFC, which will be open to all interested members of the community. These will follow the same general format as previously used for the MODIS land workshops.

We will maintain a public Web site for all of our VIIRS activities. This will include results and proceedings of the workshops; descriptions of proposed algorithm changes and calibration methodologies; results of data analyses, processing tests and product evaluations; and sample products for outside evaluation.


4.2 Systems and Facilities

The systems and facilities of Code 922/923 and University of Maryland land science group provide an effective environment for performing the assessment of NPP VIIRS land products. By leveraging an existing infrastructure developed specifically for this type of work, NASA will receive the maximum benefit for a minimum of cost. This group has demonstrated expertise in producing global products for land imaging instruments (Landsat, AVHRR and MODIS) and performing algorithm updates, testing and quality assessment on the products. The computing systems and their staff are co-located with the LDOPE (Land Data Operational Production Evaluation) team, and VIIRS land science team members. The MODAPS is high-capacity (producing upwards of 3TB per day) production-proven system that has been providing high-quality science products to the land, atmosphere and ocean communities on a daily basis since February 2000.

4.2.1 Data Acquisition and Distribution

The MODAPS currently acquires MODIS data (Terra and Aqua) through an ftp server connected to the GES DAAC but has the capability to process data from the EDOS bent pipe when more rapid production is required. Standard products produced in MODAPS at the rate of 3TB per day are archived and distributed by three DAACs. The MODAPS system also distributes approximately over 500GB of products per day to the MODIS science team through subscriptions to facilitate their quality assessment, algorithm improvement and validation efforts. In addition results from science tests are maintained online for science team analysis and experimental, interim and selected Level 3 products are also available for ordering from web-based search/order/distribution system.

A similar approach will be followed to enable the VLST to rapidly access the results of VIIRS EDR product evaluation processing and large scale testing.

4.2.2 Development

Our group has the infrastructure, knowledge, and experience to support algorithm improvements for VIIRS land products and validation activities. The modular design of the MODAPS system allows developers to easily add new product generation software without changes to the production system itself. The similarity between MODIS and VIIRS instruments and the science algorithms is such that few changes are expected to be required in the production rules that govern all aspects of how products are generated.

4.2.3 Physical Facilities

The GSFC code 922 team has a well-supported facility located in Building 32 of Goddard's Greenbelt campus. The servers reside in a 4,000 square foot raised floor computer area. The critical infrastructure required to operate a large computer facility is already in place. The room is supported by 10 air handler units, and has 250KVA of uninterruptible power fed by dual building UPSes that are backed up by diesel generators. Environmental monitoring is active 24x7 as well with automated monitoring of fire and flood sensors as well as automated monitoring of computer systems to detect abnormal conditions in the system logs related to hardware errors. Hardware and software maintenance contracts are in place and kept up to date. Critical data and software are backed up to a server located in Building 33. The computing systems within the facility are compliant with the level of IT security required for special management attention systems and undergo security audits every 6 months. The systems are supported by staff located on site.