Neural Network Based CERES/Terra ADMs

The CERES/Terra SSF dataset includes about 5% of CERES footprints with missing imager information or insufficient MODIS data for a reliable scene ID. The frequency of occurrence of these fields-of-view depends on imager viewing geometry, geographic location, and on certain cloud conditions, and can reach up to 50% of data locally or for a specific scene type (see validation plots below). In order to avoid any systematic bias in radiative budget data, it is very important to provide accurate TOA flux values for such footprints. This requires ADMs that can be used with the CERES measurements alone.

Motivation:

• Provide accurate flux values for CERES footprints with excessive "non-retrieval" imager information.
• ANN technique advantage: highly non-linear multi-dimensional continuous transfer functions.
• Application of pre-trained ANN-based ADMs to satellite data without complex scene ID over field-of-view.

Reference:

Details on building of training sets, ANN training algorithm and results on reproducing CERES ADMs are available in
• K. Loukachine and N.G. Loeb: "Application of an Artificial Neural Network Simulation for Top-of-Atmosphere Radiative Flux Estimation from CERES", Journal of Atmospheric and Oceanic Technology, vol. 20, No. 12, p. 1749-1757, 2003 ( PDF file ).
• K. Loukachine and N.G. Loeb : Top-of-Atmosphere Flux Retrievals from CERES Using Artificial Neural Networks. J. Remote Sensing of Environment , v. 93, No. 3, 381-39, 2004 ( PDF file ).

ANN-based CERES/Terra Edition-2 ADMs are available online: ANN Calculator

General ANN structure 5IN-7TS-11TS-1L (SW case)

For our purpose we use partially connected feed-forward error back-propagation multi-layer neural neworks. Generalized Delta rule with variable learning rate and a constant momentum is used as the ANN training algorithm.

L1, L2 - layers of neurons with sigmoidal activation function; L3 - linear output single-neuron layer. The first neuron layer is divided into two neuron clusters: scene ID and primary input variables. W1, W2, W3 - neuron connection weght matrices; N - normalization of the input variables; F - fuzzification of the input (for training only).

Generally, ANN algorithm can be divided into three phases:
• creating representative traning sets;
• training the ANN;
• application of ANN to the data of iterest (validation is assumed).
Training sets for Edition-2 CERES/Terra ANN-based ADMs are built on one year (2001) of RAPS and along-track data, both day-time and night-time. The data stratified in the chosen ANN input variables independently, and for every bin the mean ADM value and STD of distribution are calculated using CERES footprints with reliable imager information. The goal is to make ANN reproduce the ADM values over entire training set with minimal error. To avoid noisy configurations, each training set has an upper limit for STD of ADM value within configuration (bin). On the figure below it is illustrated for the SW ocean and dark desert scene types:

Noise Cut, Ocean and Dark Desert scene types

Standard deviation of the training set ADM versus glint angle. Every entry on the plot represents a configuration in the training set. The color scale is number of training configuratrions. Noise cut is shown with a horizontal black line. As it is expected, the noise is cut off uniformly for the land scene types, and mostly at small glint angle for the ocean.

The configurations with STD(ADM) above the limit are exluded from ANN training. As result, they are not considered for flux retrieval. All training sets for all scene types and channels (SW, LW and WN), exept the ocean scene type, consisted from 10,000 to 15,000 training configurations. The ocean (SW, LW, WN) training set consisted of about 25,000 training configurations. The off-limit configurations range from 2% to 5% in case of land scene types, and about 10% in case of ocean scene types (in sun-glint region the original Terra/CERES ADMs become noisy).

Day-Time:

Day-Time SW and LW training set stratification (scene ID group of variables indicated with blue)
	Variable SW	N bins SW	Bin Width	Variable LW	N bins LW	Bin Width
1	VZA	7	10	VZA	7	10
2	RAZ	9	20	RAZ	6	30
3	LWR	15	10	SWR	20	15
4	SZA	9	10	PW	10	1.0
5	SWR	30	10	LWR	30	5

Training was performed with 10,000 iterations for all ANN scene types and channels (SW, LW and WN). The number of iterations is a compromise between achieving the lowest possible error and computation time. Figure below illustrates training process (right) and the result at the end of training (left) for shorwave Bright Desert ANN scene type. The ANN training errors are defined as mean (bias) and STD of the difference between the training and ANN ADM values over entire set.

Shortwave Bright Desert scene type ANN training. RIGHT: Decrease of the Error Index with training iterations. LEFT: ANN value of ADM versus training set ADM value. The mean and STD values of the difference between two over a training set are considered as ANN training errors.

Day-Time ANN Scenes and Training Errors:
Index	ANN Scene	IGBP Types	SW Bias (%)	SW STD (%)	LW Bias (%)	LW STD (%)
1	EF	1, 2	0.110	3.419	0.011	0.984
2	DF	3, 4, 5	0.146	3.872	0.007	0.812
3	WS	6, 8	0.128	3.572	0.011	0.909
4	DD	7, 18	0.106	3.499	0.006	0.719
5	BD	16	0.084	2.965	0.008	0.667
6	WB	17	0.129	4.062	0.010	0.913
7	GR	9, 10, 11	0.120	3.481	0.010	0.901
8	CC	12, 13, 14	0.120	3.405	0.009	0.852
9	SN	15, 19	0.118	3.657	0.013	1.022
10	SI	20	0.122	3.826	0.005	0.620

The ANN scene types are defined by using IGBP geomap (see the table above): evergreen forests (EF), decidous forects (DF), shrubs (WS), dark desert (DD), bright desert (BD), water bodies (WB), grass (GR), crops and urban areas (CC), snow-ice land (SN), sea-ice (SI).

When training is successfully finished, ANN provides a very complex 5-dimentional transfer function. Here are examples of 2-dimentional projections in VZA and RAZ for Shortwave ANN dark desert and ocean scene types (fixed variables: LWR = 90 W/m2sr, SWR = 100 W/m2sr, SZA = 10 and 80 deg.)

Night-Time:

Night-Time LW and WN training set stratification (scene ID group of variables indicated with blue)
	Variable LW	N bins LW	Bin Width	Variable WN	N bins WN	Bin Width
1	VZA	7	10	VZA	7	10
2	RAZ	6	30	RAZ	6	30
3	SZA	9	10	SZA	9	10
4	PW	10	1.0	PW	10	1.0
5	LWR	40	3	WNR	40	1.5

Night-Time ANN Scenes and Training Errors:
Index	ANN Scene	IGBP Types	LW Bias (%)	LW STD (%)	WN Bias (%)	WN STD (%)
1	EF	1, 2	0.023	1.479	0.072	2.673
2	DF	3, 4, 5	0.013	1.016	0.032	1.735
3	WS	6, 8	0.028	1.659	0.095	3.032
4	DD	7, 18	0.015	1.087	0.036	1.929
5	BD	16	0.017	1.203	0.039	2.007
6	WB	17	0.010	0.925	0.039	1.950
7	GR	9, 10, 11	0.021	1.370	0.057	2.535
8	CC	12, 13, 14	0.014	1.151	0.042	2.076
9	SN	15, 19	0.012	1.046	0.016	1.331
10	SI	20	0.006	0.830	0.019	1.273

Validation:

The first two weeks of February 2001 CERES/Terra RAPS and along-track data are used for ANN-based ADMs validation. The ANN-derived flux is compared with CERES/Terra Edition-2A empirical ADMs flux.

Validation Plots (All-sky, Terra, February 2001, Edition-1A cloud mask):
	F(ADM) - F(ANN)	F(ADM) - F(ANN) vs Scene Type	F(ADM) - F(ANN) vs VZA	ANN Fraction
SW	plot	plot	plot	map 3.52% (Global RAPS) 5.61% (Global FAPS)
LW Day-Time	plot	plot	plot	map 5.19% (Global RAPS)
LW Night-Time	plot	plot	plot	map 3.49% (Global RAPS)

Summary:

• ANN-based CERES/Terra Edition-2 SW, LW and WN channel TOA fluxes have very small mean bias from empirically developped ADMs for all scene types.
• The instantaneous TOA flux errors for ANN-based ADMs are about factor 1.4 larger for SW, 1.2 for LW night-time, and practically the same for LW day-time. The increase in instantaneous errors is due to lack of imager information.
• It has been shown by K. Loukachine and N.G. Loeb (JTECH 2003) the ANN technique is more accurate than ERBE ADMs applied to new data.

HOME

Overview

ADMs

Validation Results

Publications

Conferences

Production Code

Relevant Links

Responsible NASA Official: Dr. Bruce A. Wielicki

Web Curator: Dr. K. Loukachine K.Loukachine@larc.nasa.gov

Last updated:

Feedback on Langley Products and Services