BEAT-II Record: GOME2_L1_radiance

Fields

The table below gives an overview of possible fields that can be returned from an ingestion of GOME2_L1_radiance data. The 'default' column indicates whether the specified data item is included in the ingestion by default or not. You can change which data items are ingested by making use of the include and exclude filter options (see below).

field name	type	dimensions	default	description
type	string		included	reference to the type of data in this record
time	double	[main]	included	The time of the measurement at end of integration time (in seconds since 2000-01-01 00:00:00)
latitude	double	[main]	included	Center latitude of the measurement
longitude	double	[main]	included	Center longitude of the measurement
corner_latitude	double	[main, 4]	included	Corner latitudes for the ground pixel of the measurement
corner_longitude	double	[main, 4]	included	Corner latitudes for the ground pixel of the measurement
spectral_radiance	double	[main, spectra]	included	Measured radiances
spectral_radiance_unit	string		included	photons/s.cm^2.sr.nm
wavelength	double	[main, spectra]	included	The nominal wavelength assignment for each of the detector pixels
wavelength_unit	string		included	nm
integration_time	double	[main]	included	The minimum integration time for all ingested bands
pixel_integration_time	double	[main, spectra]	excluded	The integration time for each detector pixel
scan_subset_counter	uint8	[main]	excluded	The relative index (0-31) of this measurement within a scan devided by 2 (giving a subset count in the range 0-15). The relative index in the scan is the relative index of the measurement in the MDR minus one (because the first readout in the MDR is from the previous scan) modulo 32
scan_direction	string	[main]	excluded	The scan direction for each measurement: 'forward', 'backward', or 'mixed' (for a measurement that consisted of both a forward and backward scan)
cloud_fraction	double	[main]	excluded	Effective cloud fraction
solar_zenith_angle_toa	double	[main]	excluded	Solar zenith angle at top of atmosphere
solar_azimuth_angle_toa	double	[main]	excluded	Solar azimuth angle at top of atmosphere
los_zenith_angle_toa	double	[main]	excluded	Solar zenith angle at top of atmosphere
los_azimuth_angle_toa	double	[main]	excluded	Solar azimuth angle at top of atmosphere

Ingestion filter options

Ingestion filter options are passed as a ',' or ';' separated string to the ingestion function. Each filter option needs to be formulated as option_name=value[ value value] (some options accept more than one value). For example, if a product has filter options 'altitude_min' and 'time_min' you can provide a filter option string "latitude_min=30,time_min=2003-01-01" to get all measurements at or above 30km and on or after January 1st 2003.

option name	type	description
product_version	integer	If multiple product format versions for this product type exist, you can use this filter option to select from which ones data should be ingested (i.e. you can provide a space-separated list of allowed product format version numbers). The list of available version numbers for this product type can be found in the CODA Product Format Definitions documentation.
data	string	You should never have to provide this filter option, since this type of record is the default type of data that is ingested for a GOME2_L1 product. You can, however, still explicitly ask for this type of record by using the filter option data=radiance.
include	string	Provide a space-separated list of field names to ingest additional data items (i.e. fields that are not ingested by default). You can use the special case include=* to indicate that you want to ingest all available data items.
exclude	string	Provide a space-separated list of field names that you don't want ingested. Mind that the exclude filter is processed after the include filter.
time_min	time*	Only include elements for which the value is greater than or equal to the provided filter option value
time_max	time*	Only include elements for which the value is less than or equal to the provided filter option value
time	time*	Only include elements that match the list of given filter option values
latitude_min	double	Only include elements for which the value is greater than or equal to the provided filter option value
latitude_max	double	Only include elements for which the value is less than or equal to the provided filter option value
latitude	double	Only include elements that match the list of given filter option values
longitude_min	double	Only include elements for which the value is greater than or equal to the provided filter option value
longitude_max	double	Only include elements for which the value is less than or equal to the provided filter option value
longitude	double	Only include elements that match the list of given filter option values
integration_time_min	double	Only include elements for which the value is greater than or equal to the provided filter option value
integration_time_max	double	Only include elements for which the value is less than or equal to the provided filter option value
integration_time	double	Only include elements that match the list of given filter option values
scan_subset_counter_min	integer	Only include elements for which the value is greater than or equal to the provided filter option value
scan_subset_counter_max	integer	Only include elements for which the value is less than or equal to the provided filter option value
scan_subset_counter	integer	Only include elements that match the list of given filter option values
scan_direction	string	Only include elements that match the list of given filter option values
cloud_fraction_min	double	Only include elements for which the value is greater than or equal to the provided filter option value
cloud_fraction_max	double	Only include elements for which the value is less than or equal to the provided filter option value
cloud_fraction	double	Only include elements that match the list of given filter option values
solar_zenith_angle_toa_min	double	Only include elements for which the value is greater than or equal to the provided filter option value
solar_zenith_angle_toa_max	double	Only include elements for which the value is less than or equal to the provided filter option value
solar_zenith_angle_toa	double	Only include elements that match the list of given filter option values
solar_azimuth_angle_toa_min	double	Only include elements for which the value is greater than or equal to the provided filter option value
solar_azimuth_angle_toa_max	double	Only include elements for which the value is less than or equal to the provided filter option value
solar_azimuth_angle_toa	double	Only include elements that match the list of given filter option values
los_zenith_angle_toa_min	double	Only include elements for which the value is greater than or equal to the provided filter option value
los_zenith_angle_toa_max	double	Only include elements for which the value is less than or equal to the provided filter option value
los_zenith_angle_toa	double	Only include elements that match the list of given filter option values
los_azimuth_angle_toa_min	double	Only include elements for which the value is greater than or equal to the provided filter option value
los_azimuth_angle_toa_max	double	Only include elements for which the value is less than or equal to the provided filter option value
los_azimuth_angle_toa	double	Only include elements that match the list of given filter option values
wavelength_min	double	Only include data where the wavelength is greater then or equal to the value provided (in nm)
wavelength_max	double	Only include data where the wavelength is less than or equal to the value provided (in nm)
band	string	Only include data from the specified bands ('band-1a', 'band-1b', 'band-2a', 'band-2b', 'band-3', 'band-4')

^* Time values can be passed as either 'yyyy-mm-dd', 'yyyy-mm-ddThh:mm:ss', 'yyyy-mm-ddThh:mm:ss.uuuuuu' or a floating point value indicating the amount of seconds since 2000-01-01 00:00:00.

Mapping description

Below a description is provided on how each data item in the ingested record was retrieved from the product.

The GOME2 spectral data in the GOME2 L1b product is stored inside MDRs. There are separate MDRs for Earthshine, Calibration, Sun, and Moon measurements. In addition there are also 'Dummy Records' (DMDR) that can be present when there is lost data in the product. With BEAT-II only Earthshine and Sun measurements can be ingested.

Each MDR roughly contains a single scan. However, an MDR does not exactly correspond 1-to-1 with a GOME-2 scan. This is an important fact to be aware of. The real situation is as follows:

Within a single scan (a scan takes 6 seconds) there are 16 Instrument Source Packets (covering 375ms each) coming from the satellite. Each ISP contains at most two readouts (there are two if the integration time for a band is 187.5ms (or 93.75ms)). The problem is that the two readouts of the first ISP of a scan contain the last measurement of the previous scan and the first measurement of the new scan. The second ISP contains data for measurements #2 and #3, the third for #4 and #5, etc. The last measurement of a scan will again be found in the first ISP of the next scan. Instead of shifting the data and grouping all data of a single scan together in a single MDR the Level 1a and Level 1b processors just place the MDR boundary at the start of the first ISP of a scan and terminate the MDR at the end of ISP 16. This means that in Level 1b (but also 1a) products the first measurement in an MDR will always be the last measurement of the previous scan.

Nearly all meta-data for a readout (time, geolocation, viewing/solar angles, etc.) in an MDR are filled taking into account this same shift. This means that for retrieving the geolocation of the first readout of an MDR from the GEO_EARTH_ACTUAL record, one will in fact get the geolocation information of the last backscan pixel of the previous scan. However, the integration time meta-data for the first readout in an MDR is not shifted this way (the GEO_EARTH information is also not shifted, by the way, and just contains the 32 geolocation pixels for the scan). As long as the integration time does not change from one scan to another this won't impact anything, but the L1 products will contain invalid metadata for the first MDR readout if there is a change of integration time between two consecutive scans. In that case the calculated geolocation, angles, etc. of the first readout are values based on the integration time of the _new_ scan instead of the _old_ scan (e.g. the ground pixel will thus either be too large or too small). The (relative) good news to this is that, if a change in integration time occurs, the last pixel readout of the final scan with the 'old' integration time will never be valid and will have undefined values in the product (this is because the instrument prematurely terminates the final readout if a scan configuration change occurs). This means that the readout that has the 'invalid' meta-data will never be a valid measurement anyway.

Because of all this, BEAT-II will exercise the following rules during ingestion:
1) the first readout of the first MDR will always be ignored (and you will never see the last readout of the last scan, because it won't be in the product)
2) the first readout after a change in measurement mode (i.e. earthshine vs. calibration vs. sun vs. moon) will be ignored
3) if a change in integration time occurs (for any of the bands) then the first readout (for all bands) of the next MDR will be ignored
4) if two MDRs are not continuous (i.e. there is a time gap) then the first readout of the second MDR will be ignored

GOME-2 uses 6 bands for the main spectra (1A, 1B, 2A, 2B, 3, and 4). Within a scan each band can have its own integration time. There will be at most 32 readouts per scan (corresponding with an integration time of 187.5ms). If the integration time is 375ms, 750ms, 1.5s, 3s or 6s there will be 16, 8, 4, 2, or 1 measurement(s) respectively for this band in a scan. Some readouts may even cover multiple scans if the integration time is larger than 6s. BEAT-II will combine the data for all bands into a single two-dimensional pixel_readout array. Because of the differences in integration time this means that for some bands there will be gaps in the pixel_readout array. These gaps will be filled with NaN values. BEAT-II will always use the minimum integration time of all ingested bands to determine the time resolution for the BEAT-II record fields. For instance, if the minimum integration time for a scan is 1.5s you will find 4 entries in the BEAT-II record fields for this scan. All meta-data, such as geolocation, angles, etc. will also be ingested for this minimum integration time (i.e. you will see co-added meta-data if the integration time is > 187.5ms). The minimum integration time is calculated based on those bands from which actual data is ingested. This means that the minimum integration time can change depending on the wavelength filter that was applied

The filtering on time and geolocation will always be performed using the 187.5ms resolution. A measurement with a higher integration time will only be included if each of its 187.5ms sub-parts have not been filtered out (this also holds for measurements with an integration time > 6s). If spectra from multiple bands with different integration times are ingested then the measurements with a high integration time will only be ingested of all subpixels of the measurements with the minimum integration time are also ingested. The measurement with a high integration time will be put in the same 'row' as the first corresponding minimum integration time measurement (i.e. measurements of different bands are aligned according to start time of the measurement).

If the band configuration changes somewhere during the orbit and a band filter is given, then we only include detector pixels that are inside the requested band for the duration of the whole orbit. i.e. detector pixels that change band during the orbit will allways be excluded when a band filter is given

field name	mapping description
time	path	/MDR[]/Earthshine/RECORD_HEADER/RECORD_START_TIME
	description	The record start time is the start time of the scan and thus the start time of the second readout in the MDR. The start time for readout i (0..31) is thus RECORD_START_TIME + (i - 1) * 0.1875 and the time at end of integration time (which is the time that is returned) is RECORD_START_TIME + i * 0.1875
latitude	path	/MDR[]/Earthshine/GEO_EARTH_ACTUAL/CENTRE_ACTUAL[INT_INDEX[band_id],]/latitude
	description	The integration time index (INT_INDEX[band_id]) is based on the id of the band with the minimum integration time (limited to those bands that are ingested)
longitude	path	/MDR[]/Earthshine/GEO_EARTH_ACTUAL/CENTRE_ACTUAL[INT_INDEX[band_id],]/longitude
	description	The integration time index (INT_INDEX[band_id]) is based on the id of the band with the minimum integration time (limited to those bands that are ingested)
corner_latitude	path	/MDR[]/Earthshine/GEO_EARTH_ACTUAL/CORNER_ACTUAL[INT_INDEX[band_id],,]/latitude
	description	The integration time index (INT_INDEX[band_id]) is based on the id of the band with the minimum integration time (limited to those bands that are ingested). The corners ABCD are reordered as BDCA
corner_longitude	path	/MDR[]/Earthshine/GEO_EARTH_ACTUAL/CORNER_ACTUAL[INT_INDEX[band_id],,]/longitude
	description	The integration time index (INT_INDEX[band_id]) is based on the id of the band with the minimum integration time (limited to those bands that are ingested). The corners ABCD are reordered as BDCA
spectral_radiance	path	/MDR[]/Earthshine/BAND_1A[]/RAD, /MDR[]/Earthshine/BAND_1B[]/RAD, /MDR[]/Earthshine/BAND_2A[]/RAD, /MDR[]/Earthshine/BAND_2B[]/RAD, /MDR[]/Earthshine/BAND_3[]/RAD, /MDR[]/Earthshine/BAND_4[]/RAD
spectral_radiance_unit	description	Will be set to "photons/s.cm^2.sr.nm"
wavelength	path	/MDR[]/Earthshine/WAVELENGTH_1A[], /MDR[]/Earthshine/WAVELENGTH_1B[], /MDR[]/Earthshine/WAVELENGTH_2A[], /MDR[]/Earthshine/WAVELENGTH_2B[], /MDR[]/Earthshine/WAVELENGTH_3[], /MDR[]/Earthshine/WAVELENGTH_4[]
wavelength_unit	description	Will be set to "nm"
integration_time	path	/MDR[]/Earthshine/INTEGRATION_TIMES[band_id]
	description	The integration time index (band_id) is the id of the band with the minimum integration time (limited to those bands that are ingested)
pixel_integration_time	path	/MDR[]/Earthshine/INTEGRATION_TIMES[]
scan_direction	path	/MDR[]/Earthshine/INTEGRATION_TIMES[band_id]
	description	When the integration time is higher than 1.5s we are dealing with a mixed pixel, otherwise the scan direction is based on the subset_counter of the measurement (0-11 = forward, 12-15 = backward)
cloud_fraction	path	/MDR[]/Earthshine/CLOUD/FIT_2[]
	description	If the minimum ingested integration time > 187.5ms then the corresponding cloud fractions will be combined using averaging. The cloud fraction will be set to NaN if FIT_MODE in the CLOUD structure is not equal to 0 or if FIT_2 is set to a fill value (even when this holds for only one of the averaged items).
solar_zenith_angle_toa	path	/MDR[]/Earthshine/GEO_EARTH_ACTUAL/SOLAR_ZENITH_ACTUAL[INT_INDEX[band_id],1,]
	description	The integration time index (INT_INDEX[band_id]) is based on the id of the band with the minimum integration time (limited to those bands that are ingested)
solar_azimuth_angle_toa	path	/MDR[]/Earthshine/GEO_EARTH_ACTUAL/SOLAR_AZIMUTH_ACTUAL[INT_INDEX[band_id],1,]
	description	The integration time index (INT_INDEX[band_id]) is based on the id of the band with the minimum integration time (limited to those bands that are ingested)
los_zenith_angle_toa	path	/MDR[]/Earthshine/GEO_EARTH_ACTUAL/SAT_ZENITH_ACTUAL[INT_INDEX[band_id],1,]
	description	The integration time index (INT_INDEX[band_id]) is based on the id of the band with the minimum integration time (limited to those bands that are ingested)
los_azimuth_angle_toa	path	/MDR[]/Earthshine/GEO_EARTH_ACTUAL/SAT_AZIMUTH_ACTUAL[INT_INDEX[band_id],1,]
	description	The integration time index (INT_INDEX[band_id]) is based on the id of the band with the minimum integration time (limited to those bands that are ingested)