remote sensing Article
Quality Assurance Framework Development Based on Six New ECV Data Products to Enhance User Confidence for Climate Applications Joanne Nightingale 1, *, Klaas Folkert Boersma 2,3 ID , Jan-Peter Muller 4 ID , Steven Compernolle 5 , Jean-Christopher Lambert 5 , Simon Blessing 6 , Ralf Giering 6 , Nadine Gobron 7 , Isabelle De Smedt 5 ID , Pierre Coheur 8 , Maya George 9 , Jörg Schulz 10 and Alexander Wood 11 1 2 3 4 5
6 7 8 9 10 11
*
National Physical Laboratory, Teddington TW11 0LW, UK Royal Netherlands Meteorological Institute, Utrechtseweg 297, 3731 GA De Bilt, The Netherlands;
[email protected] Wageningen University, 6700 AA Wageningen, The Netherlands Imaging Group, Mullard Space Sciences Laboratory, University College London, Department of Space and Climate Physics, Holmbury, St Mary RH5 6NT, UK;
[email protected] Royal Belgian Institute for Space Aeronomy (BIRA-IASB), Ringlaan-3-Avenue Circulaire, B-1180 Brussels, Belgium;
[email protected] (S.C.);
[email protected] (J.-C.L.);
[email protected] (I.D.S.) FastOpt GmbH, Schanzenstraße 36, D-20357 Hamburg, Germany;
[email protected] (S.B.);
[email protected] (R.G.) European Commission, Joint Research Centre (JRC), Via E. Fermi, 2749, 21027 Ispra VA, Italy;
[email protected] Atmospheric Spectroscopy, Quantum Chemistry and Photophysics, Université Libre de Bruxelles, 50 avenue F. D. Roosevelt, B-1050 Brussels, Belgium;
[email protected] Laboratoire Atmosphères, Milieux, Observations Spatiales (LATMOS)/IPSL, boîte 102, Sorbonne Université, 4 place Jussieu, 75252 Paris Cedex 05, France;
[email protected] European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT), Eumetsat Allee 1, D-64295 Darmstadt, Germany;
[email protected] CGI, Keats House, The Office Park, Springfield Drive, Leatherhead KT22 7LP, UK;
[email protected] Correspondence:
[email protected]; Tel.: +44-777-554-5275
Received: 8 June 2018; Accepted: 31 July 2018; Published: 9 August 2018
Abstract: Data from Earth observation (EO) satellites are increasingly used to monitor the environment, understand variability and change, inform evaluations of climate model forecasts, and manage natural resources. Policymakers are progressively relying on the information derived from these datasets to make decisions on mitigating and adapting to climate change. These decisions should be evidence based, which requires confidence in derived products, as well as the reference measurements used to calibrate, validate, or inform product development. In support of the European Union’s Earth Observation Programmes Copernicus Climate Change Service (C3S), the Quality Assurance for Essential Climate Variables (QA4ECV) project fulfilled a gap in the delivery of climate quality satellite-derived datasets, by prototyping a generic system for the implementation and evaluation of quality assurance (QA) measures for satellite-derived ECV climate data record products. The project demonstrated the QA system on six new long-term, climate quality ECV data records for surface albedo, leaf area index (LAI), fraction of absorbed photosynthetically active radiation (FAPAR), nitrogen dioxide (NO2 ), formaldehyde (HCHO), and carbon monoxide (CO). The provision of standardised QA information provides data users with evidence-based confidence in the products and enables judgement on the fitness-for-purpose of various ECV data products and their specific applications.
Remote Sens. 2018, 10, 1254; doi:10.3390/rs10081254
www.mdpi.com/journal/remotesensing
Remote Sens. 2018, 10, 1254
2 of 21
Keywords: essential climate variables; climate data records; earth observation satellites; quality assurance; traceability; user requirements; climate applications; surface albedo; LAI; FAPAR; NO2 ; HCHO; CO
1. Introduction Climate change mitigation and adaptation have risen to the top of the agenda for many governments and international organisations [1]. In particular, the Paris Agreement from 2015 [2], aiming at strengthening the global response to the threat of climate change, is requesting systematic observation of the climate system. This has led to the establishment of space and research agency programs dedicated to increasing scientific understanding of the Earth system and its response to natural and/or human-induced changes. Key to this is the derivation of quantitative variables describing the chemical and physical properties of the biosphere using Earth observation (EO) satellites. More recently, emphasis has been placed on the development of climate data records (CDRs) of essential climate variables (ECVs) to characterise and monitor long-term (20+ years) trends and fluctuations. There are currently 54 ECVs defined by the Global Climate Observing System (GCOS) spanning the atmospheric, oceanic, and terrestrial domains [3,4]. These variables can be derived directly from in situ observations or indirectly from remote-sensing instruments flown on airborne or satellite platforms. Fundamental to the scientific understanding of the Earth system, and its response to change and progress in policymaking, is a rigorous quantification of the accuracy and validity of these CDRs produced from EO satellites [3–5]. Although EO data and products are widely available, it is rare for them to have reliable and fully traceable information concerning their generation process and their quality. Setting achievable accuracy requirements that can be quantified with confidence is a challenging task that is dependent on the intended use of the data. Both the GCOS and the World Meteorological Organisation’s (WMO) observing systems capability analysis and review (OSCAR) tool publish and regularly review the quality requirements that satellite-derived variables should satisfy to support the work of the United Nations Framework Convention on Climate Change (UNFCCC), the Intergovernmental Panel on Climate Change (IPCC), and the WMO programmes. But these requirements do not specifically address specific applications, and their usage to quantify confidence in existing data products remains difficult. The situation is exacerbated, because different versions of the same ECV parameter are offered by various data providers. For example, an internet search (2018) revealed that upward of 30 satellite-derived leaf area index (LAI) and fraction of absorbed photosynthetically active radiation (FAPAR) data products are available for download, with even more for other ECVs such as sea surface temperature (~55), soil moisture (~62), and ozone and aerosols (~180). Further, most operationally derived ECV products adhere to different definitions and assumptions, which are not standardised among the international EO and ecological communities. These data products are created with independent or multiple sources of EO data using an array of retrieval algorithms and assumptions. They are also provided at different spatial and temporal resolutions over varying time periods. Regulatory frameworks requiring EO data and product producers to be held accountable for ensuring the quality, accuracy, and validity of the information provided do not currently exist nor do the standards against which data quality should be monitored [6]. However, given the increasingly prominent role that quantitative EO products assume in climate monitoring applications, it is inevitable that the quality of these data will come under increasing scrutiny in the future [6]. There is a clear requirement for continued investment by data product providers for the following: (1) detailed assessment of EO data product quality including characterisation of their associated uncertainties; (2) provision of this product quality information in a standardised and comprehensible format to help data users navigate the wealth of ECV data products available to them and ensure they are applying
Remote Sens. 2018, 10, 1254
3 of 21
the best data for their application; and (3) progression of internationally endorsed methods and good practices for this purpose. 2. Quality Assurance for Essential Climate Variables (QA4ECV) Quality Assurance for Essential Climate Variables (QA4ECV) was a European Union (EU) Seventh Framework Programme (FP7) funded project (2014–2018) comprising of a partnership of key European scientists, data providers, and developers of future climate services, as well as a national metrology institute. The partners have significant roles in the following: international metrology; EO coordinating bodies such as the Committee on Earth Observation Satellites (CEOS) and the Coordination Group for Meteorological Satellites (CGMS); space programs such as the European Space Agency’s Climate Change Initiative (ESA CCI), the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT) operational provision of CDRs, and international satellite data validation activities; ground-based reference measurement networks such as the Network for the Detection of Atmospheric Composition Change (NDACC); and various related FP7 and H2020 projects such as CHARMe (CHARacterization of Metadata), GAIA-CLIM (Gap Analysis for Integrated Atmospheric ECV Climate Monitoring and FIDUCEO (Fidelity and uncertainty in climate data records from Earth Observations). The project had three main objectives: (1) the development of a generic system for quality assurance of satellite data products that can be applied to many ECVs as a prototype of a sustainable service; (2) the generation of multi-decadal CDRs for atmospheric and terrestrial ECVs that are based on inter-satellite calibrated data and state-of-the-art retrievals and are traceable with uncertainty metrics; and (3) engagement with stakeholders, governance bodies, and end-users to demonstrate how trusted satellite data and a reliable means of interoperability can facilitate users in judging the fitness-for-purpose of the ECV CDRs. All project information is available online at: http://www. qa4ecv.eu/. 3. QA4ECV QA System The purpose of developing and implementing a quality assurance (QA) system is twofold: (1) to provide ECV data product producers/science teams with the necessary resources (internationally endorsed tools, standards, methodologies) to develop products with embedded QA information that is presented in a clear and common format throughout the EO community; and (2) to provide data users with robust QA information as a means to quantitatively assess uncertainty and fitness-for-purpose of the data and derived products. The provision of such QA information demonstrates the traceability of products and simplifies comparisons between the same ECV produced by independent science teams. It also provides data users with evidence-based confidence in the products and enables judgement on the fitness-for-purpose of various ECV CDRs for their specific applications. Figure 1 outlines the QA system framework. Essentially, all the data and methodologies used to derive these data products (i.e., satellite, ancillary (climate/elevation models, etc.), and reference (in situ or model data used to calibrate and validate the algorithms)) should go through a quality checking process before being made available for climate data usage. The QA service components are highlighted in the grey box and will be described in more detail throughout the following sections. The utility of this QA system is demonstrated on the six QA4ECV data products, which are described in Section 4: albedo, leaf area index (LAI), fraction of absorbed photosynthetically active radiation (FAPAR), nitrogen dioxide (NO2 ), formaldehyde (HCHO), and carbon monoxide (CO). 3.1. QA System Development The QA4ECV QA system has been developed over a four-year period and is described in detail within the service specification document [7]. The initial requirements were scoped within a user requirements activity that employed a survey to gauge the current state of and need for quality assurance in satellite-derived data products [8]. The QA4ECV QA system framework aligns with IPCC guidelines [9] and builds upon other relevant and successful EU projects that consider EO data
Remote Sens. 2018, 10, 1254
4 of 21
quality and provenance issues, for example CHARMe [10] and CORE-CLIMAX (COordinating Earth observation data validation for RE-analysis for CLIMAte ServiceS) (http://www.coreclimax.eu/), as well as international coordination bodies including the CEOS Working Group on Calibration and Validation (WGCV), the joint CEOS-CGMS Working Group on Climate, and GCOS, among others. In order to ensure content is current and captures relevant findings and user-endorsed methods from other initiatives, a review of 12 projects and initiatives dedicated to improving the quality of satellite-derived data streams was conducted [11]. The QA4ECV system collates the following types of quality indicators (QIs) associated with EO-derived ECV products: product details; algorithm traceability; quality flags; validation; Remote Sens. 2018, 10, x FOR PEER REVIEW uncertainties; and assessment against standards. 4 of 22
Figure 1. The Quality Assurance for Essential Climate Variables (QA4ECV) quality assurance (QA) Figure 1. The Quality Assurance for Essential Climate Variables (QA4ECV) quality assurance (QA) system framework framework overview. overview. The The system system comprises comprises aa set set of of six six quality quality indicator indicator (QI) (QI) categories categories to to system extract meaningful quality information about the data products that users should take into account extract meaningful quality information about the data products that users should take into account when applying applying the the data data for for climate climateapplications. applications. when
3.1. QA System Development 3.1.1. Product Details The QA4ECV QA system has been developed over a four-year period and is described in detail Product details summarise the basic meta-information about the product, including for within the service specification document [7]. The initial requirements were scoped within a user example: product DOI (Digital Object Identifier), spatial resolution, temporal resolution, spatial requirements activity that employed a survey to gauge the current state of and need for quality coverage, temporal coverage, in situ/reference datasets, and satellite and/or airborne datasets used. assurance in satellite-derived data products [8]. The QA4ECV QA system framework aligns with Documentation, which describes the data product, including the algorithm theoretical basis document IPCC guidelines [9] and builds upon other relevant and successful EU projects that consider EO data (ATBD) and product user manual (PUM)/product user guide (PUG)/product specification document quality and provenance issues, for example CHARMe [10] and CORE-CLIMAX (COordinating Earth (PSD), is also captured here. observation data validation for RE-analysis for CLIMAte ServiceS) (http://www.coreclimax.eu/), as well as international coordination 3.1.2. Algorithm Traceability Chain bodies including the CEOS Working Group on Calibration and Validation (WGCV), the joint CEOS-CGMS Working Group on Climate, and GCOS, among others. A QA4ECV traceability chain is a diagrammatic and partly interactive representation of the In order to ensure content is current and captures relevant findings and user-endorsed methods from processing steps taken to produce the final data product. It shows sub-processing chains and other initiatives, a review of 12 projects and initiatives dedicated to improving the quality of satelliteintermediate products/parameters, as well as provides a short description of each step and where derived data streams was conducted [11]. The QA4ECV system collates the following types of quality to find more detail on the process implemented. The traceability chain aids a user in understanding indicators (QIs) associated with EO-derived ECV products: product details; algorithm traceability; the data production and the assumptions that are made during implementation. It also helps quality flags; validation; uncertainties; and assessment against standards. producers identify and understand potential sources of discrepancies between two similar data products produced by different methods or algorithms. The traceability chains for the six QA4ECV 3.1.1. Product Details Product details summarise the basic meta-information about the product, including for example: product DOI (Digital Object Identifier), spatial resolution, temporal resolution, spatial coverage, temporal coverage, in situ/reference datasets, and satellite and/or airborne datasets used. Documentation, which describes the data product, including the algorithm theoretical basis document (ATBD) and product user manual (PUM)/product user guide (PUG)/product specification
Remote Sens. 2018, 10, 1254
5 of 21
demonstrator products are shown in Section 4, and interactive versions can be found at: http: //www.qa4ecv.eu/ecvs/. 3.1.3. Quality Flags Quality flags (QFs) are provided as a product data layer and indicate quality information about the product at the ground pixel level. There are currently no standards specifying if and what QFs should be provided with a product, and therefore, the content can vary in complexity and detail of information for different data products. Commonly implemented flags have been recommended in the QA system to provide information such as the following: number of observations used in the calculation; snow/cloud cover; back-up algorithm implementation; fill-values utilised; pixel-based uncertainty estimates, and so on. 3.1.4. Validation and Inter-Comparison Validation is the process of assessing by independent means the quality of the data products derived from the system outputs [12]. The content of the validation section is dependent on the ECV domain chosen; however, all data producers are asked to provide the validation report(s), comparisons with independent reference data, and comparisons with other satellite-derived ECV datasets (i.e., inter-comparisons). Given the ambiguity associated with determining if a global data product is “validated”, a hierarchical approach to classify validation stages was adopted by CEOS through a consensus of the land product validation (LPV) community [13]. While for the atmospheric domain, the data producer is asked to provide information about the validation protocol steps, to which the product has been subjected (i.e., those developed in the context of CEOS initiatives and ESA projects and tailored in the prototype QA/Validation Service for Atmospheric ECV Precursors—Detailed Processing Model [14]. Building on this prototype, the QA4ECV Atmosphere ECV Validation Server, available online at https://qa4ecv-dev.stcorp.nl, was implemented to validate all QA4ECV atmospheric ECV datasets and transferred recently to the Sentinel-5p Mission Performance Centre for the operational validation service for TROPOMI (the TROPOspheric Monitoring Instrument) atmospheric data products (see http://www.tropomi.eu). 3.1.5. Uncertainties The QA system captures details about the uncertainty information and derivation of the uncertainty estimates associated with the ECV data products. This includes information on how the uncertainties from each dataset have been included into the final uncertainty estimate and how uncertainties introduced at each stage of the processing have been accounted for. The concept of metrological traceability with EO data and products is a multi-faceted problem that—though it is being tackled with greater detail in dedicated projects such as FIDUCEO (www.fiduceo.eu) and others—requires considerable further research. 3.1.6. Assessment against Standards This QI of the QA System seeks to determine the fitness-for-purpose of the data product for various applications, including climate-based applications, and the degree to which the data producers follow good practises (e.g., for uncertainty estimation and validation in generating the data products). Two key internationally recognised standardised references are addressed, including the GCOS ECV product requirements for climate applications [3], as well as the core climax system maturity matrix scores [15]. GCOS—The ECV product target numerical requirements for climate applications can be found within GCOS-200 [3]. The requirements specify the physical quantities (products), as well as the frequency, resolution, uncertainty, and stability, necessary to meet climate monitoring needs. System maturity matrix (SMM)—The maturity model for assessing the completeness of CDR production systems developed within the CORE-CLIMAX project has been applied [15].
Remote Sens. 2018, 10, x FOR PEER REVIEW
6 of 22
System maturity matrix (SMM)—The maturity model for assessing the completeness of CDR production systems developed within the CORE-CLIMAX project has been applied [15]. The CORECLIMAX maturity model is an adaptation of the model of Bates et al. (2012) [16], which was revised Remote Sens. 2018, 10, 1254 6 of 21 to be more generic so that it can be applied not only for satellite data sets but for all CDRs (in situ, combined satellite and in situ, and reanalyses). The aim of adopting the SMM in the QA4ECV The CORE-CLIMAX maturity model is an adaptation of of thethe model Bates to et al. (2012)that [16],they which (described in [15]) is to evaluate the production process ECVofCDRs ensure follow was revised to be more generic so that it can be applied not only for satellite data sets but for all best practices for science, engineering, and utilisation; it is not to assess the quality of the data itself. CDRs (in situ, combined satellite and in situ, and reanalyses). The aim of adopting the SMM in the The results of this exercise help the data user to build confidence in data producers that systematically QA4ECV (described in [15]) is to evaluate the production process of the ECV CDRs to ensure that apply best practises. In addition, the maturity scores help to determine the strengths and weaknesses they follow best practices for science, engineering, and utilisation; it is not to assess the quality of the in the data record generation Thisthe information useful for data record producers data itself. The results of this process. exercise help data user to is build confidence in data producers that and agencies responsible for such data products to steer activities to mitigate weaknesses in the process systematically apply best practises. In addition, the maturity scores help to determine the strengths (e.g.,and insufficient validation activities). This should indeed lead to improved data quality. weaknesses in the data record generation process. This information is useful for data record producers and agencies responsible for such data products to steer activities to mitigate weaknesses in
3.2. QA System Process the process (e.g., insufficient validation activities). This should indeed lead to improved data quality. In order for the QA4ECV QA system to be successful and widely adopted, it must be simple and 3.2. QA System Process intuitive, offer a wide range of tools and resources relevant to multiple EO disciplines, be In order for the QA4ECV QA system to be successful and widely adopted, it must be simple and documented appropriately, and ensure that the evaluation process is streamlined and follows a userintuitive, offer a wide range of tools and resources relevant to multiple EO disciplines, be documented friendly ‘checklist’-type strategy. Figure 2 outlines theis process of the QA system, the appropriately, and ensure that the evaluation process streamlined andQA4ECV follows a user-friendly specific detail of each component is discussed below. To effectively develop, implement, and operate ‘checklist’-type strategy. Figure 2 outlines the process of the QA4ECV QA system, the specific detail the system, organisational structure skill sets of specific personsand involved be defined of each the component is discussed below.and To the effectively develop, implement, operatemust the system, for athe suite of tasks. This section theofrequirements for the QAmust evaluation organisation organisational structure and outlines the skill sets specific persons involved be defined for a suite of that tasks. This section outlines the requirements for the QA evaluation organisation that should implement should implement the QA4ECV framework, as well as the responsibilities of the ECV CDR the QA4ECV framework, as well as the responsibilities of the ECV CDR developing organisation. developing organisation.
Figure 2. Overview of the QA4ECV includingthe the role product producers Figure 2. Overview of the QA4ECVQA QAsystem system process, process, including role of of product producers and and the QA administration. Within the QA4ECV project, the QA administration refers to domain the QA administration. Within the QA4ECV project, the QA administration refers to domain experts experts for for land products(National (National Physical Laboratory, NPL) and atmosphere products (Royal Belgian land products Physical Laboratory, NPL) and atmosphere products (Royal Belgian Institute for Space Aeronomy, BIRA-IASB). The revision loop provides product producers the opportunity Institute for Space Aeronomy, BIRA-IASB). The revision loop provides product producers the to improve their QA evidence for each category to ensure that as much product QA information is QA opportunity to improve their QA evidence for each category to ensure that as much product captured is as possible. QA reports will beQA made available to be the public theto QA4ECV information capturedPublished as possible. Published reports will made through available the public website. Future iterations of the QA reports will be made available through the Copernicus Climate through the QA4ECV website. Future iterations of the QA reports will be made available through the Change Service (C3S) Climate Data Store. Copernicus Climate Change Service (C3S) Climate Data Store.
Remote Sens. 2018, 10, 1254
7 of 21
3.2.1. A Dedicated QA Administration A dedicated QA administration provides the backbone for the operation of the QA system. The office is responsible for a range of tasks including the following: preparing QA evaluation criteria; ensuring the tools, information, templates, and training modules provided within the QA system are current, state of the art, and comply with higher level requirements or recommendations; offering guidance and support for use of the QA system; conducting the independent evaluation (“audit”) by at least two domain expert evaluation officers; providing evaluation feedback to the product producer in an iterative process; and awarding endorsement for QA compliance. Noting that this type of QA system and evaluation activity is ‘new territory’ for the EO community, the framework set out within this document has been designed to allow ‘levels of QA compliance’, which will be key in encouraging ECV developers to continually improve their product QA evidence through time as the algorithm and evaluation work matures and to aid in data users’ understanding of the products. For the purpose of the prototype QA system, land and atmosphere measurement experts from the National Physical Laboratory (NPL) in the United Kingdom (UK) and the Royal Belgian Institute for Space Aeronomy (BIRA-IASB) performed the role of administrators and product evaluators. 3.2.2. Product Producers The data product producers are responsible for completing all of the required information within each of the quality indicators. This includes uploading references and documentation, providing justification for processes and statements made concerning the development and validation of the data product, and being responsive to constructive analysis and recommendations for improvement provided by the QA office assessor. This is because the product producer knows their product best. The ECV product producer is consulted through the evaluation process, and producer consent must be gained before a QA evaluation report is made publicly available. ECV product producers need to be well motivated to complete the QA report (i.e., they need to understand that systematic assessment of product quality and the processes to generate them is advantageous for further evolution of the data products). It needs to be clear to them that the whole process, including multiple interaction and feedback, takes approximately 2–4 h to complete. 3.3. Training and Guidance The training developed to accompany the QA system consists of guidance to support product developers in navigating and completing the required fields of quality information. A ~3-min video provides a visual overview of the QA system categories. A short 5-page quick start guide is provided to accompany the video. While a detailed QA system user manual has been produced [17]. This documentation, as well as the video, can be found on the QA4ECV website: http://www.qa4ecv.eu/qa-system/training. Further, face-to-face and web conferencing training sessions have been run by NPL and BIRA-IASB to demonstrate functionality and use of the QA system. 3.4. QA Evaluation (“Auditing”) Process This section describes a formalised process undertaken by the QA office to ensure effective and traceable implementation of the QA4ECV framework. This includes consideration of the level of compliance to be achieved, the process for undertaking each stage of compliance demonstration, and how the framework is driven by continual improvement. An overview of the process is given in Figure 2. The QA4ECV service uses the term “audit” (a systematic, independent, and documented process for obtaining objective evidence and evaluating it objectively to determine the extent to which the evaluation criteria are fulfilled: ISO9001:2015 [18]). The aim of checking quality records is to ensure the consistency of information between ECV datasets. Once the ECV data product producer has completed the QA system QI categories to the best of their ability, they are prompted to submit their report for evaluation by the QA office. The iterative
Remote Sens. 2018, 10, x FOR PEER REVIEW
8 of 22
derived, contains Remote Sens. which 2018, 10, 1254
three levels of increasing compliance (amount of detail/justification 8 of 21 provided), for each quality indicator. The three levels include the following:
Basic—Some information is providedinon the quality of the evaluation product to checklist allow thehas users to process for checking records is demonstrated Figure 2. A quality been make a simple distinction between the product and others. (Light grey). derived, which contains three levels of increasing compliance (amount of detail/justification provided), Intermediate—Detailed is the provided on the product, allowing the user to for each quality indicator. The threeinformation levels include following: understand how it was made and the quality and uncertainty information available to them. Basic—Some information is provided on the quality of the product to allow the users to make a (Blue). simple distinction between the product and others. (Light grey). on the product, providing the user Advanced—Significant detailed information is provided Intermediate—Detailed information provided decision on the product, allowing the usershould to understand with enough information to make anisinformed about how the product be used. how it was made and the quality and uncertainty information available to them. (Blue). (Green). Advanced—Significant detailed information is provided on the product, providing the user with Further, the QA system generates a QA label, which will be applied to a dataset. This label enough information to make an informed decision about how the product should be used. (Green). provides a quick visual overview of the quality ranking a product has achieved for each QI (basic—light grey; Thebelabel is based on the GEO label Further, the QAintermediate—blue; system generates aadvanced—green). QA label, which will applied to a dataset. This label utilised by GEOSS (Global Earth Observation System of Systems) in their datasets provides a quick visual overview of the quality ranking a product has achieved for each QI (basic—light (http://www.geolabel.info/). grey; intermediate—blue; advanced—green). The label is based on the GEO label utilised by GEOSS TheEarth QA evaluation/audit should be undertaken bydatasets at least two independent product experts and (Global Observation System of Systems) in their (http://www.geolabel.info/). consolidated by an impartial QA officer to ensure abyfair and two robust assessment. Once the QA The QA evaluation/audit should be undertaken at least independent product experts assessment has been completed, invited review the audit. This and consolidated by an impartialthe QAproduct officer producer to ensure is a fair andto robust assessment. Onceprovides the QA them with the to improve, orproducer provide further justification When assessment haschance been completed, theupdate, product is invited to review for the their audit.answers. This provides both parties arechance in agreement withupdate, the evaluation, the further final product qualityfor summary report will be them with the to improve, or provide justification their answers. When madeparties publicly The QA summary reports the are final generated for quality two main purposes: (1) towill allow both areavailable. in agreement with the evaluation, product summary report be the data producers to gauge howsummary well theyreports are achieving standardised quality assessment criteria, made publicly available. The QA are generated for two main purposes: (1) to allow anddata where they maytoneed to how focuswell theirthey efforts; (2) for standardised data product users to assessment use the reports and the producers gauge are and achieving quality criteria, identify suitable datasets their their requirements and/or discover information and where they may need for to focus efforts; and (2) for data product users about to use the the existing reports datasets theysuitable use to improve and value of applications. information regarding and identify datasets knowledge for their requirements and/or discoverDetailed information about the existing the QA evaluation be found and in [7]. The of development QA summary reportsregarding for multiple datasets they use toprocess improvecan knowledge value applications.ofDetailed information the data products will also have the added benefit of signalling key research gaps for future funding QA evaluation process can be found in [7]. The development of QA summary reports for multiple data efforts. will also have the added benefit of signalling key research gaps for future funding efforts. products 4. 4. Six New ECV Climate Data Records Records The second second key key objective objective of the the QA4ECV QA4ECV project project was was to to generate generate multi-decadal multi-decadal CDRs for atmospheric atmospheric and terrestrial terrestrial ECVs ECVs that that are are based based on on inter-satellite inter-satellite calibrated calibrated data, data, state-of-the-art state-of-the-art retrievals and are fully traceable with uncertainty metrics. Each CDR is described below, and the retrievals and are fully traceable with uncertainty metrics. the static static top-level top-level traceability traceability chain chain is is shown. shown. The traceability chain key is is shown shown in in Figure Figure 3. 3. Dynamic Dynamic versions versions of of these these traceability traceability chains chains along along with with access access to to the eight data products products produced produced within within the the QA4ECV QA4ECV project project are are available available at at http://www.qa4ecv.eu/. http://www.qa4ecv.eu/.
Figure 3. 3. Traceability Chain Key. Key. Figure Traceability Chain
4.1. Broadband Broadband Albedo Albedo 4.1. The QA4ECV is is based on on processing a 35-year (1982–2016) dailydaily time The QA4ECV broadband broadbandalbedo albedoproduct product based processing a 35-year (1982–2016) seriesseries of 0.05° polar orbiting National Oceanic and Atmospheric Administration advanced Very ◦ polar time of 0.05 orbiting National Oceanic and Atmospheric Administration advanced High-Resolution Radiometer (NOAA-AVHRR) and re-projected geostationary (Meteosat, GOES Very High-Resolution Radiometer (NOAA-AVHRR) and re-projected geostationary (Meteosat, GOES (Geostationary Operational Operational Environmental Environmental Satellite Satellite system), system), GMS GMS (Geostationary (Geostationary Meteorological Meteorological (Geostationary Satellite)) level-2 surface visible and near-infrared (NIR) bidirectional reflectance factors (BRFs) into into Satellite)) level-2 surface visible and near-infrared (NIR) bidirectional reflectance factors (BRFs) top-of-canopy bidirectional reflectance distribution functions (BRDFs). These are then integrated into top-of-canopy bidirectional reflectance distribution functions (BRDFs). These are then integrated into surface albedo. The daily polar orbiting (BRFs) were generated within the AVHRR long-term
Remote Sens. Sens. 2018, 2018, 10, 10, 1254 x FOR PEER REVIEW Remote
of 21 22 99 of
surface albedo. The daily polar orbiting (BRFs) were generated within the AVHRR long-term data data record (LTDR) V5 developed byThe [19].geostationary The geostationary BRFs were generated the record (LTDR) V5 developed by [19]. BRFs were generated using theusing standard standard SCOPE-CM (Sustained, Coordinated Processing of Environmental Satellite Data for SCOPE-CM (Sustained, Coordinated Processing of Environmental Satellite Data for Climate Climate Monitoring) processing scheme The processing used to the retrieve theuses BRDFs Monitoring) processing scheme [20]. The[20]. processing used to retrieve BRDFs the uses ESA the ESA GlobAlbedo processing chain, whichoptimal employs optimal [21] estimation [21] daily to generate daily GlobAlbedo processing chain, which employs estimation to generate 0.05° (≈5 ×5 ◦ (≈5 × 5 km) products over the 35-year time period. In addition, as part of the QA4ECV 0.05 km) products over the 35-year time period. In addition, as part of the QA4ECV processing chain processing automated machinemethods learning-based methods were rigorously tested automated chain machine learning-based were developed anddeveloped rigorouslyand tested to screen out to screen out clouds, flag snow, and sea-ice over shallow water and water bodies in order to focus on clouds, flag snow, and sea-ice over shallow water and water bodies in order to focus on land only land onlyThe pixels. The AVHRR channels and 22were converted to 3 broadbands: visible pixels. AVHRR BRFs BRFs fromfrom channels 1 1and were converted to 3 broadbands: (VIS, 0.4–0.7 NIR (0.7–3 andµm), shortwave (SW, 0.4–3 µm). Theµm). GEOThe (geostationary satellite) visible (VIS, µm), 0.4–0.7 µm), NIRµm), (0.7–3 and shortwave (SW, 0.4–3 GEO (geostationary BRFs were converted to SW (0.4–3 µm) as only bands were used, which straddle satellite) BRFs were converted to SW (0.4–3 µm)the as panchromatic only the panchromatic bands were used, which the red edge at 0.7 µm.atFor setannual of BRDF/albedo retrievals,retrievals, 18 months18 of months input low straddle the red edge 0.7each µm.annual For each set of BRDF/albedo of Earth input orbit satellites BRFs (derived from AVHRR from seven NOAANOAA spacecrafts) were low Earth orbit (LEO) satellites (LEO) BRFs (derived from AVHRR fromdifferent seven different spacecrafts) ◦ latitude, GEO employed for each daily product within the central 12 months. For the area within ± 60 were employed for each daily product within the central 12 months. For the area within ±60° latitude, top-of-atmosphere data were by EUMETSAT to SW-BRF, and these were included in the joint GEO top-of-atmosphere dataprocessed were processed by EUMETSAT to SW-BRF, and these were included in retrieval [22]. A background dataset consisting of broadband daily climatology in the 3 wavelength the joint retrieval [22]. A background dataset consisting of broadband daily climatology in the 3 regions, derived from 16 years of MODIS (MODerate resolution Imagingresolution Spectro-radiometer) BRDFs, wavelength regions, derived from 16 years of MODIS (MODerate Imaging Spectrowas employed to ensure there were no gaps when persistent or during cloud polar radiometer) BRDFs, wasthat employed to ensure that therethere werewas no gaps whencloud therecover was persistent ◦ daily and monthly night. From these daily products, using energy conservation for upscaling, 0.5 cover or during polar night. From these daily products, using energy conservation for upscaling, 0.5° products produced. Forwere each produced. and every pixel, an estimate ofpixel, the uncertainty using the daily andwere monthly products For each and every an estimateproduced of the uncertainty processes within the traceability chain was generated. These uncertainties consisted producedshown using the processes shown within the traceability chain pixel-level was generated. These pixel-level of standard errors called a cross-product covariance term calledcovariance “alpha” between the uncertainties consisted of “sigma” standardand errors called “sigma” and a cross-product term called VIS and between NIR channels, were subsequently for the two-stream package “alpha” the VISwhich and NIR channels, whichemployed were subsequently employed inversion for the two-stream (TIP) processing (see(TIP) Section 4.4). Output measurements, as a inversion package processing (see products Section also 4.4). include Outputquality products also includesuch quality weighted number ofas samples and number a relative to the entropy influence of thetoMODIS prior, measurements, such a weighted of entropy samples related and a relative related the influence in to snow water body flags. This QA4ECV broadband albedo product is the first ever of addition the MODIS prior,and in addition to snow and water body flags. This QA4ECV broadband albedo fused product from GEO + LEO and the longest time series ever produced of the Earth’s land surface product is the first ever fused product from GEO + LEO and the longest time series ever produced of albedo. It has been extensively our extensively collaborators at thebyLudwig MaximilianatUniversity of the Earth’s land surface albedo.tested It hasby been tested our collaborators the Ludwig Munich for fitness-for-purpose in climate models (papers in in climate preparation). QA4ECV broadband Maximilian University of Munich for fitness-for-purpose modelsThe (papers in preparation). albedo productbroadband top-level traceability chaintop-level is showntraceability in Figure 4.chain is shown in Figure 4. The QA4ECV albedo product
Figure 4. 4. QA4ECV QA4ECV broadband broadband albedo albedo product product top-level top-level traceability traceability chain. chain. AVHRR: advanced very very Figure AVHRR: advanced high-resolution radiometer; radiometer; BRF: high-resolution BRF: bidirectional bidirectional reflectance reflectance factor; factor; GEO: GEO: geostationary geostationary satellite; satellite; ToC: ToC: top of canopy; BBRF: broadband reflectance factor; BBDR; broadband directional reflectance; top of canopy; BBRF: broadband reflectance factor; BBDR; broadband directional reflectance; MODIS: MODIS: moderate resolution resolution imaging imaging spectro-radiometer; spectro-radiometer; BB; BB; broadband; broadband; and and BRDF: BRDF: bidirectional moderate bidirectional reflectance reflectance distribution function. distribution function.
4.2. Spectral Albedo 4.2. Spectral Albedo In the the ESA ESA GlobAlbedo GlobAlbedo project project (www.GlobAlbedo.org), (www.GlobAlbedo.org), surface surface spectral spectral BRFs BRFs were were calculated calculated In from medium-resolution imaging spectrometer (MERIS) and VEGETATION sensors. These spectral from medium-resolution imaging spectrometer (MERIS) and VEGETATION sensors. These spectral BRFs were then converted into visible, NIR, and shortwave broadbands using narrow-to-broadband
Remote Sens. 2018, 10, 1254
10 of 21
Remote Sens. 2018, 10, x FOR PEER REVIEW
10 of 22
BRFs were then converted into visible, NIR, and shortwave broadbands using narrow-to-broadband coefficients coefficients calculated calculated at at Freie Freie Universität Universität Berlin Berlin [23]. [23]. In In QA4ECV, QA4ECV, the the time time series series of of MODIS MODIS spectral spectral ◦ albedos (specifically daily MCD43C (MODIS BRDF/Albedo Product) at 0.05 ) was extended albedos (specifically daily MCD43C (MODIS BRDF/Albedo Product) at 0.05°) was extended back back in in time using VEGETATION as the primary sensor. This process started by generating a set of spectral time using VEGETATION as the primary sensor. This process started by generating a set of spectral coefficients, coefficients, which which allowed allowed VEGETATION VEGETATION spectral spectral BRFs BRFs to to be be converted converted into into their their equivalent equivalent for for MODIS. This was performed by matching up millions of MODIS spectral BRFs from MOD09 (MODIS MODIS. This was performed by matching up millions of MODIS spectral BRFs from MOD09 (MODIS Surface Surface Reflectance Reflectance Product) Product) with with the the closest closest possible possible matchups matchups in in view view and and solar solar angles angles from from MERIS MERIS and VEGETATION spectral band BRFs to determine the sensor-to-MODIS spectral band mapping. and VEGETATION spectral band BRFs to determine the sensor-to-MODIS spectral band mapping. For very well forfor thethe first 6 spectral bands of For VEGETATION, VEGETATION,with withaa1.6 1.6µm µmband, band,this thismapping mappingworks works very well first 6 spectral bands MODIS, but it does not function correctly for MODIS band 7 ( ≈ 2.13 µm). This matchup also works of MODIS, but it does not function correctly for MODIS band 7 (≈2.13 µm). This matchup also works surprisingly all all the the MERIS bands, even though there arethere no bands above 1 µm. The GlobAlbedo surprisinglywell wellforfor MERIS bands, even though are no bands above 1 µm. The processing chain was modified to process all the input VEGETATION only for 1998–2000 data GlobAlbedo processing chain was modified to process all the input VEGETATION onlydaily for 1998– and to test this for future use with Sentinel-3. This was tested for 2005 with MERIS + VEGETATION, 2000 daily data and to test this for future use with Sentinel-3. This was tested for 2005 with MERIS + as employed in GlobAlbedo. then compared MODIS spectral albedos the VEGETATION, as employed We in GlobAlbedo. Wethese thensynthesised compared these synthesised MODISwith spectral actual MODIS albedos and found extremely high correlations. A spectral version of the aforementioned albedos with the actual MODIS albedos and found extremely high correlations. A spectral version of MODIS BRDF climatology employed in thewas optimal estimation scheme. The same the aforementioned MODISwas BRDF climatology employed in the retrieval optimal estimation retrieval technique could also be applied to generate a long-time series of MERIS-like or ocean and land colour scheme. The same technique could also be applied to generate a long-time series of MERIS-like or instrument (OLCI)-like channels going back to 1998. Uncertainties were per band ocean and land colour spectral instrument (OLCI)-like spectral channels going back to calculated 1998. Uncertainties and between per bands, as and this was too computationally challenging. Only data for 16challenging. tiles over Europe werenot calculated band not between bands, as this was too computationally Only were processed for the same reason. The QA4ECV spectral albedo product top-level traceability chain data for 16 tiles over Europe were processed for the same reason. The QA4ECV spectral albedo is showntop-level in Figuretraceability 5. product chain is shown in Figure 5.
Figure 5. 5. QA4ECV albedo product product top-level top-level traceability traceability chain. chain. VEGETATION; MERIS: Figure QA4ECV spectral spectral albedo VEGETATION; MERIS: medium-resolution imaging imaging spectrometer; spectrometer; PROBA-V: PROBA vegetation vegetation sensor; sensor; TOA; top of of medium-resolution PROBA-V: PROBA TOA; top atmosphere; and SBRFs: surface bidirectional reflectance factors. atmosphere; and SBRFs: surface bidirectional reflectance factors.
4.3. Sea-Ice Albedo 4.3. Sea-Ice Albedo Sea-ice albedo is a key climate change indicator as there is strong feedback between sea-ice Sea-ice albedo is a key climate change indicator as there is strong feedback between sea-ice albedo and direct radiative forcing. Up until now, models have been employed for sea-ice albedo albedo and direct radiative forcing. Up until now, models have been employed for sea-ice albedo retrieval using instruments such as AVHRR [24] for shortwave only and at low resolution (25 km) on retrieval using instruments such as AVHRR [24] for shortwave only and at low resolution (25 km) on weekly time-steps. Sea-ice packs many kilometres in size move at up to 15 km/day, so any method weekly time-steps. Sea-ice packs many kilometres in size move at up to 15 km/day, so any method such as time-composting smears out each individual albedo record, rendering it unfit for retrieval of such as time-composting smears out each individual albedo record, rendering it unfit for retrieval sea-ice albedo. What was needed was an instantaneous measurement of spectral BRF to allow an of sea-ice albedo. What was needed was an instantaneous measurement of spectral BRF to allow an instantaneous retrieval of spectral albedo. The NASA (National Aeronautics and Space instantaneous retrieval of spectral albedo. The NASA (National Aeronautics and Space Administration) Administration) multi-angle imaging spectro-radiometer (MISR) instrument is the only such multi-angle imaging spectro-radiometer (MISR) instrument is the only such instrument in orbit, which instrument in orbit, which records information at sufficiently high spatial resolution (1.1 km in all 4 records information at sufficiently high spatial resolution (1.1 km in all 4 spectral bands of blue, green, spectral bands of blue, green, red, and NIR), albeit with a narrow 380-km swath. Surface spectral red, and NIR), albeit with a narrow 380-km swath. Surface spectral BRFs were specially processed BRFs were specially processed at NASA Langley over the Arctic and Antarctic regions from ±60° of at NASA Langley over the Arctic and Antarctic regions from ±60◦ of latitude to the northernmost latitude to the northernmost point at 83° due to the inclination of the NASA Terra orbit. The cloud point at 83◦ due to the inclination of the NASA Terra orbit. The cloud mask derived from MISR is mask derived from MISR is not yet sufficiently robust to be able to differentiate cloud from sea-ice, so a separate orbital 1-km sea-ice product derived from MODIS, called MOD29 [25], was employed to mask out the clouds and sea-only areas from the MISR-derived BRF and albedos. This land surface
Remote Sens. 2018, 10, 1254
11 of 21
not yet sufficiently robust to be able to differentiate cloud from sea-ice, so a separate orbital 1-km Remote Sens. 2018, 10, x FOR PEER REVIEW 11 of 22 sea-ice product derived from MODIS, called MOD29 [25], was employed to mask out the clouds and sea-only the MISR-derived BRF andin albedos. Thisthe land surface BRF and albedo BRF andareas MISRfrom albedo product is described [26], and special product is MISR described in product is described in [26], and the special product is described in Kharbouche and Muller (in review). Kharbouche and Muller (in review). The 1-km, 5-km and 25-km products are processed every orbit The 5-km and 25-km products are processed orbit andon then integrated over ± 24 h, ±3 days, and 1-km, then integrated over ±24 h, ±3 days, ±7 days,every and monthly a daily time-step. Although each ± 7 days, and monthly on a daily time-step. Although each retrieved spectral BRF and albedo has an retrieved spectral BRF and albedo has an uncertainty, this is not employed to generate the global uncertainty, this is not employed to generate the global product. Instead, for the time-composited product. Instead, for the time-composited products, a standard deviation of albedo is employed. This products, a standard deviation of albedo is employed. This product has used to generate a 16-year product has been used to generate a 16-year time series (2000–2016) forbeen April–September of each year ◦ . This time time series (2000–2016) for April–September of each year when the solar zenith angle ≤ 70 when the solar zenith angle ≤70°. This time series has been validated using data from a towerseries has been validatedwhen using converted data from ato tower-mounted albedometer when converted to shortwave mounted albedometer shortwave bi-hemispherical diffuse reflectance (BHR, bi-hemispherical (BHR, sky”cloud albedo), as well asradiometer data from sometimes calleddiffuse “whitereflectance sky” albedo), as sometimes well as datacalled from“white the NASA absorption the NASA cloud absorption (CAR) instrument usingisthe methods in [27]. across There (CAR) instrument using theradiometer methods described in [27]. There huge interestdescribed in this product is huge interest in this product across the climate–cryosphere community. The QA4ECV sea-ice albedo the climate–cryosphere community. The QA4ECV sea-ice albedo product top-level traceability chain product chain is shown in Figure 6. is showntop-level in Figuretraceability 6.
Figure6.6.QA4ECV QA4ECVsea-ice sea-icealbedo albedo product product top-level top-level traceability traceability chain. Figure chain. MISR: MISR:multi-angle multi-angleimaging imaging spectro-radiometer; MOD29: MODIS sea-ice products; and MOD03: MODIS Geolocation DataSet. Set. spectro-radiometer; MOD29: MODIS sea-ice products; and MOD03: MODIS Geolocation Data
4.4. TIP LAI/FAPAR 4.4. TIP LAI/FAPAR Leaf area index (LAI) and fraction of absorbed photosynthetically active radiation (FAPAR) Leaf area index (LAI) and fraction of absorbed photosynthetically active radiation (FAPAR) ECVs ECVs along with their per pixel uncertainty were consistently retrieved using the two-stream along with their per pixel uncertainty were consistently retrieved using the two-stream inversion inversion package (TIP) [28,29] applied to visible (VIS) and near-infrared (NIR) broadband albedos package (TIP) [28,29] applied to visible (VIS) and near-infrared (NIR) broadband albedos from the from the QA4ECV project. The TIP is the inversion of the two-stream model developed by [30], which QA4ECV project. The TIP is the inversion of the two-stream model developed by [30], which implements the two-stream approximation of radiative transfer for a homogeneous one dimensional implements the two-stream approximation of radiative transfer for a homogeneous one dimensional canopy (“1D-canopy”). The 1D radiative transfer model is potentially consistent with large-scale canopy (“1D-canopy”). The 1D radiative transfer model is potentially consistent with large-scale climate and Earth system models and does not require assumptions about other factors (e.g., biome climate and Earth system models and does not require assumptions about other factors (e.g., biome type) to be made. Owing to the 1D approach, TIP-LAI is an effective quantity, describing the optical type) to be made. Owing to the 1D approach, TIP-LAI is an effective quantity, describing the optical effects of the leaves. The implementation used in QA4ECV (TIP5D) uses the full variance–covariance effects of the leaves. The implementation used in QA4ECV (TIP5D) uses the full variance–covariance matrix of the BHRs, which is an enhancement beyond previous applications of the TIP [31,32], while matrix of the BHRs, which is an enhancement beyond previous applications of the TIP [31,32], while maintaining reproducibility of the results. maintaining reproducibility of the results. In QA4ECV, TIP LAI, and FAPAR were produced globally for 0.5-degree and 0.05-degree In QA4ECV, TIP LAI, and FAPAR were produced globally for 0.5-degree and 0.05-degree regular regular grids, for each day of 1982–2016. Full per pixel processing information and extra quality grids, for each day of 1982–2016. Full per pixel processing information and extra quality information is information is available through the provided retrieval flags. This product is best suited for use in available through the provided retrieval flags. This product is best suited for use in soft constraint soft constraint data assimilation into dynamic models, which use a similar radiative transfer scheme, data assimilation into dynamic models, which use a similar radiative transfer scheme, yielding yielding maximum gain from the consistency of LAI, FAPAR, and the albedos and their uncertainties maximum gain from the consistency of LAI, FAPAR, and the albedos and their uncertainties (e.g., [33]). (e.g., [33]). As the uncertainties can be quite large, they should always be taken into account. As the uncertainties can be quite large, they should always be taken into account. Depending on Depending on the application it may be advisable to mask out some data according to the retrieval the application it may be advisable to mask out some data according to the retrieval flag (see the flag (see the product user guide for details [34]); for instance, trend analysis should not use data that product user guide for details [34]); for instance, trend analysis should not use data that were filled were filled with the albedo prior (most notably in late 1994, where no AVHRR data is available). with the albedo prior (most notably in late 1994, where no AVHRR data is available). However, However, version 1.0.1 suffers from artefacts introduced by problems further up the processing chain, as detailed in the uncertainty and validation document [35]. The QA4ECV TIP LAI/FAPAR product top-level traceability chain is shown in Figure 7.
Remote Sens. 2018, 10, 1254
12 of 21
version 1.0.1 suffers from artefacts introduced by problems further up the processing chain, as detailed in the uncertainty and validation document [35]. The QA4ECV TIP LAI/FAPAR product top-level traceability chain shown Figure 7. Remote Sens. 2018, 10, xisFOR PEERin REVIEW 12 of 22
Figure 7. QA4ECV two-stream inversion package (TIP) leaf area index (LAI)/fraction of absorbed Figure 7. QA4ECV two-stream inversion package (TIP) leaf area index (LAI)/fraction of absorbed photosynthetically active radiation (FAPAR) product top-level traceability chain. BHR: biphotosynthetically active radiation (FAPAR) product top-level traceability chain. BHR: bi-hemispherical hemispherical diffuse reflectance; PDF: probability density function; VIS: visible; NIR: near-infrared; diffuse reflectance; PDF: probability density function; VIS: visible; NIR: near-infrared; and LUT: look andtable. LUT: look up table. up
4.5. AVHRR FAPAR 4.5. AVHRR FAPAR Joint Research Centre (JRC) methodology was used to compute daily fraction of absorbed Joint Research Centre (JRC) methodology was used to compute daily fraction of absorbed photosynthetically active radiation (FAPAR) from daily spectral measurements acquired by photosynthetically active radiation (FAPAR) from daily spectral measurements acquired by advanced advanced very high-resolution radiometer (AVHRR) onboard a series of National Oceanic and very high-resolution radiometer (AVHRR) onboard a series of National Oceanic and Atmospheric Atmospheric Administration (NOAA) platforms, namely 07, 09, 11, 14, and 16. The methodology Administration (NOAA) platforms, namely 07, 09, 11, 14, and 16. The methodology itself is based on itself is based on previous JRC-FAPAR algorithms, such as the ones developed for the mediumprevious JRC-FAPAR algorithms, such as the ones developed for the medium-resolution instrument resolution instrument sensor (MERIS) and the ocean land colour instrument (OLCI) [36,37], except sensor (MERIS) and the ocean land colour instrument (OLCI) [36,37], except surface reflectances in surface reflectances in Band 1 and Band 2 were used as inputs data instead of top-of-atmosphere ones Band 1 and Band 2 were used as inputs data instead of top-of-atmosphere ones [19]. The retrieval [19]. The retrieval method assumes that the leaves are alive and photosynthesising, hence the ‘green’ method assumes that the leaves are alive and photosynthesising, hence the ‘green’ FAPAR is assumed. FAPAR is assumed. Also, contrary to the TIP FAPAR section 4.4), the values correspond to Also, contrary to the TIP FAPAR Section 4.4), the values correspond to instantaneous definition instantaneous definition (i.e., under direct illumination). The QA4ECV products span from 1982 to (i.e., under direct illumination). The QA4ECV products span from 1982 to 2006 at 0.05◦ × 0.05◦ . 2006 at 0.05° × 0.05°. In addition to daily products, 10-day and monthly products were provided as In addition to daily products, 10-day and monthly products were provided as well over a coarser well over a coarser resolution for biosphere changes studies (e.g., 0.5° × 0.5°). The products contain resolution for biosphere changes studies (e.g., 0.5◦ × 0.5◦ ). The products contain several uncertainty several uncertainty metrics, such as error propagation derived from inputs uncertainties and both metrics, such as error propagation derived from inputs uncertainties and both temporal and spatial temporal and spatial standard deviation for re-gridded products. These products are unique, as they standard deviation for re-gridded products. These products are unique, as they are the only ones are the only ones containing three types of uncertainties and can be used together with SeaWiFS (Seacontaining three types of uncertainties and can be used together with SeaWiFS (Sea-Viewing Wide Viewing Wide Field-of-View Sensor), MERIS, and Sentinel-3 FAPAR products using the same Field-of-View Sensor), MERIS, and Sentinel-3 FAPAR products using the same retrieval algorithm and retrieval algorithm and definition. However, despite the recent calibration and atmospheric definition. However, despite the recent calibration and atmospheric correction performances made correction performances made by [19], the products still contain, at the end of a few NOAA satellites, by [19], the products still contain, at the end of a few NOAA satellites, some artefacts that must be some artefacts that must be corrected for global changes studies [38]. This will be conducted in the corrected for global changes studies [38]. This will be conducted in the new version. The QA4ECV new version. The QA4ECV AVHRR FAPAR product top-level traceability chain is shown in Figure AVHRR FAPAR product top-level traceability chain is shown in Figure 8. 8.
Remote Sens. 2018, 10, 1254 Remote Sens. 2018, 10, x FOR PEER REVIEW
13 of 21 13 of 22
Figure 8. 8. QA4ECV FAPAR product product top-level top-level traceability traceability chain. chain. NDVI; NDVI; normalised normalised difference difference Figure QA4ECV AVHRR AVHRR FAPAR vegetation index; index; and and RPV RPV model: model: Rahman–Pinty–Verstraete Rahman–Pinty–Verstraetemodel. model. vegetation
4.6. NO NO2 4.6. 2 The QA4ECV QA4ECV NO NO₂ ECV precursor product contains harmonised vertical NO₂ columns from the The 2 ECV precursor product contains harmonised vertical NO2 columns from the ERS-2 GOME (Global Ozone Monitoring Experiment), Experiment), Envisat Envisat SCIAMACHY SCIAMACHY (SCanning (SCanning Imaging Imaging ERS-2 GOME (Global Ozone Monitoring AbsorptionSpectroMeter SpectroMeter for Atmospheric CHartographY), Aura OMI (Ozone Instrument), Monitoring Absorption for Atmospheric CHartographY), Aura OMI (Ozone Monitoring Instrument), and MetOp-A GOME-2(A) sensors. The main product is the tropospheric vertical and MetOp-A GOME-2(A) sensors. The main product is the tropospheric vertical column density. column density. The datasets cover the period of July 1995–December 2017, a 22+ year record. The The datasets cover the period of July 1995–December 2017, a 22+ year record. The spatial resolution 2 2 spatialfrom resolution 320 × 40 13 × in 24nadir), km (OMI inmid-morning nadir), with mid-morning 2 (GOME) varies 320 ×varies 40 kmfrom to km 13 ×(GOME) 24 km2 to (OMI with (10:00, local (10:00, local time) overpasses for GOME, SCIAMACHY, and GOME-2 and early(13:40, afternoon time) overpasses for GOME, SCIAMACHY, and GOME-2 and early afternoon local(13:40, time) local time) for OMI. Global coverage is achieved every 1–6 days, depending on the instrument fieldfor OMI. Global coverage is achieved every 1–6 days, depending on the instrument field-of-view of-view and measurement conditions (e.g., presence of clouds, snow, or ice). TheQA4ECV QA4ECVNO NO2 ECV ECV and measurement conditions (e.g., presence of clouds, snow, or ice). The 2 precursor product contains detailed information on retrieval uncertainty (from uncertainty precursor product contains detailed information on retrieval uncertainty (from uncertainty propagation propagation embedded calculations embedded the retrieval algorithm) and main quality flags. The main calculations in the retrieval in algorithm) and quality flags. The uncertainty metric is uncertainty metric is the uncertainty in the tropospheric NO 2 column, but the dataset also includes a the uncertainty in the tropospheric NO2 column, but the dataset also includes a breakdown of the breakdowncontributions of the individual contributions to the overall budget (i.e., from detector noise, individual to the overall uncertainty budgetuncertainty (i.e., from detector noise, fitting techniques, fitting techniques, radiative transfer calculations, and assumptions made on ancillary data). A full radiative transfer calculations, and assumptions made on ancillary data). A full description of description of the retrieval approach, uncertainty analysis and auxiliary data, and some the retrieval approach, uncertainty analysis and auxiliary data, and some preliminarypreliminary validation validation provided in data [39]. The data has product been registered using DOIs unique for the four is providedisin [39]. The product been has registered using unique forDOIs the four sensor sensor subsets as in What [40]). is What is unique the QA4ECV NO2 record data record is itthat it isfirst the subsets (e.g., as(e.g., in [40]). unique about about the QA4ECV NO2 data is that is the first cross-calibrated, multi-sensor dataset, with very detailed quality information embedded and that cross-calibrated, multi-sensor dataset, with very detailed quality information embedded and that it it spans a period more than years. Tropospheric 2 columns are being used widely, especially spans a period of of more than 20 20 years. Tropospheric NONO 2 columns are being used widely, especially for for estimating NO x emissions (e.g., [41]), for improving the estimates and attribution of ozone and estimating NOx emissions (e.g., [41]), for improving the estimates and attribution of ozone and aerosols aerosols (e.g., for [42,43]), trend analyses (e.g., and for reanalysis studies (e.g., [45]). are (e.g., [42,43]), trend for analyses (e.g., [44]), and[44]), for reanalysis studies (e.g., [45]). Users areUsers advised advised use the tropospheric NOby 2 columns by taking into account detailed information on to use thetotropospheric NO2 columns taking into account detailed information on measurement measurement flags, spatio-temporal representativeness, and vertical sensitivity. For more on flags, spatio-temporal representativeness, and vertical sensitivity. For more detail on this and detail practical this and practical recommendations on how to use the data, we refer the readers to [46]. The QA4ECV recommendations on how to use the data, we refer the readers to [46]. The QA4ECV NO2 product NO2 product top-level traceability chain is shown top-level traceability chain is shown in Figure 9. in Figure 9. 4.7. HCHO The QA4ECV HCHO ECV precursor product contains harmonised HCHO tropospheric vertical column densities for the period of 1996–2016. The HCHO ECV data provides geophysical information for every ground pixel observed by each satellite sensor (GOME, SCIAMACHY, OMI, and GOME-2A). Global Earth coverage is achieved within 1–6 days, depending on the sensor and on the observation conditions. In addition to the vertical HCHO column densities, the product contains intermediate
Remote Sens. 2018, 10, 1254
14 of 21
results for every ground pixel, such as the result of the spectral fit, fitting diagnostics, the averaging kernel, cloud information, uncertainty estimates detailed for each retrieval step, and quality flags. A full description of the retrieval algorithm, uncertainty analysis, and auxiliary data is provided in [47]. Satellite HCHO observations are widely used to gain knowledge on non-methane volatile organic compounds (NMVOC) emissions, tropospheric ozone formation, and biogenic aerosols [48]. Uncertainties in satellite HCHO observations are dominated by their random components. Users are therefore advised to average the data in space and/or in time, in order to reduce this contribution. The QA4ECV algorithm is now being transferred to the TROPOMI sensor, offering a significantly improved signal-to-noise ratio and extending the 20-years QA4ECV HCHO dataset. The QA4ECV HCHO product top-level Remote Sens. 2018, 10, x FOR traceability PEER REVIEW chain is shown in Figure 10. 14 of 22
Figure 9. QA4ECV 2 product top-level traceability chain. GOME; SCIAMACHY; OMI; DOAS: Figure 9. QA4ECV NONO 2 product top-level traceability chain. GOME; SCIAMACHY; OMI; DOAS: Differential Optical Absorption Spectroscopy; and AMF: appropriate air mass factors. Remote Sens. 2018, 10, x Absorption FOR PEER REVIEW 15 of 22 Differential Optical Spectroscopy; and AMF: appropriate air mass factors.
4.7. HCHO The QA4ECV HCHO ECV precursor product contains harmonised HCHO tropospheric vertical column densities for the period of 1996–2016. The HCHO ECV data provides geophysical information for every ground pixel observed by each satellite sensor (GOME, SCIAMACHY, OMI, and GOME-2A). Global Earth coverage is achieved within 1–6 days, depending on the sensor and on the observation conditions. In addition to the vertical HCHO column densities, the product contains intermediate results for every ground pixel, such as the result of the spectral fit, fitting diagnostics, the averaging kernel, cloud information, uncertainty estimates detailed for each retrieval step, and quality flags. A full description of the retrieval algorithm, uncertainty analysis, and auxiliary data is provided in [47]. Satellite HCHO observations are widely used to gain knowledge on non-methane volatile organic compounds (NMVOC) emissions, tropospheric ozone formation, and biogenic aerosols [48]. Uncertainties in satellite HCHO observations are dominated by their random components. Users are therefore advised to average the data in space and/or in time, in order to reduce this contribution. The QA4ECV algorithm is now being transferred to the TROPOMI sensor, offering a significantly improved signal-to-noise ratio and extending the 20-years QA4ECV HCHO dataset. The QA4ECV HCHO product top-level traceability chain is shown in Figure 10.
10. QA4ECV formaldehyde (HCHO)product product top-level top-level traceability chemical FigureFigure 10. QA4ECV formaldehyde (HCHO) traceabilitychain. chain.CTM: CTM: chemical transport model. transport model.
4.8. CO The QA4ECV CO ECV precursor product consists of a 10-year archive of CO total columns from the IASI (Infrared Atmospheric Sounding Interferometer) sensor (2008–2017). The columns are calculated from the CO profiles, retrieved from IASI day and night level 1C radiances with the FORLI software (Fast Optimal Retrievals on Layers for IASI v20100815+v20140922), on 19 vertical layers in
Figure 10. QA4ECV formaldehyde (HCHO) product top-level traceability chain. CTM: chemical Remotetransport Sens. 2018,model. 10, 1254 15 of 21
4.8. CO 4.8. CO The QA4ECV CO ECV precursor product consists of a 10-year archive of CO total columns from The QA4ECV ECV precursor product consists of asensor 10-year(2008–2017). archive of CO columns the IASI (Infrared CO Atmospheric Sounding Interferometer) Thetotal columns are from the IASI (Infrared Atmospheric Sounding Interferometer) sensor (2008–2017). The columns calculated from the CO profiles, retrieved from IASI day and night level 1C radiances with the FORLI are calculated the Retrievals CO profiles, day and night levelon 1C19 radiances with the software (Fast from Optimal onretrieved Layers forfrom IASIIASI v20100815+v20140922), vertical layers in FORLI software (Fast Optimalare Retrievals onwith Layers forestimates IASI v20100815+v20140922), onnative 19 vertical the troposphere. The columns provided error and quality flags at the IASI layers in the troposphere. Theelliptical columnspixels are provided errorfrom estimates and flags the resolution (i.e., for individual with sizeswith ranging IASI 12 kmquality by 12 km at at nadir native IASI resolution (i.e., for individual elliptical pixels withcoverage sizes ranging from IASI km by to 20 km by 39 km at the largest angles). IASI provides bi-daily of the Earth, with12 overpass 12 kmatat around nadir to9:30 20 km 39 km at the largest angles). to IASI bi-daily coverage of the times andby 21:30. Generally, the sensitivity theprovides boundary layer is better for Earth, with overpass times at around 9:30 and 21:30. Generally, the sensitivity to the boundary daytime observations, as documented in [49], and the varying sensitivity should therefore be layer is better for the observations, as series documented in [49], A and the varying sensitivity carefully accounted fordaytime when analysing the time in the columns. general description of the should therefore foranwhen analysing the time series in the columns. A general retrieval softwarebeiscarefully providedaccounted in [50], and algorithm technical basis document has been generated description retrieval software is provided in [50], and an algorithm technical basis document in the of the context of the EUMETSAT Satellite Application Facility has been generated in the context of the EUMETSAT Satellite Application (https://acsaf.org/ (https://acsaf.org/products/iasi_co.html). A product description, with firstFacility analyses and consistency products/iasi_co.html). product description, withinfirst and consistency with the check with the MOPITT A (Measurement of Pollution the analyses Troposphere) data recordcheck is provided in MOPITT Pollution in the Troposphere) data record is provided in [51]. The QA4ECV [51]. The (Measurement QA4ECV dataofrecord is available from the French Atmosphere Infrastructure (AERIS). data record is available from the Frenchin Atmosphere Infrastructure (AERIS). Thereback are to remaining There are remaining non-homogeneities the time series, which have been traced changes non-homogeneities in the time series, which have been traced back to changes in the meteorological in the meteorological input parameters. The data record from IASI-A is being extended from 2012 inputIASI-B, parameters. The is data record from IASI-A being extended 2012 withthe IASI-B, there is with and there no bias between the twoismissions; IASI-Cfrom will continue recordand after 2018. no bias between the two is missions; IASI-C willreal-time continue applications the record after 2018. The IASI CO productvia is The IASI CO product supporting near (operational dissemination supporting near applications (operational dissemination via EUMETCast and assimilation in EUMETCast andreal-time assimilation in the Copernicus Atmospheric Monitoring Service (CAMS)), emission the Copernicus Service (CAMS)), emission tropospheric inventories, andAtmospheric tropospheric Monitoring chemistry models [52]. The QA4ECV COinventories, Product topand level traceability chemistry models [52]. The chain is shown in Figure 11.QA4ECV CO Product top level traceability chain is shown in Figure 11.
Figure 11. QA4ECV carbon monoxide (CO) product top-level traceability chain.
5. QA4ECV Product Quality Reports The summary QA4ECV product QA reports are hosted on a public-facing area of the QA system. Login information is not required for general users to view the completed product QA reports. At the time of product evaluation, the QA4ECV products were still under development and not scheduled for completion until the end of the project (early 2018). Therefore, key quality indicators, such as validation and product inter-comparison studies, have not been conducted and evaluation of this QI could not be done. The maturity matrix assessment has been repeated at the end of the project and resulted in some distinct improvements for individual data records, in particular for increased completeness of validation activities and documentation. The QA evaluations for each of the eight QA4ECV products are shown below. Each product achieved varying levels of quality based on the defined criteria [7] (note grey = basic; blue = intermediate, and green = advanced quality information). This process highlights the need for an iterative and flexible QA evaluation approach, which gets vital product QA information to the user community but allows the product producer to improve the QA as further research is conducted.
this QI could not be done. The improvements maturity matrixfor assessment has been repeated at the end for of the project the of product evaluation, the QA4ECV products were still development and not and time resulted in some distinct individual data records, in particular increased such as validation anddistinct productimprovements inter-comparison studies, have notrecords, beenunder conducted and evaluation of and resulted in some for individual data in particular for increased scheduled for completion until the end of the project (early 2018). Therefore, key quality indicators, completeness of validation activities and documentation. The QA evaluations for each of the eight this QI could not be done. The maturity matrix assessment has been repeated at the end of the project completeness of validation activities and Thevarying QAbeen evaluations forand eachevaluation of theon eight such as validation and product inter-comparison studies, have not conducted of QA4ECV products are shown below. Eachdocumentation. product achieved levels of quality based the and resulted in some distinct improvements for individual data records, in particular for increased QA4ECV products are shown below. Each product achieved varying levels of quality based on the this QI could not be(note done.grey The matrix assessmentand hasgreen been repeated at the end ofofthe defined criteria [7] =maturity basic; blue = intermediate, = advanced quality information). completeness of[7] validation activities and documentation. The QA evaluations for eachinformation). theproject eight defined criteria (note grey =improvements basic; blue = intermediate, and green = advanced quality and resulted some distinct for individual data records, in particular for increased Remote Sens. 2018, 10, 1254in 16 of 21 This process highlights the need for an iterative and flexible QA evaluation approach, which gets QA4ECV products are shown below. Each product achieved varying levels of quality based on the This process highlights the need for an iterative and flexible QA evaluation approach, which gets completeness of validation activities and documentation. The QA evaluations for each of the eight vital product QA information to the user community but allows the product producer to improve the defined criteria [7]information (note grey =to basic; bluecommunity = intermediate, and green advanced qualitytoinformation). vital product QA the user but allows the =product producer improve the QA4ECV products are shown below. Each product achieved varying levels of quality based on the QA as further research is conducted. This process highlights the need for an iterative and flexible QA evaluation approach, which gets QA as further research conducted. defined criteria (noteisgrey =tobasic; bluecommunity = intermediate, and green = advanced qualitytoinformation). Broadband Albedo vital product QA[7]information the user but allows the product producer improve the This process highlights the need for an iterative and flexible QA evaluation approach, which gets Broadband Albedo QA as further research is conducted. Broadband Albedo The QA4ECV AVHRR + GEO broadband product produced byallows UCL the product producer to improve the vital product QA information toalbedo the user community but (University College London, MSSL (Mullard Space Science and QA further research is conducted. Theas QA4ECV AVHRR + GEO broadband albedoLaboratory) product produced by Broadband Albedo The QA4ECV AVHRR + GEO broadband product produced by Geography) Brockmann Consult has achieved analbedo advanced status for the UCLand (University College London, MSSL (Mullard Space Science (University College London, MSSL (Mullard Space Science product UCL details, traceability chain, and assessment against standards; Laboratory)Albedo and Geography) and Brockmann Consult has achieved an Broadband The QA4ECV AVHRR + GEO broadband albedo product produced by Laboratory) and Geography) and Brockmann Consult achieved an intermediate statusstatus for quality and basic status for uncertainty assessment advanced for theflags; product details, traceability chain,has and assessment UCL (University College London, MSSL (Mullard Space Science and validation. advanced status for the product details, traceability chain, and assessment against standards; intermediate status for quality flags; and basic status for The QA4ECV AVHRR + GEOand broadband albedo product produced by Laboratory) and Geography) Brockmann Consult hasbasic achieved an against standards; intermediate status for quality flags; and status for uncertainty assessment and validation. UCL (University College London, MSSL (Mullard Space Science advanced status for the product details, traceability chain, and assessment uncertainty assessment and validation. Laboratory) and Geography) Brockmann has achieved an against standards; intermediateand status for qualityConsult flags; and basic status for advanced status for the product details, traceability chain, and assessment Spectral Albedo uncertainty assessment and validation. SpectralSpectral Albedo Albedo against standards; intermediate status for quality flags; and basic status for uncertainty assessment validation. The QA4ECV spectral and albedo product produced by UCL (MSSL and Spectral Albedo The QA4ECV spectral albedo product produced by UCL (MSSL and Geography) The QA4ECV albedo product UCL (MSSL Geography) andspectral Brockmann Consult has produced achieved anbyadvanced statusand for and Brockmann Consult has achievedConsult an advanced status foranthe product status detailsfor Geography) and Brockmann has achieved advanced the product details and traceability chain; intermediate status for Spectral Albedo The QA4ECV spectraland albedo product produced by UCL (MSSL and and traceability chain; intermediate status for assessment against standards; and the product details traceability chain;status intermediate status for assessment against standards; and basic for quality flags, Geography) and Brockmann Consult has achieved an advanced status for basic status for quality flags, uncertainty assessment, and validation. assessment against standards; and basic status for quality flags, uncertainty assessment, and validation. The QA4ECV spectraland albedo product produced by UCL (MSSL and the product details uncertainty assessment, andtraceability validation. chain; intermediate status for Geography) and Brockmann Consult achieved an advanced status for assessment against standards; andhasbasic status for quality flags, the product details and traceability chain; intermediate status for Sea-Ice Albedo uncertainty assessment, and validation. Sea-Ice Albedo assessment against standards; and basic status for quality flags, uncertainty assessment, and validation. The QA4ECV sea-ice albedo product produced by UCL (MSSL and Sea-Ice Sea-Ice Albedo Albedo sea-ice albedo product produced by UCL (MSSL and The QA4ECV Geography) and Brockmann Consult has achieved an advanced status for Geography) and chain Brockmann Consult hasagainst achievedstandards; an advanced status for the traceability and assessment intermediate Sea-Ice Albedo The QA4ECV sea-ice albedo product produced UCL (MSSL Geography) The QA4ECV sea-ice albedo productbyproduced by and UCL (MSSL and the traceability chain and assessment against standards; intermediate status for product details; and statusstatus for quality flags, uncertainty and Brockmann Consult achieved anbasic advanced for traceability Geography) and has Brockmann Consult achieved anthe advanced status for status for product details; and basic has status for quality flags, uncertainty assessment, and validation. The QA4ECV sea-ice albedo product produced by UCL (MSSL and chain and assessment against standards; intermediate status for product details; the traceability and assessment against standards; intermediate assessment, and chain validation. and basic status for quality flags, uncertainty assessment, and validation. Geography) and Brockmann Consult has achieved an advanced status for status for product details; and basic status for quality flags, uncertainty the traceability chain and assessment against standards; intermediate TIP LAI/FAPAR assessment, and validation. TIP LAI/FAPAR status for product details; and basic status for quality flags, uncertainty assessment, and validation.product produced by FastOpt has achieved an The QA4ECV FAPAR/LAI TIP LAI/FAPAR The QA4ECV FAPAR/LAI produced by FastOpt has achieved an advanced status for the product information provided for product details, advanced status for the information provided for product details, traceability, quality flags, and assessment against standards; intermediate TIP LAI/FAPAR TIP LAI/FAPAR The QA4ECV FAPAR/LAIand product produced by FastOpt has intermediate achieved an traceability, quality assessment standards; status for uncertaintyflags, assessment; and basicagainst status for validation. advanced status forassessment; the information provided product details, status FAPAR/LAI for uncertainty and basic status has for for validation. The QA4ECV product product produced by FastOpt achieved an The QA4ECV FAPAR/LAI produced by FastOpt hasintermediate achieved an traceability, quality flags, and assessment against standards; advanced status forstatus the information provided for product details, traceability, advanced for the information provided for product details, status for uncertainty assessment; and basic status for validation. quality flags, and assessment against intermediate status forintermediate traceability, quality flags, andstandards; assessment against standards; uncertainty assessment; and basic status for validation. status for uncertainty assessment; and basic status for validation.
Remote Sens. 2018, 10, x FOR PEER REVIEW
AVHRRAVHRR FAPAR FAPAR The QA4ECV AVHRR AVHRR FAPAR product by JRC has FAPARproduced product produced byachieved JRC has an achieved an The QA4ECV advanced status forstatus the traceability intermediate for information advanced for the chain; traceability chain;status intermediate status for providedinformation for product provided details, quality flags, and assessment against and for product details, quality flags,standards; and assessment basic status for uncertainty assessment and validation. against standards; and basic status for uncertainty assessment and
validation. NO2 The QA4ECV NO2 ECV precursor product has achieved advanced status for the information provided for product details, traceability, quality flags, and assessment against standards; intermediate status for uncertainty assessment; and basic status for validation.
HCHO
17 of 22
AVHRR Remote Sens.FAPAR 2018, 10, x FOR PEER REVIEW
17 of 22
AVHRR FAPAR product produced by JRC has achieved an The QA4ECV Remote Sens. 2018,AVHRR 10, x FORFAPAR PEER REVIEW
17 of 22
advanced status for the traceability chain; intermediate status for AVHRR FAPAR product produced JRC has an The QA4ECV information provided for product details, qualitybyflags, andachieved assessment Remote Sens. 2018, 10,FAPAR 1254 AVHRR advanced status forand thebasic traceability chain; intermediate status and for against standards; status for uncertainty assessment information provided product details, qualityby flags, assessment validation. AVHRRfor FAPAR product produced JRC and has achieved an The QA4ECV against standards; and basic status for uncertainty assessment and advanced status for the traceability chain; intermediate status for NO 2 NO2 validation. information provided for product details, quality flags, and assessment
17 of 21
against standards; and basic status for uncertainty assessment and The QA4ECV NO2 ECV precursor product product has achieved advancedadvanced status forstatus the NO 2 QA4ECV The NO 2 ECV precursor has achieved validation. information provided for product details, traceability, quality flags, and for the information provided for product details, traceability, quality flags, assessment against standards; intermediate status for assessment; The NOagainst 2 ECV precursor product hasuncertainty achieved status and assessment standards; intermediate status advanced for uncertainty NO2QA4ECV and basic status for validation. for the information provided for validation. product details, traceability, quality flags, assessment; and basic status for and assessment against standards; intermediate status for uncertainty The QA4ECV NO 2 ECV precursor product has achieved advanced status assessment; and basic status for the information providedfor forvalidation. product details, traceability, quality flags, HCHO and assessment against standards; intermediate status for uncertainty assessment; and basic status for validation. The QA4ECV HCHO ECV precursor product has achieved advanced HCHOHCHO status for the information provided for product details, traceability, quality The QA4ECV HCHO ECV precursor product has achieved advanced status for for The QA4ECV HCHO ECV precursor product has achieved advanced flags, and assessment against standards; intermediate status HCHO the information provided for product details, traceability, quality flags, and status for the information provided for product details, traceability, quality uncertainty assessment; and basic status for validation. assessment against intermediate status for uncertainty assessment; for flags, and standards; assessment against standards; The QA4ECV HCHO ECV precursor productintermediate has achievedstatus advanced and basic status for validation. uncertainty assessment; and basic status for validation. status for the information provided for product details, traceability, quality CO flags, and assessment against standards; intermediate status for uncertainty assessment; and basic status for validation. CO The IASI FORLI CO product (version 20140922 and version 20100815) has achieved advanced status for the information provided for traceability and The FORLI CO product (version 20140922details; and version 20100815) quality intermediate status for product and basic status has for CO CO IASIflags; achieved advanced status for the information provided for standards. traceability and uncertainty assessment, validation, and assessment against quality flags; intermediate status for product details; and basic status has for The IASI FORLI CO product (version 20140922 and version has The IASI FORLI CO product (version 20140922 and 20100815) version 20100815) uncertainty assessment, validation, and assessment against standards. achieved advanced status for the information provided for traceability and achieved advanced status for the information provided for traceability and
Conclusions quality6.flags; status forstatus product basicand status for status for qualityintermediate flags; intermediate for details; productand details; basic uncertainty assessment, validation, and assessment against standards. uncertainty assessment, validation, and assessment against standards.
Here, we present the service specification for a prototype, pre-operational quality assurance 6. Conclusions (QA) system for ECV data records and services, based on the experiences within the FP7-QA4ECV service specification for complex a prototype, pre-operational quality assurance Here, project. Thewe QApresent systemthe is designed to translate the information about data products that is 6. Conclusions (QA) system for ECV data records and services, based on the experiences within the FP7-QA4ECV 6. Conclusions contained in algorithm theoretical basis documents (ATBDs), product user guides (PUGs), product project. Theheads, QApresent system is designed to translate the information data products thatfor is producers and other documents into a standard format based on aabout simple set of questions the service specification forcomplex a prototype, pre-operational quality assurance Here, we Here, we present the service specification for a prototype, pre-operational quality assurance (QA) contained in algorithm theoretical basis documents (ATBDs), product user guides (PUGs), product each quality indicator. Standardisation of this based information between data products enables fair (QA) system for ECV data records and services, on the experiences within the FP7-QA4ECV system for ECV data records and services, based on the experiences within the FP7-QA4ECV project. producers heads, andproducts other documents into a guidance standard format based a about simple set of questions comparison of data and facilitates on best use ofonthe data products for climate QA system is designed to translate the complex information data products thatfor is The QAproject. systemThe is designed to translate the complex information about data products that is contained each quality indicator. Standardisation of this knowledge information dataguides products enables fair applications. It also helpstheoretical identify gaps indocuments current tobetween drive forward scientific advancement contained in algorithm basis (ATBDs), product user (PUGs), product in algorithm theoretical basis documents (ATBDs), product user guides (PUGs), product producers comparison of data products and facilitates on best based use ofon thea simple data products for climate and good practice. producers heads, and other documents into based aguidance standard format ofquality questions for heads, and other documents into a standard format on a simple set of questions for set each applications. It also helps identify gaps in current knowledge to drive forward scientific advancement Feedback on the operational utility ofofthethis prototype QA4ECV QA system was sourced from fair the each quality indicator. Standardisation information between data products enables indicator. Standardisation of this information between data products enables fair comparison of data and good practice. following: the product producers [53]; the independent QAofoffice auditors (NPLfor andclimate IASBcomparison ofQA4ECV data products and facilitates guidance onfor best use the data products products and facilitates guidance on best use of the data products climate applications. It also helps Feedback on the operational utility of the prototype QA4ECV QA system was sourced from BIRA) [53]; and a selected number of in product “champion” werescientific identified during the It also helps identify gaps current knowledge tousers drivethat forward identifyapplications. gaps in current knowledge to drive forward scientific advancement and good practice.advancement following: the QA4ECV product producers [53]; the independent QA office auditors (NPL IASBproject [54]. The QA4ECV product producers agreed that the QA system is streamlined andand relatively and good Feedback onpractice. the operational utility of the prototype QA4ECV QA system was sourced from BIRA) a the selected number of product “champion” users QA that were identified during the easy Feedback to[53]; use, andonlikewise, champion users were positive concerning thesystem product QA report content, operational utility of[53]; the the prototype QA4ECV was sourced from the the following: the QA4ECV product producers independent QA office auditors (NPL and project [54]. The QA4ECV product producers agreed that the QA system is streamlined and relatively traceability chains, andproduct accessibility of this standardised information. reviewing the following: the QA4ECV producers [53]; the independent QA officeHowever, auditors (NPL and IASBIASB-BIRA) [53]; and a selected number of product “champion” users that were identified during the easy to [53]; use, and likewise, champion users were producers positive concerning theQA product QArevealed report content, quality information provided by the product within the system several BIRA) and a selected number of product “champion” users that were identified during the project [54]. The QA4ECV product producers agreed that the QA system is streamlined and relatively traceability chains, and accessibility of this standardised information. However, reviewing the improvements that can be made to the system architecture, content, and governance. While project [54]. The QA4ECV product producers agreed that the QA system is streamlined and relatively easy to use, and likewise, champion users were positive concerning the product QA report content, quality information provided by the product within the QA system revealed several feedback is detailed in [7,54], some key elements for improvement arethe outlined below. easy to use, and likewise, champion users wereproducers positive concerning product QA report content, traceability chains, and accessibility of this standardised information. However, reviewing the quality improvements thatusers can made toa the architecture, content, andHowever, While The product requested different report layoutinformation. streamlining ifgovernance. information was the not traceability chains, and be accessibility of system this standardised reviewing the information provided by the product producers within the QA system revealed several improvements feedback is detailed in [7,54], some key elements for improvement are outlined below. available, provision of more detail in sections related to the quality flags, validation, and data quality information provided by the product producers within the QA system revealed several that can be made to the system architecture, content, and governance. While the feedback is detailed The product different report layout streamlining informationWhile was not improvements thatusers can requested be made toa the system architecture, content, andifgovernance. the in [7,54], some key elements for improvement are outlined below. available, of[7,54], more some detailkey in elements sections related to the quality flags, validation, and data feedback isprovision detailed in for improvement are outlined below. The product users requested a different report layout streamlining if information was not available, The product users requested a different report layout streamlining if information was not provision of more detail in sections related to the quality flags, validation, and data uncertainties, available, provision of more detail in sections related to the quality flags, validation, and data as well as more product usability case studies to be presented. Further, concern was raised about the
Remote Sens. 2018, 10, 1254
18 of 21
usability of the GCOS requirements and the system maturity matrix information in the QA context. The SMM should only be used as an overarching management tool by funding organisations to track the maturity of their data products, and consideration as to how relevant this information is to data users, needs to be investigated further. The QA system architecture could be improved through a more robust software framework, as well as enhancing the evaluation “audit” functionality and communication exchanges between reviewer and product producer. The “audit” functionality could become part of a review process that leads to the release authorisation of a data record to the public. Such reviews are common practise with operational data providers, such as EUMETSAT, and would lead to mandatory provision of information to the QA system by the data record producers. Such a review process is important for operational activities, such as the Copernicus Climate Change Service, to ensure that published data products are good and mature enough to support the authoritative character of the C3S services that make official statements about climate change on behalf of the EU. The system content may be improved by tailoring content more specifically to each ECV domain (land, ocean, and atmosphere) and re-evaluating the audit categories and evaluation scheme, as well as integrating the traceability chains with the algorithm uncertainty information. The QA4ECV QA system provides a solid architecture for the concepts of QA for ECV data products derived from EO satellites. The system addresses the science and product quality attributes; however, additional quality dimensions, such as that of stewardship and services, to the overall quality of these ECV datasets should be investigated further and incorporated into future iterations of CDR evaluation and quality control processes [55,56]. In particular, this relates to the standardisation of QI information and subsequent requirement for data producers to incorporate all relevant field within their product’s metadata. This issue is highly relevant to international coordination bodies, such as CEOS, and this forum should be used to encourage product development and funding organisations to ensure quality information standardisation and provision within all data products. The QA system was applied successfully to the six QA4ECV data products. A summary quality report has been generated for each data product and made available to data users via the QA system to aid in fitness-for-purpose assessments for different application requirements. Throughout the project, product producers and external QA4ECV “champion users” have provided positive and constructive feedback on the architecture, content, and governance of the prototype QA4ECV QA system presented. All product producer and champion user feedback has been consolidated and will be taken into account for future iterations of the QA system that will be applied in a revised form as part of the evaluation and quality control functionality of the European Copernicus Climate Change Service (C3S). Supplementary Materials: The QA4ECV QA system, including training materials, as well as the QA4ECV Data products are available online at http://www.qa4ecv.eu/. Author Contributions: J.N. is the main author and represents NPL, who led the development of the QA4ECV QA system. F.B. is the lead of the QA4ECV project and responsible for the development and description of the NO2 climate data record. J.-P.M. is responsible for the development and description of the surface albedo products. S.C. and J.-C.L. brought atmospheric expertise in the development and application of the QA4ECV QA system, and they led the development of the QA4ECV atmosphere ECV validation server. S.B. and R.G. are responsible for the development and description of the TIP-LAI/FAPAR climate data record. N.G. is responsible for the development and description of the AVHRR FAPAR climate data record. I.D.S. is responsible for the development and description of the HCHO climate data record. P.C. and M.G. are responsible for the development and description of the CO climate data record. J.S. is responsible for the contribution of geostationary data to the surface albedo products and was assessing QA4ECV data products using the maturity matrix. A.W. is responsible for the software programming of the QA4ECV QA system. Funding: This research was funded by the EU FP7 Project Quality Assurance for Essential Climate Variables (QA4ECV), grant no. 607405. The European Commission is further acknowledged for having supported cross-fertilisation meetings among FP7 (CLIP-C, ERACLIM-2, EUCLEIA, EUPORIAS, UERRA) and H2020 (GAIA-CLIM, FIDUCEO) climate service related projects. Acknowledgments: Though not authors of this paper, we would like to acknowledge the work of our QA4ECV colleagues across 17 individual EU institutions, who contributed to the development of the QA system. We would also like to acknowledge the champion data users, who took time to review and provide valuable feedback on the QA system quality reports.
Remote Sens. 2018, 10, 1254
19 of 21
Conflicts of Interest: The authors declare no conflict of interest.
References 1.
2. 3.
4.
5. 6. 7.
8.
9. 10.
11.
12. 13.
14.
15. 16. 17. 18. 19.
Stocker, T.F.; Qin, D.; Plattner, G.K.; Tignor, M.; Allen, S.K.; Boschung, J.; Nauels, A.; Xia, Y.; Bex, V.; Midgley, P.M. IPCC Summary for Policymakers; Cambridge University Press: Cambridge, UK; New York, NY, USA, 2013. UNFCCC. The Paris Agreement-2015. 2017, p. 27. Available online: http://unfccc.int/paris_agreement/ items/9485.php (accessed on 6 August 2018). GCOS-200. The Global Observing System for Climate: Implementation Needs; World Meteorological Organisation: Geneva, Switzerland, 2016; Available online: https://library.wmo.int/opac/doc_num.php?explnum_id= 3417 (accessed on 6 August 2018). Bojinski, S.; Verstraete, M.; Peterson, T.; Richter, C.; Simmons, A.; Zemp, M. The Concept of Essential Climate Variables in Support of Climate Research, Applications, and Policy. Bull. Am. Meteorol. Soc. 2014, 95, 1431–1443. [CrossRef] Cooke, R.; Golub, A.; Wielicki, B.; Young, D.; Mlynczak, M.; Baize, R. Using the social cost of carbon to value earth observing systems. Clim. Policy 2016, 17, 330–345. [CrossRef] Widlowski, J.-L. Conformity testing of satellite-derived quantitative surface variables. Environ. Sci. Policy 2015, 51, 149–169. [CrossRef] Nightingale, J.; Compernolle, S.; Lambert, J.-C.; Boersma, F.; Muller, J.-P.; Scanlon, T.; Douglas, S.; Fox, N. Framework for a Prototype QA Service in Support of C3S (Version 3.0). National Physical Laboratory, 2018. Available online: http://www.qa4ecv.eu/ (accessed on 6 August 2018). Nightingale, J.; de Rudder, A.; Boersma, F.; Scanlon, T.; Farquhar, C.; Muller, J.-P.; Fox, N. Results from the QA4ECV User Requirements Survey on Quality Assurance in Satellite Data Products; National Physical Laboratory: London, UK, 2015. IPCC. Chapter 8: Quality Assurance and Quality Control; IPCC: Montreal, QC, Canada, 2001. Henry, A.; Wood, A.; Mustafee, I.; Alegre, R.; Blower, J.D.; Kershaw, P.; Nagni, M.; Harwood, P.; Phipps, R. CHARMe: Earth Observation Metadata and the Semantic Web. In Proceedings of the 65th International Astronautical Congress, Toronto, Canada, 29 September–3 October 2014. Scanlon, T.; Douglas, S.; Nightingale, J.; Compernolle, S.; Lambert, J.-C. Linkages between the QA Service of QA4ECV and Other Projects and Initiatives. National Physical Laboratory, 2017. Available online: http://www.qa4ecv.eu/ (accessed on 6 August 2018). Justice, C.; Belward, A.; Morisette, J.; Lewis, P.; Privette, J.; Baret, F. Developments in the validation of satellite sensor products for the study of the land surface. Int. J. Remote Sens. 2000, 21, 3383–3390. [CrossRef] Nightingale, J.; Schaepman-Strub, G.; Nickeson, J. Assessing Satellite Derived Land Product Quality for Earth System Science Applications: Overview of the CEOS LPV Sub-Group. In Proceedings of the International Symposium on Remote Sensing of Envrionment, Sydney, Australia, 10–15 April 2011. Compernolle, S.; Lambert, J.-C.; Niemeijer, S. Prototype QA/Validation Service for Atmospheric ECV Precursors: Detailed Processing Model Version 2. 2016. Available online: http://www.qa4ecv.eu/ (accessed on 6 August 2018). Schulz, J.; John, V.; Kaiser-Weiss, A.; Tan, D.; Swinnen, E.; Roebeling, R. CORE-CLIMAX European ECV CDR Capacity Assessment Report; EUMETSAT: Darmstadt, Germany, 2015. Bates, J.J.; Privette, J.L. A maturity model for assessing the completeness fo climate data records. EOS Trans. Am. Geophys. Union 2012, 93, 441. [CrossRef] Farquhar, C.; Milnes, M.; Hurst, K.; Scanlon, T. Quality Assurance System User Guide. 2017. Available online: http://www.qa4ecv.eu/ (accessed on 6 August 2018). ISO9001. Quality Management Systems—Fundamentals and Vocabulary; International Organization for Standardization: Geneva, Switzerland, 2015. Franch, B.; Vermote, E.; Roger, J.-C.; Murphy, E.; Becker-Reshef, I.; Justice, C.; Claverie, M.; Nagol, J.; Csisza, I.; Meyer, D.; et al. A 30+ Year AVHRR Land Surface Reflectance Climate Data Record and Its Application to Wheat Yield Monitoring. Remote Sens. 2017, 9, 296. [CrossRef]
Remote Sens. 2018, 10, 1254
20.
21.
22. 23.
24.
25. 26.
27. 28. 29.
30.
31.
32.
33.
34. 35.
36.
37. 38.
20 of 21
Lattanzio, A.; Schulz, J.; Matthews, J.; Okuyama, A.; Theodore, B.; Bates, J.; Knapp, K.; Kosaka, Y.; Schuller, L. Land Surface Albedo from Geostationary Satelites: A Multiagency Collaboration within SCOPE-CM. Bull. Am. Meteorol. Soc. 2013, 94, 205–214. [CrossRef] Lewis, P.; Guanter, L.; Saldana, G.L.; Muller, J.-P.; Watson, G.; Shane, N.; Kennedy, T.; Fisher, J.; Domenech, C.; Preusker, R.; et al. The ESA GlobAlbedo Project: Algorithm. In Proceedings of the 2012 IEEE International Geoscience and Remote Sensing Symposium (IGARSS12), Munich, Germany, 22–27 July 2012; pp. 5745–5748. EUMETSAT. Meteosat Surface Albedo Retrieval: Algorithm Theoretical Basis Document. 2014. Available online: http://www.qa4ecv-land.eu/docs/BRDFalbedo/PDF_MSA_ATBD.pdf (accessed on 6 August 2018). Muller, J.P.; Kennedy, T.; Lewis, P.; Kharbouche, S.; Fisher, D.; North, P.; Fisher, J.; Preusker, R.; Brockmann, C.; Kramer, U.; et al. GlobAlbedo Algorithm Theoretical Basis Document Version 4.12. 2013. Available online: http://www.globalbedo.org/docs/GlobAlbedo_Albedo_ATBD_V4.12.pdf (accessed on 6 August 2018). Karlsson, K.G.; Anttila, K.; Trentmann, J.; Stengel, M.; Meirink, J.; Devasthale, A.; Hanschmann, T.; Kothe, S.; Jääskeläinen, E.; Sedlar, J.; et al. CLARA-A2: The second edition of the CM SAF cloud and radiation data record from 34 years of global AVHRR data. Atmos. Chem. Phys. 2017, 17, 5809–5828. [CrossRef] Riggs, G.; Hall, D. MODIS Sea Ice Products User Guide to Collection 6. 2015. Available online: https://nsidc. org/sites/nsidc.org/files/files/modis-sea-ice-user-guide-C6%5B1%5D.pdf (accessed on 6 August 2018). Muller, J.-P.; Kharbouche, S. Variation of Arctic’s Sea-ice Albedo between 2000 and 2016 by fusion of MISR and MODIS data. In Proceedings of the EGU General Assembly Conference Abstracts, Vienna, Austria, 23–28 April 2017. Kharbouche, S.; Muller, J.-P.; Gatebe, C.; Scanlon, T. Assessment of Satellite-Derived Surface Reflectances by NASA’s CAR Airborne Radiometer over Railroad Valley Playa. Remote Sens. 2017, 9, 562. [CrossRef] Voßbeck, M.; Clerici, M.; Kaminski, T.; Lavergne, T.; Pinty, B.; Giering, R. An inverse radiative transfer model of the vegetation canopy based on automatic differentiation. Inverse Probl. 2010, 26, 095003. [CrossRef] Peng, J.; Blessing, S.; Giering, R.; Muller, B.; Gobron, N.; Nightingale, J.; Boersma, F.; Muller, J.-P. Quality-assured long-term satllite-based leaf area index product. Glob. Chang. Biol. 2017, 23, 5027–5028. [CrossRef] [PubMed] Pinty, B.; Lavergne, T.; Dickinson, R.; Widlowski, J.-L.; Gobron, N.; Verstraete, M. Simplifying the interaction of land surfaces with radiation for relating remote sensing products to climate models. J. Geophys. Res. Atmos. 2006, 111, 20. [CrossRef] Disney, M.; Muller, J.-P.; Kharbouche, S.; Kaminski, T.; Vossbeck, M. fAPAR/LAI Product Validation Report. ESA, Ed.; 2013, p. 50. Available online: http://aramis.obspm.fr/~jimenez/Docs/WACMOSET/ WACMOSET_WP2230_approved.pdf (accessed on 6 August 2018). Muller, J.-P.; Lewis, P.; Disney, M. Design of the Albedo/fAPAR/LAI Products. Technical Report WP2210-2220. 2013, p. 24. Available online: http://aramis.obspm.fr/~jimenez/Docs/WACMOSET/ WACMOSET_WP2210_WP2220_approved.pdf (accessed on 6 August 2018). Kaminski, T.; Knorr, W.; Scholze, M.; Gobron, N.; Pinty, B.; Giering, R.; Mathieu, P.-P. Consistent assimilation of MERIS FAPAR and atmospheric CO2 into a terrestrial vegetation model and interactive mission benefit analysis. Biogeosciences 2012, 9, 3173–3184. [CrossRef] UCL/MSSL; Brockmann Consult; EUMETSAT; FastOpt; JRC. D7.5–Product User Guide Document. 2018. Available online: http://www.qa4ecv-land.eu/document.php (accessed on 6 August 2018). Muller, J.P.; Kharbouche, S.; Watson, G.; Danne, O.; Blessing, S.; Giering, R.; Gobron, N.; Govaerts, Y.; Schulz, J.; Lattanzio, A.; et al. Quality assessment of Land ECV Data Products. 2018. Available online: http://www.qa4ecv.eu/sites/default/files/D5.4_v1.0.pdf (accessed on 6 August 2018). Gobron, N.; Pinty, B.; Aussedat, O.; Taberner, M.; Faber, O.; Melin, F.; Lavergne, T.; Robustelli, M.; Snoeij, P. Uncertainty Estimates for the FAPAR Operational Products Derived from MERIS—Impact of Top-of-Atmosphere Radiance Uncertainties and Validation with Field Data. Remote Sens. Environ. 2008, 112, 1871–1883. [CrossRef] Gobron, N.; Belward, A.; Pinty, B.; Knorr, W. Monitoring Biosphere Vegetation 1998–2009. Geophys. Res. Lett. 2010, 37. [CrossRef] Gobron, N. Terrestrial vegetation dynamics. Bull. Am. Meteorol. Soc. 2017, 98, S57. [CrossRef]
Remote Sens. 2018, 10, 1254
39.
40.
41. 42. 43.
44. 45.
46.
47.
48. 49.
50.
51.
52.
53.
54. 55. 56.
21 of 21
Boersma, K.F.; Eskes, H.H.J.; Richter, A.A.; de Smedt, I.I.; Lorente, A.A.; Beirle, S.; van Geffen, J.; Zara, M.; Peters, E.; van Roozendael, M.; et al. Improving algorithms and uncertainty estimates for satellite tropospheric NO2 retrievals: Results from the Quality Assurance for Essential Climate Variables (QA4ECV) project. Atmos. Meas. Tech.. in preparation. Boersma, K.F.; Eskes, H.; Richter, A.; de Smedt, I.; Lorente, A.; Beirle, S.; van Geffen, J.; Peters, E.; van Roozendael, M.; Wagner, T. QA4ECV NO2 Tropospheric and Stratospheric Vertical Column Data from OMI (Version 1.1); Royal Netherlands Meteorological Institute (KNMI): De Bilt, The Netherlands, 2017. Vinken, G.C.M.; Boersma, K.F.; Maasakkers, J.D.; Adon, M.; Martin, R.V. Worldwide biogenic soil NOx emissions inferred from OMI NO2 observations. Atmos. Chem. Phys. 2014, 14, 10363–10381. [CrossRef] Verstraeten, W.W.; Neu, J.L.; Williams, J.E.; Bowman, K.W.; Worden, J.R.; Boersma, K.F. Rapid increases in tropospheric ozone production and export from China. Nat. Geosci. 2015, 8, 690–695. [CrossRef] Xu, X.; Wang, J.; Henze, D.K.; Qu, W.; Kopacz, M. Constraints on aerosol sources using GEOS-Chem adjoint and MODIS radiances, and evaluation with multisensor (OMI, MISR) data. Geophys. Res. Atmos. 2013, 118, 6396–6413. [CrossRef] Castellanos, P.; Boersma, K.F. Reductions in nitrogen oxides over Europe driven by environmental policy and economic recession. Sci. Rep. 2012, 2, 265. [CrossRef] [PubMed] Inness, A.; Baier, F.; Benedetti, A.; Bouarar, I.; Chabrillat, S.; Clark, H.; Clerbaux, C.; Coheur, P.; Engelen, R.J.; Errera, Q.; et al. The MACC reanalysis: An 8 yr data set of atmospheric composition. Atmos. Chem. Phys. 2013, 13, 4073–4109. [CrossRef] Boersma, K.F.; Vinken, G.C.M.; Eskes, H.J. Representativeness errors in comparing chemistry transport and chemistry climate models with satellite UV–Vis tropospheric column retrievals. Geosci. Model Dev. 2016, 9, 875–898. [CrossRef] De Smedt, I.; Theys, N.; Yu, H.; Danckaert, T.; Lerot, C.; Compernolle, S.; van Roozendael, M.; Richter, A.; Hilboll, A.; Peters, E.; et al. Algorithm Theoretical Baseline for formaldehyde retrievals from S5P TROPOMI and from the QA4ECV project. Atmos. Meas. Tech. 2018, 11, 2395–2426. [CrossRef] Stavrakou. Impact of climate variability on volatile organic compounds emissions assessed using OMI formaldehyde observations. Geophys. Res. Lett. 2018, in press. Bauduin, S.; Clarisse, L.; Theunissen, M.; George, M.; Hurtmans, D.; Clerbaux, C.; Coheur, P.-F. IASI’s sensitivity to near-surface carbon monoxide (CO): Theoretical analyses and retrievals on test cases. J. Quant. Spectrosc. Radiat. Transf. 2017, 189, 428–440. [CrossRef] Hurtmans, D.; Coheur, P.-F.; Wespes, L.; Clarisse, L.; Scharf, O.; Clerbaux, C.; Hadji-Lazaro, J.; George, M.; Turquety, S. FORLI radiative transfer and retrieval code for IASI. J. Quant. Spectrosc. Radiat. Transf. 2012, 113, 1391–1408. [CrossRef] George, M.; Clerbaux, C.; Bouarar, I.; Coheur, P.-F.; Deeter, M.; Edwards, D.; Francis, G.; Gille, J.; Hadji-Lazaro, J.; Hurtmans, A.; et al. An examination of the long-term CO records from MOPITT and IASI: Comparison of retrieval methodology. Atmos. Meas. Tech. 2015, 8, 4313–4328. [CrossRef] Müller, J.-F.; Stavrakou, T.; Bauwens, M.; George, M.; Hurtmans, D.; Coheur, P.-F.; Clerbaux, C.; Sweeney, C. Top-Down CO Emissions Based On IASI Observations and Hemispheric Constraints on OH Levels. Geophys. Res. Lett. 2018, 45, 1621–1629. [CrossRef] Nightingale, J.; Compernolle, S.; Boersma, F.; Muller, J.-P.; Blessing, S.; George, M.; de Smedt, I.; Danne, O.; Kharbouche, S.; Gobron, N.; et al. Audit Reports for the QA of the Six QA4ECV ECV Products. QA4ECV Deliverable D2.8. 2018. Available online: http://www.qa4ecv.eu/ (accessed on 6 August 2018). Nightingale, J.; Douglas, S.; Compernolle, S.; Boersma, F.; Muller, J.-P. Update of the User Requirements Report. QA4ECV Deliverable D1.3. 2018. Available online: http://www.qa4ecv.eu/ (accessed on 6 August 2018). Lee, Y.; Strong, D.; Kahn, B.; Wang, R. AIMQ: A methodology for information quality assessment. Inf. Manag. 2002, 40, 133–146. [CrossRef] Peng, J.; Privette, J.; Tilmes, C.; Bristol, S.; Maycock, R.; Bates, J.; Hausman, S.; Brown, O.; Kearns, E. A Conceptual Enterprise Framework for Managing Scientific Data Stewardship. Data Sci. J. 2018, 17. [CrossRef] © 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
