Home | CoMet Toolkit

Met4EO Training on Uncertainty Analysis for Earth Observation Datasets 2026

Fri, 01 May 2026 00:00:00 +0000

ARIA 2026 Workshop

Tue, 14 Apr 2026 00:00:00 +0000

Wavelength 2026 Workshop

Tue, 07 Apr 2026 00:00:00 +0000

curepy

Tue, 31 Mar 2026 00:00:00 +0000

❔ What is curepy?

curepy is a Python package designed to propagate uncertainties through inverse problems. The name curepy stands for “Comet Uncertainties for REtrievals in PYthon”.

Currently, only a beta version is available for curepy. Please use with caution.

📍 Where can curepy be found?

Curepy can be installed via pip, and is available through:

Github here
curepy documentation here
Example jupyter notebook in the example section.

📋 What can curepy be used for?

curepy provides tools to solve inverse problems in which a state vector and its associated uncertainties is retrieved from measurements, optional prior information, and optionally taking into account any required ancillary parameters.

In inverse problems, uncertainties arise not only from measurement noise but also from the ill‑posedness of the retrieval. To obtain meaningful results, these uncertainties must be rigorously characterised and propagated through the inversion process. This includes handling error-correlations in the measurement data, as well as uncertainties introduced by priors, forward‑model assumptions, and model sensitivities. curepy quantifies how each of these factors affects the retrieved state, providing a robust framework for assessing retrieval reliability and uncertainty.

To make this process straightforward, curepy implements tools that allows users to propagate uncertainties through any measurement or forward model function written in Python, ranging from simple analytical expressions to full numerical processing chains (including external components such as radiative‑transfer models), as long as they can be wrapped in a Python function.

Uncertainty propagation through the retrieval can be performed using:

Markov Chain Monte Carlo (MCMC)
Optimal Estimation (OE)

Measurement values and uncertainties can be supplied manually or via an obsarray dataset. curepy also provides tools to analyse retrieval outputs, quantify uncertainties, compare results, and export them again as obsarray datasets.

Imperial training

Thu, 19 Mar 2026 00:00:00 +0000

PANGEOS training

Thu, 19 Mar 2026 00:00:00 +0000

Spring 2026 Workshops

Sun, 01 Mar 2026 00:00:00 +0000

CoMet has a booked and busy Spring 2026 - with five workshops planned across the season.

These sessions are delivered with our partners across academic and professional programmes, more information about each organisation/event is linked below.

For any events where public registration is available, we will post the details closer to the date!

1. Imperial College London

🗓 When: Thursday 19th March

📍 Where: Imperial College London (London, UK)

2. Pangeos

🗓 When: Monday 23rd March

📍 Where: Online

🔗 More information: https://pangeos.eu/

3. Wavelength Conference 2026

🗓 When: Thursday 9th April

📍 Where: National Physical Laboratory (Teddington, UK)

🔗 More information: https://rspsoc.org.uk/

4. ARIA

🗓 When: Wednesday 15th April

📍 Where: National Physical Laboratory (Teddington, UK)

🔗 More information: https://aria.org.uk/

5. Met4EO

🗓 When: 20th-22nd May

📍 Where: National Physical Laboratory (Teddington, UK)

🔗 More information: https://www.qa4eo.org/met4eo/

🗸 In these tutorial, we will:

Introduce the key concepts behind metrological uncertainty propagation
Walk through the core components of the toolkit:
- punpy – for robust uncertainty propagation with support for error-correlation structures
- obsarray – for managing and storing uncertainty metadata in a self-describing, traceable format
Work through a sensor calibration example using Google Colab-based Jupyter notebooks

We’ll also provide support to get you started using CoMet with your own example use-case if time permits — so feel free to bring a Python example from your work involving measurement uncertainty.

Whether you’re working in satellite Cal/Val, EO data processing, or any other measurement-driven field, this session will help you implement rigorous, traceable uncertainty handling in your processing chains.

🗸 No installation is required beforehand – all examples will run in Google Colab!

MetEOR Toolkit

Sat, 01 Nov 2025 00:00:00 +0000

❔ What is MetEOR?

The MetEOR Toolkit (Metrology for Earth Observation and Radiometry) is an open-source Python toolkit for the comparison of satellite and reference measurements.

It supports scalable, reproducible, and uncertainty-aware Earth Observation (EO) calibration and validation workflows, helping users work consistently across large and diverse EO data archives.

MetEOR is designed as a modular toolkit, allowing users to adopt individual components or build complete workflows.

Core functionality includes:

Identification of satellite matchup opportunities
Product search, filtering, and retrieval across EO catalogues
Harmonised access to satellite and reference data
Automated construction of comparison-ready datasets
Support for uncertainty handling and analysis
brdf and atmospheric corrections

The toolkit integrates key steps of EO comparison into a coherent and reproducible workflow.

📍 Where can the MetEOR Toolkit be accessed?

MetEOR is available on Github, with code, documentation (click on package name below), and training materials available to support uptake by the EO community. Each of the open source tools is also installable via pip.

Currently, the following tools are available as open source:

orbitx: Propagates satellite orbits using Two Line Element (TLE) data to identify potential matchup events by finding spatiotemporal intersections between satellite ground tracks.
scrappi: Queries, retrieves, and organises Earth observation products from multiple APIs and catalogues using a unified interface, enabling metadata-based filtering before download.
eoio: Provides a harmonised data-access framework that reads diverse EO products, extracts collocated regions of interest, and outputs standardised datasets with measurements, metadata, and uncertainties.
eomatch: Acts as the orchestration layer that links matchup discovery, product association, and catalogue representation into a unified, reusable workflow for EO comparison analyses.
pydirectional: Models and corrects bidirectional reflectance effects by simulating and fitting BRDF behaviour to account for viewing and illumination geometry differences in comparisons.
processor_tools: A set of modular processing utilities within EO data workflows that apply transformations such as interpolation, coordinate generation, and unit conversion to enrich datasets.
matheo: The matheo module is a Python package providing mathematical tools for Earth observation data, including functionality for spectrally integrating measurements using sensor spectral response functions. Provide your feedback on BizChat

There are also two tools under development, which are not yet open source:

scene_forge: Scene-modelling component intended to support radiative transfer and synthetic scene generation within comparison workflows.
eoalign: A python package for preparing comparison samples for uncertainty-quantified comparisons of satellite and reference data.

💡Why MetEOR?

Comparison of EO measurements is central to calibration and validation, but generating robust matchup datasets across modern archives is often complex, computationally intensive, and fragmented.

MetEOR was developed to address this by providing a single, flexible framework that enables:

Systematic and scalable generation of matchup datasets
Consistent, uncertainty-aware comparison workflows
Integration with modern EO data standards and catalogues
Accessible, reusable tools for the wider community

📋 Demonstrated Capability

MetEOR has been applied to both large-scale and targeted comparison studies, including Sentinel-2 and Landsat-8 analyses.

These demonstrate that the toolkit can:

Identify large volumes of cloud-free matchups
Generate high-quality subsets for calibration studies
Produce standardised, analysis-ready outputs

This enables consistent comparison workflows across a wide range of EO applications. All the tools in the matchup pipeline used for the CEOS-PVP, are part of the MetEOR toolkit.

📍 Impact

The MetEOR Toolkit lowers the barrier to rigorous, reproducible EO comparison studies, supporting:

Improved consistency across satellite missions
More traceable calibration and validation workflows
Scalable analysis of multi-mission datasets

👋 Acknowledgements & Citations

Developed by: National Physical Laboratory (NPL)

Funding: The development of the MetEOR toolkit was funded through a number of different project, most notably the European Space Agency (ESA) Met4EO project.

This work was also supported by the UK’s Natural Environment Research Council [NERC grant reference number NE/X019071/1, “UK EO Climate Information Service”], the ESA-funded TRUTHS science studies (TRUTHS Mission Accompanying Consolidation (TMAC) and TRUTHS mission Accompanying Consolidation towards Operations Study (TACOS) contracts) and the National Measurement System programme of the UK Government’s Department for Science, Innovation and Technology.

Citation: Hunt S. E., De Vis P., Stedman M. et al. MetEOR Toolkit. [online] National Physical Laboratory. Available at: (https://www.comet-toolkit.org/meteor)

VH-RODA training

Wed, 01 Oct 2025 00:00:00 +0000

VH-RODA workshop 2025

Wed, 01 Oct 2025 00:00:00 +0000

We’re pleased to invite you to a hands-on tutorial session on the CoMet (Community Metrology) Toolkit, taking place during ESA’s Very High-resolution Radar & Optical Data Assessment (VH-RODA) 2025 Workshop which will be held at ESA-ESRIN, Frascati (Italy), from 17-21 November 2025. The tutorial itself will take place:

🗓 Wednesday 19th November

🕒 12:50 – 13:40

📍 ESA-ESRIN (Frascatti, Italy), Big hall

The CoMet Toolkit (www.comet-toolkit.org) is an open-source suite of Python tools for handling and propagating uncertainties and error-correlation in measurement data. Originally designed for Earth Observation (EO) applications, CoMet’s flexible tools can be applied to any dataset involving uncertainties — making it highly relevant for anyone working with measurement functions and data analysis in Python.

🗸 In this tutorial, we will:

Introduce the key concepts behind metrological uncertainty propagation
Walk through the core components of the toolkit:
- punpy – for robust uncertainty propagation with support for error-correlation structures
- obsarray – for managing and storing uncertainty metadata in a self-describing, traceable format
Work through a sensor calibration example using Google Colab-based Jupyter notebooks

This session will be tailored to implement rigorous, traceable uncertainty handling in satellite Cal/Val processing chains.

🗸 No installation is required beforehand – all examples will run in Google Colab!

LPS training

Fri, 20 Jun 2025 00:00:00 +0000

New Website live!

Wed, 18 Jun 2025 00:00:00 +0000

The CoMet Website Has a Fresh New Look!

We’re excited to share that the CoMet website has been updated — with a cleaner design, refreshed content, and plenty of new material to explore.

Here’s what’s new:

🧭 A new projects page showcases a growing list of projects using the CoMet Toolkit.
🚀 A dedicated getting started page is now available to help you begin working with CoMet quickly and confidently.
📘 A theoretical background section has been added, including introductions to uncertainties, QA4EO, processing chains, and error correlation.
🔍 A case studies section now features practical applications — starting with HYPERNETS and GRUAN.
📂 The examples section has been reorganised under the User Guide, and includes several new examples.
🎓 A new training section has launched, currently hosting materials from our first CoMet tutorial held at ESA’s Living Planet Symposium.

Explore the updated site and let us know what you think!

Projects & Partner Organisations!

Wed, 18 Jun 2025 00:00:00 +0000

Here’s the ever-growing list of the projects and organisations which have utilised CoMet’s capabilities!

🗸 QA4EO

📋 Overview: QA4EO is a Quality Assurance framework for Earth Observation (EO) data, providing guidance and best practices across the EO community.

🔗 URL: QA4EO Website

☄️ Involvement: The CoMet toolkit received funding from QA4EO for its development and provides the tools for the practical implementation of some of the QA4EO steps to an uncertainty budget.

🗸 HYPERNETS

📋 Overview: HYPERNETS developed an automated network of hyperspectral radiometers for validating surface reflectance of water and land for satellite missions.

🔗 URL: HYPERNETS Website

☄️ Involvement: The CoMet toolkit is used within the HYPERNETS_processor to propagate uncertainties from lab calibration through each of its products. See also this case study page.

🗸 CHIME L2

📋 Overview: The CHIME L2 project performs atmospheric correction for the Copernicus Hyperspectral Imaging Mission for the Environment (CHIME).

☄️ Involvement: CoMet is used for the uncertainty propagation through the atmospheric correction algorithms.

🗸 FLEX validation

📋 Overview: FLEX validation supports the calibration and validation of ESA’s Fluorescence Explorer mission using ground-based and airborne measurements.

☄️ Involvement: The CoMet toolkit is used to ensure the validation results are accompanied by uncertainties, and thus can be reliably interpreted.

🗸 LIME

📋 Overview: LIME is the Lunar Irradiance Model of the European Space Agency (ESA), which aims to determine an improved lunar irradiance model with sub-2% radiometric uncertainties.

🔗 URL: LIME Website

☄️ Involvement: These uncertainties are propagated through the LIME toolbox using CoMet.

🗸 Met4EO

📋 Overview: Met4EO aims to develop, apply and extend a metrological approach to the uncertainty analysis and comparison of satellite data products.

☄️ Involvement: The CoMet toolkit will be used in a range of its case studies.

🗸 TACOS

📋 Overview: The TRUTHS mission Accompanying Consolidation Towards Operations Study (TACOS) performs scientific studies in preparation of the TRUTHS mission.

☄️ Involvement: The CoMet toolkit is used for uncertainty propagation in various parts.

🗸 FRM4SOC

📋 Overview: FRM4SOC sets out to define above-water ocean colour radiometry best practice for measurement and uncertainty propagation, for two radiometers in common usage, being implementing into an open source community processor.

🔗 URL: FRM4SOC Website

☄️ Involvement: CoMet has been used heavily to propagate uncertainties through a complicated measurement system and provide uncertainty outputs for L1 and L2 products.

🗸 RPV4PICS

📋 Overview: RPV4PICS advances reflectance product validation for satellite data using improved BRDF and aerosol modelling over Pseudo Invariant Calibration Sites (PICS).

☄️ Involvement: Uncertainties were propagated using CoMet.

🗸 MACRAD

📋 Overview: - The Metrological Analysis of CIMR Radiometry (MACRAD) study.

☄️ Involvement: Uses CoMet to propagate uncertainties for the Copernicus Imaging Microwave Radiometer (CIMR) mission.

🗸 FDR4ATMOS

📋 Overview: - FDR4ATMOS develops methodologies for full‐disclosure radiometry in atmospheric remote sensing to support climate monitoring.

🗸 DTE-S2GOS

📋 Overview: The Digital Twin Earth Synthetic Scene Generator and Observation Simulator (DTE-S2GOS) project focuses on developing a new pre-operational service to be implemented as a component of the ESA DestinE Platform.

🔗 URL: S2GOS Website

☄️ Involvement: Validation of this service needs a metrological approach, which is implemented using CoMet.

🗸 DEFLOX

📋 Overview: DEFLOX is a project supporting the development of the FloX system for Solar induced Chlorophyll Fluorescence observation.

☄️ Involvement: The CoMet Toolkit is being used to develop uncertainty propagation through the FLOX processing chain.

🗸 FRM4FLUO

📋 Overview: - FRM4FLUO establishes standards for the calibration and validation of satellite-derived fluorescence measurements of vegetation.

🗸 GRUAN

📋 Overview: Global Climate Observing System (GCOS) ensures access to long-term climate observations. Within GCOS, the GCOS Reference Upper-Air Network (GRUAN) plays a key role by providing reference-quality atmospheric balloon data.

🔗 URL: GRUAN Website

☄️ Involvement: A case study where the CoMet Toolkit has been successfully implemented with GRUAN data to acquire covariance information of GRUAN data. Read more on the case study page.

🗸 St3TART-FO

📋 Overview: The St3TART-FO project is aimed at providing an operational framework for Fiducial Reference Measurements (FRM) in support of the validation activities of the Sentinel-3 (S3) radar altimeter over land surfaces of interest, including inland water bodies (lakes, reservoirs, rivers including estuarian areas), as well as sea ice and land ice areas (ice caps, mountain glaciers, etc.).

🔗 URL: St3TART Website

☄️ Involvement: CoMet has been used to implement uncertainty propagation within the St3TART-FO project.

Introduction to uncertainties

Sun, 15 Jun 2025 00:00:00 +0000

Navigating the world of uncertainties can be quite tricky, as often terms such as errors and uncertainties are used interchangeably, even though they do not describe the same thing.

In this brief introduction to uncertainties and their propagation, we will highlight the most important concepts and signpost you to useful resources.

By providing learning materials covering relevant topics at varying levels of depth, we want to ensure that you can fully understand and therefore implement the capabilities of this toolkit.

❔ Why do we care about uncertainties?

Uncertainties are a vital aspect of the measurement science, as they provide credibility and trust in measured data.

To ensure that everyone is on the same page, a great place to start is this Measurement Good Practice Guide put together by the National Physical Labpratory (NPL).

It covers all the relevant terminology, best practises, and examples for understanding the uncertainty of measurements.

As mentioned at the beginning of this article, we can confirm that uncertainties and errors are, in fact, not the same thing:

Uncertainty is a quantification of the doubt about the measurement result.

Error is the difference between the measured value and the “true value” of the thing being measured.

📜 Metrology & the Guide to Uncertainties in Measurements

Metrology, the science of measurement, is the discipline that maintains the SI and the associated system of measurement, ensuring measurements are stable over time and measurement standards equivalent worldwide. These properties are supported by the principles of metrological traceability: uncertainty analysis and comparison.

The Guide to the expression of Uncertainty in Measurement or GUM for short, is the ultimate resource to refer to, when you are evaluating measurement uncertainties.

All the official documentation of GUM as well as a guide to the International Vocabulary of Metrology (VIM) are available here.

🏆 Best practice framework for EO

To ensure credible and reliable interpretation of environmental observations from satellites and in-situ measurements, Committee on Earth Observation Satellites (CEOS) has established and endorced the Quality Assurance framework for Earth Observation (QA4EO).

This framework requires that associated uncertainty information is provided for all measurements. Additionally, it highlights the importance of understanding the error-covariances in the data.

Approaches defined within QA4EO enable the EO community to develop quantitative characterisation of uncertainty in EO data.

QA4EO developed a 5-steps approach to do an uncertainty budget. These 5 steps are described on this CoMet page and in the QA4EO process document.

However, practically implementing these methods is not trivial and can be time consuming.

As a way to facilitate this, the CoMet Toolkit was developed as a means to store and propagate uncertainty and error-correlation information.

✔️Other useful resources for Uncertainty in Earth Observations (EO)

A great resource that describes all the various aspects of uncertainty propagation for satellite EO data is FIDUCEO.

This project provides a comprehensive guide to understanding and implementing:

QA4EO uncertainty budget

Sat, 14 Jun 2025 00:00:00 +0000

QA4EO is a best practice framework, based on the principles of metrology, to establish The Global Earth Observation System of Systems — based on coordinated and harmonised processes and activities that enable interoperability. QA4EO was established and endorsed by the Committee on Earth Observation Satellites (CEOS). QA4EO ensures credible and reliable interpretation of environmental observations from satellites and in-situ measurements by requiring that associated uncertainty information is provided. A set of guidelines was developed on how to apply the QA4EO principles to generate metrologically-rigorous data products and to perform uncertainty analysis.

There are five steps towards a metrological uncertainty analysis. These steps are described in detail in the QA4EO Process document which provides Step-by-step guidance on implementing a metrological approach to uncertainty analysis. The 5 steps are: • Step 1: Define the measurand and measurement model • Step 2: Establish the traceability with a diagram (see example at the top of this page) • Step 3: Evaluate each source of uncertainty and fill out an effects table • Step 4: Calculate the data product and uncertainties • Step 5: Record information about the uncertainty analysis for long term data preservation purposes (implicit above) and summarise for today’s users

The CoMet toolkit is particularly useful for steps 4, (see our page on propagating uncertainties through a measurement function) and step 5 (see our page on error correlation and how to store it).

As discussed in the QA4EO Process document, Effects Tables (step 3) are a useful way to record and report the information required to fully parameterise an error-correlation effect. However, to use this information in a processing chain, it must be provided digitally. The CoMet Toolkit defines a mechanism for this, with a metadata standard (see UNC specification) that enables the creation of Digital Effects Tables stored in NetCDF files. In this way, uncertainty information can be written, read,

Punpy interfaces with obsarray to make uncertainty propagation as efficient and easy to use as possible. The digital effects tables produced with obsarray can be propagated through measurement functions using punpy, without the need for providing additional information. The data has thus been encoded with all relevant error-covariance information, though users do not need to interact with it. Together these tools enable both the experienced and inexperienced user to efficiently include uncertainties throughout their data processing, and thus make their datasets more reliable and interpretable. Optional keywords provide the user with the flexibility to deal with all kinds of complex use cases. For further info, we refer to the punpy and obsarray documentation.

Propagating uncertainties through a processing chain

Fri, 13 Jun 2025 00:00:00 +0000

When determining the value of a given measurand (i.e. the quantity intended to be measured), some processing typically needs to be performed to calculate the measurand based on some input quantities. In some cases, this processing is relatively simple, and can be done with a simple analytical function. In other cases, a full processing chain is needed, which can contain a whole range of computational processing steps. In metrology (GUM), this relationship between the measurand and the input quantities is refered to as a measurement model or a measurement function. For some further information, we refer to this FIDUCEO tutorial.

In order to calculate the uncertainty on the measurand, one needs to propagate uncertainties on the input quantities through the measurement model. There are various methods that can be used for this. Two commonly used ones are the Monte Carlo (MC) method and the Law of Propagation of Uncertainties. For further detail on these methods, see the GUM and its supplements, the QA4EO metrology document, or the punpy ATBD.

The CoMet toolkit implements both the Monte Carlo and the Law of Propagation of Uncertainty methodologies for uncertainty propagation. The CoMet toolkit interface to both of these methodlogies is exactly the same, so the user can easily switch between them and compare the results. There are two ways the input data and their uncertainties can be ingested in the CoMet toolkit. Either they can be ingested manually as numpy arrays, or they can be ingested using `Digital Effects Tables’, defined with obsarray.

In order to propagate uncertainties using the CoMet toolkit, one needs to be able to write the measurement function as a Python function which takes the input quantities as arguments and returns the measurand. Inside this measurement function, there can be a varying range of complexity. It could be a simple analytical function, or a full processing chain, including calls to other external software. The CoMet toolkit treats this measurement function somewhat as a blackbox and simply modifies the inputs, and analyses the outputs.

Error correlation and how to store it

Thu, 12 Jun 2025 00:00:00 +0000

See this FIDUCEO tutorial for further information on how to evaluate error-correlation information.

CoMet Toolkit vs. Atmospheric Weather Balloons

Tue, 13 May 2025 00:00:00 +0000

🎈 Background

Global Climate Observing System (GCOS) ensures access to long-term climate observations. Within GCOS, the GCOS Reference Upper-Air Network (GRUAN) plays a key role by providing reference-quality atmospheric balloon data.

GRUAN data includes measurements such as temperature, humidity and pressure. This data is traceable to SI standards, and published with detailed uncertainty and metadata. At present, GRUAN does not report detailed error correlation across its data products.

This article outlines a study which has successfully deployed CoMet Toolkit to acquire covariance information of GRUAN data.

🗸 Aims

Investigate GRUAN data product uncertainty components.
Obtain covariance matrices for temperature (T) and relative humidity (RH).
Assess how different data processing strategies impact covariance.

📋 Data

GRUAN data is gathered via radiosondes mounted on weather balloons, recording vertical atmospheric profiles at high resolution (1–2 seconds per point) reaching altitudes of 30–35 km.

We selected two radiosonde launches from Lindenberg, Germany (GRUAN’s lead centre) on January 1st, 2024 at 00:00 and 12:00 UTC. The data is available in NetCDF format, and includes full metadata and GRUAN-validated uncertainty components.

Each balloon ascent collects thousands of measurements. To reduce the number of data points, this data is typically layered. For instance, 137-layer ECMWF pressure coordinate system, is commonly used in atmospheric modelling.

🗸 Uncertainties

For each measurement point, GRUAN reports the combined uncertainty as well as three uncertainty components:

Uncorrelated: random, no relationship between points exists.
Spatially Correlated: data within a single vertical profile is spatially related
Temporally Correlated: data across different soundings and/or sites is temporally related (e.g., due to shared calibration).

GRUAN assumes full (r = 1) correlation for spatial and temporal components—an important simplification carried into the analysis.

☄️CoMet Setup

Covariance and correlation matrices were calculated using CoMet.

obsarray: used to re-structure weather balloon measurements, their uncertainty, and metadata (units, dimensions)
comet_maths: performed the mathematical operations to compute covariance matrices, correlation information

🗂️ Results

Using CoMet, raw covariance matrices were computed from the unprocessed midday profile.

Temperature and RH matrices show strong diagonals (variance) and structured off-diagonal patterns (systematic uncertainty).
The RH covariance scale is notably higher, especially due to sensitivity to solar radiation during daytime launches.

In layered heatmaps:

White regions indicate missing data beyond balloon burst.
Mean-layered data shows reduced variance (weaker diagonal).
Subsampled data retains more variance but still shows strong off-diagonal structure due to full assumed correlation.

Notably, the Upper Troposphere–Lower Stratosphere (UTLS) region exhibits distinct covariance patterns.

🗸 Layered Relative Humidity Covariance

Layered RH matrices differ significantly:

Covariance values are up to 100× greater than for temperature.
Heatmaps show a checkerboard pattern, indicating high inter-layer correlation.
Averaging reduces the variance, and temporal correlations dominate the structure, reflecting RH’s rapid environmental sensitivity.

These findings underscore the need to treat RH and temperature differently in uncertainty-aware models.

🗸 Summary of Findings

This project:

Developed and validated a method for computing covariance matrices from GRUAN RS-41 data using CoMet,
Tested multiple data processing strategies (mean, subsample, integration),
Provided practical recommendations through a GRUAN Technical Note.

The results help researchers better account for correlated uncertainty when working with radiosonde data.

🗸 Future Work

Planned next steps include:

Incorporating Tier 2 uncertainty sources,
Implementing partial correlation handling,
Expanding to other variables (e.g., pressure, wind, radiation).

These developments will enhance climate models’ treatment of uncertainty and ensure even greater accuracy in long-term climate monitoring.

Living Planet Symposium 2025

Thu, 01 May 2025 00:00:00 +0000

We’re pleased to invite you to a hands-on tutorial session on the CoMet (Community Metrology) Toolkit, taking place during ESA’s Living Planet Symposium 2025:

🗓 Sunday 22nd June

🕒 15:30 – 16:50

📍 Vienna, Living Planet Symposium (Room TBC)

The CoMet Toolkit (www.comet-toolkit.org) is an open-source suite of Python tools for handling and propagating uncertainties and error-correlation in measurement data. Originally designed for Earth Observation (EO) applications, CoMet’s flexible tools can be applied to any dataset involving uncertainties — making it highly relevant for anyone working with measurement functions and data analysis in Python.

🗸 In this tutorial, we will:

Introduce the key concepts behind metrological uncertainty propagation
Walk through the core components of the toolkit:
- punpy – for robust uncertainty propagation with support for error-correlation structures
- obsarray – for managing and storing uncertainty metadata in a self-describing, traceable format
Work through a sensor calibration example using Google Colab-based Jupyter notebooks

We’ll also provide support to get you started using CoMet with your own example use-case if time permits — so feel free to bring a Python example from your work involving measurement uncertainty.

🗸 No installation is required beforehand – all examples will run in Google Colab!

CoMet Toolkit used in HYPERNETS Processor

Fri, 14 Jun 2024 00:00:00 +0000

🎈 Background

The LANDHYPERNET and WATERHYPERNET networks (which together make up the HYPERNETS network) consist of a set of autonomous hyperspectral spectroradiometers (HYPSTAR®) acquiring fiducial reference measurements of surface reflectance at various sites covering a wide range of surface types (both land and water) for use in satellite Earth observation validation and remote sensing applications.

The HYPERNETS_PROCESSOR is a Python software package to process the HYPERNETS in-situ hyperspectral raw data to reflectance and other variables.

🗸 Aims

In order to achieve fiducial reference measurement quality, and be optimally useful as a satellite validation reference, uncertainties need to be propagated through each step of the HYPERNETS processing chain, taking into account temporal and spectral error-covariance.

📋 Data

The HYPERNETS instruments measure downwelling hemispherical irradiance, and upwelling radiance. The raw data is provided as digital numbers, and combined with gains and non-linearity coefficients from Lab calibrations. The irradiance and radiance data are combined to calculate reflectance. These data are stored in different product files from L0-L2B.

🗸 Uncertainties

Three uncertainty contributions are tracked throughout the processing.

Random uncertainty: Uncertainty component arising from the noise in the measurements, which does not have any error-correlation between different wavelengths or different repeated measurements (scans/series/sequences).
Systematic independent uncertainty: Uncertainty component combining a range of different uncertainty contributions in the calibration. Only the components for which the errors are not correlated between radiance and irradiance are included.
Systematic uncertainty correlated between radiance and irradiance: Uncertainty component combining a range of different uncertainty contributions in the calibration. Only the components for which the errors are correlated between radiance and irradiance are included. This error-correlation means this component will become negligible when taking the ratio of radiance and irradiance (i.e., in the L2A reflectance products), which is why we separate it from the systematic independent uncertainty.

The two systematic uncertainty components are both based on lab calibrations. Since the same lab calibration is used within the HYPERNETS_ PROCESSOR for repeated measurements (scans/series/ sequences), the errors in the systematic uncertainty components are assumed to be fully systematic (errorcorrelation made up of ones) with respect to different scans/series/sequences. With respect to wavelength, we combine the different error-correlations of the different contributions and calculate a custom error-correlation matrix between the different wavelengths.

CoMet Setup

The HYPERNETS_PROCESSOR uses a Monte Carlo (MC) approach, to propagate uncertainties and error-correlations between product levels. This MC approach is implemented using punpy.

The HYPERNETS products themselves are stored as digital effects tables using obsarray.

Results

An example of the propagated uncertainties on the HYPERNETS radiances is shown below.

These uncertainties can be propagated throughout further processing chains (see e.g. this example)

Summary of Findings

The HYPERNETS_processor is operationally producing product files stored as digital effects tables. Uniquely for this type of processing, multiple different types of uncertainty (including error-correlations) are propagated through each of the processing levels.

These uncertainties have been very useful for the validation of satellite data (e.g. FLEX, CHIME, ENMAP, S2, …).

Future Work

There are a number of uncertainty components that are still missing from the HYPERNETS uncertainty budget (currently a placeholder uncertainty of 2% is added instead). Dedicated studies will be performed to better quantify each of the missing components.

CoMet v1 release

Mon, 15 Apr 2024 00:00:00 +0000

🎉 CoMet Toolkit v1.0 Released!

We’re excited to announce the release of version 1.0 — the first stable release of the CoMet Toolkit!

This milestone includes three core Python packages:

All three are now available on GitHub and can be installed directly via Python package index (pip):

- pip install comet_maths
- pip install punpy
- pip install obsarray

📋 These tools have been validated and tested, and are accompanied by full documentation:

Further updates and improvements will continue to be released on both pip and GitHub as the toolkit evolves!

Getting started with CoMet

Thu, 21 Mar 2024 00:00:00 +0000

Welcome 👋

In this brief guide we will walk you through the basic steps and prerequisites to get started with CoMet.

1.💡 Get familiar with the toolkit and its capabilities.

All the relevant information regarding the aims and functionality of CoMet Toolkit is outlined in the About Section.

But, in a nutshell,

CoMet stands for Community Metrology Toolkit and it a set of software tools that handle, process, and store measurement data uncertainties and error-correlation information.

It accounts for case- and source-specific characteristics of the measurements, and can be used to quantify uncertainties and the uncertainty budget, create digital effects tables, and overall validate measurements.

At this time, there are three individual tools:

1. obsarray
2. punpy
3. comet_maths

but more modules are planned to be developed and included in the future. For more detail, refer to the Tools Section.

2. 🗃️ Characterise the data/measurements that require the uncertainty propagation.

The main purpose of these tools, is to propagate uncertainties. To do that, you must have an overall understanding of the type of data/measurements you are working with.

For a general approach on determining an uncertainty budget, we refer to the 5-step QA4EO approach. See this page in our theory section, or the QA4EO process document.

To help you identify all the relevant information from your dataset, we have compiled a list of relevant questions and tips.

🗸 General

❔ What kind of data do you have?
❔ Does it require any pre-processing or filtering?
❔ How many datapoints do you have? Is the data memory-heavy?

🗸 Quantifying uncertainties on input quantities

❔ Can you list all the input quantities of your measurements?
❔ Can you identify all the error sources?
There are three types of errors, each with their own characteristics:
1. Random
2. Systematic
3. Structured

❕ Typically, each of the input quantities will be affected by one or more error effect!

🗸 Defining measurement function

❔ What is the analytic expression (i.e. measurement function) of your data?
❔ Do you have a more complex processing chain using external software?
❔ Can your measurement function be written as a Python function that has the input quantities as arguments, and returns the measurand?

Read more about the importance and functionality of measurement functions in our page on propagating uncertainties through a measurement function.

🗸 Determining error correlation

Once you have identified the various errors and their types, that are present in your measurements, it’s important to consider how these values and errors correlate with one another.

As defined by this FIDUCEO article on “The origin of error correlation”,

Correlation is a statistical measure of how two, or more, variables vary together.

To learn more about error correlation structures and examples in the context of Earth Observations, refer our page on error correlation and how to store it.

3. 🧾 Identify similarities between your specific requirements and the available examples.

Look through the available examples and documentation.
Plan out how the toolkit can be applied to your specific case study.
❔ Which tools and in what order will you use?

4. 🖥️ Install the tools

All the available tools are available on GitHub and installable via pip:

- pip install comet_maths
- pip install punpy
- pip install obsarray

Installing punpy will automatically install comet-maths and obsarray.

5. ✔️ Perform the uncertainty estimation and interpret the results.

After defining a measurement function, installing and importing all the relevant packages and data, it’s time to benefit from the power of CoMet!

🗸 Method breakdown

A general overview of the various capabilities and methods are compiled bellow.

store uncertainty and error correlation information
1. machine readable digital effects tables
2. UNC specification
Propagate uncertainties
1. 🎲 Monte Carlo (MC)
2. ⚖️ Law of Propagation of Uncertainty (LPU)
Interpolate data & uncertainties
1. Linear
2. Quadratic
3. Cubic
4. Gaussian Process Regression (GPR)
5. Extrapolate data

Several of the methods listed above apply to more than one of the applications. For more information refer to examples.

6. 📈 Advanced use.

In this section, we have highlighted certain tips for advanced use of the toolkit.

🗸 Managing memory and runtime

Certain products may have large RAM requirements, and the MC approach that is often used in CoMet can increase the RAM and runtime requirements by one or more orders of magnitude.

There are ways to manage memory and runtime, as described in the punpy documentation.

For example, often storing the error correlation between all the measurements along all dimensions in a dataset is often prohibitively memory intensive. Instead, it is usually possible to store the error correlation separately between different dimensions.

E.g. the HYPERNETS L2A surface reflectance data, has a wavelength and series dimension for which the error correlation are stored separately.
When propagating this information, using error correlation dictionaries can be useful, (see e.g. the end of this jupyter notebook example).

comet_maths

Tue, 19 Mar 2024 00:00:00 +0000

❔ What is comet_maths?

comet_maths is a python module with useful mathematical algorithms (including interpolation with uncertainties) for general use as well as for use in the other tools in the CoMet toolkit.

📍 Where can comet_maths be found?

The comet_maths documentation is available here. There are also jupyter notebooks available in the example section.

📋 What can comet_maths tool be used for?

There are quite a range of different functionalities within comet_maths. There are currently three submodules. One for linear algebra (mainly used for matrix operations in both obsarray and punpy), one for random generators (mainly used for sample generation in punpy) and one for interpolation (for general use).

✔️ Interpolation using comet_maths

The interpolation submodules focuses on two aspects. First, it aims to provide interpolation uncertainties that are as realistic as possible, and include both a contribution from the uncertainty on the input data point, as well as a contribution from the uncertainty in the model used for interpolation. Second, the interpolation module has functionality to interpolate between some low-resolution data points following a high resolution example. The example spectrum gets scaled in order to go through the low-resolution data points to form a sensible interpolation. For more info, see the comet_maths documentation.

Examples

Tue, 19 Mar 2024 00:00:00 +0000

Below, we have compiled a list of relevant examples and linked their corresponding jupyter notebooks with detailed commentary. All the available tools can be used on their own or in conjunction with other modules. For additional information regarding individual packages, refer to the signposted sources throughout the website.

📦 Punpy as a Standalone Package

🗸 General use cases

An example showcasing the capabilitites of the punpy package can be found here.

This jupyter notebook covers the following concepts:

Calibration of L0 data to L1
Propagation of various types of uncertainties: - uncorrelated (random) uncertainties - fully correlated (systematic) uncertainties - uncertainties associated with structured errors
Correlation along one, two, or more dimensions of a variable
Multidimensional input quantities, where a certain correlation structure is known along one dimension, while the other dimensions are random or systematic.

🗸 Punpy vs. NIST

We also have compiled some validation examples here, where the punpy results are compared against the NIST uncertainty machine.

Here, we have replicated the following examples available on the NIST uncertainty machine user manual.

End-gauge calibration
Dynamic viscosity
Resistance
Stefan-Boltzmann constant
Voltage reflection coefficient

❕ All the obtained results are fully consistent with the results of the NIST uncertainty machine.

📋 Digital Effects Tables (DET)

🗸 Defining DET

A notebook containing examples that define digital effects tables is available here.

It covers the following concepts:

How obsarray can be used as a templater for efficiently making xarray datasets (both with and without uncertainties)?
How, using obsarray’s special variable types (uncertainties and flags), datasets including detailed uncertainty and covariance information as well as quality flags can be created?
An example for a digital effects table quantifying the uncertainties and error-correlation of the gas temperature, pressure, and the number of moles. Here, the uncertainties can be efficiently and easily propagated through a measurement function using punpy (learn more).

🗸 Utilising obsarray & punpy

An example showcasing the application of obsarray can be found here.
An example of using punpy with digital effects tables created with obsarray is explained here.

This notebook outlines how digital effects tables that are created with obsarray, can be propagated through a measurement function using punpy.

At first, we calculate the uncertainties in a volume of gas, using the ideal gas law and a digital effects table.
Then we quantify the uncertainties and error-correlation of the gas temperature, pressure and amount of substance.

☄️ Comet_maths interpolation

🗸 How to interpolate data with uncertainties?

A jupyter notebook for interpolation with uncertainties can be found here.

This example covers the following concepts:

Interpolation
1. Linear
2. Quadratic
3. Cubic
Unknown input uncertainties (e.g. model uncertainties)
Known measurment uncertainties
Monte Carlo uncertainty propagation
Extrapolation
1D interpolation along high-resolution example

☄️ Curepy retrieval

🗸 How to obtain uncertainties for inverse problems?

A jupyter notebook for retrievals with uncertainties can be found here.

This example allows you to get familiar with the curepy tool, and covers the following concepts:

Propagate uncertainties through a basic inverse problem (harmonisation of two sensors).
Pass all relevant inputs to curepy
1. Measurements
2. Measurement function
3. Prior
4. Ancillary params
Run curepy retrieval to obtain state vector with uncertainties and error correlation
1. MCMC
2. Optimal Estimation

🗂️ Project specific examples

In this section, we have compiled a list of external projects and examples that have utilised the CoMet Toolkit.

🗸 HYPERNETS example

🛰️ LANDHYPERNET flags and uncertainty propagation (through band integration over S2 SRF) is available here.

obsarray

Tue, 19 Mar 2024 00:00:00 +0000

❔ What is obsarray?

obsarray is an extension to xarray for

defining
storing
interfacing

with uncertainty and measurement error-covariance information in NetCDF files using standardised metadata.

These datasets that include standardised uncertainty and error-covariance information in so-called ‘digital effects tables’.

📍 Where can obsarray be found?

The obsarray documentation is available here.
There are also jupyter notebooks available in the example section.

📋 What can obsarray tool be used for?

Using methods defined by the CoMet UNC Specification (uncertainty metadata naming conventions), obsarray enables users to parameterise their error-covariance information by storing it as attributes to uncertainty variables - creating ‘digital effects tables’.

One important aim of the CoMet Toolkit is to abstract away the complexity of dealing with error-covariances. Using measurement datasets defined in this way using obsarray, you can for example:

read/write datasets in a way that error-correlation information is preserved.
propagate dataset uncertainty using punpy – which can directly use the ‘digital effects tables’, so that users typically never have to interact with it.

punpy

Tue, 19 Mar 2024 00:00:00 +0000

❔ What is punpy?

punpy is a tool that stands for “Propagation of UNcertainties in Python”.

It propagates uncertainties on input quantities through any python function, evaluating the uncertainty on the output. These input data uncertainties can be defined using obsarray.

📍 Where can punpy be found?

The punpy documentation is available here, including some examples for standalone punpy and for using punpy with the digital effects tables.
There are also jupyter notebooks available in the example section.

📋 What can punpy tool be used for?

When data is processed through a processing chain (or through a measurement function in metrological terms), the uncertainties on the input quantities need to be propagated to the ouputs (the measurand). This uncertainty propagation needs to take into account error-correlation information. Standard metrological (science of measurement) methods from the Guide to the expression of Uncertainty in Measurement (GUM) can be used to propagate the uncertainties from the input quantities to uncertainties on the measurand (the processed data).

punpy aims to make this simple for users. It allows users to propagate uncertainties through any given measurement function, using either:

the Monte Carlo (MC) method
the law of propagation of uncertainty.

In this way, dataset uncertainties can be propagated through any measurement function that can be written as a python function – including simple analytical measurement functions, as well as full numerical processing chains (which might e.g. include external radiative transfer simulations), as long as these can be wrapped inside a python function.

Both methods have been validated against analytical calculations as well as other tools such as the NIST uncertainty machine.

✔️ punpy together with obsarray

punpy can be used as a standalone tool, or in combination with obsarray digital effects tables. When using punpy as a standalone tool, the input quantities and their uncertainties are manually specified.

Alternatively, digital effects tables defined with obsarray can be used. punpy and obsarray have been designed to interface with each other. All the uncertainty information in the obsarray products can be automatically parsed and passed to punpy.

A typical approach would be to separately propagate the random uncertainties (potentially multiple components combined), systematic uncertainties and structured uncertainties, and return them as an obsarray dataset that contains the measurand, the uncertainties and the covariance information of the measurand.

By combining these tools, handling uncertainties and covariance information has become as straightforward as possible, without losing flexibility.

Welcome to the CoMet Toolkit!

Tue, 19 Mar 2024 00:00:00 +0000

❔ What is the CoMet Toolkit?

The Community Metrology (CoMet) Toolkit is a set of open-source software tools that can handle, process, and store measurement data uncertainties and error-correlation information.

The main feature of this toolkit lies in its abilities to deal with the complexities of combining individual uncertainties from various sources, propagating these through any Python measurement function, and quantify and store uncertainty and error correlation information on the outputs.

This is done in a way that allows the user to use quality assured code, while most of the complexities are handled behind the scenes. This simplifies dealing with uncertainties for experienced and less experienced users alike.

💡 Why is the CoMet Toolkit relevant?

To ensure credible and reliable interpretation of data, the associated uncertainty information ought to be provided. Oftentimes it is made up of a multitude of sources combined through the processing chain. Each source affects the final product at varying levels.

When multiple measurements with uncertainties are combined throughout the processing chain (e.g. performing temporal or spatial averages or integrals, or when fitting a model to the data), it is also critical to take into account the error-correlation information.

Depending on the error-correlation, the output uncertainty will be different (e.g., random uncertainties are reduced by averaging, but systematic uncertainties are not). To get a correct uncertainty on the final measurand from the combined measurements, the error correlation thus needs to be taken into account for each relevant dimension.

CoMet Toolkit accounts for case- and source-specific characteristics of the measurement uncertainties. It can handle:

any measurement function that can be written as a Python function
data of any dimension (float/1D/2D/3D/…)
data with multiple sources of uncertainties
a wide range of different error correlation structures
different probability distribution functions
many more

📋 What can the CoMet Toolkit be used for?

The CoMet toolkit can be used to:

define measurement functions in Python
propagate uncertainties
create digital effects table (DTE)
Automatically parse and propagate uncertainties in DTE
propagating uncertainties through temporal or spatial averaging
combining random and systemic uncertainties
handling random and systemic uncertainties separately
calculate the uncertainty budget
many more

CoMet was designed to fulfil the Quality Assurance framework for Earth Observation (QA4EO) requirements. According to these guidelines, all Satellite Earth Observations (EO) and in-situ measurements require their corresponding uncertainty information. Although the CoMet toolkit was designed with the requirements of the EO community in mind, it can be applied to any field that requires measurement uncertainty propagation.

📍 Where can CoMet Toolkit be accessed?

The CoMet Toolkit is available on GitHub with packages installable via pip from the Python Package Index. Examples demonstrating the capabilities of this toolkit are available here.

👋 Acknowledgements & Citations

Developed by: National Physical Laboratory (NPL)

Funded by:

IDEAS-QA4EO: Instrument Data quality Evaluation and Assessment Service - Quality Assurance for Earth Observation (IDEAS-QA4EO) contract funded by ESA-ESRIN (n. 4000128960/19/I-NS)
NMS: The UK’s Department for Business, Energy and Industrial Strategy’s (BEIS) National Measurement System (NMS) programme

Citation: De Vis, P. & Hunt, S. E. CoMet Toolkit. [online] National Physical Laboratory. Available at: (https://www.comet-toolkit.org)

Privacy

Fri, 01 Dec 2023 00:00:00 +0000

Add your company privacy policy here…

Terms of Service

Fri, 01 Dec 2023 00:00:00 +0000

Add your company legal terms here…

HYPERNETS Science Conference

Mon, 07 Nov 2022 00:00:00 +0000

The HYPERNETS project has developed a new hyperspectral radiometer integrated in automated networks of water and land bidirectional reflectance measurements for satellite validation. A detailed uncertainty budget was measured in the lab for these instruments, and these uncertainties are propagated from product to product using the CoMet toolkit within the hypernets_processor. All HYPERNETS products are provided as obsarray-compatible digital effects tables. For the HYPERNETS vicarious calibration study, the uncertainties in these files are also propagated to TOA using punpy.

❔ What is HYPERNETS?

The HYPERNETS project has developed a new hyperspectral radiometer integrated in automated networks of water and land bidirectional reflectance measurements for satellite validation.

❕ How is the CoMet Toolkit contributing to HYPERNETS?

A detailed uncertainty budget was measured in the lab for the HYPERNETS instruments. These uncertainties were then propagated from product to product using the CoMet Toolkit within the hypernets_processor.

All HYPERNETS products are provided as obsarray-compatible digital effects tables. For the HYPERNETS vicarious calibration study, the uncertainties in these files are also propagated to TOA using punpy.

UK EO conference

Tue, 06 Sep 2022 00:00:00 +0000

The UK National Earth Observation Conference 2022 – the premier annual event for the Earth Observation, Remote Sensing and Photogrammetry community took place on the 6-8th September 2022, providing a great opportunity for people to meet, network and learn about the latest developments in scientific research and space-enabled technology. The CoMet toolkit was presented in a session on “Python for Earth Observation and Climate Sciences”, which was an excellent fit. There was significant interest in our toolkit, and it was also very interesting to see all the other work people are doing in python for EO.

Living Planet Symposium

Thu, 12 May 2022 00:00:00 +0000

ESA’s Living Planet Symposium (LPS) is amongst the biggest Earth observation (EO) conferences in the world. There has been a growing interest in the EO comunity towards using approached from metrology (the measurement of science) to better quantify the uncertainties and error-correlation information on satellite and in-situ EO data. Having accurate uncertainties is paramount for making these data reliable, interpretable and actionable, which is key in understanding and reducing climate change. The CoMet poster being presented at LPS will provide a first introduction to our new toolkit, which aims to enable easier handling of covariance information. The 2022 LPS will be held at the World Conference Center, in Bonn, Germany on 23-27 May 2022. It promises to be bigger and wider ranging than ever before. We look forward to discussing our tools with such an extensive audience.

CoMet is up and running!

Wed, 11 May 2022 00:00:00 +0000

After two years of development of the tools, we are very excited we can now make our tools available as open-source on github and the comet-toolkit webpage. This is an important milestone for CoMet, and will allow the tools to be used by the wider scientific community.

VH-RODA workshop

Wed, 11 May 2022 00:00:00 +0000

The Very High-resolution Radar & Optical Data Assessment (VH-RODA) will provide an open forum (for the new space, commercial and institutional space sectors) for presenting and discussing about the current status and future developments related to Earth Observation (EO) data quality, calibration and validation of space-borne very high-resolution SAR and Optical sensors and data products, with a dedicated focus on commercial EO data providers and related Cal/Val activities, synergies between optical and SAR communities, presentation of standards and best practices for data quality. Uncertainties are of paramount importance to this field, and the CoMet toolkit could be a very useful tool for this community.

Mon, 01 Jan 0001 00:00:00 +0000

Meet the Team

Mon, 01 Jan 0001 00:00:00 +0000