Theory | CoMet Toolkit

Introduction to uncertainties

Sun, 15 Jun 2025 00:00:00 +0000

Navigating the world of uncertainties can be quite tricky, as often terms such as errors and uncertainties are used interchangeably, even though they do not describe the same thing.

In this brief introduction to uncertainties and their propagation, we will highlight the most important concepts and signpost you to useful resources.

By providing learning materials covering relevant topics at varying levels of depth, we want to ensure that you can fully understand and therefore implement the capabilities of this toolkit.

❔ Why do we care about uncertainties?

Uncertainties are a vital aspect of the measurement science, as they provide credibility and trust in measured data.

To ensure that everyone is on the same page, a great place to start is this Measurement Good Practice Guide put together by the National Physical Labpratory (NPL).

It covers all the relevant terminology, best practises, and examples for understanding the uncertainty of measurements.

As mentioned at the beginning of this article, we can confirm that uncertainties and errors are, in fact, not the same thing:

Uncertainty is a quantification of the doubt about the measurement result.

Error is the difference between the measured value and the “true value” of the thing being measured.

📜 Metrology & the Guide to Uncertainties in Measurements

Metrology, the science of measurement, is the discipline that maintains the SI and the associated system of measurement, ensuring measurements are stable over time and measurement standards equivalent worldwide. These properties are supported by the principles of metrological traceability: uncertainty analysis and comparison.

The Guide to the expression of Uncertainty in Measurement or GUM for short, is the ultimate resource to refer to, when you are evaluating measurement uncertainties.

All the official documentation of GUM as well as a guide to the International Vocabulary of Metrology (VIM) are available here.

🏆 Best practice framework for EO

To ensure credible and reliable interpretation of environmental observations from satellites and in-situ measurements, Committee on Earth Observation Satellites (CEOS) has established and endorced the Quality Assurance framework for Earth Observation (QA4EO).

This framework requires that associated uncertainty information is provided for all measurements. Additionally, it highlights the importance of understanding the error-covariances in the data.

Approaches defined within QA4EO enable the EO community to develop quantitative characterisation of uncertainty in EO data.

QA4EO developed a 5-steps approach to do an uncertainty budget. These 5 steps are described on this CoMet page and in the QA4EO process document.

However, practically implementing these methods is not trivial and can be time consuming.

As a way to facilitate this, the CoMet Toolkit was developed as a means to store and propagate uncertainty and error-correlation information.

✔️Other useful resources for Uncertainty in Earth Observations (EO)

A great resource that describes all the various aspects of uncertainty propagation for satellite EO data is FIDUCEO.

This project provides a comprehensive guide to understanding and implementing:

QA4EO uncertainty budget

Sat, 14 Jun 2025 00:00:00 +0000

QA4EO is a best practice framework, based on the principles of metrology, to establish The Global Earth Observation System of Systems — based on coordinated and harmonised processes and activities that enable interoperability. QA4EO was established and endorsed by the Committee on Earth Observation Satellites (CEOS). QA4EO ensures credible and reliable interpretation of environmental observations from satellites and in-situ measurements by requiring that associated uncertainty information is provided. A set of guidelines was developed on how to apply the QA4EO principles to generate metrologically-rigorous data products and to perform uncertainty analysis.

There are five steps towards a metrological uncertainty analysis. These steps are described in detail in the QA4EO Process document which provides Step-by-step guidance on implementing a metrological approach to uncertainty analysis. The 5 steps are: • Step 1: Define the measurand and measurement model • Step 2: Establish the traceability with a diagram (see example at the top of this page) • Step 3: Evaluate each source of uncertainty and fill out an effects table • Step 4: Calculate the data product and uncertainties • Step 5: Record information about the uncertainty analysis for long term data preservation purposes (implicit above) and summarise for today’s users

The CoMet toolkit is particularly useful for steps 4, (see our page on propagating uncertainties through a measurement function) and step 5 (see our page on error correlation and how to store it).

As discussed in the QA4EO Process document, Effects Tables (step 3) are a useful way to record and report the information required to fully parameterise an error-correlation effect. However, to use this information in a processing chain, it must be provided digitally. The CoMet Toolkit defines a mechanism for this, with a metadata standard (see UNC specification) that enables the creation of Digital Effects Tables stored in NetCDF files. In this way, uncertainty information can be written, read,

Punpy interfaces with obsarray to make uncertainty propagation as efficient and easy to use as possible. The digital effects tables produced with obsarray can be propagated through measurement functions using punpy, without the need for providing additional information. The data has thus been encoded with all relevant error-covariance information, though users do not need to interact with it. Together these tools enable both the experienced and inexperienced user to efficiently include uncertainties throughout their data processing, and thus make their datasets more reliable and interpretable. Optional keywords provide the user with the flexibility to deal with all kinds of complex use cases. For further info, we refer to the punpy and obsarray documentation.

Propagating uncertainties through a processing chain

Fri, 13 Jun 2025 00:00:00 +0000

When determining the value of a given measurand (i.e. the quantity intended to be measured), some processing typically needs to be performed to calculate the measurand based on some input quantities. In some cases, this processing is relatively simple, and can be done with a simple analytical function. In other cases, a full processing chain is needed, which can contain a whole range of computational processing steps. In metrology (GUM), this relationship between the measurand and the input quantities is refered to as a measurement model or a measurement function. For some further information, we refer to this FIDUCEO tutorial.

In order to calculate the uncertainty on the measurand, one needs to propagate uncertainties on the input quantities through the measurement model. There are various methods that can be used for this. Two commonly used ones are the Monte Carlo (MC) method and the Law of Propagation of Uncertainties. For further detail on these methods, see the GUM and its supplements, the QA4EO metrology document, or the punpy ATBD.

The CoMet toolkit implements both the Monte Carlo and the Law of Propagation of Uncertainty methodologies for uncertainty propagation. The CoMet toolkit interface to both of these methodlogies is exactly the same, so the user can easily switch between them and compare the results. There are two ways the input data and their uncertainties can be ingested in the CoMet toolkit. Either they can be ingested manually as numpy arrays, or they can be ingested using `Digital Effects Tables’, defined with obsarray.

In order to propagate uncertainties using the CoMet toolkit, one needs to be able to write the measurement function as a Python function which takes the input quantities as arguments and returns the measurand. Inside this measurement function, there can be a varying range of complexity. It could be a simple analytical function, or a full processing chain, including calls to other external software. The CoMet toolkit treats this measurement function somewhat as a blackbox and simply modifies the inputs, and analyses the outputs.

Error correlation and how to store it

Thu, 12 Jun 2025 00:00:00 +0000

See this FIDUCEO tutorial for further information on how to evaluate error-correlation information.