Metrics¶

Bases: SicOnlyMetricMixin, BaseDailyMetric

Euclidean distance (in pixels) between the predicted and target centroids.

The centroid of a (batch, time) frame is its value-weighted center of mass over the spatial dimensions, summed across channels. Frames whose target has (near-)zero total mass have an undefined centroid and are excluded from the average.

Bases: BaseIceAreaMetric

Distance-averaged Integrated Ice Edge Error (DIIEE), in km, per lead time.

The total misclassified-ice area (overestimation + underestimation, i.e. IIEE, see IntegratedIceEdgeErrorPerForecastDay) is normalised by the combined length of the predicted and true ice edges, giving an average displacement distance in km: roughly, how far the ice edge would need to move to reconcile the two fields.

DIIEE = 2 * (over_area + under_area) / (pred_edge_length + true_edge_length)

Edge length is approximated on the raster grid as (edge cell count) * pixel_size; this is coarser than a true vector polygon perimeter, and needs no vector geometry, though an optional land_mask can be supplied to exclude land/ice boundaries from the edge count (see __init__). Lead times where both fields are entirely ice or entirely ice-free (combined edge length zero) are undefined and reported as NaN, rather than the -9999.99 sentinel used upstream, to compose correctly with tensor reductions (e.g. nanmean).

Bases: SicOnlyMetricMixin, LandMaskMixin, AccumulatorMixin, Metric

FractionalSkill Score (FSS) of the sea-ice edge, for use at multiple lead times.

Computes the FSS of the sea-ice edge at a fixed neighbourhood size, following Roberts and Lean (2008) and its application to sea-ice edge position by Melsom et al. (2019, https://doi.org/10.5194/os-15-615-2019). Adapted from the effectiveres_icenetv2_FSS notebook's step-by-step computation.

Each field is first reduced to a binary ice-edge map (cells that are ice but border a non-ice cell). The local fraction of edge cells is then computed within a fixed neighbourhood_size x neighbourhood_size window around every cell. FSS compares the mean squared error (MSE) between the predicted and true fraction fields to a reference (worst-case) MSE:

FSS = 1 - MSE / MSE_ref

FSS is 1 for a perfect match and 0 (or below) for no better than the worst-case reference. To assess effective resolution, instantiate this metric once per neighbourhood size of interest and compare where the resulting curve crosses 0.5.

Bases: SicOnlyMetricMixin, LandMaskMixin, AccumulatorMixin, Metric

Binary accuracy metric for use at multiple leadtimes.

Adapted from the IceNet implementation at: - https://github.com/icenet-ai/icenet-notebooks/blob/main/pytorch/1_icenet_forecast_unet.ipynb

Bases: MeanIceAreaMetric

Integrated Ice Edge Error (IIEE) metric (in km^2) for use at multiple lead times.

IIEE is the area of the symmetric difference between the predicted and true ice extent: the total area where the two disagree on ice presence, following Goessling et al. (2016, https://doi.org/10.1002/2015GL067232). Sea ice presence is defined by having a probability greater than the threshold value.

Unlike SeaIceExtentErrorPerForecastDay, which is a signed difference of extents and can cancel out over- and under-estimation, IIEE always accumulates disagreement and is therefore always >= |SIEError|.

Bases: BaseDailyMetric

Mean Absolute Error per forecast lead time.

Bases: BaseDailyMetric

Root Mean Squared Error per forecast lead time.

Bases: MeanIceAreaMetric

Sea Ice Extent error (SIEError) metric (in km^2) for use at multiple lead times.

The SIE error is calculated as the signed difference between the predicted and true sea ice extent for each forecast day. Sea ice presence is defined by having a concentration greater than the threshold value.

Bases: BaseDailyMetric

Land-masked spatial-mean ground-truth value per forecast lead time.

Paired with SpatialMeanPredictionPerForecastDay to trace spatial-mean prediction versus ground truth across forecast steps, e.g. to spot a systematic bias or a collapse toward a constant value that per-pixel error metrics would not by themselves reveal.

Bases: BaseDailyMetric

Land-masked spatial-mean prediction value per forecast lead time.

Paired with SpatialMeanGroundTruthPerForecastDay to trace spatial-mean prediction versus ground truth across forecast steps.

Bases: BaseDailyMetric

Structural Similarity Index (SSIM) per forecast lead time.

Follows the standard SSIM definition from Wang et al. (2004), "Image quality assessment: from error visibility to structural similarity", IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612. Adapted (channels-first, PyTorch) from the Gaussian-filtered dm_pix/cloudcasting implementation at: https://github.com/openclimatefix/cloudcasting/blob/main/src/cloudcasting/metrics.py

Each field is locally compared to the other within a Gaussian-weighted filter_size x filter_size window around every cell, following the standard SSIM formulation. Note that the true SSIM is only defined on grayscale; this implementation does not perform any colourspace transform, so multi-channel inputs are averaged as if each channel were an independent greyscale image.