autoemulate.calibration.history_matching

autoemulate.calibration.history_matching#

class HistoryMatching(observations, threshold=3.0, model_discrepancy=0.0, rank=1, device=None)[source]#

Bases: TorchDeviceMixin

History Matching class for model calibration.

History matching is a model calibration method, which uses observed data to rule out implausible parameter values. The implausibility metric is:

\[I_i(\bar{x_0}) = \frac{|z_i - \mathbb{E}(f_i(\bar{x_0}))|} {\sqrt{\text{Var}[z_i - \mathbb{E}(f_i(\bar{x_0}))]}}\]

Queried parameters above a given implausibility threshold are ruled out (RO) whereas all other parameters are marked as not ruled out yet (NROY).

get_nroy(implausibility, x=None)[source]#

Get indices of NROY points from implausibility scores.

If x is provided, returns parameter values at NROY indices.

Parameters:

  • implausibility (TensorLike) – Tensor of implausibility scores for tested input parameters.

  • x (Tensorlike | None) – Optional tensor of scored input parameters.

Returns:

Indices of NROY points or x parameters at NROY indices.

Return type:

TensorLike

get_ro(implausibility, x=None)[source]#

Get indices of RO points from implausibility scores.

If x is provided, returns parameter values at RO indices.

Parameters:

  • implausibility (TensorLike) – Tensor of implausibility scores for tested input parameters.

  • x (Tensorlike | None) – Optional tensor of scored iput parameters.

Returns:

Indices of RO points or x parameters at RO indices.

Return type:

TensorLike

calculate_implausibility(pred_means, pred_vars)[source]#

Calculate implausibility scores.

Parameters:

  • pred_means (TensorLike) – Tensor of prediction means [n_samples, n_outputs]

  • pred_vars (TensorLike) – Tensor of prediction variances [n_samples, n_outputs].

Returns:

Tensor of implausibility scores.

Return type:

TensorLike

static generate_param_bounds(nroy_x, buffer_ratio=0.05, param_names=None, min_samples=1)[source]#

Generate lower/upper parameter bounds as min/max of NROY samples.

Parameters:

  • nroy_x (TensorLike) – A tensor of NROY parameter samples [n_samples, n_inputs].

  • buffer_ratio (float) – A scaling factor used to expand the bounds of the (NROY) parameter space. It is applied as a ratio of the range (max_val - min_val) of each input parameter to create a buffer around the NROY minimum and maximum values.

  • param_names (list[str] | None) – Optional list of parameter names. If None, uses default [“x1”, …, “xn”].

  • min_samples (int) – Minimum number of samples needed to generate new bounds.

Returns:

The generated [lower, upper] parameter bounds. Returns None if there are not enough samples to generate bounds from.

Return type:

dict[str, [float, float]] | None

class HistoryMatchingWorkflow(simulator, result, observations, threshold=3.0, model_discrepancy=0.0, rank=1, train_x=None, train_y=None, calibration_params=None, device=None, random_seed=None, log_level='progress_bar')[source]#

Bases: HistoryMatching

History Matching Workflow class.

Run history matching workflow: - sample parameter values to test from the current NROY parameter space - use emulator to rule out implausible parameter samples - run simulations for a subset of the NROY parameters - refit the emulator using the simulated data

cloud_sample(n, scaling_factor=0.1)[source]#

Generate n additional parameter samples using cloud sampling.

Handles fixed parameters (min == max) by not sampling those. The constant values are inserted at the correct indices in the sampled tensor.

Parameters:

  • n (int) – The number of samples to generate.

  • scaling_factor (float) – The standard deviation of the Gaussian to sample from in cloud sampling is set to: parameter range * scaling_factor.

Returns:

A tensor of sampled (and potentially constant) parameters [n, in_dim].

Return type:

TensorLike

generate_samples(n, scaling_factor=0.1)[source]#

Generate parameter samples and evaluate implausibility.

Draw n samples either from the simulator min/max parameter bounds or using cloud sampling centered at NROY samples. Evaluate sample implausability using emulator predictions.

Parameters:

  • n (int) – The number of parameter samples to generate.

  • scaling_factor (float) – The standard deviation of the Gaussian used in cloud sampling is set to: parameter range * scaling_factor.

Returns:

A tensor of tested input parameters and their implausability scores.

Return type:

tuple[TensorLike, TensorLike]

sample_tensor(n, x)[source]#

Randomly sample n rows from x.

Parameters:

  • n (int) – The number of samples to draw.

  • x (TensorLike) – The tensor to sample from.

Returns:

A tensor of samples with n rows.

Return type:

TensorLike

simulate(x)[source]#

Simulate x parameter inputs and filter out failed simulations.

Parameters:

x (TensorLike) – A tensor of parameters to simulate [n_samples, n_inputs].

Returns:

Tensors of succesfully simulated input parameters and predictions.

Return type:

tuple[TensorLike, TensorLike]

refit_emulator(x, y)[source]#

Refit the emulator on the provided data.

Parameters:

  • x (TensorLike) – Tensor of input data to refit the emulator on.

  • y (TensorLike) – Tensor of output data to refit the emulator on.

run(n_simulations=100, n_test_samples=10000, max_retries=3, scaling_factor=0.1, refit_emulator=True, refit_on_all_data=True)[source]#

Run a wave of the history matching workflow.

Parameters:

  • n_simulations (int) – Number of simulations to run.

  • n_test_samples (int) – Number of input parameters to test for implausibility with the emulator. Parameters to simulate are sampled from this NROY subset.

  • max_retries (int) –

    Maximum number of times to try to generate n_simulations NROY parameters. That is the maximum number of times to repeat the following steps:

    • draw n_test_samples parameters (use cloud sampling if possible)

    • use emulator to make predictions for those parameters

    • score implausability of parameters given predictions

    • identify NROY parameters within this set

  • scaling_factor (float) – The standard deviation of the Gaussian to sample from in cloud sampling is set to: parameter range * scaling_factor.

  • refit_emulator (bool) – Whether to refit the emulator at the end of the run. Defaults to True.

  • refit_on_all_data (bool) – Whether to refit the emulator on all available data or just the data available from the most recent simulation run. Defaults to True.

Returns:

A tensor of tested input parameters and their implausibility scores from which simulation samples were then selected.

Return type:

tuple[TensorLike, TensorLike]

run_waves(n_waves=5, frac_nroy_stop=0.9, n_simulations=100, n_test_samples=10000, max_retries=3, scaling_factor=0.1, refit_emulator_on_last_wave=True, refit_on_all_data=True)[source]#

Run multiple waves of the history matching workflow.

Refits the emulator after each wave (except the last), using all available data.

Parameters:

  • n_waves (int) – The maximum number of waves to run.

  • frac_nroy_stop (float) – Fraction of NROY samples to stop at. If less than this fraction of NROY samples is reached, the workflow stops.

  • n_simulations (int) – Number of simulations to run in each wave.

  • n_test_samples (int) – Number of input parameters to test for implausibility with the emulator. Parameters to simulate are sampled from this NROY subset.

  • max_retries (int) –

    Maximum number of times to try to generate n_simulations NROY parameters. That is the maximum number of times to repeat the following steps:

    • draw n_test_samples parameters (use cloud sampling if possible)

    • use emulator to make predictions for those parameters

    • score implausibility of parameters given predictions

    • identify NROY parameters within this set

  • scaling_factor (float) – The standard deviation of the Gaussian to sample from in cloud sampling is set to: parameter range * scaling_factor.

  • refit_emulator_on_last_wave (bool) – Whether to refit the emulator after the last wave. Defaults to True.

  • refit_on_all_data (bool) – Whether to refit the emulator on all available data after each wave or just the data from the most recent simulation run. Defaults to True.

Returns:

A tensor of tested input parameters and their implausibility scores.

Return type:

tuple[TensorLike, TensorLike]

plot_run(test_parameters, impl_scores, set_simulator_axis_limits=True, ref_val=None, title='History Matching Results', fname=None)[source]#

Plot results of a single history matching run.

Parameters:

  • test_parameters (TensorLike) – A tensor of tested input parameters [n_samples, n_inputs].

  • impl_scores (TensorLike) – A tensor of implausibility scores for the tested input parameters.

  • set_simulator_axis_limits (bool) – Whether to keep the simulator parameter ranges as axis limits.

  • ref_val (dict[str, float] | None) – Optional dictionary of true parameter values to mark on the plots.

  • title (str) – Title for the plot.

  • fname (str | None) – Optional filename to save the plot to. If None, the plot is displayed.

Returns:

If fname is provided, saves the plot to the file and returns None. If fname is None, displays the plot and returns the plot figure.

Return type:

None | Figure

plot_wave(wave, set_simulator_axis_limits=True, ref_val=None, fname=None)[source]#

Plot results for a specific wave.

Parameters:

  • wave (int) – The wave number to plot (0-indexed).

  • set_simulator_axis_limits (bool) – Whether to keep the simulator parameter ranges as axis limits.

  • ref_val (dict[str, float] | None) – Optional dictionary of true parameter values to mark on the plots.

  • fname (str | None) – Optional filename to save the plot to. If None, the plot is displayed.

Returns:

If fname is provided, saves the plot to the file and returns None. If fname is None, displays the plot and returns the plot figure.

Return type:

None | Figure

get_wave_results(wave)[source]#

Get results for a specific wave.

Parameters:

wave (int) – The wave number to get results for (0-indexed).

Returns:

A tensor of tested input parameters and their implausibility scores.

Return type:

tuple[TensorLike, TensorLike]

plot_wave_evolution(param, ref_val=None, fname=None)[source]#

Plot evolution of parameter distributions across all waves.

Parameters:

  • param (str) – The parameter to plot the evolution for.

  • ref_val (dict[str, float] | None) – Optional dictionary of true parameter values to mark on the plots.

  • fname (str | None) – Optional filename to save the plot to. If None, the plot is displayed.

Returns:

If fname is provided, saves the plot to the file and returns None. If fname is None, displays the plot and returns the plot figure.

Return type:

None | Figure

Contents