from autoemulate.emulators import GaussianProcess
from autoemulate.core.compare import AutoEmulate
from autoemulate.simulations.projectile import Projectile
from autoemulate.calibration.history_matching import HistoryMatching
History Matching#
In this tutorial we demonstrate the use of History Matching to determine which points in the input space are plausible given a set of observations.
Performing History Matching requires:
a fit emulator that predicts uncertainty (e.g., Gaussian Process) and
an observation associated with the simulator output.
The emulator is used to efficiently estimate the simulator output, accounting for all uncertainties. The emulated output is then compared with the observation and parameter inputs that are unlikely to produce the observation can then be “ruled out” as implausible, reducing the input space.
1. Simulate data and train an emulator#
In this example we use the Projectile
simulator. It returns distance travelled by a projectile given a drag coefficient c
and initial velocity v0
.
simulator = Projectile(log_level="error")
x = simulator.sample_inputs(1000)
y = simulator.forward_batch(x)
Since we need an uncertainty aware emulator, we restrict AutoEmulate to only train a Gaussian Process.
ae = AutoEmulate(x, y, models=[GaussianProcess], log_level="error")
We can verify that the fitted emulator performs well on both the train and test data.
ae.summarise()
model_name | x_transforms | y_transforms | params | rmse_test | r2_test | r2_test_std | r2_train | r2_train_std | |
---|---|---|---|---|---|---|---|---|---|
0 | GaussianProcess | [StandardizeTransform()] | [StandardizeTransform()] | {'mean_module_fn': <function constant_mean at ... | 6.625579 | 0.999962 | 0.000024 | 0.999988 | 0.000002 |
model = ae.best_result().model
2. Calibrate#
To instantiate the HistoryMatching
object, we need an observed mean and, optionally, variance for each simulator output.
# observed data: (mean, variance)
hm = HistoryMatching(observations={"distance": (2500, 100)})
We also need predictions for a set of query points using the trained emulator. These must have uncertainty estimates.
x_new = simulator.sample_inputs(10)
pred = model.predict(x_new)
Primary use of the HistoryMatching
class is the calculate_implausibility
method, which returns the implausibility metric (a number of standard deviations between the emulator mean and the observation) for the queried points.
implausability = hm.calculate_implausibility(pred.mean, pred.variance)
implausability
tensor([[42.3653],
[ 9.5001],
[10.1569],
[57.3149],
[10.3270],
[ 5.5752],
[ 8.9227],
[10.3307],
[10.2595],
[ 8.9826]])
The get_nroy
method can be used to filter the queried points given the implausability scores and only retain those that have not been ruled out yet (NROY).
hm.get_nroy(implausability, x_new)
tensor([], size=(0, 2))