autoemulate.core.model_selection#
- evaluate(y_pred, y_true, metric=<class 'torchmetrics.regression.r2.R2Score'>)[source]#
Evaluate Emulator prediction performance using a torchmetrics.Metric.
- cross_validate(cv, dataset, model, model_params, x_transforms=None, y_transforms=None, device='cpu', random_seed=None)[source]#
Cross validate model performance using the given cv strategy.
- Parameters:
cv (BaseCrossValidator) – Provides split method that returns train/val Dataset indices using the specified cross-validation strategy (e.g., KFold, LeaveOneOut).
dataset (Dataset) – The data to use for model training and validation.
model (Emulator) – An instance of an Emulator subclass.
model_params (ModelParams) – Model parameters to be used to construct model upon initialization. Passing an empty dictionary {} will use default parameters.
device (DeviceLike) – The device to use for model training and evaluation.
random_seed (int | None) – Optional random seed for reproducibility.
- Returns:
Contains r2 and rmse scores computed for each cross validation fold.
- Return type:
- bootstrap(model, x, y, n_bootstraps=100, device='cpu')[source]#
Get bootstrap estimates of R2 and RMSE.
- Parameters:
model (Emulator) – An instance of an Emulator subclass.
x (TensorLike) – Input features for the model.
y (TensorLike) – Target values corresponding to the input features.
n_bootstraps (int) – Number of bootstrap samples to generate. Defaults to 100.
device (str | torch.device) – The device to use for computations. Default is “cpu”.
- Returns:
((r2_mean, r2_std), (rmse_mean, rmse_std))
- Return type: