autoemulate.emulators.conditional_neural_process

autoemulate.emulators.conditional_neural_process#

class ConditionalNeuralProcess(hidden_dim=64, latent_dim=64, hidden_layers_enc=3, hidden_layers_dec=3, min_context_points=3, max_context_points=10, n_episode=32, max_epochs=100, lr=0.005, batch_size=16, activation=<class 'torch.nn.modules.activation.ReLU'>, optimizer=<class 'torch.optim.adamw.AdamW'>, normalize_y=True, device='cpu', random_state=None, attention=False)[source]#

Bases: RegressorMixin, BaseEstimator

Conditional Neural Process (CNP) Regressor.

This model integrates a meta-learning approach into a scikit-learn regressor. The fit() method accepts standard X, y inputs. A Conditional Neural Process (CNP) operates using context points and target points in episodes. For each episode, the model samples n_episode data points, a subset of which are designated as context points. All points from the episode, including the context points, are used as target points. Each episode thus represents a single sample in a meta-learning framework. During each epoch, we sample X.shape[0] times, ensuring that each data point is part of different contexts across episodes. The model can predict with a single input X, similar to other scikit-learn models. However, since the CNP requires context points for prediction, it stores the original X, y data as attributes, which are then used as context points for predicting new X data.

Parameters:

  • hidden_dim (int, default=64) – The number of hidden units in the neural network layers.

  • latent_dim (int, default=64) – The dimensionality of the latent space.

  • hidden_layers_enc (int, default=3) – The number of hidden layers in the encoder.

  • hidden_layers_dec (int, default=3) – The number of hidden layers in the decoder.

  • min_context_points (int, default=3) – The minimum number of context points to use during training.

  • max_context_points (int, default=10) – The maximum number of context points to use during training.

  • n_episode (int, default=32) – The number of episodes to sample during each epoch.

  • max_epochs (int, default=100) – The maximum number of epochs to train the model.

  • lr (float, default=0.01) – The learning rate for the optimizer.

  • batch_size (int, default=16) – The number of samples per batch.

  • activation (callable, default=torch.nn.ReLU) – The activation function to use in the neural network layers.

  • optimizer (callable, default=torch.optim.AdamW) – The optimizer to use for training the model.

  • device (str, default="cpu") – The device to use for training. Options are “cpu” or “cuda”.

  • random_state (int, default=None) – The seed used by the random number generator.

References

[1] Garnelo, M., Rosenbaum, D., Maddison, C., Ramalho, T., Saxton, D., Shanahan, M., Teh, Y.W., Rezende, D., & Eslami, S.M.A. (2018).

Conditional Neural Processes. In International Conference on Machine Learning (pp. 1704-1713). PMLR.

input_dim_#

The number of features in the input data.

Type:

int

output_dim_#

The number of targets in the output data.

Type:

int

model_#

The neural network model used for regression.

Type:

skorch.NeuralNetRegressor

X_train_#

The training input samples.

Type:

ndarray of shape (n_samples, n_features)

y_train_#

The target values (real numbers) in the training set.

Type:

ndarray of shape (n_samples,) or (n_samples, n_outputs)

fit(X, y)[source]#

Fit the model to the training data.

predict(X, return_std=False)[source]#

Predict using the trained model.

Examples

>>> import numpy as np
>>> from autoemulate.emulators.cnp import ConditionalNeuralProcess
>>> X = np.random.rand(100, 10)
>>> y = np.random.rand(100, 1)
>>> cnp = ConditionalNeuralProcess(hidden_dim=128, latent_dim=128, hidden_layers=4, min_context_points=3, max_context_points=10, n_episode=32, max_epochs=100, lr=0.01, batch_size=16, activation=torch.nn.ReLU, optimizer=torch.optim.AdamW, device="cpu", random_state=42)
>>> cnp.fit(X, y)
>>> y_pred = cnp.predict(X)
>>> y_pred.shape
(100, 1)
set_predict_request(*, return_std='$UNCHANGED$')#

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to predict.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

return_std (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for return_std parameter in predict.

Returns:

self – The updated object.

Return type:

object