Machines

Recall from Getting Started that a machine binds a model (i.e., a choice of algorithm + hyperparameters) to data (see more at Constructing machines below). A machine is also the object storing learned parameters. Under the hood, calling fit! on a machine calls either MLJBase.fit or MLJBase.update, depending on the machine's internal state (as recorded in private fields old_model and old_rows). These lower-level fit and update methods, which are not ordinarily called directly by the user, dispatch on the model and a view of the data defined by the optional rows keyword argument of fit! (all rows by default).

Warm restarts

If a model update method has been implemented for the model, calls to fit! will avoid redundant calculations for certain kinds of model mutations. The main use-case is increasing an iteration parameter, such as the number of epochs in a neural network. To test if SomeIterativeModel supports this feature, check iteration_parameter(SomeIterativeModel) is different from nothing.

tree = (@load DecisionTreeClassifier pkg=DecisionTree verbosity=0)()
forest = EnsembleModel(model=tree, n=10);
X, y = @load_iris;
mach = machine(forest, X, y)
fit!(mach, verbosity=2);
Machine trained 1 time; caches data
  model: ProbabilisticEnsembleModel(model = DecisionTreeClassifier(max_depth = -1, …), …)
  args: 
    1:	Source @670 ⏎ `Table{AbstractVector{Continuous}}`
    2:	Source @376 ⏎ `AbstractVector{Multiclass{3}}`

Generally, changing a hyperparameter triggers retraining on calls to subsequent fit!:

julia> forest.bagging_fraction=0.5
0.5

julia> fit!(mach, verbosity=2);
[ Info: Updating machine(ProbabilisticEnsembleModel(model = DecisionTreeClassifier(max_depth = -1, …), …), …).
[ Info: Truncating existing ensemble.

However, for this iterative model, increasing the iteration parameter only adds models to the existing ensemble:

julia> forest.n=15
15

julia> fit!(mach, verbosity=2);
[ Info: Updating machine(ProbabilisticEnsembleModel(model = DecisionTreeClassifier(max_depth = -1, …), …), …).
[ Info: Building on existing ensemble of length 10
[ Info: One hash per new atom trained: 
#####

Call fit! again without making a change and no retraining occurs:

julia> fit!(mach);
[ Info: Not retraining machine(ProbabilisticEnsembleModel(model = DecisionTreeClassifier(max_depth = -1, …), …), …). Use `force=true` to force.

However, retraining can be forced:

julia> fit!(mach, force=true);
[ Info: Training machine(ProbabilisticEnsembleModel(model = DecisionTreeClassifier(max_depth = -1, …), …), …).

And is re-triggered if the view of the data changes:

julia> fit!(mach, rows=1:100);
[ Info: Training machine(ProbabilisticEnsembleModel(model = DecisionTreeClassifier(max_depth = -1, …), …), …).
julia> fit!(mach, rows=1:100);
[ Info: Not retraining machine(ProbabilisticEnsembleModel(model = DecisionTreeClassifier(max_depth = -1, …), …), …). Use `force=true` to force.

If an iterative model exposes its iteration parameter as a hyperparameter, and it implements the warm restart behavior above, then it can be wrapped in a "control strategy", like an early stopping criterion. See Controlling Iterative Models for details.

Inspecting machines

There are two principal methods for inspecting the outcomes of training in MLJ. To obtain a named-tuple describing the learned parameters (in a user-friendly way where possible) use fitted_params(mach). All other training-related outcomes are inspected with report(mach).

X, y = @load_iris
pca = (@load PCA verbosity=0)()
mach = machine(pca, X)
fit!(mach)
Machine trained 1 time; caches data
  model: PCA(maxoutdim = 0, …)
  args: 
    1:	Source @934 ⏎ `Table{AbstractVector{Continuous}}`
julia> fitted_params(mach)
(projection = [-0.3615896773814499 0.6565398832858295 0.5809972798276163; 0.08226888989221413 0.7297123713264987 -0.5964180879380993; -0.8565721052905274 -0.1757674034286529 -0.07252407548696002; -0.358843926248216 -0.07470647013503473 -0.5490609107266097],)

julia> report(mach)
(indim = 4,
 outdim = 3,
 tprincipalvar = 4.545608248041777,
 tresidualvar = 0.023683027126000233,
 tvar = 4.569291275167777,
 mean = [5.843333333333334, 3.0540000000000003, 3.758666666666667, 1.198666666666667],
 principalvars = [4.2248407683201075, 0.24224357162751498, 0.0785239080941545],)
MLJModelInterface.fitted_paramsFunction
fitted_params(model, fitresult) -> human_readable_fitresult # named_tuple

Models may overload fitted_params. The fallback returns (fitresult=fitresult,).

Other training-related outcomes should be returned in the report part of the tuple returned by fit.

fitted_params(mach)

Return the learned parameters for a machine mach that has been fit!, for example the coefficients in a linear model.

This is a named tuple and human-readable if possible.

If mach is a machine for a composite model, such as a model constructed using @pipeline, then the returned named tuple has the composite type's field names as keys. The corresponding value is the fitted parameters for the machine in the underlying learning network bound to that model. (If multiple machines share the same model, then the value is a vector.)

using MLJ
@load LogisticClassifier pkg=MLJLinearModels
X, y = @load_crabs;
pipe = @pipeline Standardizer LogisticClassifier
mach = machine(pipe, X, y) |> fit!

julia> fitted_params(mach).logistic_classifier
(classes = CategoricalArrays.CategoricalValue{String,UInt32}["B", "O"],
 coefs = Pair{Symbol,Float64}[:FL => 3.7095037897680405, :RW => 0.1135739140854546, :CL => -1.6036892745322038, :CW => -4.415667573486482, :BD => 3.238476051092471],
 intercept = 0.0883301599726305,)

Additional keys, machines and fitted_params_given_machine, give a list of all machines in the underlying network, and a dictionary of fitted parameters keyed on those machines.

```

MLJBase.reportFunction
report(mach)

Return the report for a machine mach that has been fit!, for example the coefficients in a linear model.

This is a named tuple and human-readable if possible.

If mach is a machine for a composite model, such as a model constructed using @pipeline, then the returned named tuple has the composite type's field names as keys. The corresponding value is the report for the machine in the underlying learning network bound to that model. (If multiple machines share the same model, then the value is a vector.)

using MLJ
@load LinearBinaryClassifier pkg=GLM
X, y = @load_crabs;
pipe = @pipeline Standardizer LinearBinaryClassifier
mach = machine(pipe, X, y) |> fit!

julia> report(mach).linear_binary_classifier
(deviance = 3.8893386087844543e-7,
 dof_residual = 195.0,
 stderror = [18954.83496713119, 6502.845740757159, 48484.240246060406, 34971.131004997274, 20654.82322484894, 2111.1294584763386],
 vcov = [3.592857686311793e8 9.122732393971942e6 … -8.454645589364915e7 5.38856837634321e6; 9.122732393971942e6 4.228700272808351e7 … -4.978433790526467e7 -8.442545425533723e6; … ; -8.454645589364915e7 -4.978433790526467e7 … 4.2662172244975924e8 2.1799125705781363e7; 5.38856837634321e6 -8.442545425533723e6 … 2.1799125705781363e7 4.456867590446599e6],)

Additional keys, machines and report_given_machine, give a list of all machines in the underlying network, and a dictionary of reports keyed on those machines.

```

report(fitresult::CompositeFitresult)

Return a tuple combining the report from fitresult.glb (a Node report) with the additions coming from nodes declared as report nodes in fitresult.signature, but without merging the two.

A learning network signature is an intermediate object defined when a user constructs a learning network machine, mach. They are named tuples whose values are the nodes consitituting interface points between the network and the machine. Examples are

(predict=yhat, )
(transform=Xsmall,)
(predict=yhat, transform=W, report=(loss=loss_node,))

where yhat, Xsmall, W and loss_node are nodes in the network.

If a key k is the name of an operation (such as :predict, :predict_mode, :transform, inverse_transform) then k(mach, X) returns n(X) where n is the corresponding node value. Each such node must have a unique origin (length(origins(n)) === 1).

The only other allowed key is :report, whose corresponding value must be a named tuple

(k1=n1, k2=n2, ...)

whose keys are arbitrary, and whose values are nodes of the network. For each such key-value pair k=n, the value returned by n() is included in the named tuple report(mach), with corresponding key k. So, in the third example above, report(mach).loss will return the value of loss_node().

Private method

Training losses and feature importances

Training losses and feature importances, if reported by a model, will be available in the machine's report (see above). However, there are also direct access methods where supported:

training_losses(mach::Machine) -> vector_of_losses

Here vector_of_losses will be in historical order (most recent loss last). This kind of access is supported for model = mach.model if supports_training_losses(model) == true.

feature_importances(mach::Machine) -> vector_of_pairs

Here a vector_of_pairs is a vector of elements of the form feature => importance_value, where feature is a symbol. For example, vector_of_pairs = [:gender => 0.23, :height => 0.7, :weight => 0.1]. If a model does not support feature importances for some model hyperparameters, every importance_value will be zero. This kind of access is supported for model = mach.model if reports_feature_importances(model) == true.

If a model can report multiple types of feature importances, then there will be a model hyper-parameter controlling the active type.

Constructing machines

A machine is constructed with the syntax machine(model, args...) where the possibilities for args (called training arguments) are summarized in the table below. Here X and y represent inputs and target, respectively, and Xout is the output of a transform call. Machines for supervised models may have additional training arguments, such as a vector of per-observation weights (in which case supports_weights(model) == true).

model supertypemachine constructor callsoperation calls (first compulsory)
Deterministic <: Supervisedmachine(model, X, y, extras...)predict(mach, Xnew), transform(mach, Xnew), inverse_transform(mach, Xout)
Probabilistic <: Supervisedmachine(model, X, y, extras...)predict(mach, Xnew), predict_mean(mach, Xnew), predict_median(mach, Xnew), predict_mode(mach, Xnew), transform(mach, Xnew), inverse_transform(mach, Xout)
Unsupervised (except Static)machine(model, X)transform(mach, Xnew), inverse_transform(mach, Xout), predict(mach, Xnew)
Staticmachine(model)transform(mach, Xnews...), inverse_transform(mach, Xout)

All operations on machines (predict, transform, etc) have exactly one argument (Xnew or Xout above) after mach, the machine instance. An exception is a machine bound to a Static model, which can have any number of arguments after mach. For more on Static transformers (which have no training arguments) see Static transformers.

A machine is reconstructed from a file using the syntax machine("my_machine.jlso"), or machine("my_machine.jlso", args...) if retraining using new data. See Saving machines below.

Lowering memory demands

For large data sets, you may be able to save memory by suppressing data caching that some models perform to increase speed. To do this, specify cache=false, as in

machine(model, X, y, cache=false)

Constructing machines in learning networks

Instead of data X, y, etc, the machine constructor is provided Node or Source objects ("dynamic data") when building a learning network. See Composing Models for more on this advanced feature. One also uses machine to wrap a machine around a whole learning network; see Learning network machines.

Saving machines

Users can save and restore MLJ machines using any external serialization package by suitably preparing their Machine object, and applying a post-processing step to the deserialized object. This is explained under Using an arbitrary serializer below.

However, if a user is happy to use Julia's standard library Serialization module, there is a simplified workflow described first.

The usual serialization provisos apply. For example, when deserializing you need to have all code on which the serialization object depended loaded at the time of deserialization also. If a hyper-parameter happens to be a user-defined function, then that function must be defined at deserialization. And you should only deserialize objects from trusted sources.

Using Julia's native serializer

MLJModelInterface.saveFunction
MLJ.save(filename, mach::Machine)
MLJ.save(io, mach::Machine)

MLJBase.save(filename, mach::Machine)
MLJBase.save(io, mach::Machine)

Serialize the machine mach to a file with path filename, or to an input/output stream io (at least IOBuffer instances are supported) using the Serialization module.

To serialise using a different format, see serializable.

Machines are deserialized using the machine constructor as shown in the example below.

The implementation of save for machines changed in MLJ 0.18 (MLJBase 0.20). You can only restore a machine saved using older versions of MLJ using an older version.

Example

using MLJ
Tree = @load DecisionTreeClassifier
X, y = @load_iris
mach = fit!(machine(Tree(), X, y))

MLJ.save("tree.jls", mach)
mach_predict_only = machine("tree.jls")
predict(mach_predict_only, X)

# using a buffer:
io = IOBuffer()
MLJ.save(io, mach)
seekstart(io)
predict_only_mach = machine(io)
predict(predict_only_mach, X)
Only load files from trusted sources

Maliciously constructed JLS files, like pickles, and most other general purpose serialization formats, can allow for arbitrary code execution during loading. This means it is possible for someone to use a JLS file that looks like a serialized MLJ machine as a Trojan horse.

See also serializable, machine.

Using an arbitrary serializer

Since machines contain training data, serializing a machine directly is not recommended. Also, the learned parameters of models implemented in a language other than Julia may not have persistent representations, which means serializing them is useless. To address these two issues, users:

  • Call serializable(mach) on a machine mach they wish to save (to remove data and create persistent learned parameters)

  • Serialize the returned object using SomeSerializationPkg

To restore the original machine (minus training data) they:

  • Deserialize using SomeSerializationPkg to obtain a new object mach
  • Call restore!(mach) to ensure mach can be used to predict or transform new data.
MLJBase.serializableFunction
serializable(mach::Machine)

Returns a shallow copy of the machine to make it serializable. In particular, all training data is removed and, if necessary, learned parameters are replaced with persistent representations.

Any general purpose Julia serializer may be applied to the output of serializable (eg, JLSO, BSON, JLD) but you must call restore!(mach) on the deserialised object mach before using it. See the example below.

If using Julia's standard Serialization library, a shorter workflow is available using the save method.

A machine returned by serializable is characterized by the property mach.state == -1.

Example using JLSO

using MLJ
using JLSO
Tree = @load DecisionTreeClassifier
tree = Tree()
X, y = @load_iris
mach = fit!(machine(tree, X, y))

# This machine can now be serialized
smach = serializable(mach)
JLSO.save("machine.jlso", machine => smach)

# Deserialize and restore learned parameters to useable form:
loaded_mach = JLSO.load("machine.jlso")[:machine]
restore!(loaded_mach)

predict(loaded_mach, X)
predict(mach, X)

See also restore!, save.

MLJBase.restore!Function
restore!(mach::Machine)

Restore the state of a machine that is currently serializable but which may not be otherwise usable. For such a machine, mach, one has mach.state=1. Intended for restoring deserialized machine objects to a useable form.

For an example see serializable.

Internals

For a supervised machine, the predict method calls a lower-level MLJBase.predict method, dispatched on the underlying model and the fitresult (see below). To see predict in action, as well as its unsupervised cousins transform and inverse_transform, see Getting Started.

Except for model, a Machine instance has several fields which the user should not directly access; these include:

  • model - the struct containing the hyperparameters to be used in calls to fit!

  • fitresult - the learned parameters in a raw form, initially undefined

  • args - a tuple of the data, each element wrapped in a source node; see Learning Networks (in the supervised learning example above, args = (source(X), source(y)))

  • report - outputs of training not encoded in fitresult (eg, feature rankings), initially undefined

  • old_model - a deep copy of the model used in the last call to fit!

  • old_rows - a copy of the row indices used in the last call to fit!

  • cache

The interested reader can learn more about machine internals by examining the simplified code excerpt in Internals.

API Reference

MLJBase.machineFunction
machine(model, args...; cache=true, scitype_check_level=1)

Construct a Machine object binding a model, storing hyper-parameters of some machine learning algorithm, to some data, args. Calling fit! on a Machine instance mach stores outcomes of applying the algorithm in mach, which can be inspected using fitted_params(mach) (learned paramters) and report(mach) (other outcomes). This in turn enables generalization to new data using operations such as predict or transform:

using MLJModels
X, y = make_regression()

PCA = @load PCA pkg=MultivariateStats
model = PCA()
mach = machine(model, X)
fit!(mach, rows=1:50)
transform(mach, selectrows(X, 51:100)) # or transform(mach, rows=51:100)

DecisionTreeRegressor = @load DecisionTreeRegressor pkg=DecisionTree
model = DecisionTreeRegressor()
mach = machine(model, X, y)
fit!(mach, rows=1:50)
predict(mach, selectrows(X, 51:100)) # or predict(mach, rows=51:100)

Specify cache=false to prioritize memory management over speed.

When building a learning network, Node objects can be substituted for the concrete data but no type or dimension checks are applied.

Checks on the types of training data

A model articulates its data requirements using scientific types, i.e., using the scitype function instead of the typeof function.

If scitype_check_level > 0 then the scitype of each arg in args is computed, and this is compared with the scitypes expected by the model, unless args contains Unknown scitypes and scitype_check_level < 4, in which case no further action is taken. Whether warnings are issued or errors thrown depends the level. For details, see default_scitype_check_level, a method to inspect or change the default level (1 at startup).

Learning network machines

machine(Xs; oper1=node1, oper2=node2, ...)
machine(Xs, ys; oper1=node1, oper2=node2, ...)
machine(Xs, ys, extras...; oper1=node1, oper2=node2, ...)

Construct a special machine called a learning network machine, that wraps a learning network, usually in preparation to export the network as a stand-alone composite model type. The keyword arguments declare what nodes are called when operations, such as predict and transform, are called on the machine. An advanced option allows one to additionally pass the output of any node to the machine's report; see below.

In addition to the operations named in the constructor, the methods fit!, report, and fitted_params can be applied as usual to the machine constructed.

machine(Probabilistic(), args...; kwargs...)
machine(Deterministic(), args...; kwargs...)
machine(Unsupervised(), args...; kwargs...)
machine(Static(), args...; kwargs...)

Same as above, but specifying explicitly the kind of model the learning network is to meant to represent.

Learning network machines are not to be confused with an ordinary machine that happens to be bound to a stand-alone composite model (i.e., an exported learning network).

Examples of learning network machines

Supposing a supervised learning network's final predictions are obtained by calling a node yhat, then the code

mach = machine(Deterministic(), Xs, ys; predict=yhat)
fit!(mach; rows=train)
predictions = predict(mach, Xnew) # `Xnew` concrete data

is equivalent to

fit!(yhat, rows=train)
predictions = yhat(Xnew)

Here Xs and ys are the source nodes receiving, respectively, the input and target data.

In a unsupervised learning network for clustering, with single source node Xs for inputs, and in which the node Xout delivers the output of dimension reduction, and yhat the class labels, one can write

mach = machine(Unsupervised(), Xs; transform=Xout, predict=yhat)
fit!(mach)
transformed = transform(mach, Xnew) # `Xnew` concrete data
predictions = predict(mach, Xnew)

which is equivalent to

fit!(Xout)
fit!(yhat)
transformed = Xout(Xnew)
predictions = yhat(Xnew)

Including a node's output in the report

The return value of a node called with no arguments can be included in a learning network machine's report, and so in the report of any composite model type constructed by exporting a learning network. This is useful for exposing byproducts of network training that are not readily deduced from the reports and fitted_params of the component machines (which are automatically exposed).

The following example shows how to expose err1() and err2(), where err1 are err2 are nodes in the network delivering training errors.

X, y = make_moons()
Xs = source(X)
ys = source(y)

model = ConstantClassifier()
mach = machine(model, Xs, ys)
yhat = predict(mach, Xs)
err1 = @node auc(yhat, ys)
err2 = @node accuracy(yhat, ys)

network_mach = machine(Probabilistic(),
                       Xs,
                       ys,
                       predict=yhat,
                       report=(auc=err1, accuracy=err2))

fit!(network_mach)
r = report(network_mach)
@assert r.auc == auc(yhat(), ys())
@assert r.accuracy == accuracy(yhat(), ys())

See also fit!, default_scitype_check_level, MLJBase.save, serializable.

StatsAPI.fit!Function
fit!(mach::Machine, rows=nothing, verbosity=1, force=false)

Fit the machine mach. In the case that mach has Node arguments, first train all other machines on which mach depends.

To attempt to fit a machine without touching any other machine, use fit_only!. For more on the internal logic of fitting see fit_only!

fit!(N::Node;
     rows=nothing,
     verbosity=1,
     force=false,
     acceleration=CPU1())

Train all machines required to call the node N, in an appropriate order, but parallelizing where possible using specified acceleration mode. These machines are those returned by machines(N).

Supported modes of acceleration: CPU1(), CPUThreads().

fit!(mach::Machine{<:Surrogate};
     rows=nothing,
     acceleration=CPU1(),
     verbosity=1,
     force=false))

Train the complete learning network wrapped by the machine mach.

More precisely, if s is the learning network signature used to construct mach, then call fit!(N), where N is a greatest lower bound of the nodes appearing in the signature (values in the signature that are not AbstractNode are ignored). For example, if s = (predict=yhat, transform=W), then call fit!(glb(yhat, W)).

See also machine

MLJBase.fit_only!Function
MLJBase.fit_only!(mach::Machine; rows=nothing, verbosity=1, force=false)

Without mutating any other machine on which it may depend, perform one of the following actions to the machine mach, using the data and model bound to it, and restricting the data to rows if specified:

  • Ab initio training. Ignoring any previous learned parameters and cache, compute and store new learned parameters. Increment mach.state.

  • Training update. Making use of previous learned parameters and/or cache, replace or mutate existing learned parameters. The effect is the same (or nearly the same) as in ab initio training, but may be faster or use less memory, assuming the model supports an update option (implements MLJBase.update). Increment mach.state.

  • No-operation. Leave existing learned parameters untouched. Do not increment mach.state.

Training action logic

For the action to be a no-operation, either mach.frozen == true or or none of the following apply:

  • (i) mach has never been trained (mach.state == 0).

  • (ii) force == true.

  • (iii) The state of some other machine on which mach depends has changed since the last time mach was trained (ie, the last time mach.state was last incremented).

  • (iv) The specified rows have changed since the last retraining and mach.model does not have Static type.

  • (v) mach.model has changed since the last retraining.

In any of the cases (i) - (iv), mach is trained ab initio. If only (v) fails, then a training update is applied.

To freeze or unfreeze mach, use freeze!(mach) or thaw!(mach).

Implementation details

The data to which a machine is bound is stored in mach.args. Each element of args is either a Node object, or, in the case that concrete data was bound to the machine, it is concrete data wrapped in a Source node. In all cases, to obtain concrete data for actual training, each argument N is called, as in N() or N(rows=rows), and either MLJBase.fit (ab initio training) or MLJBase.update (training update) is dispatched on mach.model and this data. See the "Adding models for general use" section of the MLJ documentation for more on these lower-level training methods.