Model Signing
Description
Model signing lets the publisher of a machine-learning model attach a cryptographic signature to the model's files, so that anyone who later loads them can check who released them and that no file has changed since. Under the Open Source Security Foundation's Model Signing (OMS) specification and its reference tool, model-signing, the publisher computes a hash (a short fingerprint that changes if a single byte of the file changes) of every file in the model directory, such as weights, configuration, tokeniser and any dataset files kept with them, lists the hashes in a sorted manifest and signs the manifest, producing a detached signature file kept beside the model. The signature can be made with the publisher's own key, with a certificate from an organisation's certificate authority, or through Sigstore, which issues a short-lived certificate tied to a verified online identity, such as a person's account or a build pipeline's, and records each signing event in a transparency log, an append-only record that can be inspected afterwards. Before loading, the consumer verifies the signature against the signer it expects, recomputes every hash and rejects the model if any listed file differs or is missing or, unless that check is switched off, an unlisted file has appeared, so the evidence is a pass or fail tied to a named signer and an exact set of files. Unlike the unsigned file hashes kept by data-versioning tools, it shows who stands behind the files, and unlike a watermark, which is trained into the weights so that it survives in an altered copy, it is broken by any change; it says nothing about how the model behaves or what it was trained on.
Example Use Cases
Security
A bank's machine-learning platform team signs each model at the end of its training pipeline, using the pipeline's own workload identity through Sigstore, and configures its model-serving cluster to load only models whose signature verifies against that identity. Its security team watches the transparency log for signing events under that identity that match no approved release, to decide whether a model reached production outside the release process.
A vehicle maker's perception team downloads a pretrained object detector from a supplier catalogue that signs every model it publishes. Before fine-tuning, it verifies the download against the supplier's published certificate and files the verification output in its supplier-acceptance record, so later reviews can show exactly which files the work started from.
A hospital's imaging department receives quarterly updates to a vendor's chest X-ray triage model. Its clinical engineering team verifies each update against the vendor's certificate and rejects any update whose signature fails, or that contains a file the signature does not cover, before it enters the clinical validation environment.
A public research institute releases an open model together with the test set behind its published benchmark, signing both in one bundle, so that a laboratory re-running the benchmark can confirm that the weights and the test data are byte for byte the ones the institute released before comparing its results with the published ones.
Limitations
- It shows who released the files and that they have not changed, nothing more. A model carrying a malicious pickle payload, a backdoor or the effects of poisoned training data verifies perfectly if its publisher signed it, whether through carelessness or because the publisher's own pipeline was compromised before signing. The originating paper states that signing tells the user nothing about what a model does or how it was produced, so content checks are still needed.
- A valid signature is only as good as the consumer's check of who made it. A model signed with an attacker's own identity, or by an account named to look like a well-known organisation (a documented attack on model hubs), verifies cleanly unless the verifier insists on the specific identity, certificate or public key it expects. Consumers need a maintained list of trusted signers for each model source.
- A stolen signing key, or a compromised account at the identity provider used for Sigstore signing, produces valid signatures. With Sigstore every signature is recorded in a transparency log, so a publisher that monitors the log for its own identity can spot signatures it did not make; with the key and certificate methods the specification does not require a log entry, so misuse of a key may go unnoticed.
- Any change to the files breaks verification, including legitimate ones such as quantisation, conversion to another file format or adding a file. Each derived model needs a new signature from whoever produced it, and that signature says nothing about how the derivative relates to the signed original; recording that relationship needs a separate provenance record, such as the pedigree section of an ML-BOM.
- It protects only where verification is enforced. Most models on public hubs are unsigned (fewer than 2% of Hugging Face models used the hub's GPG commit-signing option, per studies the originating paper cites), and an unchecked signature protects nobody. Verification re-hashes every file, and hashing dominates the cost: a 1 TB file took about 13 minutes on a 128-vCPU server in the originating paper's benchmark, which matters if very large models are verified at every load.
Resources
Research Papers
Position: Machine Learning Models Have a Supply Chain Problem
Powerful machine learning (ML) models are now readily available online, which creates exciting possibilities for users who lack the deep technical expertise or substantial computing resources needed to develop them. On the other hand, this type of open ecosystem comes with many risks. In this paper, we argue that the current ecosystem for open ML models contains significant supply-chain risks, some of which have been exploited already in real attacks. These include an attacker replacing a model with something malicious (e.g., malware), or a model being trained using a vulnerable version of a framework or on restricted or poisoned data. We then explore how Sigstore, a solution designed to bring transparency to open-source software supply chains, can be used to bring transparency to open ML models, in terms of enabling model publishers to sign their models and prove properties about the datasets they use.