Trigger-Set Model Watermarking

Description

Trigger-set model watermarking lets the owner of a trained classifier show later that a suspect model was copied from theirs, by planting a secret behaviour that only the owner can demonstrate. Before release, the owner chooses a small secret trigger set (in the originating scheme, 100 abstract images), gives each input an arbitrary label unrelated to its content, and trains or fine-tunes the model to return those labels while keeping its accuracy on the real task. In the originating scheme the owner also records cryptographic commitments to the trigger inputs and their labels, which fix the watermark at release so that a trigger set cannot be invented once a dispute has begun. To verify, the owner, or an arbiter shown the opened commitments, sends the trigger inputs to the suspect model through its ordinary prediction interface and counts how many return the assigned labels; an independently trained model matches them at about chance level, so a match rate above a preset threshold is evidence of copying. The technique produces the watermarked model, the commitments and, for each verification, the suspect model's trigger-set accuracy against the threshold.

Example Use Cases

Security

A medical-imaging company that licenses a chest X-ray classifier for installation on hospital servers, where the weights leave its control, embeds a trigger set before each release and lodges the commitments with a timestamping service. When a near-identical model appears in a rival's product, it queries that product with the trigger images to decide whether there is enough evidence to pursue a licence breach.

Before relying on the watermark in its licence terms, the security team of a retailer that licenses a visual product-search model to partner stores to run on their own servers runs the fine-tuning and pruning attacks in its threat model against the watermarked model, and reports the trigger-set accuracy that survives each, to set the verification threshold and state what the watermark can and cannot evidence.

In a licensing dispute over a satellite-imagery land-use classifier used by a local authority, the developer opens its pre-release commitments to an independent expert, who queries the disputed model's prediction service with the trigger images and reports the match rate against a threshold agreed in advance, without either party handing over model weights.

Limitations

Removal attacks defeat it in practice. A systematic evaluation of published schemes, trigger-set schemes among them, found none robust against adaptive removal attacks or known ways of deriving a surrogate model, and in the originating paper re-training all layers sharply reduced trigger-set accuracy in models watermarked by fine-tuning. A failed verification does not show a model was not copied, and a robustness claim holds only for the attacks tested.
It evidences copying of the model, not theft through its prediction interface, and needs query access to the suspect. A copy trained on the deployed model's answers (model extraction or distillation) need never see the trigger inputs, and a systematic evaluation found these schemes fail against such surrogates, so extraction needs other defences. A copy used only inside its owner's organisation, or not exposed to queries, cannot be tested at all.
Verification reveals the secret. Sending trigger inputs to a suspect's interface, or showing them to an arbiter, exposes them; an adversary who learns them can fine-tune them away or filter those queries, so verification is in effect single-use unless several trigger sets were embedded or a zero-knowledge protocol is used, both of which the originating paper describes and both of which add work.
Without commitments lodged before release the evidence is weak: whoever holds a copy can embed a second trigger set of their own and claim ownership. In the originating paper's experiments the second set was much less durable under fine-tuning than the first, but that helps only an owner who can show its commitments existed before the disputed model did.
The scheme and its evaluations cover image classifiers that the owner can train or fine-tune with access to the training data. Generative and language models need different designs, and the threshold must reflect the number of classes: an unwatermarked model matched 7.0% of trigger labels on 10-class CIFAR-10 but 1.0% on 100-class CIFAR-100, so chance agreement is far higher with few classes.

Resources

Research Papers

Protecting Intellectual Property of Deep Neural Networks with Watermarking

Jialong Zhang et al.•Jan 1, 2018

Deep learning technologies, which are the key components of state-of-the-art Artificial Intelligence (AI) services, have shown great success in providing human-level capabilities for a variety of tasks, such as visual analysis, speech recognition, and natural language processing and etc. Building a production-level deep learning model is a non-trivial task, which requires a large amount of training data, powerful computing resources, and human expertises. Therefore, illegitimate reproducing, distribution, and the derivation of proprietary deep learning models can lead to copyright infringement and economic harm to model creators. Therefore, it is essential to devise a technique to protect the intellectual property of deep learning models and enable external verification of the model ownership. In this paper, we generalize the "digital watermarking'' concept from multimedia ownership verification to deep neural network (DNNs) models. We investigate three DNN-applicable watermark generation algorithms, propose a watermark implanting approach to infuse watermark into deep learning models, and design a remote verification mechanism to determine the model ownership. By extending the intrinsic generalization and memorization capabilities of deep neural networks, we enable the models to learn specially crafted watermarks at training and activate with pre-specified predictions when observing the watermark patterns at inference. We evaluate our approach with two image recognition benchmark datasets. Our framework accurately (100%) and quickly verifies the ownership of all the remotely deployed deep learning models without affecting the model accuracy for normal input data. In addition, the embedded watermarks in DNN models are robust and resilient to different counter-watermark mechanisms, such as fine-tuning, parameter pruning, and model inversion attacks.

Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring

Yossi Adi et al.•Jan 1, 2018

Software Packages

Watermark-Robustness-Toolbox

Aug 9, 2021

The official implementation of the IEEE S&P`22 paper "SoK: How Robust is Deep Neural Network Image Classification Watermarking".

WatermarkNN (PyTorch implementation of Adi et al. 2018)

Sep 16, 2026

Watermarking Deep Neural Networks (USENIX 2018)

Related Techniques

Name	Description	Assurance Goals
Model Signing	Model signing lets the publisher of a machine-learning model attach a cryptographic signature to the model's files, so that anyone who later loads them can check who released them and that no file has changed since. Under the Open Source Security Foundation's Model Signing (OMS) specification and its reference tool, model-signing, the publisher computes a hash (a short fingerprint that changes if a single byte of the file changes) of every file in the model directory, such as weights, configuration, tokeniser and any dataset files kept with them, lists the hashes in a sorted manifest and signs the manifest, producing a detached signature file kept beside the model. The signature can be made with the publisher's own key, with a certificate from an organisation's certificate authority, or through Sigstore, which issues a short-lived certificate tied to a verified online identity, such as a person's account or a build pipeline's, and records each signing event in a transparency log, an append-only record that can be inspected afterwards. Before loading, the consumer verifies the signature against the signer it expects, recomputes every hash and rejects the model if any listed file differs or is missing or, unless that check is switched off, an unlisted file has appeared, so the evidence is a pass or fail tied to a named signer and an exact set of files. Unlike the unsigned file hashes kept by data-versioning tools, it shows who stands behind the files, and unlike a watermark, which is trained into the weights so that it survives in an altered copy, it is broken by any change; it says nothing about how the model behaves or what it was trained on.	Security
Membership Inference Attack Testing	Membership inference attack testing checks whether someone with access to a trained model can tell if a particular record was in its training data. The tester plays the attacker, using the model's confidence scores, its prediction patterns or its loss on a record, often with shadow models, which are copies trained on similar data whose members are known. The attack is scored on records known to be members and records known to be non-members, which the tester must keep apart. The key result is the share of true members the attack identifies when it wrongly flags only a tiny share of non-members (for example 0.1%), because overall accuracy can hide an attack that identifies a few records with near certainty. The results guide the choice of mitigations, such as differential privacy or regularisation.	Privacy Security
AutoAttack	AutoAttack is a fixed suite of four automated adversarial attacks, introduced by Croce and Hein (ICML 2020), that measures how much of an image classifier's test accuracy survives small, deliberately chosen changes to its input images at inference time. The evaluator states the threat model, a norm and radius that bound the change (for example, no pixel value changed by more than 8/255 under the L-infinity norm), and the standard version then attacks each correctly classified test image in turn with APGD-CE (projected gradient descent that sets its own step size, on the cross-entropy loss), a targeted APGD on the authors' Difference of Logits Ratio loss, a targeted form of the FAB minimum-norm attack, and the Square Attack, which needs only the model's outputs and so still works when gradients are uninformative. An image counts as robust only if none of the four finds a misclassified version within the bound, and because every attack setting is fixed in advance the evaluator tunes nothing, which removes a common cause of overstated robustness: an attack too weakly tuned for the model. The output is robust accuracy, the share of test images still classified correctly under attack at the stated norm and radius, reported beside clean accuracy; published models are compared on the RobustBench leaderboard, which uses AutoAttack as its standard evaluation. Because AutoAttack is not built around any one defence, it can supply the attack step for a wider judgement of a defence, such as whether adversarial training holds up against attacks it was not trained on; unlike red teaming, it involves no human attacker or open-ended search.	Security