Data Version Control
Description
Data Version Control (DVC) is an open-source tool that records which version of a dataset or model was used at each point in a machine learning project. It works alongside Git, the standard system for tracking changes to code. DVC keeps large data and model files in separate storage, such as a cloud storage service, and puts a small pointer file into Git that holds a fingerprint (hash) of the file's contents. A team can also describe a pipeline, meaning the ordered steps from raw data through preprocessing to a trained model, and DVC then records fingerprints of each step's inputs, settings and outputs. Committed to Git, these files give a dated history of how data and models changed and let the team restore an earlier version, but only for what was added to DVC and committed.
Example Use Cases
Transparency
A hospital research group trains a chest X-ray classifier. When a regulator asks which images and preprocessing produced the version it submitted, the group checks out the tagged Git commit, shows the dataset fingerprints and pipeline steps recorded in it, and restores that exact data to confirm.
A flood-forecasting agency versions its river-sensor calibration data and the feature-extraction steps that use it. When a forecast is challenged, an independent reviewer can see from the version history who changed the calibration data and when, and which model release was trained on each version.
General
A lender's credit-scoring team sees approval rates shift after a quarterly data refresh. Using DVC with Git, they list the files that changed between the datasets behind the last two model releases, trace the shift to one refresh, and decide whether to revert to the earlier data version.
A manufacturer's quality team retrains a defect-detection model each month and tags each release in Git with the DVC record of its training images. When a customer reports a missed defect, the team retrieves the exact images behind the model then in use and sees what was added or removed since the previous release.
Limitations
- DVC records only what has been added to it and committed to Git. Data edited outside DVC, or changes that were never committed, leave no trace, and nothing in the history shows what is missing, so the record is only as complete as the team's habits.
- Lineage exists only for steps defined as DVC pipeline stages. A study of 56 open-source projects found that 85.7% used only DVC's versioning and tracking feature, so a project that uses DVC may hold no record of how its data was preprocessed or its model trained.
- Restoring an earlier model takes more than DVC's files. The code, the software environment and the stored data must all still exist, and the pipeline lock file records commands, file fingerprints and parameter values but not library versions, so a restored pipeline can still give different results.
- Scale costs time and storage. A changed data file is stored again as a new copy, and the documentation notes that operations such as pulling become slow with tens of thousands of files, pointing teams that need to branch and merge large datasets in object storage to a different tool.
- When several people change the same data files or pipeline, merge conflicts in DVC's tracking files have to be resolved by hand. Without agreed branching rules the history becomes hard to trust.