Data Version Control (DVC) is an open-source command-line tool that extends Git-style version control to large datasets, trained models, and the multi-step pipelines that produce them. For research groups running computational or machine-learning-driven studies, it addresses a gap that Git itself was never designed to close: Git tracks source code well, but it was built for small text files, not multi-gigabyte datasets or binary model weights, and it has no built-in concept of “run this pipeline and verify the output matches what was recorded.” DVC sits alongside Git to close that gap, and because of that it has become a common building block in labs and data-science-adjacent research groups trying to make their computational pipelines auditable and reproducible.
This guide explains what DVC actually does, how it fits into the broader research data lifecycle, and where it stops — because DVC is a version-control and pipeline-automation tool, not a repository, and research administrators evaluating it for a lab or a data management plan need to be clear about that distinction before committing to it as infrastructure.
What DVC Is (and What It Isn’t)
DVC is free, open-source software released under the Apache License 2.0, originally developed by Iterative, Inc. and maintained as a community open-source project on GitHub. It is best understood as “Git for data”: it borrows Git’s mental model — commits, diffs, branches, remotes — and applies it to artifacts Git can’t handle efficiently, namely large data files, model checkpoints, and the outputs of multi-step processing pipelines.
DVC is not:
- A replacement for Git — it runs alongside an existing Git repository and depends on it for versioning the small metadata files it creates.
- A data repository or archive — it doesn’t provide long-term preservation, curation, discoverability, or a DOI. A data repository and a DVC-tracked Git project solve different problems, and using DVC does not substitute for depositing a dataset somewhere findable and citable at the end of a project.
- Cloud storage — DVC does not host your data itself. It requires you to already have (or provision) a storage backend — commonly a cloud object store, an institutional SSH server, or a shared network filesystem — that it then tracks versions against.
Why Git Alone Doesn’t Work for Research Data
Git stores every version of every file in its history inside the repository itself, and it computes line-by-line diffs — both of which work well for source code and badly for research data. Committing a large dataset directly into Git bloats the repository permanently (Git never really discards old blobs), and diffing a binary file or a compressed dataset produces no useful information. Research groups that try to force large data into Git typically end up either excluding data from version control entirely (breaking reproducibility — nobody can tell which data version a given result used) or checking in enormous binary blobs that make clone and fetch operations impractically slow.
DVC’s approach: instead of storing the data itself in Git, it computes a content hash for each tracked file or directory, stores the actual data in a local cache and an optional remote store, and commits only a small, human-readable pointer file (a .dvc file) to Git. The pointer file records the hash, so Git’s own history now accurately tracks which version of the data was present at any given commit, without the repository ever growing by the size of the data itself.
How DVC Versions Data: Cache, Pointer Files, and Content Addressing
The core workflow is deliberately close to Git’s:
dvc initsets up DVC inside an existing Git repository.dvc add data/raw_dataset.csvmoves the file into DVC’s local cache (organized by content hash, the same content-addressable-storage principle Git itself uses internally) and generatesdata/raw_dataset.csv.dvc, a small YAML pointer file recording the hash and file size.- The
.dvcpointer file — not the dataset — gets committed withgit add/git commit, so every Git commit that touches the dataset now has a verifiable, hash-linked record of exactly which version of the data was in use. - When the dataset changes and
dvc addruns again, DVC computes a new hash, caches the new version alongside the old one, and updates the pointer file — so any prior Git commit can still reconstruct the exact data version it referenced, viadvc checkout.
This is the mechanism that makes DVC useful for research provenance in the same sense that PROV-O and other provenance vocabularies are useful at the metadata level: a DVC-tracked project makes it possible to answer “what exact version of the data produced this figure/model/result?” retroactively, from Git history alone, rather than relying on a lab member’s memory or an informally-named file like data_final_v3_USE_THIS.csv. See CASRAI’s guide to data provenance for the broader documentation practice this fits into.
Setting Up Remote Storage for a Shared Dataset
The local cache alone only helps one person on one machine. To collaborate, a DVC project needs a remote — a storage location DVC pushes hashed data to and pulls it back from, configured with dvc remote add. DVC supports a range of backends, including major cloud object stores (Amazon S3, Google Cloud Storage, Azure Blob Storage), SSH/SFTP servers, HDFS, WebDAV, Google Drive, and plain local or network-mounted directories — which means an institution can point DVC at storage it already operates (an institutional S3-compatible bucket, a research computing cluster’s shared filesystem) rather than adopting a new vendor.
dvc push uploads newly cached data versions to the configured remote; dvc pull retrieves them. A collaborator who clones the Git repository gets the small .dvc pointer files immediately, then runs dvc pull to fetch only the actual data versions referenced by the commit they’ve checked out — so nobody needs to download the entire history of every dataset version just to work on the current state of the project.
Defining Reproducible Pipelines with dvc.yaml
Beyond raw file versioning, DVC’s pipeline feature is arguably the part most directly relevant to computational reproducibility. A dvc.yaml file defines a pipeline as a set of named stages, each with:
- Dependencies — the input files, scripts, and parameters a stage reads.
- Outputs — the files or directories it produces.
- A command — the actual script or program that runs.
DVC assembles these stages into a directed acyclic graph (DAG) of the whole pipeline — for example: raw data → preprocessing → feature extraction → model training → evaluation. Running dvc repro walks that graph, hashes every dependency, and re-runs only the stages whose inputs, code, or parameters have actually changed since the last run — a stage whose dependencies are unchanged is reported as already up to date and skipped. The result of each run is recorded in a dvc.lock file, which pins the exact hashes of every dependency and output for that run — the reproducibility record a collaborator or reviewer can use to confirm a given pipeline execution matches what’s claimed.
This maps closely onto the distinction CASRAI’s computational reproducibility and computational environment definitions draw between having the original data and having a fully specified, re-runnable process: DVC’s pipeline layer captures the process side, while its data-versioning layer captures the data side, and dvc.lock ties both together for a given run. It’s a practical, tool-level complement to formal reproducibility assessment frameworks like ACM’s Artifact Review and Badging, which evaluate whether a paper’s artifacts are actually available, functional, and reusable — a DVC-tracked pipeline is one concrete way to make an artifact meet that bar.
Tracking Experiments Without Drowning in Commits
Iterative model development typically means running the same pipeline dozens or hundreds of times with different parameters, and creating a full Git commit for every run would flood the repository’s history with noise. DVC’s experiment-tracking layer (dvc exp run) addresses this: it runs a pipeline variant — often with parameters overridden from a tracked params.yaml file — without immediately creating a Git commit, records the resulting metrics and parameters, and lets a researcher compare runs (dvc exp show) before deciding which, if any, is worth promoting to a permanent, committed version of the project. Only the runs worth keeping become part of the permanent Git/DVC history; the rest can be discarded without ever having cluttered the record.
Where DVC Fits in the Research Data Lifecycle
DVC operates almost entirely within the earlier, active stages of research data handling — creation, processing, and analysis — rather than the later stages of deposit and long-term access. It’s useful to place it explicitly against a lifecycle framework: it has no role in the “ingest” or “preservation action” stages as a certified repository would perform them, but it directly supports the “create,” “appraise and select,” and — critically for computational work — the ongoing “access, use, and reuse” stages within an active project, by making every intermediate version of a dataset or model reconstructable and every processing step re-runnable.
For a lab writing a data management plan, DVC is infrastructure that can be named as the mechanism for in-project version control and provenance tracking — but it does not, on its own, satisfy a funder’s data-sharing or preservation commitment. A Data Management Plan (DMP) that commits to a named repository at project end still needs that deposit step; DVC just makes what gets deposited more trustworthy, because its provenance is documented rather than reconstructed after the fact. CASRAI’s DMP template and structure guide and taxonomy of research data types are useful starting points for where DVC-tracked datasets fit into a compliant plan.
DVC and Repository Deposit Are Not the Same Thing
A recurring point of confusion worth stating plainly for research administrators: pushing data to a DVC remote is not the same act as depositing data in a certified repository, and it should not be described that way in a DMP or compliance report. A DVC remote is typically private, access-controlled cloud or institutional storage with no independent curation, no persistent identifier issuance, and no certification against a standard like CoreTrustSeal. When a funder or journal requires open, FAIR-compliant, citable data at publication, that requirement is met by depositing the final dataset in an actual repository — with a real DOI and a documented preservation commitment — not by pointing a reviewer at a DVC remote’s access credentials.
Where DVC-tracked projects do connect usefully to the open-data side of the lifecycle is at final deposit: a DVC pipeline’s final output, together with its dvc.yaml/dvc.lock pipeline definition, is exactly the kind of “how this dataset was produced” documentation that pairs well with structured packaging formats built for FAIR reuse, such as RO-Crate, or with ML-dataset-specific metadata formats like Croissant when the deposited artifact is itself a training dataset. A dataset’s FAIR compliance and its metadata schema (see CASRAI’s guide to choosing a metadata schema, and the DataCite metadata schema commonly used at deposit) are decided at that final deposit step, separately from whatever version-control tooling was used during active work on it. Training datasets and models deposited for reuse or citation should also carry a real persistent identifier — see CASRAI’s guide on persistent identifiers and citation for AI models and training datasets.
Considerations for Research Administrators and Data Stewards
- Cost: the DVC command-line tool itself is free and open source (Apache 2.0). The real cost is the storage backend it pushes to — cloud object storage, institutional storage, or research-computing allocation — which an institution needs to budget and provision separately, and account for in a DMP’s cost estimate.
- Adoption burden: DVC assumes comfort with Git and the command line. It’s realistic infrastructure for computational/data-science-heavy labs already using Git for code, and a harder sell for groups without that baseline, where the training overhead needs to be weighed against the reproducibility gain.
- Access control and compliance: because DVC remotes are typically plain cloud or SSH storage, any sensitive or regulated data (human-subjects data, controlled-access genomic data, etc.) tracked through DVC inherits whatever access controls the underlying storage backend provides — DVC itself adds no additional governance layer, encryption policy, or audit trail beyond what the storage backend and institutional IT policy supply.
- Not a substitute for a preservation plan: a DVC remote can be deleted, migrated, or lose funding for its storage tier without any of the certification or continuity commitments a repository evaluated under a framework like CoreTrustSeal is expected to provide. A DMP or data-sharing agreement should name the actual deposit repository, not the DVC remote, as the preservation commitment.
Frequently Asked Questions
Is DVC the same thing as Git-LFS?
No, though they solve an overlapping problem. Git Large File Storage (Git-LFS) also replaces large files in a Git repository with lightweight pointers, but it’s essentially a storage-substitution extension to Git with no pipeline or experiment-tracking layer. DVC adds a full pipeline-definition and reproducibility system (dvc.yaml, dvc repro, experiment tracking) on top of the same basic pointer-file principle, and supports a broader range of storage backends without requiring a Git-LFS-compatible server.
Does using DVC make a project FAIR-compliant?
Not by itself. DVC improves the Reusability and, indirectly, the provenance/documentation quality of a dataset within an active project, but Findability and Accessibility under the FAIR Data Principles require the dataset to actually be deposited somewhere discoverable with a persistent identifier and clear access/license terms — steps DVC does not perform.
Do I need a paid product to use DVC?
The core DVC tool and its command-line workflow are free and open source under the Apache 2.0 license. The maintainers behind DVC also offer separate, commercially licensed hosted tooling aimed at team-scale experiment visualization and collaboration; that product is optional and distinct from the open-source core described in this guide, and a lab can get the full data-versioning and pipeline-reproducibility functionality without it.
Can DVC version data stored on an institutional research-computing cluster?
Yes — DVC’s remote-storage backends include SSH/SFTP and local/network-mounted filesystems, so a lab can configure a DVC remote pointing at storage already provisioned on an institutional HPC or research-computing system, without needing a commercial cloud account.
DVC is not a replacement for a data management plan, a certified repository, or a formal metadata schema — it’s version-control and pipeline-automation infrastructure that operates during the active, computational phase of a research project. Used well, it makes the eventual deposit step easier to do correctly, because the provenance of the final dataset or model is already documented rather than reconstructed from memory. See CASRAI’s broader research data management pillar for how data versioning fits alongside planning, metadata, repository selection, and licensing across the full research data lifecycle.
