Tag: national data repository

  • National Data Repository Mandates: UK, US, EU

    National data repository requirements now differ sharply by jurisdiction: the UK coordinates through UKRI’s Concordat on Open Research Data and a planned National Data Library, the US relies on agency-specific mandates such as the NIH Data Management and Sharing Policy layered on the OPEN Government Data Act, and the EU binds Horizon Europe funding to mandatory FAIR data management plans routed through the European Open Science Cloud. All three converge on the FAIR principles as the technical baseline, but they diverge sharply on enforcement, centralisation and what “as open as possible” means in practice.

    A national data repository is a government- or funder-endorsed infrastructure (or federated network of infrastructures) for depositing, curating and providing persistent access to datasets produced by publicly funded research, so that they meet the FAIR standard of being Findable, Accessible, Interoperable and Reusable. No single global rulebook defines what such a repository must look like — which is precisely why the UK, US and EU have built three structurally different systems around the same FAIR foundation.

    What counts as a national data repository?

    A national data repository is infrastructure, endorsed at government or funder level, that stores research datasets with persistent identifiers, standardised metadata and defined reuse licences. The FAIR data principles — first formalised in Scientific Data in 2016 — define the technical bar: data and metadata must be findable via persistent identifiers, accessible over open protocols, interoperable through shared vocabularies, and reusable under clear provenance and licensing.

    Crucially, FAIR does not mean unconditionally open. The dominant policy language across all three jurisdictions is some variant of “as open as possible, as closed as necessary” — datasets with legitimate privacy, security or intellectual-property constraints can remain FAIR while access to the raw data itself stays restricted, provided the metadata is still discoverable.

    How does the UK mandate research data repositories?

    The UK’s approach is coordinated centrally through UK Research and Innovation (UKRI) rather than fragmented across individual funders. The Concordat on Open Research Data, agreed by UK funders and sector bodies, sets the expectation that publicly funded research data should be made openly available with as few restrictions as possible, in a timely and responsible manner.

    UKRI has been developing a harmonised open research data policy to replace the varying requirements previously set by its individual research councils, with a more explicit alignment to FAIR principles than the original Concordat text. The UK does not run one single mandatory repository for all disciplines; instead it combines a cross-disciplinary resource — the UK Data Service, holding the country’s largest collection of economic, population and social research data — with discipline-specific data centres. A National Data Library initiative is also under development. Enforcement runs through grant conditions rather than statute.

    How does the US enforce data-sharing requirements?

    The US combines a government-wide legal baseline with agency-specific enforcement, producing a federated rather than centralised system. The OPEN Government Data Act codifies the principle that federal government data — including federally funded research outputs captured by agencies — should be open and machine-readable by default, operationalised through the Data.gov catalogue.

    The sharpest enforcement sits with individual funding agencies. Under the NIH Data Management and Sharing (DMS) Policy, effective since January 2023, NIH-funded researchers must submit a DMS Plan describing how scientific data will be managed and shared, with FAIR principles strongly encouraged. The National Science Foundation requires a Data Management Plan for all proposals and supports deposit through disciplinary repositories and its own NSF Public Access Repository (NSF-PAR). This gives communities flexibility to choose fitting repositories, at the cost of one unified national research-data repository.

    How does the EU mandate FAIR data through Horizon Europe?

    The EU operates the most centrally binding framework of the three. The Directive on open data and the re-use of public sector information requires member states to establish national policies for open access to publicly funded research data on an “open by default” basis, explicitly aligned with FAIR principles. For research funded under Horizon Europe, making data FAIR is a mandatory grant condition, not a recommendation: funded projects must produce a Data Management Plan and comply with FAIR requirements as a condition of the award, under the same “as open as possible, as closed as necessary” test used elsewhere.

    Infrastructure is built around the European Open Science Cloud (EOSC), described by the European Commission as a federated environment intended to become a “web of FAIR data and services” spanning all scientific disciplines. Within that federation, researchers commonly deposit through the general-purpose repository Zenodo — built and operated with CERN — while the Community Research and Development Information Service (CORDIS) serves as the EU’s public repository of record for funded project information.

    Where do the three approaches converge and diverge?

    All three jurisdictions treat FAIR as the technical baseline and all three qualify openness with a “necessary restriction” clause. The differences lie in enforcement mechanism, degree of centralisation, and whether a single flagship repository exists.

    Feature UK US EU
    Primary instrument UKRI Concordat on Open Research Data (evolving to a harmonised FAIR-explicit policy) OPEN Government Data Act; NIH DMS Policy; NSF Public Access Policy EU Open Data Directive; Horizon Europe grant conditions
    Legal basis Funder policy condition Federal statute plus agency policy Legally binding directive plus grant condition
    FAIR status Increasingly explicit in new UKRI policy Encouraged, embedded in agency plans Mandatory for Horizon Europe-funded projects
    Data management plan required Yes, for UKRI funding Yes, for NIH and NSF funding Yes, mandatory for Horizon Europe
    Repository model Centralised flagship (UK Data Service) plus disciplinary centres Federated (Data.gov, NSF-PAR, disciplinary repositories) Federated supranational (EOSC, Zenodo, CORDIS)

    Common questions on national data repository mandates

    What are the FAIR data principles required by UKRI?

    UKRI requires funded researchers to make outputs Findable, Accessible, Interoperable and Reusable, aligned with its Concordat on Open Research Data. UKRI councils frame this as maximising the impact, visibility and citation of research while applying the “as open as possible, as restricted as necessary” test to data with legitimate sensitivities.

    Does the NIH require a data management and sharing plan?

    Yes. Since 25 January 2023, the NIH Data Management and Sharing (DMS) Policy requires funded researchers to submit a DMS Plan describing how scientific data will be preserved and shared. NIH strongly encourages applying FAIR principles when selecting repositories and structuring metadata for that plan.

    Is FAIR data mandatory under Horizon Europe?

    Yes, unlike the UK’s evolving policy and the US’s encouraged-but-agency-specific approach, Horizon Europe makes FAIR data management a binding grant condition. Funded projects must submit a Data Management Plan and comply with FAIR requirements, subject to the same necessary-restriction exceptions used across all three jurisdictions.

    Is there one single national data repository researchers must use?

    No jurisdiction mandates a single universal repository. The UK combines a flagship service (UK Data Service) with disciplinary centres; the US runs a federated system across Data.gov and agency repositories such as NSF-PAR; the EU federates access through EOSC, Zenodo and CORDIS. Researchers typically choose the repository matching their discipline and funder requirements.

    What this means for institutions and researchers

    For research administrators managing multi-jurisdictional funding, a single data management plan template cannot satisfy all three regimes. Compliance teams must map deposit requirements per funder rather than assume FAIR-labelled data automatically meets every mandate’s specific repository, licensing and metadata conditions.

    The trend line points toward convergence. The UK’s move to a harmonised, more explicitly FAIR-aligned UKRI policy and the EU’s EOSC federation both signal a shift from fragmented rules toward unified infrastructure. The US remains the outlier: its federal open-data statute operates largely independently of agency-specific mandates from NIH and NSF.

    Institutions should treat “FAIR” and “open” as related but distinct compliance targets. A dataset can be fully FAIR — persistently identified, well-described, licensed — while remaining access-restricted for legitimate reasons in every jurisdiction covered here. Repository choice and data management plan content should be checked against the specific funder mandate, not a generic FAIR checklist.

  • UK Data Service vs ICPSR: Choosing an Archive

    The UK Data Service and ICPSR are the two largest social-science data archives in the English-speaking research world, and the right choice usually depends on jurisdiction and funder mandate rather than feature parity. The UK Data Service is the ESRC-funded national repository for UK social, economic and population data, while ICPSR is a US-based, membership-funded consortium archive at the University of Michigan. Researchers outside the biomedical repository ecosystem — where PubMed-linked mandates dominate — need to weigh deposit workflow, restricted-access tiers and citation practice before picking either as a home for a dataset.

    The UK Data Service is the largest digital repository for quantitative and qualitative social science and humanities research data in the United Kingdom, formed in October 2012 when the Economic and Social Research Council (ESRC) consolidated the UK Data Archive — established at the University of Essex in 1967 — with several university partners. ICPSR, by contrast, is a membership consortium of academic and research institutions that has archived social and behavioural science data since 1962. Both are listed in re3data.org, the global Registry of Research Data Repositories, and both hold CoreTrustSeal certification for trustworthy digital repositories.

    What Are the UK Data Service and ICPSR?

    The UK Data Service is a national data repository funded through UKRI’s Economic and Social Research Council (ESRC) and led by the UK Data Archive at the University of Essex, in partnership with the University of Manchester, Jisc, EDINA and University College London. It holds more than 6,000 datasets, including UK Census data, the Labour Force Survey, the Millennium Cohort Study and cross-national surveys such as the European Social Survey.

    ICPSR — the Inter-university Consortium for Political and Social Research — is a membership-funded archive based at the University of Michigan, serving several hundred member institutions worldwide alongside non-member depositors and users. Its holdings span large-scale US and international surveys, criminal justice, education and ageing data, and it runs openICPSR as a self-publishing companion repository for rapid dissemination.

    How Do Deposit Workflows Compare?

    Both archives run a curated deposit model rather than a bare-metal upload box: staff review documentation, check disclosure risk and enhance metadata before release. The UK Data Service’s ESRC funding creates a contractual hook — grant holders are required to offer their data for archiving as a condition of the ESRC Research Data Policy — which ICPSR’s membership model does not replicate for non-US funders.

    • UK Data Service: two routes — the main curated collection for large, complex or sensitive studies, and ReShare, a lighter self-deposit repository for smaller datasets, code and syntax files.
    • ICPSR: two routes — the standard curated deposit process, and openICPSR, a self-publishing repository for researchers who want faster turnaround with lighter-touch review.

    Depositors submitting to either service should expect a documentation checklist covering variable-level metadata, consent and ethics evidence, and a data management plan — the same categories UKRI and NSF grant terms typically require regardless of which archive receives the deposit.

    How Do Restricted-Access Tiers Differ?

    Access tiering is where the two services diverge most for researchers working with confidential or disclosive social-science data. The UK Data Service operates a published three-tier model; ICPSR uses a comparable but differently named structure built around its Virtual Data Enclave.

    Access dimension UK Data Service ICPSR
    Open tier No registration; Open Government Licence data Public-use files via free MyData account
    Standard tier Safeguarded — registration plus End User Licence Member-institution access under consortium terms
    Restricted tier Controlled — SecureLab, requiring accredited-researcher training under the Five Safes Framework Restricted-use data via secure Virtual Data Enclave or encrypted physical media, subject to a data security plan
    Governance standard Accredited under the Digital Economy Act 2017 by the UK Statistics Authority (2020) Institutional Review Board and data-use-agreement based review

    The UK Data Service’s Five Safes Framework — safe people, projects, settings, data and outputs — was developed with HMRC DataLab and the Office for National Statistics Secure Research Services, and now underpins the SafePod Network launched in 2021 for wider geographical access to sensitive data. ICPSR’s restricted-data pathway achieves an equivalent security outcome through its enclave model but does not use the Five Safes terminology, which matters for UK researchers writing data management plans against ESRC or UKRI templates that reference it explicitly.

    How Do Citation Practices Compare?

    Both archives assign persistent identifiers and expect formal data citation, but their machinery differs. The UK Data Service works with DataCite and the British Library to issue DOIs and promotes an easy-to-use citation tool, framing its approach around the FAIR data principles — Findable, Accessible, Interoperable, Reusable — and its open-source QAMyData tool, which gives depositors a health check for numeric data before release.

    ICPSR similarly issues persistent identifiers for deposited studies and expects citation in publications that reuse its data, but its emphasis sits more on bibliography-style study citations tied to its own numbering system than on a dedicated public FAIR-compliance tool. For researchers publishing in journals that enforce data-availability statements — a growing requirement under funder open-science mandates — the practical difference is smaller than the access-tier gap: both produce a citable, resolvable record, but only the UK Data Service publishes a named QA tool for pre-citation data quality.

    Which Archive Should Researchers Outside Biomedicine Choose?

    For most projects the decision is jurisdictional rather than qualitative. A research data repository choice driven by funder mandate removes ambiguity immediately: ESRC-funded UK researchers must offer data to the UK Data Service, while NSF- or NIH-adjacent US social-science grants more commonly point toward ICPSR or openICPSR.

    • Choose the UK Data Service if your funder is UKRI/ESRC, your data concerns UK administrative, census or longitudinal panel data, or you need SecureLab/Five Safes access to controlled government microdata.
    • Choose ICPSR if your institution is a consortium member, your data is US-focused or cross-national with US partners, or you want the faster openICPSR self-publishing route.
    • Consult both catalogues before depositing internationally comparable survey data (e.g. European Social Survey, Eurobarometer) — coverage overlaps, and the UK Data Service can facilitate UK-based access to ICPSR holdings.

    Institutions building or reviewing a data management plan should treat this as a data repository for research compliance question first and a discoverability question second: a technically excellent dataset deposited in the wrong repository for its funder mandate creates avoidable rework at grant closeout.

    Answer-First Questions Researchers Ask

    What Is the UK Data Service?

    The UK Data Service is the ESRC-funded national repository for UK economic, population and social research data, led by the UK Data Archive at the University of Essex. It holds over 6,000 datasets, including census, survey and longitudinal study data, and operates under the OAIS digital-preservation reference model.

    How Do You Access Data on the UK Data Service?

    Access runs through three published tiers: Open data requiring no registration, Safeguarded data requiring registration and an End User Licence, and Controlled data requiring SecureLab accreditation under the Five Safes Framework. Most researchers start with the free data catalogue and register once they identify a specific study.

    Is the UK Data Service Free?

    Yes — the service is free to data owners depositing studies and free at the point of use for non-commercial research and teaching. Commercial users may incur administrative fees, and controlled-tier access requires accredited-researcher training rather than a monetary charge.

    Implications for Research Administrators

    Data management plans reviewed by institutional research offices, ARMA and INORMS-aligned research administrators, and funder compliance teams increasingly treat repository choice as an auditable field, not a footnote. A UK-funded study archived outside the UK Data Service without documented justification can trigger ESRC compliance queries at final reporting; a US consortium study left undeposited with ICPSR can weaken an institution’s case for renewed membership funding. Neither archive competes with domain-specific biomedical repositories governed by NISO, ICMJE or COPE norms — this comparison sits squarely in the national data repository space for social science, distinct from that ecosystem.

    As open-science mandates from UKRI, cOAlition S and equivalent US funders converge on FAIR-by-default expectations, the operational gap between the UK Data Service and ICPSR is narrowing to jurisdiction, access-tier terminology and citation tooling rather than underlying trustworthiness — both hold CoreTrustSeal certification and both sit inside the CESSDA/re3data recognised-repository landscape that funders now check by default.