Written and maintained by CASRAI Editorial Board
Last updated
The NIH All of Us Researcher Workbench is the cloud-based Trusted Research Environment (TRE) built by the National Institutes of Health All of Us Research Program—and operated by the Data and Research Center (DRC) at Vanderbilt University Medical Center in partnership with the Broad Institute and Verily—to provide authorized investigators with programmatic access to longitudinal multimodal data from a diverse U.S. cohort of over one million target participants. Built on top of the Terra cloud platform on Google Cloud Platform (GCP), the Researcher Workbench brings computation directly to the data rather than allowing downloads of individual-level participant records.
What Is the NIH All of Us Researcher Workbench and How Does its Cloud Architecture Work?
In traditional biobanks (or file-download repositories governed by dbGaP), approved investigators download terabytes of phenotypic CSV files and genomic VCFs onto local university clusters. By contrast, the All of Us Researcher Workbench operates as a centralized cloud enclave similar in security philosophy to the UK Biobank Research Analysis Platform (UKB-RAP):
- Curated Data Repository (CDR): Approximately once or twice per year, the DRC freezes, cleans, de-identifies, and publishes a versioned snapshot of the cohort in Google BigQuery alongside companion genomic object buckets in Google Cloud Storage (GCS). Because each CDR version is immutable, analyses pinned to a specific CDR release remain reproducible over time.
- Cohort Builder, Dataset Builder, and Concept Sets: A graphical interface that lets researchers define inclusion/exclusion criteria across EHR diagnoses, labs, surveys, physical measurements, and genomic availability, then generates reproducible SQL queries against the underlying BigQuery CDR tables.
- Cloud Analysis Environments: Within a shared project workspace, collaborators spin up on-demand virtual machines or Dataproc Spark clusters running Jupyter Notebooks (Python and R), RStudio, SAS Studio, or batch genomic workflow engines (Cromwell / WDL, Hail, and PLINK).
What Is the Difference Between the Public, Registered, and Controlled Tiers in All of Us?
The program implements a three-tier passport data access model (detailed from a governance perspective in our guide on All of Us Researcher Workbench tiered access) that determines which BigQuery datasets and GCS buckets a workspace can mount:
| Access Tier | Included Data Modalities | De-identification & Date/Geography Rules |
|---|---|---|
| Public Tier (Research Hub Data Browser) | Aggregate summary counts and distributions across EHR conditions, drugs, labs, surveys, and common genomic variants. | Open to the public with no login; counts rounded to multiples of 20 and small cells suppressed. |
| Registered Tier (Researcher Workbench) | Individual-level EHR, Participant-Provided Information (PPI) surveys, baseline physical measurements, and Fitbit wearable summaries. | All dates for a participant shifted backward by a consistent random integer from 1 to 365 days; no genomic data or geographic identifiers. |
| Controlled Tier (Researcher Workbench) | All Registered Tier data plus short-read and long-read Whole-Genome Sequencing (WGS), genotyping arrays, real unshifted dates, and 3-digit ZIP / state geocodes. | Real clinical and wearable timestamps restored; suppressed rare demographic/ICD combinations restored; 3-digit ZIP codes linked to area-level social determinants of health (SDOH). |
How Are EHR, Survey, Fitbit Wearable, and Whole-Genome Data Structured in OMOP on the Workbench?
Inside the Researcher Workbench BigQuery environment, diverse phenotypic streams are harmonized into a unified OMOP Common Data Model schema linked by person_id, supplemented by auxiliary multimodal tables:
1. Electronic Health Record (EHR) Data
Participating Healthcare Provider Organizations (HPOs) and participant-mediated Fast Healthcare Interoperability Resources (FHIR) connections submit clinical histories mapped to standard OMOP tables: condition_occurrence (coded in SNOMED CT with source ICD-9-CM/ICD-10-CM retained in condition_source_concept_id), drug_exposure (RxNorm), measurement (LOINC laboratory tests and baseline study-visit anthropometrics/blood pressure), procedure_occurrence (CPT-4, HCPCS, ICD-10-PCS, SNOMED CT), and visit_occurrence.
2. Participant-Provided Information (PPI) Surveys
Unlike standard hospital OMOP deployments that contain only clinical EHR data, All of Us participants complete baseline and longitudinal self-report surveys (covering overall health, lifestyle, personal and family medical history, healthcare access, and social determinants of health). All survey questions and participant answers are stored in the OMOP observation table (and pre-joined helper view ds_survey) using the custom PPI vocabulary registered in the OMOP concept table, where `observation_source_concept_id` identifies the survey question and `value_source_concept_id` (or `value_as_number` / `value_as_string`) records the participant’s response.
3. Fitbit Wearable Device Streams
Through the Bring-Your-Own-Device (BYOD) and program-provided wearable initiatives, longitudinal consumer wearable data are exposed in four dedicated BigQuery tables linked by person_id:
activity_summary: Daily calorie burn, elevation, floors, sedentary minutes, lightly/fairly/very active minutes, and total daily steps.steps_intraday: Minute-level step count time series.heart_rate_summary&heart_rate_minute_level: Daily heart rate zone summaries (Out of Range, Fat Burn, Cardio, Peak) and minute-level beats-per-minute recordings.sleep_daily_summary&sleep_level: Daily sleep efficiency/duration metrics and stage-level intervals (wake, light, deep, REM).
4. Genomic Data in the Controlled Tier
In the Controlled Tier, genomic data generated by the All of Us Genome Centers (including Illumina Global Diversity Array genotyping, ≥30x short-read Whole-Genome Sequencing, and PacBio long-read sequencing subsets) are stored in Google Cloud Storage buckets accessible directly from Workbench environments:
- Hail MatrixTables (
.mt): Pre-indexed distributed variant matrices queried across Dataproc Spark clusters using Python and Hail for genome-wide association studies (GWAS) and rare-variant burden testing. - VCF, BGEN, and PLINK 1.9 / 2.0 files: Chromosome-split variant callsets and smaller callsets pre-filtered by allele count/frequency (such as the ACAF threshold callset and exonic regions callset) to reduce cloud compute cost for common-variant analyses, alongside pre-computed genetic ancestry principal components and relatedness kinship tables.
Frequently Asked Questions
Why are EHR and survey records stored in the same OMOP database in All of Us?
Storing both HPO-submitted EHR clinical records and participant-completed PPI surveys inside the OMOP CDM schema allows researchers to query both data sources with a single SQL join on person_id—and to distinguish EHR-derived records from self-reported survey answers by checking the observation_type_concept_id or joining against the `concept` table where `vocabulary_id = ‘PPI’`.
What is the All of Us Data and Statistics Dissemination Policy for small cell sizes?
To prevent participant re-identification in published manuscripts, posters, or external presentations, the All of Us Data and Statistics Dissemination Policy prohibits publishing any participant count (or summary statistic derived from a subgroup) of 1 to 19 participants (including marginal row or column totals that would allow a reader to deduce a cell count of <20 by subtraction). Counts of zero (`0`) are permitted when no re-identification risk exists, and small non-zero cells must be reported as `<20`.
How do researchers minimize Google Cloud compute charges on the Researcher Workbench?
Because BigQuery queries are billed by bytes scanned and cloud VMs/Dataproc clusters accrue per-hour compute charges, best practices include filtering cohorts in BigQuery before pulling dataframes into memory, using the pre-filtered ACAF or Exome PLINK/BGEN callsets rather than the full multi-terabyte Hail MatrixTable when analyzing common variants (`MAF > 1%` or allele count `> 100`), and configuring auto-pause timeouts on idle Jupyter and Dataproc instances.
