Written and maintained by CASRAI Editorial Board
Published
On this page
Evaluating electronic health record data quality is a prerequisite for any secondary clinical research, real-world evidence (RWE) study, or clinical artificial intelligence model training pipeline. Unlike prospective clinical trial Electronic Data Capture (EDC) systems—where protocol-mandated data points are collected at fixed schedule intervals under Good Clinical Practice (GCP) monitoring—EHR data are recorded opportunistically as a byproduct of routine clinical care, hospital billing, and medico-legal documentation. Consequently, biomedical informaticists rely on formal data quality frameworks—most notably Weiskopf & Weng (2013) and Kahn et al. (2016)—to quantify data fitness for use before statistical modeling.
What Are the 5 Dimensions of EHR Data Quality in Weiskopf and Weng (2013)?
In their foundational systematic review published in the Journal of the American Medical Informatics Association (JAMIA) (2013), Nicole G. Weiskopf and Chunhua Weng synthesized the biomedical informatics literature into five core dimensions of EHR data quality and seven empirical assessment methods:
| Weiskopf & Weng Dimension | Formal Clinical Informatics Definition | Empirical Assessment Methods & EHR Example |
|---|---|---|
| 1. Completeness | Whether a truth about a patient that exists in clinical reality is actually present in the EHR repository (Fundamental dimension). | Measured across four operational definitions (Weiskopf et al., 2013): Documentation (vital sign recorded at visit), Breadth (all 5 core data types present), Density (longitudinal visit frequency), and Predictive completeness. |
| 2. Correctness (Accuracy) | Whether an element recorded in the EHR is true and free of error relative to the patient’s actual clinical state (Fundamental dimension). | Assessed via Gold Standard Comparison (manual clinician chart abstraction or pathology biopsy confirmation vs. ICD-10 billing codes). |
| 3. Currency (Timeliness) | Whether a recorded EHR observation is a representative reflection of the patient’s state at the relevant point in time (Fundamental dimension). | Assessed via Log Review and timestamp lag analysis (e.g., verifying that problem list diagnoses or active medication lists were reconciled at the index encounter rather than copied forward from years prior). |
| 4. Concordance | Whether there is structural and clinical agreement between two or more elements within the EHR or across separate data sources (Proxy dimension). | Assessed via Data Element Agreement (e.g., patients with an ICD-10 code for Type 1 Diabetes also having insulin orders in RxNorm) or Data Source Agreement (EHR vs. payer claims). |
| 5. Plausibility | Whether an EHR value makes biological and clinical sense given other medical knowledge about the patient (Proxy dimension). | Assessed via Validity Checks and Distribution Comparison (e.g., flagging adult body height of 450 cm, serum pH of 2.1, or prostate cancer diagnoses in female patients). |
Crucially, Weiskopf and Weng demonstrated that Completeness, Correctness, and Currency are the three irreducible, fundamental properties of EHR data quality, whereas Concordance and Plausibility serve as computable methodological proxies when an external gold standard is unavailable.
How Does Kahn et al. (2016) Classify Conformance, Completeness, and Plausibility?
To standardize automated data quality checking across multi-site distributed research networks—including PCORnet, the NIH All of Us Research Program, and the OMOP Common Data Model—Michael G. Kahn and colleagues (eGEMs, 2016) developed the Harmonized Data Quality Assessment Terminology and Framework. The Kahn framework crosses three intrinsic data quality categories with two evaluation contexts:
- Category 1 — Conformance (Structural & Syntactic Compliance): Evaluates whether data values adhere to internal or external formatting, typing, and relational specifications:
- Value Conformance: Data types, field lengths, and allowed terminology bindings (e.g., `gender_concept_id` belongs to the OMOP Gender domain).
- Relational Conformance: Primary key uniqueness and foreign key referential integrity (e.g., every `person_id` in `CONDITION_OCCURRENCE` exists in the `PERSON` table).
- Computational Conformance: Calculated fields match their deterministic formula (e.g., calculated BMI matches $text{weight (kg)} / text{height (m)}^2$).
- Category 2 — Completeness (Absence of Missing Values): Evaluates the frequency of missing or null data attributes at a single time point (atemporal completeness) or across longitudinal follow-up windows (temporal completeness), without assuming why the value is absent.
- Category 3 — Plausibility (Clinical Believability & Truthfulness): Evaluates whether recorded data values are credible:
- Uniqueness Plausibility: Absence of duplicated clinical records or redundant entity keys.
- Atemporal Plausibility: Biological range checks and cross-variable clinical consistency (e.g., systolic blood pressure must exceed diastolic blood pressure; numeric lab values fall within physiological bounds).
- Temporal Plausibility: Chronological sequence logic and longitudinal stability (e.g., clinical encounters cannot occur before `birth_datetime` or after `death_datetime`; seasonal disease spikes follow expected secular trends).
- Verification vs. Validation Contexts: Every check in the Kahn 3×2 matrix is executed either as Verification (evaluating internal consistency against local schema rules and historical distributions) or Validation (evaluating alignment against external benchmarks, published epidemiological prevalence, or multi-site network distributions). In the OHDSI ecosystem, the open-source R package
DataQualityDashboard(DQD) executes more than 3,300 automated SQL checks mapped directly to Kahn’s Conformance, Completeness, and Plausibility matrix, complementing broader regulatory guidance such as the EMA/HMA Real-World Data Quality Framework (RW-DQF) and general research data quality frameworks.
Why Is EHR Missingness Informative (MNAR) Rather Than Missing Completely at Random?
The single greatest statistical hazard in secondary EHR analysis is treating missing clinical data as Missing Completely at Random (MCAR) or Missing at Random (MAR) (see MCAR, MAR, and MNAR). In an EHR, the clinical decision to order a laboratory test, record a vital sign, or schedule an encounter is driven directly by the patient’s underlying illness severity and clinician suspicion—a phenomenon termed informative presence or informative observation (Goldstein et al., JAMIA 2016; Agniel et al., BMJ 2018):
- Informative Missingness (MNAR) in Laboratory Ordering: A healthy outpatient rarely has an arterial blood gas, serum lactate, or cardiac troponin ordered. Consequently, the mere presence of a laboratory order (or the time of day it was drawn, such as 03:00 AM inpatient draws) is often a stronger predictor of 30-day mortality than the physiological numeric value itself (Agniel et al., 2018). Performing naive complete-case analysis (`dropna`) restricts the study cohort to sicker, heavily monitored patients and introduces severe selection bias.
- Absence of Evidence vs. Evidence of Absence (Closed vs. Open Healthcare Systems): In structured EHR diagnosis tables, there is no explicit row for ”patient does not have heart failure.” Informaticists must distinguish between a patient who truly lacks a condition and a patient who suffered care fragmentation (receiving acute care at an outside hospital outside the EHR catchment area). Defining a ”loyalty cohort” (filtering for patients with regular primary care encounters and baseline routine labs inside the health system) reduces false-negative misclassification before running EHR phenotyping algorithms.
- Mitigating Informative Presence in Statistical Models: Rather than blindly imputing missing lab values across the entire population, biostatisticians explicitly adjust for healthcare utilization intensity (such as prior-year outpatient visit count, inpatient days, or total distinct lab orders), include missingness indicator variables when training predictive models (while auditing for shortcut learning), and perform sensitivity analyses under explicit MNAR selection models.
Frequently Asked Questions
What are the 5 dimensions of EHR data quality in Weiskopf and Weng (2013)?
Weiskopf and Weng (2013) defined five core dimensions of Electronic Health Record data quality: Completeness (whether a clinical truth about a patient is recorded in the EHR), Correctness (whether the recorded EHR data are accurate and free of error), Currency (whether the recorded data reflect the patient’s state at the relevant time), Concordance (agreement between related data elements or separate data sources), and Plausibility (whether values are biologically and clinically believable).
How does Kahn et al. (2016) classify Conformance, Completeness, and Plausibility?
Kahn’s Harmonized Data Quality Framework organizes EHR data checks into a 3×2 matrix crossing three categories—Conformance (value, relational, and computational compliance with data model standards), Completeness (atemporal and temporal presence of expected non-null values), and Plausibility (uniqueness, atemporal physiological range checks, and temporal chronological sequences)—against two evaluation contexts: Verification (internal database checks) and Validation (comparison against external gold standards or multi-site benchmarks).
Why is EHR missingness informative (MNAR) rather than Missing Completely at Random?
EHR data are Missing Not At Random (MNAR) because clinicians order diagnostic tests and record observations only when clinically indicated by symptoms or disease severity. As a result, whether a lab test is present or missing—and how frequently a patient interacts with the health system (”informative presence”)—is directly correlated with the patient’s unobserved health status, meaning naive complete-case deletion or standard MAR imputation can severely bias epidemiological estimates.







