Skip to main content
v2026.11,986 entries · CC-BY 4.0

medspaCy Clinical NLP Toolkit in Python: ConText Negation & Temporality, Clinical Sectionizer, TargetMatcher, and EHR Workflows

A practical engineering and research guide to medspaCy, the open-source Python clinical Natural Language Processing (cNLP) library built on spaCy, detailing PyRuSH sentence boundary detection, TargetMatcher rule extraction, ConText negation/temporality/experiencer assertion, clinical sectionization, and EHR-to-OMOP/FHIR pipelines.

Written and maintained by CASRAI Editorial Board

Published

On this page

medspaCy is an open-source Python library for clinical Natural Language Processing (cNLP) developed by biomedical informatics researchers at the University of Utah and the U.S. Department of Veterans Affairs (VA) Informatics and Computing Infrastructure (VINCI). Built directly on top of the industrial-strength spaCy architecture (spacy.Language), medspaCy provides modular, deterministic, and explainable pipeline components engineered specifically for unstructured Electronic Health Record (EHR) clinical notes—solving the core clinical text challenges where general-domain NLP models fail: telegraphic sentence boundaries (PyRuSH), clinical header scoping (Sectionizer), rule- and UMLS-based concept extraction (TargetMatcher / QuickUMLS), and contextual negation, temporality, uncertainty, and family-history assertion (ConText).

Ask CASRAI · free to tryQuestion about this guide? Ask CASRAI answers with linked sources. 2 free questions a day, no account.

Ask your first question freeSubscribe

What Is medspaCy and How Is It Used in Clinical Research and Healthcare?

When clinical researchers, computational phenotyping teams, or AI medical coding engineers extract diagnoses and symptoms from narrative SOAP note Subjective and Assessment sections, naive keyword or dictionary matching produces unacceptably high false-positive rates. In clinical prose, a mention of “pneumonia” or “colon cancer” is frequently negated (“chest X-ray negative for pneumonia”), historical (“history of childhood pneumonia”), uncertain (“rule out pulmonary embolism vs. pneumonia”), hypothetical (“return to ED if fever or signs of pneumonia develop”), or experienced by a family member rather than the patient (“Father died of colon cancer at age 58”).

Standard spaCy pipelines trained on web news (such as en_core_web_sm) mis-split sentences at clinical abbreviations (e.g., Pt., Dr., q.i.d., 38.5 C.) and lack native clinical assertion logic. medspaCy bridges this gap by registering five specialized spaCy pipeline components that read and write custom underscore attributes on spaCy Doc, Span, and Token objects:

medspaCy Component Name Underlying Algorithm / Module Clinical NLP Function & spaCy Extension Attributes
1. medspacy_pyrush PyRuSH (Python Rule-based Sentence Segmenter using Hashing) Replaces standard punctuation sentence splitters with a fast hash-table state machine trained on clinical notes so vital signs, numbered problem lists, and medical shorthand do not fracture sentence boundaries (critical because ConText operates within sentence bounds).
2. medspacy_target_matcher TargetMatcher & TargetRule (plus optional QuickUMLS) Extracts clinical target entities into doc.ents using case-insensitive literal strings, regular expressions, or spaCy token-attribute patterns, assigning custom ICD-10-CM, SNOMED CT, or UMLS Concept Unique Identifiers (CUIs) via TargetRule.metadata.
3. medspacy_context ConText Algorithm (Harkema et al., 2009 extension of Chapman’s NegEx) Scans forward and backward within the sentence from modifier trigger phrases (ConTextRule) to set boolean assertion flags on each extracted entity: ent._.is_negated, ent._.is_historical, ent._.is_uncertain, ent._.is_hypothetical, and ent._.is_family.
4. medspacy_sectionizer Sectionizer & SectionRule Detects clinical note headers (e.g., Chief Complaint:, HPI:, PMH:, Family History:, Allergies:, Assessment/Plan:) and tags every entity with ent._.section_category and ent._.section_title.
5. medspacy_postprocessor Postprocessor & PostprocessingRule Applies deterministic post-extraction business logic—for example, automatically setting ent._.is_family = True if an entity appears inside a family_history section, or removing entities matched inside a medication allergy list.

How to Build a Clinical NLP Pipeline with medspaCy in Python (Step-by-Step Code)

Below is a complete, reproducible Python implementation showing how to install medspacy, define custom TargetRule, ConTextRule, and SectionRule patterns, and extract structured clinical phenotypes from an unstructured EHR progress note:

# Install in a Python 3.9–3.12 virtual environment:
# pip install medspacy

import medspacy
from medspacy.ner import TargetRule
from medspacy.context import ConTextRule
from medspacy.section_detection import SectionRule
from medspacy.visualization import visualize_ent, visualize_dep

# 1. Load default medspaCy pipeline (tokenizer, pyrush, target_matcher, context)
nlp = medspacy.load()

# Add the clinical Sectionizer component to the pipeline
sectionizer = nlp.add_pipe("medspacy_sectionizer")
sectionizer.add([
    SectionRule("HISTORY OF PRESENT ILLNESS:", "hpi"),
    SectionRule("PAST MEDICAL HISTORY:", "pmh"),
    SectionRule("FAMILY HISTORY:", "family_history"),
    SectionRule("ASSESSMENT AND PLAN:", "assessment_plan"),
])

# 2. Register clinical concepts with TargetMatcher (mapping to ICD-10 / SNOMED)
target_matcher = nlp.get_pipe("medspacy_target_matcher")
target_matcher.add([
    TargetRule("pneumonia", "CONDITION", attributes={"icd10": "J18.9"}),
    TargetRule("community-acquired pneumonia", "CONDITION", attributes={"icd10": "J18.9"}),
    TargetRule("pulmonary embolism", "CONDITION", attributes={"icd10": "I26.99"}),
    TargetRule("PE", "CONDITION", pattern=[{"LOWER": "pe"}], attributes={"icd10": "I26.99"}),
    TargetRule("type 2 diabetes", "CONDITION", attributes={"icd10": "E11.9"}),
    TargetRule("coronary artery disease", "CONDITION", attributes={"icd10": "I25.10"}),
])

# 3. Inspect or extend ConText modifier rules if needed
context = nlp.get_pipe("medspacy_context")
context.add([
    ConTextRule("ruled out for", "NEGATED_EXISTENCE", direction="FORWARD"),
])

# 4. Run the pipeline on a multi-section clinical note
clinical_note = """
HISTORY OF PRESENT ILLNESS:
67 y.o. male presents with fever and productive cough. CTA chest ruled out for pulmonary embolism.

PAST MEDICAL HISTORY:
History of type 2 diabetes.

FAMILY HISTORY:
Father had coronary artery disease at age 54.

ASSESSMENT AND PLAN:
1. Community-acquired pneumonia - start empiric ceftriaxone and azithromycin.
Return to ED immediately if chest pain or signs of PE develop.
"""

doc = nlp(clinical_note)

# 5. Iterate over extracted entities and evaluate assertion & section context
for ent in doc.ents:
    print({
        "text": ent.text,
        "label": ent.label_,
        "section": ent._.section_category,
        "is_negated": ent._.is_negated,
        "is_historical": ent._.is_historical,
        "is_uncertain": ent._.is_uncertain,
        "is_hypothetical": ent._.is_hypothetical,
        "is_family": ent._.is_family,
        "is_active_patient_condition": not any([
            ent._.is_negated,
            ent._.is_uncertain,
            ent._.is_hypothetical,
            ent._.is_family,
        ]),
    })

In the example above, medspaCy accurately distinguishes the four mentions: “pulmonary embolism” in the HPI is flagged is_negated = True; “type 2 diabetes” in the PMH is flagged is_historical = True within section pmh; “coronary artery disease” is flagged is_family = True within section family_history; “PE” in the discharge precautions is flagged is_hypothetical = True; and only “Community-acquired pneumonia” in the Assessment and Plan is asserted as an active, present diagnosis.

How ConText Negation and Scope Termination Work Under the Hood

The medspacy_context component implements the ConText algorithm using four distinct rule classes stored in its knowledge base (default_context_rules.json):

  1. Forward Modifiers (direction="FORWARD"): Trigger phrases such as “no evidence of”, “denies”, “negative for”, “rule out”, or “history of” modify all target clinical entities that appear after the trigger within the same sentence, until a termination token is reached.
  2. Backward Modifiers (direction="BACKWARD"): Post-nominal phrases such as “was ruled out”, “is unlikely”, or “not seen” modify target entities appearing before the trigger in the sentence (e.g., “Pulmonary embolism and pneumothorax were ruled out”).
  3. Bidirectional Modifiers (direction="BIDIRECTIONAL"): Apply across both sides of the trigger within the active scope window.
  4. Pseudo-Triggers (direction="PSEUDO") and Terminators (direction="TERMINATE"):
    • Pseudo-triggers prevent false negations on phrases that contain a negation substring but mean the opposite (for example, “no change in”, “not ruled out”, or “gram negative”).
    • Conjunction Terminators (such as “but”, “however”, “except”, “although”, and “secondary to”) halt the forward scope of a negation modifier mid-sentence. For instance, in the sentence “The patient denies chest pain or hemoptysis but complains of severe dyspnea,” the terminator “but” stops the scope of “denies” so that “chest pain” and “hemoptysis” are marked is_negated = True while “dyspnea” remains positive (is_negated = False).

Comparing medspaCy with scispaCy, Apache cTAKES, and ClinicalBERT

Health informatics architects frequently combine deterministic rule engines like medspaCy with neural models such as ClinicalBERT or BioGPT depending on regulatory explainability requirements and hardware constraints:

Evaluation Dimension medspaCy scispaCy (AllenAI) Apache cTAKES ClinicalBERT / Bio_ClinicalBERT
Primary Text Domain EHR Clinical Notes (H&Ps, SOAP notes, radiology/pathology reports, VA VINCI notes). Biomedical Literature (PubMed abstracts, PMC full-text scientific articles). EHR Clinical Notes (Java UIMA pipeline originally built at Mayo Clinic). EHR Clinical Notes (pre-trained on MIMIC-III ICU notes).
Negation, Family & Section Logic Native, built-in ConText, Sectionizer, and PyRuSH. Not built in (requires pairing with external `negspacy` or `medspacy_context` pipes). Built-in NegEx / Assertion annotator and clinical section segmenter in Java. Requires supervised fine-tuning on labeled assertion datasets (e.g., i2b2 2010).
Auditability & Explainability 100% deterministic and inspectable (`ent._.context_graph` shows exact trigger rule that negated an entity). Statistical spaCy NER + UMLS k-NN character 3-gram linker. Rule-based + statistical machine learning hybrid. Neural transformer attention weights (requires post-hoc explainability).
Compute & Deployment Footprint Lightweight CPU execution (thousands of notes/minute on standard CPU cores; zero GPU required). CPU execution (moderate RAM footprint when loading full UMLS linker). Heavyweight JVM memory footprint and XML descriptor configuration. GPU recommended for large-scale batch inference; 512-subword token context limit.

How Does medspaCy Integrate with EHR and Health Informatics Workflows?

In enterprise clinical data warehouses and i2b2 / OMOP research repositories, medspaCy is deployed across three standard health informatics workflows:

  • Populating the OMOP CDM NOTE_NLP Table: Using the companion medspacy_io utilities, each extracted entity is written directly into the OHDSI OMOP Common Data Model NOTE_NLP table—mapping the target concept to note_nlp_concept_id, the section header to section_concept_id, and the ConText boolean flags (is_negated, is_historical, is_family) to the standardized OMOP term_exists, term_temporal, and term_modifiers columns.
  • FHIR DocumentReference to Condition / Observation Extraction: When ingesting unstructured clinical notes via FHIR DocumentReference, a medspaCy microservice parses the decoded note text and generates structured FHIR Condition resources (`verificationStatus = confirmed` vs. `refuted` when `is_negated == True`) or FamilyMemberHistory resources (when `is_family == True`).
  • Hybrid Rule + LLM Guardrails: Under ONC HTI-1 (`45 CFR § 170.315(b)(11)`) and HIPAA minimum-necessary governance, health systems run deterministic medspaCy pipelines on premise (inside the hospital firewall with zero external cloud egress) either to pre-filter cohorts before LLM summarization or to verify that an ambient AI scribe did not invert a negated symptom into a positive diagnosis.

Frequently Asked Questions

What is medspaCy and how is it used in clinical research and healthcare?

medspaCy is an open-source Python clinical Natural Language Processing (cNLP) toolkit built on the spaCy framework by researchers at the University of Utah and VA VINCI. It is used in clinical research and healthcare to extract diagnoses, symptoms, and medications from unstructured EHR clinical notes while accurately handling clinical sentence segmentation (PyRuSH), note section headers (Sectionizer), and negation, historical temporality, uncertainty, and family history assertions (ConText).

What standards or regulatory requirements govern medspaCy?

When deployed on Protected Health Information (PHI) in U.S. health systems, medspaCy pipelines operate under the HIPAA Security and Privacy Rules (45 CFR § 164.312 and § 164.514), map clinical sections and concepts to HL7 LOINC document section codes, SNOMED CT, ICD-10-CM, RxNorm, and UMLS CUIs, and populate standardized research schemas such as the OHDSI OMOP Common Data Model NOTE_NLP table and HL7 FHIR Condition and Observation resources.

How does medspaCy integrate with EHR and health informatics workflows?

medspaCy integrates into EHR and research workflows by ingesting raw clinical notes from enterprise data warehouses (such as Epic Clarity/Caboodle or FHIR DocumentReference feeds), executing deterministic PyRuSH + Sectionizer + TargetMatcher + ConText pipelines on local CPU servers without external cloud PHI exposure, and exporting assertion-tagged clinical entities into OMOP NOTE_NLP tables, i2b2 observation facts, or computer-assisted coding (CAC) queues.

Follow CASRAI

Research-administration guidance, standards updates and independent tool reviews.

Ask CASRAI · free to try

Ask about medspaCy Clinical NLP Toolkit in Python: ConText Negation & Temporality, Clinical Sectionizer, TargetMatcher, and EHR Workflows

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →