Skip to main content
v2026.11,986 entries · CC-BY 4.0
CASRAIAsk CASRAI · AI assistant for researchAsk a research question. Answers come with their sources.Funder rules, compliance, methods and publishing. Answers link the sources they rest on.

Editorial · CASRAI · AI and ML research outputs

Researchers Test LLM Agents for Inducing Clinical Common Data Elements From Abstracts

Researchers tested LLM agents to induce Common Data Elements across 24,588 sarcoma abstracts, achieving 94.9% extraction agreement despite schema hurdles.

Published 16 Nov 2026· 6 minute read

Ask CASRAI · free to try

Ask about this story

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

CASRAI is the reference for research administration — bookmark it for the next question.

On November 13, 2026, clinical informatics researchers from Bern University of Applied Sciences, the University of Geneva, Roche, and partner institutions published a preprint demonstrating automated agentic induction of Common Data Elements (CDEs) from clinical literature. Analyzing 24,588 PubMed sarcoma case report abstracts, the open-weight LLM agent pipeline derived a standardized schema and extracted structured variables with 94.9% agreement, addressing chronic bottlenecks in oncology data curation.

Multi-Agent Schema Induction Across 24,588 Sarcoma Abstracts

The preprint, titled Agentic induction of common data elements and information extraction from sarcoma case report abstracts (arXiv:2611.18506) and authored by Daniel Reichenpfader, Kwaku Appiagyei, Manovah Nadarajah, Ahmed Reda Bahr, Irina Filchenko, and Fabio Dennstädt, evaluates whether multi-agent LLM systems can automatically derive standardized schemas from unstructured literature. In rare cancers such as sarcomas, crucial clinical observations remain scattered across individual case reports that lack shared data dictionaries.

The team retrieved 58,265 PubMed records, of which 42,018 included abstracts. An initial LLM classifier classified 24,588 abstracts as single-patient case reports, filtering out multi-patient series and reviews. From this corpus, an agentic proposal workflow generated 72,909 candidate CDEs. Exact question-type deduplication compressed these proposals to 7,788 unique elements.

Using the open-source AG2 multi-agent framework, the researchers executed two consolidation passes. The first pass reduced the set to 2,183 CDEs, and the second yielded a master schema of 1,570 CDEs (560 numeric and 1,010 categorical). An agentic standardization routine revised 626 of the 1,010 categorical vocabularies (62.0%) and recorded 63 synonym relationships across 39 CDEs. Applying a minimum support threshold of 50 source abstracts yielded 53 retained CDEs (14 numeric and 39 categorical)—representing 3.4% of the consolidated schema—for corpus-wide information extraction.

Extraction Performance and Hardware Architecture

The 53 retained CDEs were deployed in an automated extraction pipeline across all 24,588 abstracts. Extraction succeeded for all 53 elements in 19,482 abstracts (79.2%), including valid responses recording non-mention. For the remaining 5,106 abstracts (20.8%), at least one element could not be extracted, affecting 6,287 of 1,303,164 abstract-CDE pairs (0.5%). Failures stemmed primarily from ungrounded evidence spans not matching source text (60.9%), proposed labels outside the value set (24.3%), non-atomic labels (8.8%), and missing evidence citations (5.9%).

To assess accuracy, the authors evaluated 100 randomly sampled abstracts (5,300 abstract-CDE pairs). Both the human reference and model were empty in 4,184 pairs. In the 1,116 pairs with a non-empty answer, the pipeline matched human reference values in 1,059 pairs, achieving 94.9% exact agreement (bootstrap 95% CI: 93.0% to 96.4%).

Clinical Common Data Element Matches Non-Matches Compared Exact Agreement (%)
Histopathologic diagnosis (biopsy) 50 0 50 100.0%
Tumour recurrence during follow-up 37 0 37 100.0%
Age at diagnosis 73 1 74 98.6%
Patient sex 75 2 77 97.4%
Distant metastases present 44 2 46 95.7%
Surgical resection performed 54 3 57 94.7%
Metastatic tumour location 18 6 24 75.0%
Latency to secondary tumour 7 3 10 70.0%
Primary lesion location 27 12 39 69.2%
Granulocytic-sarcoma anatomical sites 11 5 16 68.8%
All Evaluated CDEs Combined 1,059 57 1,116 94.9%

Agreement was high for demographic and binary clinical variables, but dropped significantly on spatial and temporal fields: primary tumour location fell to 69.2% and latency to secondary tumour to 70.0%.

The workflow ran locally on two Apple Mac Studio systems (512 GB unified memory each) running the open-weight gpt-oss-120b model with embeddinggemma-300m for embeddings, routed via LiteLLM with strict JSON schema validation. Logged token usage reached 21.3 million tokens for screening, 259.0 million for CDE proposal, and 1,159.4 million for extraction, exceeding 1.44 billion generative tokens.

Background in Clinical Common Data Elements

Establishing standardized schemas has long been a manual bottleneck. Initiatives like the NIH Common Data Element Repository and the NCI Cancer Data Standards Registry and Repository (caDSR) define standardized variables to make clinical datasets comparable. Under NIH Notice NOT-OD-20-138, NIH-funded studies are encouraged to adopt validated CDEs.

However, consensus-based CDE definition takes years and often omits rare disease presentations. As research institutions upgrade their clinical data management infrastructure, deriving schemas directly from narrative literature offers a viable alternative. Unlike previous NLP approaches that mapped text to predefined dictionaries, this study evaluated bottom-up schema induction, reflecting the architectural trade-offs between autonomous agents and deterministic pipelines highlighted in Anthropic agent systems research.

Clinical Review Identifies Schema Gaps

Despite high extraction accuracy, independent clinical review revealed significant schema limitations. Two clinical experts assessed the 53 retained CDEs on relevance, clarity, and correctness.

Dimension Reviewer 1 Reviewer 2 Agreement Kappa (κ) Dual Approval
Relevance 48 / 53 (90.6%) 47 / 53 (88.7%) 46 / 53 (86.8%) 0.29 44 / 53 (83.0%)
Clarity 39 / 53 (73.6%) 50 / 53 (94.3%) 36 / 53 (67.9%) -0.10 36 / 53 (67.9%)
Correctness 48 / 53 (90.6%) 30 / 53 (56.6%) 35 / 53 (66.0%) 0.24 30 / 53 (56.6%)

Only 19 of 53 CDEs (35.8%) received dual approval across all three criteria. Inter-rater reliability was low (kappa between -0.10 and 0.29), reflecting differing clinical expectations. Reviewers criticized ambiguous value sets—such as mixing metric dimensions with subjective labels like ‘Large’—and persistent redundancy. The retained set contained three separate age variables (two numeric, one categorical) and duplicate primary location questions. High extraction fidelity inside an induced schema does not ensure the schema itself asks clinically sound questions.

What It Means for Researchers and Research Offices

The findings carry practical implications for health faculties and trial offices:

  • Accelerated Registry Bootstrapping: Research offices establishing a clinical data registry for orphan conditions can draft comprehensive candidate dictionaries rapidly from published case literature.
  • Mandatory Human Curation Gates: Automated induction produces noisy schemas; dual-expert consensus reached only 35.8%. Clinical curators must review schemas before corpus extraction, adhering to standard practices for how to choose a metadata schema for a dataset.
  • Privacy-Preserving Local Infrastructure: Open-weight models on workstation clusters eliminate the need to transmit clinical text to closed commercial cloud APIs.
  • FAIR Data Compliance: Converting unstructured narratives into evidence-linked variables advances FAIR data principles, unlocking legacy publications for systematic evidence synthesis.

What Happens Next in Agentic Data Engineering

The authors identified several prerequisites for production deployment. First, post-induction filters must automate deduplication, unit standardization, and hierarchical categorization before human curation. Second, the pipeline must scale from abstracts to full-text publications and multi-institutional electronic health records. Finally, induced schemas require automated mapping to federated registries such as NCI caDSR and NLM CDE repositories to preserve interoperability across clinical research networks.

Sources

  • arXiv:2611.18506 — Reichenpfader, D., et al. (2026). Agentic induction of common data elements and information extraction from sarcoma case report abstracts. Checked 2026-11-16.
  • NIH Common Data Element Repository — National Library of Medicine, National Institutes of Health. Checked 2026-11-16.
  • NIH Notice NOT-OD-20-138 — National Institutes of Health. NIH Policy on Common Data Elements in NIH-funded Research. Checked 2026-11-16.
  • Anthropic Research — Anthropic. Building Effective Agents: Workflows vs Autonomous Systems. Checked 2026-11-16.
  • AG2 Multi-Agent Framework — AG2 AI. Open-source agent orchestration repository. Checked 2026-11-16.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →