Written and maintained by CASRAI Editorial Board
Last updated
Health IT standards architects, EHR interoperability engineers, and clinical research informatics leads (participating in HL7 Vulcan, Da Vinci, or US Core tracks) use this guide to navigate HL7 FHIR Connectathons, CMS HL7 FHIR Connectathons, and HL7 FHIR DevDays, satisfy FHIR Maturity Model (FMM) progression criteria, and execute automated conformance testing with AEGIS Touchstone and ONC Inferno.
What Happens at an HL7 or CMS FHIR Connectathon?
An HL7 FHIR Connectathon is a multi-day, hands-on technical interoperability testing event organized by Health Level Seven International (HL7) where software developers, EHR vendors, payer engineers, clinical research sponsors, and federal agencies test draft and published FHIR specifications against live reference servers and peer implementations in real time. Rather than passive lecture tracks, a Connectathon is structured around 25 to 40 parallel Testing Tracks (each governed by a formal Track Proposal on HL7 Confluence and led by designated Track Leads).
Across the annual calendar (including the 2022, 2023, 2025, and 2026 cycles), the FHIR developer community convenes across three distinct event formats:
| Event Format | Cadence & Organizer | Primary Technical Focus & Track Scope | Typical Output & Artifacts |
|---|---|---|---|
| HL7 WGM FHIR Connectathon | Tri-annual (January, May, and September), held the weekend immediately preceding each HL7 Working Group Meeting (WGM) | Broad standards maturity testing across all HL7 FHIR Accelerators (Argonaut, Da Vinci, Vulcan clinical research, Gravity, CodeX, FAST) and core FHIR R4/R5/R6 resources | Track report-outs, Jira ballot tracker tickets (jira.hl7.org), and formal FMM advancement evidence |
| CMS HL7 FHIR Connectathon | Annual summer event co-hosted by the Centers for Medicare & Medicaid Services (CMS) and HL7 International | Regulatory implementation testing for CMS rules (`CMS-9115-F` and `CMS-0057-F`): Patient Access API, Provider Access API, Payer-to-Payer API, and Prior Authorization (CRD, DTR, PAS) | Payer-provider sandbox validation, reference implementation demos, and CMS enforcement readiness metrics |
| HL7 FHIR DevDays | Annual international developer conference co-organized by Firely and HL7 International | Developer education, architectural deep-dives, “Let’s Build!” hands-on coding tutorials, and community Startup/Student tracks alongside informal hackathons | Open-source reference code, tutorial slide decks, and validator tooling updates |
How Do FHIR Connectathons Advance Implementation Guides Through the FHIR Maturity Model?
HL7 governs the stability of every FHIR resource (`StructureDefinition`) and Implementation Guide (IG) using the FHIR Maturity Model (FMM 0 through FMM 5, culminating in Normative status), modeled on the Capability Maturity Model (CMM). Formal HL7 FHIR Connectathon testing is a mandatory gate for advancing an artifact along this maturity ladder:
- FMM 0 (Draft): The artifact has been published on the current build server (`build.fhir.org`) with no formal maturity guarantees.
- FMM 1 (Build Quality): The responsible HL7 Work Group considers the artifact substantially complete and ready for implementation testing, and it compiles with zero warnings in the FHIR IG Publisher.
- FMM 2 (Connectathon Tested): The artifact has been tested at at least one HL7-managed FHIR Connectathon by at least three independently developed systems exercising at least 80% of the core data elements across the defined exchange scenarios, with issues logged and resolved.
- FMM 3 (Work Group Verified & Balloted): The artifact has passed formal HL7 Standard for Trial Use (STU) ballot reconciliation and has been verified by the sponsoring Work Group.
- FMM 4 (Published & Production-Prototyped): The artifact is tested across multiple Connectathons, published in a formal release, and implemented in multiple prototype/pilot projects.
- FMM 5 (Production Deployed) & Normative: The artifact has been deployed in at least five independent production systems across more than one country/regime and passes an HL7 Normative ballot, locking backward-compatibility rules.
How Are Touchstone and Inferno Used to Test FHIR Servers During Connectathons?
To ensure reproducible, machine-verifiable results during HL7 and CMS Connectathons, participants execute automated conformance suites against their endpoints using two primary test harnesses:
1. AEGIS Touchstone and FHIR TestScript Resources
AEGIS Touchstone is the cloud-based continuous interoperability testing platform widely used by the HL7 Da Vinci Project, CARIN Alliance, and FAST accelerators during Connectathons:
- Native FHIR
TestScriptExecution: Touchstone executes standard HL7 FHIRTestScriptresources (`setup`, `test`, `teardown` operations and `assert` evaluations) against registered client and server endpoints. - Peer-to-Peer Proxy Inspection: When System A (e.g., an EHR SMART on FHIR client) exchanges payloads with System B (e.g., a payer PAS server), traffic routes through the Touchstone proxy, which validates HTTP headers, OAuth 2.0 scopes, `CapabilityStatement` declarations, and US Core FHIR Profile `StructureDefinition` invariants in real time.
- Org-Level Conformance Matrices: Track leads monitor visual heatmaps showing pass/fail percentages across every participating organization and test script.
2. ONC Inferno Framework
Inferno (`inferno.healthit.gov`), developed by MITRE and stewarded by the Assistant Secretary for Technology Policy / Office of the National Coordinator for Health Information Technology (ASTP/ONC), is an open-source Ruby testing framework used both for regulatory certification and Connectathon track validation:
- ONC `(g)(10)` Standardized API Test Kit: Validates SMART App Launch (v1.0.0 and v2.0.0), Bulk Data Access (`$export`), and US Core STU 3.1.1 / STU 6.1.0 / STU 7.0.0 resource read/search interactions required under the ONC HTI-1 Final Rule (45 CFR § 170.315(g)(10)).
- Accelerator Test Kits: Provides modular, locally runnable Docker test suites for Da Vinci Clinical Data Exchange (CDex), Burden Reduction (CRD/DTR/PAS), Plan-Net, and HL7 Vulcan clinical research tracks (such as mapping EHR data to electronic case report forms via FHIR Questionnaire and SDC workflows or querying HL7 FHIR Resources).
How Clinical Research Teams Participate (HL7 Vulcan Accelerator Tracks)
Academic medical centers, pharmaceutical sponsors, and contract research organizations (CROs) participate in HL7 FHIR Connectathons primarily through the HL7 Vulcan FHIR Accelerator tracks, testing Real-World Data (RWD) retrieval, Schedule of Activities (SoA) representation (`ResearchStudy` and `PlanDefinition`), Adverse Event reporting (`AdverseEvent` to FDA FAERS/E2B(R3)), and phenotypic cohort discovery.
Frequently Asked Questions
What happens at an HL7 or CMS FHIR Connectathon?
At an HL7 or CMS FHIR Connectathon, software engineers from EHR vendors, health systems, payers, and research institutions join topic-specific Testing Tracks over 2 to 3 days to connect live FHIR client and server implementations, exchange sample patient payloads, log specification defects in HL7 Jira, and present interoperability demonstrations.
How do FHIR Connectathons advance Implementation Guides through the FHIR Maturity Model?
Under the HL7 FHIR Maturity Model (FMM), an artifact cannot advance to FMM 2 until it has been successfully tested at an HL7-managed FHIR Connectathon by at least three independently developed systems covering at least 80% of its core elements, providing the empirical feedback required before STU ballot reconciliation (FMM 3) and production maturity (FMM 4–5).
How are Touchstone and Inferno used to test FHIR servers during Connectathons?
Participants register their FHIR server base URLs in AEGIS Touchstone or run ONC Inferno Test Kits to execute automated FHIR TestScript assertions—verifying SMART on FHIR OAuth 2.0 authentication, HTTP status codes, search parameters, and US Core or Da Vinci StructureDefinition profile conformance.
Sources
- HL7 International. FHIR Connectathons Confluence Portal & FHIR Maturity Model (FMM). https://hl7.org/fhir/versions.html#maturity (2026-11-09).
- Centers for Medicare & Medicaid Services (CMS). CMS Interoperability and HL7 FHIR Connectathon Overview. https://www.cms.gov/priorities/key-initiatives/burden-reduction/interoperability (2026-11-09).
- Assistant Secretary for Technology Policy / ONC. Inferno Health IT Testing Framework. https://inferno.healthit.gov/ (2026-11-09).
