Choosing an Implementation Science Framework: Process, Determinant, or Evaluation

The question “which implementation science framework should I use?” is usually the wrong question. Frameworks are not competitors for the same job. Per Nilsen’s 2015 taxonomy — still the field’s standard map — theories, models and frameworks in implementation science serve three overarching aims: “describing and/or guiding the process of translating research into practice (process models); understanding and/or explaining what influences implementation outcomes (determinant frameworks, classic theories, implementation theories); and evaluating implementation (evaluation frameworks).”

Those are three different jobs. A well-specified project normally needs at least two of them, and the frameworks that get named in a protocol should map onto the questions the protocol actually asks. The failure mode is not picking the “wrong” framework — it is picking one, naming it in the background section, and never letting it touch the data collection plan.

Start from the question, not from the framework

Before you compare CFIR with RE-AIM, write down which of these you are actually being asked to answer:

• “What do we do first, and then what?” — you need a process model. It gives you an ordered set of stages and activities.
• “Why is this not sticking here?” — you need a determinant framework. It gives you a checklist of contextual factors to assess as barriers and enablers.
• “Did implementation succeed, and on what dimensions?” — you need an evaluation framework. It gives you the outcome constructs you will report against.
• “Why would this strategy change behaviour at all?” — you need a classic theory or an implementation theory. These are the two categories in Nilsen’s taxonomy that most projects skip, and skipping them is why so many implementation reports can describe what happened but not why.

Nilsen is explicit about the limit of the descriptive categories: “Frameworks do not provide explanations; they only describe empirical phenomena by fitting them into a set of categories,” and “neither models nor frameworks specify the mechanisms of change; they are typically more like checklists of factors relevant to various aspects of implementation.” If your grant reviewer asks for a mechanism and you answer with a determinant framework, you have answered a different question.

Process models: the temporal spine

Process models lay out implementation as a sequence. Nilsen places the Stetler Model, the Iowa Model of Evidence-Based Practice, the Ottawa Model, the Knowledge-to-Action framework, the ACE Star Model and the Quality Implementation Framework in this category.

The defining feature is time. As Nilsen puts it, “process models recognize a temporal sequence of implementation endeavours, whereas determinant frameworks do not explicitly take a process perspective.” That is the practical dividing line: if the diagram has arrows that mean “next,” it is a process model; if the boxes mean “also consider,” it is a determinant framework.

Process models are the right choice when your deliverable is a project plan, a stage-gated rollout, or an EBP change protocol on a unit. They are the wrong choice when your deliverable is an explanation of variation between sites, because a stage list will not tell you why site B stalled at stage three.

Determinant frameworks: the barrier-and-enabler checklist

Determinant frameworks enumerate the contextual factors hypothesised to influence outcomes. Nilsen’s examples include PARIHS, the Theoretical Domains Framework, the Active Implementation Frameworks, Durlak and DuPre’s ecological framework, and the Consolidated Framework for Implementation Research (CFIR).

CFIR is the field’s most-used framework and the one most likely to be misapplied. Two things about its current state matter operationally:

• It was substantially revised in 2022. The updated CFIR contains 48 constructs and 19 subconstructs across five domains — Innovation, Outer Setting, Inner Setting, Individuals and Implementation Process. If your interview guide still says “Intervention Characteristics” and “Characteristics of Individuals,” you are coding to the 2009 version.
• The vocabulary changed on purpose. The update replaced “intervention” with “innovation” throughout, replaced “patients” with “recipients,” introduced “deliverers” for the people delivering the innovation, and dropped “stakeholders.” It also added a Critical Incidents construct in Outer Setting for large-scale or unanticipated disruptions, and four equity-oriented Culture subconstructs (Human Equality-Centeredness, Recipient-Centeredness, Deliverer-Centeredness, Learning-Centeredness).

The recurring mistake with any determinant framework is treating the full construct list as a data collection instrument. Forty-eight constructs is a menu, not a questionnaire. Select the constructs your context makes plausible, say in the methods why you selected those, and report the ones you assessed and found irrelevant — that is a finding, not a gap.

Evaluation frameworks: what you will report as success

Evaluation frameworks specify the aspects of implementation to be measured. Nilsen lists RE-AIM, PRECEDE-PROCEED and the Proctor framework here.

RE-AIM defines five dimensions: Reach and Effectiveness at the individual level, Adoption at the setting and staff level, Implementation (setting-level fidelity to protocol, individual-level use of intervention strategies), and Maintenance at both levels. Its long-standing weakness is that it tells you what to measure but not what explains the result. That is what PRISM addresses — the RE-AIM group’s companion model supplies contextual factors hypothesised to influence RE-AIM outcomes across recipients, staff, organisations and systems, and the external environment.

The other standard option is Proctor’s implementation outcomes, which is the better fit when you need to separate implementation outcomes from service and client outcomes explicitly — for example when a trial has to show that a null clinical result was a delivery failure rather than an intervention failure.

Choose here on reporting audience. RE-AIM travels well with public health and health services funders and pairs naturally with pragmatic external validity arguments. Proctor’s taxonomy travels well where you must instrument implementation constructs individually and defend each measure.

The categories leak — and that is fine, if you say so

Nilsen himself notes the overlap. PARIHS is framed for “anyone either attempting to get evidence into practice, or anyone who is researching or trying to better understand implementation processes.” The Active Implementation Frameworks have “a dual aim of providing hands-on support to implement something and identifying determinants of this implementation.”

EPIS is the clearest modern example of a deliberate hybrid. It combines four phases — Exploration, Preparation, Implementation, Sustainment — with four determinant domains: outer context, inner context, bridging factors and innovation factors. Its maintainers describe it as highlighting “key phases that guide and describe the implementation process” while enumerating “common and unique factors within and across levels of outer context (system) and inner (organizational) context across phases.” Bridging factors — the things that sit between system and organisation, such as intermediary organisations and contracts — are the piece most single-level frameworks have no place for.

Similarly, RE-AIM is now used prospectively for planning as well as retrospectively for evaluation. None of this is a problem. What is a problem is claiming a framework does a job it does not do. If you use EPIS as your process spine and your determinant list, say that; do not also claim it specifies your outcomes.

Combining frameworks without producing a mess

Most real projects end up with two or three. Moullin and colleagues, in their ten recommendations for using implementation frameworks, note that “it may be necessary and desirable to use multiple frameworks” — one to understand determinants, another to assess outcomes. The Implementation Research Logic Model (IRLM) authors put the difficulty plainly: researchers “often find it necessary to use more than one to describe the various aspects of an implementation research study,” but “the conceptual connections and relationships between multiple frameworks are often difficult to describe.”

The IRLM is the most useful device for forcing those connections into the open. Its standard format runs five columns in order:

1. Determinants — barriers and facilitators (populate from your determinant framework)
2. Implementation strategies — what you will do about them
3. Mechanisms of action — why that strategy would change anything (this is where a classic or implementation theory earns its place)
4. Implementation outcomes — populate from your evaluation framework
5. Clinical outcomes

The underlying causal claim is stated by the authors as: “implementation strategies selected for a given EBI are related to implementation determinants (context-specific barriers and facilitators), strategies work through specific mechanisms of action to change the context or the behaviors of those within the context, and implementation outcomes are the proximal impacts of the strategy and its mechanisms.”

Filling in the middle column is where most framework choices are exposed as decorative. If you cannot state a mechanism, you have a determinant list and a wish, not a theory of change.

Selecting defensibly, and writing the rationale down

Two tools exist specifically for this decision.

T-CaST (the Theory Comparison and Selection Tool) organises 16 criteria into four categories: applicability (analytic level, setting, generalisability, outcome of interest), usability (inclusion of change strategies, process guidance, diagrammatic representation, simplicity, description of the change process, accessibility), testability (specificity, causal relationship specificity, falsifiability, explanatory power, logical consistency, empirical support) and acceptability (personal experience, uniqueness, approval, disciplinary origins). It was developed through concept mapping with 37 implementation scientists — 19 researchers and 18 practitioners across the USA, UK and Canada — then cognitively tested and piloted. It is free at impsci.tracs.unc.edu/tcast.

The D&I Models webtool from the University of Colorado Denver, at dissemination-implementation.org, is the complementary resource: an interactive tool for planning, selecting, combining, adapting and using dissemination and implementation TMFs, and for linking them to measures.

The reason both exist is that, left to themselves, people choose by habit. Birken and colleagues surveyed 223 implementation scientists across 12 countries and found more than 100 different theories in use, dominated by a short head: CFIR (20.63% of responses), RE-AIM (13.90%), Diffusion of Innovation (8.97%) and the Theoretical Domains Framework (5.38%). The criteria respondents reported using were analytic level (58%), logical consistency and plausibility (56%), description of the change process (54%) and empirical support (53%); the least-used were fecundity (10%), uniqueness (12%) and falsifiability (15%). The authors’ own summary is blunt: “selection of implementation theories is often haphazard or driven by convenience or prior exposure.” One respondent said simply, “selection is arbitrary…I tend to use ones that are familiar to me.”

Why the long tail of frameworks is a trap

There are far more frameworks than there is evidence about them. Strifler and colleagues’ scoping review of knowledge translation TMFs found 596 studies reporting the use of 159 distinct theories, models or frameworks — and 87% of those were used in five or fewer studies, with 60% used exactly once. Only 26 covered the full implementation spectrum. Coverage by level was badly skewed: all were applicable to individual-level behaviour change, but only 48% addressed the organisation level, 33% the community level and 17% the system level.

Two practical conclusions follow. First, a novel framework with one prior application gives you no comparability and no measurement infrastructure; there is a real cost to being the second user. Second, check the analytic level before anything else — it is both the most-cited selection criterion in the Birken survey and the dimension on which the framework population is thinnest. If your barriers are at the system level, most of the catalogue simply does not reach that far.

The failure mode that actually sinks projects

Moullin and colleagues describe the common defect precisely: suboptimal use occurs “where it is applied conceptually, but not operationalized or incorporated throughout the phases,” and “superficial use of frameworks hinders being able to use, learn from, and work sequentially to progress the field.” Their fix is that frameworks “should be used to inform decisions about what constructs to assess, data to collect, and which measures to use.”

Their ten recommendations, in order, are: select a suitable framework(s); establish and maintain community stakeholder engagement and partnerships; define the issue and develop research or evaluation questions and hypotheses; develop an implementation mechanistic process model or logic model; select research and evaluation methods; determine implementation factors and determinants; select and tailor, or develop, implementation strategies; specify implementation outcomes and evaluate implementation; use a framework at the micro level to conduct and tailor implementation; and write the proposal and report.

Note that framework selection is step one of ten. The other nine are all about whether it survives contact with the protocol. A reviewer can tell the difference in about thirty seconds: if the framework is named in the background and never appears again in the methods, the analysis plan or the results tables, it was decoration.

A workable default

For a typical applied implementation project in a health or academic setting, a defensible combination is:

• One process model or hybrid to structure phases (EPIS if you need phases and determinants together; a classic EBP process model if the work is unit-level practice change).
• One determinant framework to structure barrier assessment, with the specific constructs pre-selected and justified rather than the whole list applied.
• One evaluation framework to define reported outcomes, chosen for the audience that will read the report.
• One explicit mechanism claim per strategy, drawn from a classic or implementation theory such as Normalization Process Theory or COM-B, written into the middle column of an IRLM.
• A written selection rationale, ideally against T-CaST criteria, in the protocol — not reconstructed at write-up.

Frequently asked questions

What is the difference between a theory, a model and a framework in implementation science?

Nilsen distinguishes them by explanatory power. A theory is “a set of analytical principles or statements designed to structure our observation, understanding and explanation of the world.” A model “typically involves a deliberate simplification of a phenomenon” and is “descriptive, whereas a theory is explanatory as well as descriptive.” A framework “denotes a structure, overview, outline, system or plan consisting of various descriptive categories” that “do not provide explanations.” In practice the terms are used interchangeably in the literature, which is why the taxonomy by aim is more useful than the taxonomy by label.

Can I use CFIR and RE-AIM in the same study?

Yes, and it is a common and defensible pairing: CFIR supplies determinants, RE-AIM supplies outcomes. The requirement is to state which framework is doing which job and to link them explicitly — an Implementation Research Logic Model is the standard device for showing determinants, strategies, mechanisms and outcomes side by side rather than as two unconnected appendices.

Is CFIR a process model because it has an Implementation Process domain?

No. CFIR is classified as a determinant framework; the Implementation Process domain is a set of factors relating to how implementation is carried out, not an ordered sequence of stages you follow. The test is Nilsen’s: process models encode a temporal sequence, determinant frameworks do not.

Do I need more than one framework?

Usually, because the three overarching aims are distinct and few frameworks credibly serve all three. Moullin and colleagues note it may be necessary and desirable to use multiple frameworks. The cost of multiplying frameworks is coherence, so cap it at what you can connect in a single logic model.

How do I justify my framework choice to a grant reviewer?

Name the job each framework does, cite the criteria you applied, and show the framework operating in the methods. T-CaST’s four categories — applicability, usability, testability, acceptability — give you a citable structure for the rationale, and its authors developed it partly to promote transparent reporting of the criteria used.

Which implementation science framework is most used?

In Birken and colleagues’ survey of 223 implementation scientists, CFIR was the most frequently reported (20.63%), followed by RE-AIM (13.90%), Diffusion of Innovation (8.97%) and the Theoretical Domains Framework (5.38%). Popularity is a proxy for comparability and available measures, not for fit to your question.

What if no existing framework fits my setting?

Adapting an established framework is generally preferable to inventing one, because the long tail is already crowded — 60% of the 159 TMFs in Strifler and colleagues’ review were used exactly once. The D&I Models webtool includes an explicit adaptation pathway for this. If you do adapt, report what you changed and why, so the next user can compare.

References

• Nilsen P. Making sense of implementation theories, models and frameworks. Implementation Science 2015. PMC4406164
• Damschroder LJ et al. The updated Consolidated Framework for Implementation Research based on user feedback. Implementation Science 2022. PMC9617234
• CFIR Guide — updated constructs and definitions. cfirguide.org/constructs
• Birken SA et al. Criteria for selecting implementation science theories and frameworks: results from an international survey. Implementation Science 2017. PMC5663064
• Birken SA et al. T-CaST: an implementation theory comparison and selection tool. Implementation Science 2018. PMC6251099; tool at impsci.tracs.unc.edu/tcast
• Moullin JC et al. Ten recommendations for using implementation frameworks in research and practice. Implementation Science Communications 2020. PMC7427911
• Smith JD, Li DH, Rafferty MR. The Implementation Research Logic Model. Implementation Science 2020. PMC7523057
• Strifler L et al. Scoping review identifies significant number of knowledge translation theories, models, and frameworks with limited use. Journal of Clinical Epidemiology 2018. PubMed 29660481
• RE-AIM. What is RE-AIM? re-aim.org; What is PRISM? re-aim.org
• EPIS Framework. episframework.com
• Dissemination & Implementation Models in Health Research and Practice webtool, University of Colorado Denver. dissemination-implementation.org