An adversarial collaboration is a study designed and run jointly by researchers who hold genuinely opposing theoretical positions, with the design, predictions, and (increasingly) the arbitration of any remaining disagreement all agreed in advance and made public. It is a distinct genre from ordinary preregistration or a single-team Registered Report: the defining feature isn’t just that a protocol is locked before data collection, but that the people locking it are the opposing camps themselves, co-designing a test that neither side can later claim was rigged against them. This guide covers what the format actually requires structurally — the joint preregistered protocol, the role of a neutral arbitrator or consortium, and the arbitrated disagreement sections that let a paper stay honest when the data don’t settle everything — and points to real, verifiable published examples of the format in current use.
What Adversarial Collaboration Is (and Isn’t)
The term and the modern practice are usually traced to Daniel Kahneman and Gary Klein’s 2009 American Psychologist paper, “Conditions for Intuitive Expertise: A Failure to Disagree,” in which the two researchers — who held opposing views on when expert intuition can be trusted — worked through their disagreement collaboratively rather than in dueling publications, and published the joint result under both names. The core idea generalizes beyond that one paper: instead of each side running separate studies designed to favor its own position, then arguing past each other in the literature, the opposing researchers agree on a single design capable of discriminating between their theories, specify in advance what result each would accept as evidence against their own view, and co-author whatever comes out — a result favoring either side, a draw, or an inconclusive outcome.
It is not simply “preregistration with extra authors.” Ordinary preregistration locks a single research team’s plan to prevent that team’s own post-hoc reinterpretation. Adversarial collaboration locks a plan that two (or more) teams with opposing predictions had to actually agree on before it was locked — the negotiation itself is part of what makes the eventual result credible to both audiences, not just to neutral outsiders.
Why Researchers Use It
Long-running theoretical disputes in a field tend to produce parallel literatures: each camp runs studies favoring its own account, using methodological choices the other side can always dispute after the fact (a different stimulus set, a different exclusion rule, a different operationalization). Adversarial collaboration is a direct structural response to that dynamic. Because both sides had to sign off on the design, the sample, and the analysis plan before seeing any data, a result can’t be waved away afterward as “not a fair test of my theory” — the fairness was negotiated and documented in advance. A 2025 Nature comment piece arguing for wider adoption of the approach frames this explicitly as a way to make theoretical disagreement more collegial and more empirically resolvable, rather than leaving competing camps to talk past each other indefinitely (“Make science more collegial: why the time for ‘adversarial collaboration’ has come,” Nature, 2025).
The format also forces a discipline that looser theoretical debate often avoids: each side has to state, before the fact, what observation would count against its own theory. A theory whose proponents cannot specify a disconfirming result within the agreed design isn’t ready for an adversarial test yet — which is itself useful information about how well-specified the competing theories actually are.
The Structural Conventions
A jointly authored, preregistered protocol
The protocol is written and agreed by representatives of the competing positions together, not by one side and reviewed by the other. It specifies, in the same way any strong preregistration does — methods, sample size and power, exclusion rules, and a locked analysis plan — but with one addition specific to the genre: for each theory or position under test, the protocol states in advance exactly what pattern of results that camp accepts as disconfirming its own view. In some published cases, this joint protocol is itself published as a separate, standalone, peer-reviewed paper well before the results exist, functioning much like a Registered Report’s Stage 1 manuscript (see CASRAI’s guide to writing a Registered Report Stage 1 protocol for the mechanics of that format specifically).
A theory-neutral facilitator, arbiter, or consortium
Because the point is a test both sides trust, adversarial collaborations frequently involve a third party with no stake in either theory: a facilitating researcher, a neutral consortium, or in some cases a designated arbiter who runs or oversees the actual data collection and analysis so that neither camp’s lab is solely responsible for execution. This neutral party’s role, and how disagreements during the study are escalated to them, should itself be specified in the protocol — deciding this after the fact defeats the purpose.
Arbitrated disagreement sections
Data rarely resolve a long-running theoretical dispute cleanly. When results are genuinely ambiguous, or when the same result can be read as consistent with both theories under different auxiliary assumptions, the convention in published adversarial collaborations is not to force a single, negotiated-away conclusion. Instead, the paper includes an explicit section — sometimes framed as separate interpretive commentaries from each camp, sometimes as a structured “where we still disagree” section, occasionally with a neutral arbiter’s own read included alongside — that documents the remaining disagreement candidly rather than papering over it with vague consensus language. This is arguably the format’s most distinctive structural feature relative to a conventional paper’s Discussion section, which conventionally speaks in one authorial voice.
How to Set Up and Write an Adversarial Collaboration Paper
- Identify a genuinely falsifiable disagreement. The dispute needs to be one where a single, shared empirical design could plausibly produce a result that at least one side would accept as unfavorable to its own theory. Disputes that are actually about unfalsifiable framing, not falsifiable prediction, aren’t a fit for this format.
- Recruit proponents of the competing positions, plus a neutral party. The credibility of the eventual paper rests on the opposing authors being genuine, recognized proponents of their respective positions — not researchers sympathetic to a compromise view — and on the neutral facilitator or consortium having no stake in the outcome.
- Jointly negotiate and write the protocol. Specify the design, sample size and power justification, and analysis plan together. For each competing theory, write down explicitly what result would count as evidence against it. This negotiation is typically the slowest part of the process and shouldn’t be rushed to protect the format’s core value.
- Preregister the protocol publicly before data collection, on a registry such as OSF, or as a standalone published protocol paper, so the locked plan is independently verifiable later.
- Agree on the disagreement-arbitration process in advance. Decide, before running the study, how interpretive disagreement after the data are in will be handled — a structured joint-interpretation section, separate camp-authored commentaries, an arbiter’s independent read, or some combination — rather than leaving it to be negotiated under the pressure of an unwelcome result.
- Run the study under the locked plan, ideally with the neutral party involved in execution or oversight, and document any deviation from the preregistered plan explicitly rather than silently.
- Write and co-author the results paper regardless of outcome. All parties are listed as authors whether the result favors their own theory, the opposing theory, neither, or is genuinely inconclusive — publication was agreed to in advance, not contingent on a favorable result.
How This Differs from a Registered Report or Ordinary Preregistration
A Registered Report locks a single team’s protocol through in-principle acceptance from a journal’s reviewers before data collection; the check on researcher degrees of freedom comes from independent peer review. Ordinary preregistration locks a single team’s own plan on a public registry, with no built-in adversarial input at all. Adversarial collaboration locks a plan too, but the check on researcher degrees of freedom comes from the opposing camp itself having had to agree to the design — a materially different, and in a genuine theoretical dispute, often stronger, source of scrutiny. The formats aren’t mutually exclusive: an adversarial collaboration’s protocol can also go through formal Registered Report review, combining both checks.
Real Published Examples
Cogitate Consortium: adversarial testing of consciousness theories (Nature, 2025)
The Cogitate Consortium ran a large-scale adversarial collaboration directly testing contrasting predictions of two competing theories of consciousness, Integrated Information Theory (IIT) and Global Neuronal Workspace Theory (GNWT), through a theory-neutral consortium that worked with proponents of both theories to design the study. The joint protocol — including the specific predictions each theory made and what would count against it — was preregistered and published as a standalone paper years ahead of the results (“An adversarial collaboration protocol for testing contrasting predictions of global neuronal workspace and integrated information theory,” PLOS ONE, 2022). The results paper, using neuroimaging data from 256 participants, was published as “Adversarial testing of global neuronal workspace and integrated information theories of consciousness,” Nature, 2025, and is widely described as the first preregistered adversarial collaboration in consciousness science.
Adversarial collaboration in ecology and evolution (Trends in Ecology & Evolution, 2025)
A 2025 opinion piece, “From rivals to partners: adversarial collaboration in ecology and evolution,” Trends in Ecology & Evolution, 2025, argues that the format has been little used in ecology and evolution relative to psychology and consciousness science, identifies the practical hindrances specific to those fields (longer study timelines, less tractable experimental control over natural systems, more diffuse theoretical camps), and proposes adaptations to make joint preregistered, arbitrated designs workable for long-running ecological and evolutionary controversies. It’s a useful read for researchers outside psychology considering whether the format could translate to their own field’s disputes.
Frequently Asked Questions
What is an arbitrated disagreement section?
It’s a section of the published paper — not the ordinary single-voice Discussion — that explicitly documents where the collaborating authors still disagree about how to interpret the results, after data collection under the jointly agreed protocol. It’s a structural convention specific to adversarial collaboration, used because forcing a single negotiated interpretation would misrepresent a genuine remaining disagreement as resolved consensus.
Who arbitrates disagreements in an adversarial collaboration?
Practice varies: some collaborations use a theory-neutral third researcher or consortium with no stake in either position; others simply publish each camp’s interpretation side by side without a formal arbiter, letting readers weigh both. Whichever approach is used, the protocol should specify it in advance rather than deciding after an unwelcome result is in hand.
Can an adversarial collaboration also be a Registered Report?
Yes. The two formats check different things — adversarial collaboration checks that a design is fair to both competing theories, while Registered Report review checks that independent peer reviewers accept the design’s rigor — and a protocol can be both jointly negotiated by opposing camps and submitted through a journal’s Registered Reports track for in-principle acceptance before data collection.
What happens if the study doesn’t resolve the disagreement?
It gets published anyway, with the remaining disagreement documented candidly, typically in an arbitrated disagreement section. Because publication was agreed to before the outcome was known, an inconclusive or mixed result isn’t a failure of the collaboration — it’s still a co-authored, credible answer to “does the current evidence favor either theory,” even when that answer is “not clearly.”
Is adversarial collaboration only used in psychology?
No, though it originated there and consciousness science is currently its most visible application. It has been proposed and is beginning to be adapted for other fields with entrenched theoretical disputes, including ecology and evolution, as researchers in those fields explicitly work out how to make the format’s requirements — a shared design, advance falsification criteria, joint authorship — workable for their own kinds of studies.
