Signal Detection in Pharmacovigilance: A Practical Process Guide

Signal detection is the process of finding, in the ongoing flow of adverse-event reports a drug or biologic generates, the specific drug-event combinations that deserve a closer look. It sits upstream of case-level reporting: an adverse event (AE) or serious adverse event (SAE) is a single report about a single patient, but a signal is a pattern across many reports — evidence that a drug and an event may be connected more often than chance or background disease rates would explain. This guide walks through the practical mechanics: where the source data comes from, how disproportionality analysis (PRR, ROR, EBGM) turns raw case counts into a statistic worth reviewing, and the validation-to-recommendation pipeline that turns a statistical flag into a documented safety decision.

What counts as a signal

The pharmacovigilance field generally works from a definition close to the one developed by CIOMS Working Group VIII: information arising from one or more sources — including observations and experiments — that suggests a new, potentially causal association, or a new aspect of a known association, between an intervention and an event, and that is judged likely enough to justify verificatory action. Two things in that definition matter operationally. First, a signal is a hypothesis, not a conclusion — it says “this is worth investigating,” not “this drug causes this event.” Second, “likely enough to justify verificatory action” is a threshold decision, and different organizations draw that line in different places depending on their resources and the severity of the event involved.

Where the source data comes from

Signal detection is only as good as the case data feeding it. The main sources, roughly in order of how systematically they’re screened:

• Spontaneous reporting databases — the FDA Adverse Event Reporting System (FAERS) in the US, EudraVigilance in the EU, and VigiBase, the WHO global database maintained by the Uppsala Monitoring Centre (UMC) and populated by national pharmacovigilance centers worldwide. These are the primary engines for post-marketing quantitative screening.
• Clinical trial safety data — accumulating SAE and SUSAR data reviewed on a periodic basis (feeding into DSURs and other aggregate safety reports) during development; see our companion guide on pharmacovigilance in clinical research for how case-level trial reporting works.
• Literature surveillance — published case reports and pharmacoepidemiology studies, which regulators and marketing authorization holders are both expected to monitor as part of routine signal-detection activity.
• Other sources — registries, epidemiological studies, and increasingly, real-world data from claims and electronic health records, used more to corroborate a signal already flagged elsewhere than to generate one on their own.

Case terms in all of these sources are coded to MedDRA, which is what makes systematic, cross-report screening possible in the first place — without a controlled vocabulary, “kidney failure,” “renal failure,” and “acute kidney injury” would each be counted separately and dilute any real pattern.

Disproportionality analysis: the quantitative screening layer

Disproportionality analysis asks a narrow statistical question: for a given drug-event pair, is this combination reported more often than would be expected if the drug and the event were unrelated — based on how often that event appears for all other drugs in the database, and how often that drug appears with all other events? It does not, and cannot, establish causation on its own; it is a screening filter for prioritizing which drug-event pairs get human review, out of the millions of possible pairs in a large database.

PRR — Proportional Reporting Ratio

PRR compares the proportion of reports for a target event among reports for the drug of interest, against that same proportion among reports for all other drugs in the database. A PRR of 1 means no disproportionality; values above 1 suggest the event is reported more often with this drug than with drugs generally. A widely cited convention from the pharmacoepidemiology literature (Evans, Waller and Davis, 2001) flags a pair as worth reviewing when PRR ≥ 2, the associated chi-squared statistic is ≥ 4, and at least 3 cases have been reported — three conditions used together specifically because a high ratio built on only one or two reports is usually noise, not signal.

ROR — Reporting Odds Ratio

ROR asks a closely related question using an odds-ratio framing (the odds of the event given the drug, versus the odds of the event given all other drugs) rather than PRR’s proportion-ratio framing. The two usually move together and are interpreted the same directional way — well above 1 suggests disproportionate reporting — but ROR’s statistical properties make it somewhat more stable at low case counts, which is part of why different regulators and databases have settled on PRR or ROR as their production method rather than both.

Worked example: calculating PRR and ROR from a 2×2 table

Both PRR and ROR come from the same four-cell contingency table, comparing one drug-event pair against everything else reported in the same database over the same period:

	Event of interest	All other events	Row total
Drug of interest	a	b	a + b
All other drugs	c	d	c + d

a is reports naming both the drug and the event under review; b is every other reported event for that same drug; c is the same event reported against every other drug in the database; d is everything else. The counts below are illustrative — not pulled from an actual FAERS or VigiBase extract — chosen only to walk the arithmetic cleanly:

	Event Y	All other events	Total
Drug X	12	588	600
All other drugs	72	14,328	14,400
Total	84	14,916	15,000

PRR divides the proportion of Drug X’s reports that are Event Y by that same proportion among all other drugs: PRR = [a / (a + b)] ÷ [c / (c + d)] = [12 / 600] ÷ [72 / 14,400] = 0.020 ÷ 0.005 = 4.0. Event Y is reported four times as often, proportionally, with Drug X as with the rest of the database.

ROR uses the same four cells in an odds-ratio framing — cross-multiplying the diagonal cells: ROR = (a × d) / (b × c) = (12 × 14,328) / (588 × 72) = 171,936 / 42,336 ≈ 4.06. PRR and ROR land close together here, which is typical for a moderately common event; they diverge more as the event gets rarer.

The Evans, Waller and Davis (2001) convention also requires the chi-squared statistic computed from the same table: χ² = N(ad − bc)² ÷ [(a + b)(c + d)(a + c)(b + d)] = 15,000 × (171,936 − 42,336)² ÷ (600 × 14,400 × 84 × 14,916) ≈ 23.3. With PRR = 4.0 (≥ 2), χ² ≈ 23.3 (≥ 4), and a = 12 cases (≥ 3), this pair clears all three Evans criteria and would be flagged for review.

Why small case counts make both ratios unstable. The same formulas applied to a rarer event tell a very different story. Hold Drug X’s total reports at 5 and vary only how many of those 5 name the event in question, keeping the rest of the table’s proportions roughly the same:

• 2 of 5 reports (a = 2, b = 3): PRR ≈ 300, ROR ≈ 499
• 3 of 5 reports (a = 3, b = 2): PRR ≈ 450, ROR ≈ 1,123
• 4 of 5 reports (a = 4, b = 1): PRR ≈ 600, ROR ≈ 2,995

One additional case report — the smallest possible change — moves ROR by well over a thousand points. None of that swing reflects a real change in the underlying drug-event relationship; it’s an artifact of computing a ratio from a handful of reports. This is exactly the instability the Evans N ≥ 3 threshold exists to filter, and the reason Bayesian shrinkage methods (EBGM, IC — below) exist at all: they pull extreme, low-count estimates toward a conservative population-average value specifically to correct for this behavior, rather than reporting the raw ratio at face value.

The masking effect described later in this guide compounds the problem at the opposite end of the same table: a drug with one dominant, extremely commonly-reported event inflates its own b cell so much that a second, genuinely disproportionate event for the same drug can produce a PRR near 1 — not because no real signal exists, but because the denominator in a / (a + b) is swamped by the unrelated common event.

EBGM and IC — Bayesian shrinkage methods

PRR and ROR are simple frequentist ratios, and both become unstable — prone to wildly inflated values — when case counts are very small, which is common for rare events or newly marketed drugs with a short reporting history. Bayesian shrinkage methods correct for this by pulling extreme, low-count estimates toward a more conservative population-average value until more data accumulates. The two most established implementations are the Empirical Bayes Geometric Mean (EBGM), from FDA’s Multi-item Gamma Poisson Shrinker (MGPS) methodology, and the Information Component (IC), from WHO-UMC’s Bayesian Confidence Propagation Neural Network (BCPNN) — both used across FAERS- and VigiBase-scale screening precisely because they behave more sensibly than PRR/ROR at the long tail of rare events.

No regulator mandates a single universal threshold across all four measures — GVP-style signal-management frameworks deliberately leave the specific statistical method and threshold to the organization running the screening, on the reasoning that the right sensitivity/specificity trade-off depends on database size, therapeutic area, and the severity of the events being screened for. What’s actually inspected during an audit is whether your chosen method and threshold are defined in advance, documented, and applied consistently — not which specific number you picked.

Why disproportionality is a screening tool, not causal evidence

A few structural reasons a statistical signal can appear (or fail to appear) independent of any real causal relationship:

• Confounding by indication — a drug prescribed specifically for a condition will naturally co-occur with events related to that condition’s own natural history, inflating disproportionality without any drug effect at all.
• The masking (or cloaking) effect — a drug with one extremely commonly-reported adverse event can suppress the apparent disproportionality of a second, genuinely associated event for that same drug, because the denominator used in the ratio is dominated by the first event.
• Reporting artifacts — duplicate reports, media-driven reporting spikes (“notoriety bias”), and simple stimulated reporting following a label change or news story can all move the numbers without any change in the drug’s actual safety profile.
• The multiplicity problem — a large spontaneous-reporting database is screened across enormous numbers of drug-event pairs simultaneously; at any conventional significance threshold, a meaningful fraction of “hits” are expected by chance alone, which is exactly why validation and medical review sit downstream of the statistic rather than the statistic triggering action on its own.

The signal-management pipeline: from statistic to decision

Once a quantitative or qualitative trigger flags a candidate drug-event pair, it moves through a sequence of review stages — closely following the structure used in the EU’s GVP Module IX signal-management framework, and broadly mirrored in FDA and WHO-UMC practice — each of which should leave a documented entry in a signal tracking record (a log of every candidate signal, its current stage, owner, and disposition, which is exactly what an inspector will ask to see):

1. Detection — the trigger itself: a disproportionality statistic crossing threshold, a clustering of well-documented cases, a specific case of unusual severity or a novel event type, or an external signal (a regulator’s public notice, a literature report).
2. Validation — a first-pass sanity check, not yet a scientific judgment: is the underlying data real and sufficient to proceed? This step screens out duplicate reports, obvious data-entry errors, and cases with too little clinical detail to be informative, and confirms the event isn’t already a labeled, well-characterized effect with nothing new to add.
3. Confirmation — does the pattern hold up against a fresh data pull or an independent source? A signal that was a one-time statistical blip in last quarter’s data and doesn’t reproduce is closed out here rather than carried forward for full assessment.
4. Analysis and prioritization — confirmed signals are ranked, not treated identically: seriousness of the event, strength and consistency of the statistical and clinical evidence, whether the association is already reflected in labeling, and potential public-health impact (rare-but-severe events and widely-used drugs both raise priority) all factor into how urgently it gets full review.
5. Assessment — the substantive medical/scientific evaluation: case-by-case causality assessment, review of biological plausibility and any relevant nonclinical data, and comparison against the drug’s existing safety profile and similar drugs in its class. This is where a cross-functional team (typically medical safety, epidemiology, and regulatory affairs) does the actual weighing that a statistic alone cannot do.
6. Recommendation for action — the assessment concludes in a documented outcome: no action (with the rationale recorded), continued monitoring, a labeling update, a direct healthcare-provider communication, a post-authorization safety study, or in the most serious cases, a risk-minimization measure up to and including market restriction. Whatever the outcome, the tracking record closes the loop from the original statistic to the final decision.

This pipeline is deliberately slower and more conservative than the detection step alone, and that’s by design: the statistic’s job is to narrow millions of drug-event pairs down to a manageable review list, not to make the safety decision itself.

How this differs from case-level trial safety reporting

It’s easy to conflate signal detection with the individual-case reporting obligations covered in our pharmacovigilance in clinical research guide, but they operate at different levels. Case-level reporting — determining whether a single event meets SUSAR criteria and needs expedited submission — is about one report, on a clock measured in days. Signal detection works across the accumulated case base, on a clock measured in review cycles, looking for patterns that no single case report could reveal on its own. Both feed into the same overall pharmacovigilance system, and both ultimately answer to the same Pharmacovigilance System Master File (PSMF), but they are separate operational functions with separate timelines and separate documentation trails.

Frequently asked questions

What is a signal in pharmacovigilance?

A signal is information suggesting a possible new or changed association between a drug and an adverse event, strong enough to warrant further investigation — a hypothesis to check, not a confirmed causal finding.

What’s the difference between PRR and ROR?

Both measure disproportionate reporting of a drug-event pair against the rest of the database — PRR as a ratio of proportions, ROR as an odds ratio. They typically point the same direction; organizations generally standardize on one as their production method rather than running both.

Why do PRR and ROR sometimes give very different numbers for the same drug-event pair?

Both are computed from the same 2×2 table, but PRR compares proportions while ROR compares odds — the two calculations only converge closely when the event is common enough that “proportion” and “odds” are numerically similar. At very low case counts they can diverge sharply (see the worked example above), which is one reason some organizations prefer ROR’s somewhat more stable behavior at small N, while others standardize on PRR for its simpler interpretation.

Does a high disproportionality score mean the drug caused the reaction?

No. It means the combination was reported more often than expected from the database’s overall pattern, which can also result from confounding by indication, reporting artifacts, or chance across a large number of comparisons. That’s exactly why validation, confirmation, and medical assessment sit downstream of the statistic rather than the statistic standing alone as a decision.

 

Why use EBGM or IC instead of PRR or ROR?

PRR and ROR are simple ratios that become unstable at low case counts — common for rare events or recently marketed drugs. Bayesian shrinkage methods (EBGM, IC) correct for this by pulling low-count estimates toward a more conservative value, making them better suited to screening the long tail of infrequently reported events.

Who is responsible for signal detection — the sponsor or the regulator?

Both. Marketing authorization holders run their own continuous signal-detection activity as part of their pharmacovigilance system, documented in their PSMF, while regulators independently screen their own databases (FAERS, EudraVigilance, VigiBase) and can raise signals the sponsor hasn’t yet identified. Neither side’s screening substitutes for the other’s.