On November 12, 2026, Columbia Business School researcher Hannah Li released a theoretical and empirical study analyzing how endogenous human adaptation undermines standard evaluation methods for interactive large language model (LLM) systems. The preprint, titled When Both Sides Adapt: The Challenges of Evaluating Interactive LLM Systems (arXiv:2611.17239v1), demonstrates that as human users observe model outputs, they alter prompting strategies and task difficulty, systematically distorting offline benchmarks and live A/B testing comparisons.
Two Divergent Estimands: Common Versus Induced Distributions
Modern machine learning teams evaluate generative models through two primary methods: offline benchmark testing on static datasets and live A/B testing on randomized user cohorts. Li proves that these approaches target fundamentally distinct statistical quantities in interactive systems. Offline benchmarks evaluate candidate models on a fixed prompt distribution, isolating performance differentials on identical tasks. In contrast, live randomized trials evaluate models across prompt distributions dynamically induced by each model.
In automated pipelines, evaluators rely on an offline model evaluation suite or a static synthetic benchmark to calculate what Li terms the common-distribution effect. This estimand fixes the prompt distribution and measures computational differences. However, in live production environments, users observe completions and adapt. A user receiving a high-quality answer escalates to difficult tasks, whereas a user receiving unhelpful output lowers task difficulty, uses rigid templates, or leaves. Consequently, difference-in-means estimators in live A/B trials converge to an induced-distribution effect, which conflates model capability with endogenous shifts in task difficulty.
The Mechanics of Ranking Reversals
The study formally demonstrates that these differing estimands cause ranking reversals, where Model A outperforms Model B on every prompt category, yet Model B achieves higher aggregate satisfaction in live trials. Li illustrates this with a two-turn experiment where users request email writing assistance, beginning with routine meeting rescheduling.
| Evaluation Metric | Model M0 Performance | Model M1 Performance | Outcome and Preferred Model |
|---|---|---|---|
| Turn 1 Baseline (Routine) | 90.0% Usable (10% Excellent) | 90.0% Usable (80% Excellent) | Equal basic success; M1 provides superior tone |
| Turn 2 Induced Workload | 80 Routine, 10 Difficult | 10 Routine, 80 Difficult | Users trust M1 with complex tasks; M0 users stay on simple tasks |
| Turn 2 Observed Success | 83.3% Usable | 72.2% Usable | M0 handles easy prompts; M1 tackles difficult issues |
| Aggregate Two-Turn Average | 86.65% Success Rate | 81.10% Success Rate | Live A/B test selects M0 despite lower capability |
| Common-Distribution Benchmark | Lower on all difficult tasks (30% vs 70%) | Dominates or equals M0 on every task tier | Offline benchmark selects M1, correctly isolating capability |
As detailed above, Model M1 matches or exceeds Model M0 across every individual task type: both achieve 90% usability on routine emails, while M1 achieves 70% usability on difficult tasks compared to M0’s 30%. Because M1 produces more thoughtful completions in turn 1, 80% of its users escalate to difficult tasks in turn 2, compared to only 10% of M0 users. This compositional shift penalizes the superior model in aggregate A/B metrics, demonstrating an acute failure of classical experimental versus quasi-experimental design assumptions in interactive settings.
Variance Inflation and Evaluation Pathologies
Beyond ranking discrepancies, Li proves that interactive feedback loops inflate estimation variance. Modeling user-model conversations as a stochastic sequence, the paper shows that turn-by-turn dependencies amplify noise. In a stylized linear model with user feedback sensitivity and model response parameters, variance decays at the standard rate only when the product of user and model sensitivity remains strictly below one. When feedback strength reaches or exceeds unity, estimation variance scales linearly or exponentially with conversation length, requiring larger sample sizes.
These adaptation dynamics also explain pervasive industry challenges, including reward hacking and novelty effects. A model can inflate aggregate metrics without improving core capabilities simply by nudging users toward simpler prompts where errors are rare. Furthermore, discarding initial burn-in periods—a common industry practice to eliminate novelty bias—can inadvertently exacerbate distortion by placing greater weight on late-stage turns where prompt distributions have drifted farthest apart. To establish valid comparisons, research teams configuring a reproducible AI experiment or deploying multi-task frameworks like the HELM benchmark must account for both sides of the interaction loop.
Implications for Academic Research Offices and AI Evaluators
For universities, research administration offices, and computing facilities deploying conversational tools, Li’s findings provide critical operational guidance. Relying exclusively on high-level user ratings during pilot rollouts risks penalizing sophisticated AI assistants that encourage users to tackle ambitious research questions. Conversely, relying solely on static benchmarks risks deploying systems that falter under real-world human prompting.
Li establishes an analytical diagnostic: offline benchmarks and live A/B rankings are guaranteed to agree only when the baseline performance advantage exceeds the product of the total variation distance between induced prompt distributions and the model’s performance range across task partitions. Rather than tracking single aggregate scores, research offices and AI evaluation teams should report joint metrics: task-stratified accuracy, prompt composition shifts over time, and user task escalation trajectories.
Sources
- Hannah Li, When Both Sides Adapt: The Challenges of Evaluating Interactive LLM Systems, arXiv:2611.17239v1, submitted November 12, 2026.
- arXiv HTML Edition, Full text of arXiv:2611.17239v1, November 12, 2026.
