Paper review · October 25, 2026
Consistent but Biased: Chair Personas Trade Noise for Bias in Multi-Agent Clinical Decision-Making
Consistent but Biased: Chair Personas Trade Noise for Bias in Multi-Agent Clinical Decisions
By holding case evidence fixed and changing only the chair agent's persona, the study shows that the persona substantially changes surgical timing decisions and does not raise run-to-run agreement beyond chance level.
Problem
In clinical decision-making systems built from multiple LLM agents, a chair agent usually makes the final decision. However, whether the chair's persona prompt changes the decision, and how it affects run-to-run agreement, which some systems use as a measure of reliability, had never been tested with the cases held fixed.
Method
A virtual spine surgery department of 7 agents discussed 20 public cases of degenerative cervical myelopathy (PMC-Patients). Statements from the case presentation and risk assessment stages (Stage I–II) were generated once and reused unchanged in every run. Only the chair and vice-chair prompts were changed: no persona, consultative, autocratic, and avoidant. Each persona was written in a 'behavioral rules' version and a 'personality traits only' version. The rules version was run for 4 conditions × 20 cases × 3 runs, and the traits version for 3 conditions × 20 cases × 2 runs. The base model is DeepSeek deepseek-v4-pro. On a 10-case subset, the authors also ran replication experiments swapping the chair model (gpt-5.6-terra) and using reworded prompts.
Results
- In the rules version, the share of 'immediate surgery' decisions was 93% for autocratic, 2% for avoidant, and 57% for no persona.
- Surgical intent was maintained in 88–100% of decisions. In other words, the persona mainly changed the timing, 'now or after further tests,' rather than whether to operate. The authors report that on the 10-case subset, the persona effect was larger than the change from swapping models (3–20 percentage points per condition).
- Autocratic and avoidant personas raised the rate of full agreement across 3 runs from 50% to 85–90%. However, this was similar to the chance agreement level expected from the decision distribution (81–86%), and Fleiss κ did not rise (no persona 0.31, autocratic 0.20, avoidant 0.31).
- Prompts using only personality traits also retained 57% and 76% of the immediate-surgery effect of the autocratic and avoidant personas, respectively. Quick decisions, deferral, and additional testing were reproduced, but ignoring red flags, skipping contingency plans, and shifting decisions onto the family were not.
Limitations
Only 20 public cases of a single condition (degenerative cervical myelopathy) were used. The model-swap and rewording replications were run only once per cell on the 10-case subset. The agreement analysis is exploratory. Actual treatment records reflect only one team's choice and are not a ground truth, so the clinical appropriateness of decisions was not evaluated (blinded clinical review is future work). Because the speaking order was fixed, the effect of junior members staying silent could not be tested. Only text records were used, without imaging.
Why it matters
The study shows that in multi-agent systems, a short persona prompt for the chair agent can drive decisions as much as, or more than, the choice of model. It also shows that interpreting run-to-run agreement as reliability can lead to wrongly trusting a system that is consistently biased. The authors recommend treating personas like models, reporting and versioning them, and reporting agreement with chance correction.
Key terms
- Persona prompt
- The opening part of a prompt that gives an agent a personality or way of behaving. This study compared the effects of 'behavioral rules' and 'personality traits' types.
- Noise and bias
- Noise is how much judgments that should be the same are scattered, and bias is a systematic lean of judgments in one direction. A fixed decision policy reduces noise but carries its bias into every decision.
- Fleiss κ
- A measure of agreement among multiple ratings (here, repeated runs) computed after removing the probability of agreeing by chance. When decisions are concentrated on one side, the simple agreement rate can be high while κ does not rise.
- Fixed-evidence (replay) design
- An approach in which case presentation and risk assessment statements are generated once and reused unchanged across all conditions. It makes it possible to attribute differences in outcomes only to the persona and model randomness.
Original
Authors: Yitong Bai, Chengyi Zhang, Ziyang Wang · Posted 2026-10-22 · License arXiv 비독점 배포 라이선스 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.