GPT-4o and GPT-4o-mini collapse 'Partially Supported' clinical label in 83–100% of cases

Auto-published from arXiv:2610.01497 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.91, no flags)

Category

over-refusal

Model

GPT-4o (gpt-4o-2024-08-06), GPT-4o-mini (gpt-4o-mini-2024-07-18)

Surface

API (temperature=0, max_tokens=512)

Setup

Models audited 500 synthetic NSCLC molecular tumor board cases from the OpenMTB-Audit benchmark across four safety labels: Supported, Partially Supported, Unsupported, and Insufficient Information. 60 of the 500 cases are ground-truth 'Partially Supported' — evidence-backed therapy recommendations that still require oncologist review due to patient-specific caveats (e.g., ECOG performance status ≥3, extensive prior treatment). Models were tested in four configurations: Base, Simple RAG, RAG+EV (evidence verification), and Prompt-Checklist. Verbatim prompt templates are not reproduced in the paper body; full prompts are available in the linked repository.

Reproduction anchor

code: https://github.com/Negiiinx/OpenMTB-Audit — contains all 500 benchmark cases, full prompt templates, evaluation code, and expert annotations

Observed behavior

All eight LLM configurations (GPT-4o-mini and GPT-4o across all prompt strategies) failed to assign the 'Partially Supported' label in 83.3–100% of the 60 true Partially Supported cases, collapsing predictions to either 'Unsupported' or 'Supported'. GPT-4o-mini RAG+EV and GPT-4o Prompt-Checklist reached 100% over-refusal on this class. This produced misleadingly high aggregate safety scores (74.9–90.9) that did not reflect calibrated clinical reasoning.

Expected behavior

A correctly-behaving model should recognize that a molecularly matched therapy with patient-specific contraindications (e.g., poor ECOG status) merits the 'Partially Supported' label, flagging it for oncologist review rather than blanket rejection as 'Unsupported'.

Reproducibility

high

Threat model

In AI-assisted clinical decision support for precision oncology, systematic over-refusal causes valid treatment options to be incorrectly flagged as unsupported, potentially delaying or denying beneficial therapies for cancer patients. Inflated safety scores mask this failure from developers and regulators validating these systems.

Novelty

First benchmark to demonstrate that LLMs inflate clinical safety scores through label collapse rather than accurate reasoning — adding RAG and evidence verification (RAG+EV) made over-refusal significantly worse, not better, revealing that retrieval-augmented pipelines compound conservatism in four-class medical classification tasks.

Source

• arXiv: 2610.01497
• PDF: https://arxiv.org/pdf/2610.01497
• Categories: cs.AI, cs.LG
• Authors: Negin Ashrafi, Jia Luo, Stacey M. Frumm, Roxana Daneshjou

Triage notes (auto)

• paperType: benchmark
• estimatedCaseCount: 2
• triage reason: Benchmark of 500 clinical cases exposing reproducible over-refusal failure: all 8 LLM configurations collapse fine-grained labels (Partially Supported) in 83.3–100% of cases rather than performing calibrated clinical reasoning.

─────────────────────────────────────────────────────────────────────