World Feedback for Clinical Agents: Diagnosing RL in FHIR Environments
The paper diagnoses the challenges of applying Reinforcement Learning (RL) to clinical agents in FHIR environments, introducing MedAgentBench-v3 to address feedback flaws and proposing a hybrid SFT-RL approach.
Computer Science > Artificial Intelligence
Title:World Feedback for Clinical Agents: Diagnosing RL in FHIR Environments
View PDF HTML (experimental)Abstract:Clinical protocol-execution tasks -- checking a lab value, applying a threshold, placing a correctly structured FHIR order -- are natural candidates for RL from world feedback: once clinical SMEs encode decision logic into a verifier, that verifier grades unlimited rollouts without per-episode annotation. But applying RL requires a sound feedback channel and sufficient base capability. We audit MedAgentBench v1/v2, find a 41.7% silent-finish ceiling that makes inaction the RL dominant strategy, and construct \textbf{MedAgentBench-v3 (MAB-v3)} (508 tasks, 8.9% ceiling). Training Qwen3-8B exposes two structural barriers: a \emph{capability ceiling} (10/20 task types have 0% base performance, zero gradient) and a \emph{format-knowledge barrier} (3/20 types require exact clinical codes undiscoverable by exploration). Pure RL reaches 18.2% pass@1 vs.\ 34.1% for rule-based SFT; the 15.9~pp gap is attributable entirely to these barriers. A decision/format-knowledge/lookup taxonomy predicts RL learnability and prescribes the fix: SFT to inject codes, RL to learn conditionals.
Source: arXiv cs.AI Recent
