AGENTIC-SYSTEMSJuly 28, 20261 min read

MedLoCoMo: A Long-Context Multi-Session Medical Dialogue Benchmark for Large Language Models

MedLoCoMo is a new medical benchmark designed to evaluate LLMs' clinical reasoning over multi-session, long-context patient histories. Experiments reveal that reasoning across different hospital admissions remains particularly difficult for current AI models.

Computer Science > Artificial Intelligence

Title:MedLoCoMo: A Long-Context Multi-Session Medical Dialogue Benchmark for Large Language Models

View PDF HTML (experimental)Abstract:MedLoCoMo is a Medical Long-Context Memory benchmark for patient-specific clinical reasoning over multi-admission medical dialogue. Existing medical QA benchmarks largely test short context knowledge or single document grounding, leaving open whether LLMs can use, connect, and abstain over longitudinal patient histories. We build MedLoCoMo from deidentified MIMIC-IV and MIMIC-IV-Note records by constructing admission-level clinical packets, synthesizing grounded doctor-patient conversations, and generating evidence linked QA items over single-admission, cross-admission, and adversarial unanswerable settings. The benchmark contains 100 patient timelines averaging 1,669.8 turns, 29.7 sessions, and 74,512.2 tokens per conversation. Across the evaluated baselines, cross-admission reasoning is consistently harder than localized evidence use, even when models have long context windows or use external memory or retrieval methods. The code and MedLoCoMo benchmark release is available at this https URL for use and reproducibility.

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

Concept-based Visual Counterfactual Explanations with Diffusion Models

Researchers introduced C-VCE, a novel diffusion framework that generates human-interpretable visual counterfactual explanations by building concept bottleneck layers directly into the generative model.

agentic-systems

Synthetic Scenario Generation for Evaluation of Industry 4.0 Agents

Researchers have developed ScenarioGeneratorAgent, an automated pipeline that generates synthetic evaluation scenarios for Industry 4.0 AI agents. Tested on Smart Grid Transformers, the approach accelerates benchmark generation by 8x while maintaining high quality and adherence to IEC standards.

agentic-systems

Same Question, Different Answers: Evaluating LLM Reliability Beyond Accuracy

A new study reveals that large language models (LLMs) often produce inconsistent answers when the same question is rephrased, with mismatch rates exceeding 23%. The findings suggest that standard benchmark accuracy masks substantial instability, but a self-paraphrasing strategy can help recover latent knowledge at inference time.

agentic-systems

DeepLens Diagnosis Agent: Agentic Workflow Design Lets a Small Reasoning Model Compete with Frontier LLMs

By leveraging a structured five-stage agentic workflow, the 7-billion-parameter DeepLens Diagnosis Agent outperforms frontier models like Claude Sonnet 4.5 and Gemini 3.1 Pro on complex clinical diagnostics, demonstrating that disciplined process design can outweigh raw parameter count and API costs.

agentic-systems

SCAIR: Schema-Conditioned Agentic Iterative Reasoning for Enterprise Knowledge Graphs

Researchers have introduced SCAIR, a training-free agentic framework that enhances Knowledge Graph-based RAG (KG-RAG) for enterprise applications by injecting schema constraints into multi-hop iterative reasoning.

agentic-systems

Loss-Aware Feature-Map Pruning in Convolutional Neural Networks Using Multi-Armed Bandits

Researchers have proposed a loss-aware structured feature-map pruning framework for convolutional neural networks using multi-armed bandits. The approach significantly reduces computational complexity and storage costs while maintaining accuracy levels close to original unpruned models.