AGENTIC-SYSTEMSApril 3, 20261 min read23 views

In harmony with gpt-oss

Researchers have successfully reproduced OpenAI's gpt-oss-20b scores by reverse-engineering in-distribution tools and building a native agent harness, overcoming previous transparency issues.

Computer Science > Artificial Intelligence

Title:In harmony with gpt-oss

View PDF HTML (experimental)Abstract:No one has independently reproduced OpenAI's published scores for gpt-oss-20b with tools, because the original paper discloses neither the tools nor the agent harness. We reverse-engineered the model's in-distribution tools: when prompted without tool definitions, gpt-oss still calls tools from its training distribution with high statistical confidence -- a strong prior, not a hallucination. We then built a native harmony agent harness (this https URL) that encodes messages in the model's native format, bypassing the lossy Chat Completions conversion. Together, these yield the first independent reproduction of OpenAI's published scores: 60.4% on SWE Verified HIGH (published 60.7%), 53.3% MEDIUM (53.2%), and 91.7% on AIME25 with tools (90.4%).

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Source: arXiv cs.AI Recent

More in this category

agentic-systems

Teaching LLMs to Update Beliefs for Efficient Long-Horizon Interaction

As LLMs tackle longer tasks, retaining complete interaction histories becomes contextually and computationally expensive. The ABBEL framework addresses this by isolating summaries into natural-language 'belief states' and supervising them via belief grading, recovering accuracy while significantly cutting memory usage.

agentic-systems

DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding

Researchers have introduced DC-Leap, a training-free framework that accelerates inference in Diffusion Large Language Models (dLLMs) by up to 105x when combined with KV-Cache while maintaining generation quality.

agentic-systems

Benchmarking Large Language Models on Multi-Sensor Physical Hazard Assessment

A new benchmark evaluates five leading large language models on multi-sensor physical hazard assessments, revealing a critical vulnerability when evaluating combined sensor risks below individual thresholds. Despite near-perfect accuracy on single-sensor breaches, all models failed to issue precautionary signals for multi-sensor hazards.

agentic-systems

Benchmarking the Personalization Capabilities of Large Language Models

A new study introduces SDR-Bench and SDR-Arena to benchmark the personalization capabilities of LLMs in two-party persuasion scenarios, revealing a personalization plateau among frontier models.

agentic-systems

VeriSimpl: Robust Optimization Modeling from Natural Language using Simplification-based Verification

Researchers have introduced VeriSimpl, a novel framework that leverages LLMs and simplification-based verification to accurately translate natural language descriptions into executable optimization formulations.

agentic-systems

SonicSampler: Unified Tile-Aware Kernels for LLM Sampling and Speculative Verification

Researchers introduced SonicSampler, a unified suite of tile-aware Triton kernels designed to accelerate LLM sampling and speculative verification. By fusing the entire pipeline into a single CUDA Graph-compatible kernel, SonicSampler achieves up to 16x speedups over state-of-the-art baselines.