RefineRL: Advancing Competitive Programming with Self-Refinement Reinforcement Learning
RefineRL introduces a self-refinement reinforcement learning approach that enables compact 4B models to outperform 32B models and rival 235B giants in competitive programming tasks.
Computer Science > Artificial Intelligence
Title: RefineRL: Advancing Competitive Programming with Self-Refinement Reinforcement Learning
While large language models (LLMs) have demonstrated strong performance on complex reasoning tasks such as competitive programming (CP), existing methods predominantly focus on single-attempt settings, overlooking their capacity for iterative refinement. In this paper, we present RefineRL, a novel approach designed to unleash the self-refinement capabilities of LLMs for CP problem solving.
RefineRL introduces two key innovations:
- Skeptical-Agent: An iterative self-refinement agent equipped with local execution tools to validate generated solutions against public test cases of CP problems. This agent always maintains a skeptical attitude towards its own outputs and thereby enforces rigorous self-refinement even when validation suggests correctness.
- Reinforcement Learning (RL) Solution: A method to incentivize LLMs to self-refine with only standard RLVR data (i.e., problems paired with their verifiable answers).
Extensive experiments on Qwen3-4B and Qwen3-4B-2507 demonstrate that our method yields substantial gains: after our RL training, these compact 4B models integrated with the Skeptical-Agent not only outperform much larger 32B models but also approach the single-attempt performance of 235B models. These findings suggest that self-refinement holds considerable promise for scaling LLM reasoning, with significant potential for further advancement.
Source: arXiv cs.AI Recent
