← Key papers

Paper review · November 13, 2026

Eliciting LLM self-learning ability with reinforcement learning

Incentivizing Self-Learning in Large Language Models via Reinforcement Learning

Reinforcement learningLanguage modelsAgents Advanced Upvotes 27 cs.CL Intern Large Models

Training an LLM with reinforcement learning and fine-grained rewards to solve proof problems by navigating textbooks itself raised proof accuracy from 51.5% to 79.5%.

Problem

Solving problems in an unfamiliar field takes more than access to relevant information. The model has to understand new principles and work them into its solution. Existing RAG and learning from pre-bundled context do not handle well the interplay between knowledge acquisition and reasoning. The authors study 'self-learning': the ability to find, study and apply what is needed from long reference material on one's own.

Method

5,183 exercises were drawn from 158 math textbooks to build 'problem–textbook' pairs, with solutions removed from the textbooks visible to the model. The BookNavigator interface offers three actions: viewing the table of contents (get_toc), keyword search (search_index) and reading a section (read_section). Intern-S2-Preview-35B was trained with GRPO for 50 steps, with two rewards added to proof correctness (passing requires GLM-5.2 to judge it correct in all 8 checks). (1) An exploration bonus given when tools are used in a correct Trajectory, decayed from 1 to 0.25 on a cosine schedule. (2) A frozen copy of the initial policy rates tool use in each round as 'helpful, neutral or harmful', and the advantage is redistributed per round accordingly.

Results

Limitations

Both proof correctness judgments and tool-use ratings are left to LLM judges, so grading errors may creep in. The RL experiments involve a single model, Intern-S2-Preview-35B, at a scale of 50 rollout steps. The task is limited to proofs based on math textbooks, and textbook search is keyword-based. Gains on non-math tasks are modest, at around 1–2 percentage points. The input text was cut off partway, so the analysis of what drives the performance gains and detailed ablation numbers could not be checked.

Why it matters

It points to a direction of teaching models 'how to find and learn what they don't know on their own' rather than packing in more knowledge. When designing agents that solve problems while reading long material such as internal documents or manuals, it provides evidence that active exploration may work better than RAG that attaches fixed retrieval results.

Key terms

GRPO(Group Relative Policy Optimization)
A reinforcement learning algorithm that samples multiple responses to the same problem and updates the policy using, as the advantage, how much better each response is relative to others in the group.
Self-learning
The ability of a model, while solving a problem, to find, read, understand and apply the knowledge it needs from reference material on its own.
Credit assignment
The problem of deciding how much of the final reward to assign to individual actions within a Trajectory (here, tool calls in each round).
LLM-as-judge
An approach in which another LLM judges whether a model's output (here, a proof) is correct and uses that as an evaluation or reward signal.

Original

Authors: Lingkai Kong, Yuzhe Gu, Haiteng Zhao, Zijian Wu, Youqing Fang, Zhouqi Hua, Yanan Sun, Kai Chen, et al. · Posted 2026-10-29 · License CC BY 4.0 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.