← Key papers

Paper review · November 12, 2026

Tree-Ising: Repairing reasoning loops in language models with bit-level Ising edits

Tree-Ising: Repairing Language Models with Bit-Level Ising Edits

Language modelsEfficiency & infra Advanced Upvotes 71 cs.LG

By recasting the weight repair previously done by an LLM agent as a per-row QUBO (Ising) optimization problem, the method achieved higher accuracy at far lower cost and with fewer weight changes.

Problem

In thinking mode, Qwen3.5-0.8B falls into loops where it repeats the same sentence. On MATH-500, 93% of problems hit the 32k-token limit, and accuracy is only 7.9%. Prior work (Takehi et al.) solved this by having a GPT-6 Astra agent read and fix the weights directly. However, this took 158 turns, 89.7 million tokens and 15.6 hours, and there was no guarantee that the edits it found were optimal.

Method

BF16 weights are represented as a State Tree, a lossless structure in which groups of 64 weights share an exponent. The only allowed edits are sign flips and power-of-two scale shifts; shared exponents and mantissas are left untouched. Using only forward statistics, the method finds a "loop direction" and a "conclusion direction," and a single forward scan selects the layers to edit (15 and 19). The objective combines three terms: weaken the loop-direction output on loop tokens, preserve the output on normal tokens, and minimize the number of changed weights. This objective decomposes exactly into an independent QUBO (Ising) problem for each output row. The problems are solved on GPU with Simulated Annealing or Simulated Bifurcation.

Results

Limitations

The authors state that in the Qwen3.5-0.8B experiments they reused the layers chosen by the agent and also calibrated the edit strength (dose) to the agent's edit. The cost comparison is therefore between "the process of finding a solution" and "applying an already-determined recipe." The layer-selection rule (excluding the last layer) was set after looking at the scores. Both models share the same architecture mixing full attention and linear attention, so the matching scan results may be due to the architecture. The experiments are limited to two small Qwen3.5 models and the problem of math reasoning loops. The QAOA quantum run is an 8-variable proof of concept.

Why it matters

The work shows that model repair can be solved as a reproducible, low-cost optimization problem rather than left to the judgment of an expensive agent. Because it edits sparsely in the storage format itself without gradients, it could be a practical way to quickly fix specific defects in deployed small models.

Key terms

QUBO / Ising model
A problem format that minimizes a quadratic polynomial over binary variables. It can be solved directly with general-purpose solvers such as Simulated Annealing, or with quantum annealers and QAOA.
State Tree
A way of representing each weight as a path of discrete decisions. Weights that share decisions share nodes. Lossless BF16 storage is a special case in which 64 weights share an exponent.
Weight-space programming
An approach that changes a network's behavior by reading and modifying its weights directly, without training (gradients).
Simulated Bifurcation
An algorithm for solving Ising problems inspired by quantum systems. It can solve many problems in parallel on a GPU.

Original

Authors: Hadi Salloum, Ammar Ali · Posted 2026-11-10 · License arXiv 비독점 배포 라이선스 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.