MASS (Multi-Agent Self-Supervision): How Agents Teach Each Other

Why it was needed: the cost of human labels and the limits of self-grading

To make an AI model better, you need a signal that says "this answer is good and that one is bad." The traditional methods are data labeling, where people supply the correct answers, and RLHF (reinforcement learning from human feedback), which learns from collected human preferences. But the longer and more specialized the task (multi-step agent tasks, code, 3D design and so on), the faster the cost of people checking each item grows.

One alternative is to have the model grade its own answers. But when a model rereads its own answer, it tends to miss during grading what it missed while writing the answer. It is effectively looking twice with the same knowledge and the same habits. Sycophancy, the tendency to agree with what the user says, can also show up in self-evaluation.

MASS (multi-agent self-supervision) is an idea that comes out of the space between these two problems. AI produces the supervision signal instead of people, but rather than relying on a single model, several agents with different roles watch each other to reduce the blind spots of self-grading.

Note that the input material collected this time did not include an AI methods paper named "MASS." Most of the results were astrophysics and theoretical physics papers about "mass," so they were not used as sources. The concept explanation below stays at the level of general principles, and specific numbers or claims from papers come only from two related recent studies.

Core idea: split roles like a newsroom

A newspaper newsroom is a helpful comparison. A reporter writes the story, an editor points out gaps in logic and missing parts, and a copy editor checks facts and style. Mistakes are caught more often when several people with different perspectives look at a story than when the reporter revises it alone. Even so, fact-checking is only finished once it is compared against outside sources (original statistics, sources interviewed).

MASS has the same structure. It has a generator agent that produces answers, a critic agent that looks for weaknesses in the answers, and, where possible, a verification step that checks against standards outside the model (code execution results, physics simulators, rule checkers). The results that pass this process, along with the critique records, become the material for the next round of training, that is, the "self-supervision" signal.

Here "self-supervision" means the learning signal is produced inside the system without people attaching labels. It does not mean people are removed. Instead of grading every case, people shift to setting the standards and checking samples.

How it works

A typical MASS flow can be seen as five steps. First, define the task. It may be a single problem or a multi-step job for an agent to carry out. Second, the generator agent produces an answer or an action record (Trajectory). Sometimes several are sampled for the same task to use as material for comparison.

Third, the critic agent reads the result and points out errors, missing conditions and unsupported claims. This feedback can be sent back to the generator agent and used to produce a revised version. You can think of it as LLM-as-a-Judge (an LLM acting as evaluator) split into several roles. Fourth, if there is an external verifier, the result is actually run. Code is run against tests, and a 3D object is placed in a physics simulator to see whether it falls over or collides.

Fifth, the judgments of the agents and the verification results are combined to select only what passed, or to pair good and bad answers, and these are saved as training data. The model is retrained on this data using methods such as SFT (supervised fine-tuning) or DPO (direct preference optimization), and the improved model becomes the generator agent again. As the loop repeats, it builds up like a data flywheel.

How it developed: where three lines of thinking meet

The input material contains no AI paper that serves as the starting point for MASS, so this section does not trace a lineage between specific papers. Instead, it only outlines which ideas meet at the conceptual level.

The first line is the long-standing idea of self-supervised learning. It draws signals that act as correct answers from the data itself, and it is the basic principle that made large-scale pretraining possible without human labels. The second line is LLM-as-a-Judge, where an LLM evaluates the output of another LLM. It imitates human evaluation to cut costs, but the evaluator's own bias has been pointed out as a problem. The third line is multi-agent systems, in which several agents split roles and collaborate.

MASS ties these three together. It combines self-supervision's "creating signals without labels," LLM evaluators' "models grading models" and multi-agent systems' "division of roles," so that evaluation is not left to one model but split across different roles.

Where it is heading: adding outside standards

A notable direction in recent research is to add verification signals from outside the model to the loop instead of relying only on judgments between agents. Artisan paper is a study on generating 3D objects with movable joints. It extends an existing method, Articraft, and proposes a multi-agent pipeline that receives physics simulation feedback. Rather than objects that only look similar, it checks whether they avoid collisions when they move, whether they stay standing under gravity, and whether their mass and material properties are correct.

The study also built a benchmark for evaluation. It consists of 100 reference assets taken from ArtVIP and LightWheel, 50 stability prompts and 50 ABO objects with reference masses. Comparing more than 10 representative models, the authors report that Artisan had the lowest collision rate and the highest stability among the methods compared. When the simulator reports that "this object falls over," it can stop agents from letting it pass because it "looks fine" to each other.

Another direction uses human judgment as a reference point. PAGE paper, a framework for comics research, has a vision-language model (VLM) annotate each page by following a codebook (classification guidelines) written by people, and compares a sample of those readings against human annotations. It was applied to 3,929 American comic books and 87 Japanese manga volumes. It is not a multi-agent study, but the structure of people checking samples of readings that AI produced at scale can be used as-is when running MASS.

Limitations and open issues

The biggest risk is "the same eyes." If the generator, critic and verifier agents all come from the same base model, only their role names differ; their knowledge gaps and biases are the same. They may then agree that something is correct together rather than catching each other's errors. This is why different models are mixed or external verifiers are added.

The second is reward hacking. If the generator agent learns the format or tone the critic agent likes instead of actually producing good answers, scores go up but quality stays the same. The third is model collapse. Repeatedly training a model on data produced by models can reduce diversity and cause rare cases to be forgotten, so human data or outside signals need to be mixed in.

The fourth is cost and the reliability of evaluation. Running several agents multiplies inference costs, and if the tasks used for training overlap with the evaluation set, data contamination inflates performance. In the end, even if people do not grade every answer, they carry the final responsibility for setting standards, checking samples and catching unusual patterns.

Further study

Among the input material, Artisan paper is the recommended first read. Seeing what can be measured when an external verifier, physics simulation, is attached to a multi-agent pipeline, and how the benchmark is split into four aspects (spatial fidelity, physical plausibility, downstream task performance and overall quality), gives a feel for "designing verification signals."

Next, reading the part of PAGE paper that compares VLM readings against human annotations offers practical hints on how to make large volumes of AI-generated labels trustworthy.

On the conceptual side, reading this site's entries on LLM-as-a-Judge, multi-agent systems, synthetic data, model collapse and reward hacking together helps connect where MASS gets its strength and where it is likely to break down.