Paper review · October 9, 2026
LeCuration: A Small World Model as a Multipurpose Tool for Data Curation
LeCuration: A Tiny World Model as a Data Curation Multi-Tool
The authors trained a small JEPA-based world model with about 15 million parameters on CS:GO data in a few hours and showed qualitatively that its embeddings and prediction errors can serve as data curation signals for clustering, anomaly detection and similar tasks.
Problem
Using all physical AI training data means the model also learns abnormal segments as they are. Manual filtering is hard to scale, and rule-based filters need physics engine state that is usually not accessible. A lightweight tool is therefore needed to quickly identify what is typical, rare and surprising in a dataset.
Method
The authors built a small curation-specific model based on LeWorldModel (LeWM). Frames are first compressed by TAESD into a 4×28×28 latent (48×), and a patch-ViT encoder with about 2 million parameters produces a 196-dimensional world state (CLS token) for each frame. A causal Transformer predictor with about 13 million parameters predicts the next state from the past 6 states and actions. Training consists of pretraining with a prediction MSE and SIGReg regularization, followed by 50-frame autoregressive post-training. A DiT decoder for visualization is trained separately and is not used for curation. Embedding clustering (UMAP+HDBSCAN), nearest-neighbor search and the predicted-observed embedding distance d_t are used as curation tools.
Results
- The authors reported that the full training finished within a few hours on a single 40GB A100: 120 minutes of pretraining, 60 minutes of post-training and 30 minutes for the DiT decoder.
- Clustering 1 million CLS embeddings produced scene-based groups such as open views, close indoor spaces, smoke and scope views, without labels (qualitative evaluation). The authors showed this can be used to filter out abnormal clusters such as scope views.
- When autoregressively predicted embeddings were replaced with the nearest real frame among 1 million frames, the scene, viewpoint and lighting matched the ground-truth sequence. However, the weapon held in hand was often represented incorrectly, which led to the diagnosis that more weapon-switching and shooting data is needed.
- The predicted-observed embedding distance d_t spiked sharply in segments where a rare action teleported the scene to another location on the map, then fell as it returned about 350 frames later, suggesting it could be used as an unsupervised anomaly signal.
Limitations
As the authors state, this is a qualitative proof of concept on a single CS:GO map (de_dust2). There are no quantitative metrics for curation quality and no results from training a larger downstream model on curated data. The clusters were not compared against actual position or orientation labels. Spikes in d_t can come not only from physics anomalies but also from accumulated prediction error, out-of-distribution actions or recording dropouts. No threshold was set for separating frames to discard from frames to sample more.
Why it matters
The paper proposes a practical approach: before training a large world model for robots or game agents, use a small 'scout' model that trains in a few hours to identify skew, defects and rare segments in the data. A key advantage is that it works from video and action logs alone, without physics engine state or labels.
Key terms
- JEPA(Joint-Embedding Predictive Architecture)
- A self-supervised approach that, instead of reconstructing pixels directly, learns to predict the next frame's embedding in embedding space.
- SIGReg
- A regularization term that pushes the embedding distribution toward an isotropic Gaussian, preventing representation collapse, where all inputs get squashed into the same representation.
- TAESD
- A lightweight autoencoder with 2.4 million parameters, distilled from the Stable Diffusion autoencoder. It quickly compresses images into a small latent and reconstructs them.
- Embedding Divergence d_t
- The Euclidean distance between the next-state embedding the predictor expects and the embedding of the actual frame. A spike means the model did not anticipate that segment, so it serves as a candidate signal for anomaly detection.
Original
Authors: Mayank Sengupta, Nirmit Desai, Eric Song, Kunal Sawarkar · Posted 2026-10-07 · License CC BY-SA 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.