← Key papers

Paper review · November 19, 2026

NVIDIA Kumo Tabular: Tabular foundation models that improve both accuracy and efficiency

NVIDIA Kumo Tabular: Pushing the Accuracy-Efficiency Frontier for Tabular Prediction

Classic ML, recsys & searchEfficiency & infra Intermediate cs.LG

The authors released three in-context learning–based tabular foundation models pretrained only on synthetic tabular data, ranking first on 8 benchmarks and predicting faster than competing models.

Problem

Tabular data prediction has long been dominated by gradient-boosted trees tuned for each task. Recently, tabular foundation models (TFMs) have started to pull ahead by predicting without training, but it has been hard to combine accuracy, inference speed and the ability to handle large tables.

Method

The models extend the TabICL and TabICLv2 architectures. Cell embeddings group neighboring cells and use sin/cos features and missing-value embeddings. The table encoder alternates column attention (linear cost via inducing points) and row attention to produce row embeddings. An ICL Transformer then predicts test rows using labeled rows as context, with Test-GQA so that test rows read only 2 KV heads. Length-aware temperature keeps attention from becoming diffuse on long or wide tables. Pretraining data was generated with the Kumo prior based on random causal graphs, with added measurement noise, correlated feature blocks, missingness patterns and a learnability filter. Training ran in 3 stages, growing table size from 1,024 rows to up to 60,000 rows. Sizes are S (27M), M (61M) and L (214M).

Results

Limitations

Pretraining used only synthetic tables. The model handles up to 10 classes at a time and splits anything beyond that into subtasks. Above 200,000 rows, training rows are subsampled for each estimator, and each estimator uses at most 500 columns. A single L checkpoint takes about 11,600 GPU-hours. Ranking and speed comparisons follow each benchmark's protocol.

Why it matters

Tabular data is the most common kind of data in enterprise settings such as finance, manufacturing and healthcare. Because this is an open model (weights and training code released) that quickly produces strong predictions without tuning, it can be tried right away as an alternative or complement to AutoML or GBDT pipelines.

Key terms

Tabular foundation model (TFM)
A model pretrained on a large number of prediction tasks that, given a new table, predicts the test rows in a single forward pass without separate training.
In-context learning(ICL)
An approach that feeds labeled training rows in as input context and predicts the test rows from their patterns, without updating any weights.
Prior (synthetic data generator)
A generator that produces an unlimited number of fake tables from random causal graphs and functions. The model is trained to approximate the posterior predictive distribution over this distribution.

Original

Authors: Jingang Qu, Valter Hudovernik, Martin Jurkovič, Cedric Lorenz, David Holzmüller, Federico Lopez, Dmitry Gordeev, Akihiro Nitta, et al. · Posted 2026-11-17 · License arXiv 비독점 배포 라이선스 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.