Paper review · November 15, 2026
A systematic evaluation of training-data generation strategies for ML-based molecular and materials property prediction
Systematic Evaluation of Training Data Generation Strategies for Machine Learning-Based Molecular and Materials Property Prediction
Comparing random, uncertainty-based and diversity-based sampling across four molecular and materials datasets and several models, the study found that uncertainty sampling generally uses the fewest labels, and derived guidance for choosing a strategy.
Problem
Datasets for molecular and materials property prediction are usually built in a nearly random way. As a result, many similar structures overlap, and expensive DFT calculation labels are wasted. There has been little systematic comparison of which sampling strategies save labels while preserving accuracy, and how their effect changes with the model, representation and dataset.
Method
The study used the QM9 (HOMO-LUMO gap), QM9small (a 5% subset of QM9), Matbench MP (formation energy) and Pourbaix stability (HSE06 decomposition free energy) datasets. The models were a descriptor-based ensemble (RF·KRR·XGBoost voting) and the graph neural network SchNet. Using pool-based active learning, the authors compared random sampling (RS), uncertainty sampling (US) based on the variance of committee predictions, and diversity sampling (DS) based on TypiClust. Data efficiency was measured as the number of labels needed to first reach 1.1 times the RMSE obtained with the full dataset (N10%).
Results
- For the QM9 ensemble, N10% was about 19,000 for US (17% of the pool), lower than about 26,000 for RS (25%) and about 24,000 for DS (23%).
- On the Pourbaix stability dataset, US reached the threshold with about 3,000 labels (54%), saving labels equal to about 22% of the pool compared with RS (about 4,200, 77%).
- For the Matbench MP ensemble, all three strategies reached the threshold at small fractions: US 13%, RS 15% and DS 16%. The authors attributed this to the high redundancy of the dataset.
- With SchNet, no strategy reached the threshold within the budget (about 50,000 labels) on either QM9 or Matbench MP. On Matbench MP, DS achieved a final RMSE of 0.080 eV/atom, 11% lower than RS (0.090).
- On QM9small, DS needed about 300 more labels than RS, so the ranking of strategies changed depending on dataset size and model.
Limitations
Because of computational cost, the SchNet experiments were run only once each. SchNet was not evaluated on the two small datasets. On the large datasets, active learning was stopped at about half of the pool. The scope is limited: only three strategies (RS, US, DS) were compared, with one target property per dataset. The descriptor ensemble uses only composition information, which caps its performance on structure-dependent properties.
Why it matters
In materials and chemistry research where labels come from expensive calculations such as DFT and HSE06, the number of labels directly equals computational cost. This study provides a practical guideline, backed by numbers: use uncertainty sampling as the default, but check its effect for the given dataset size and model.
Key terms
- Active learning
- A training approach in which the model picks samples that seem likely to be informative and requests labels for them, repeating this to improve performance with fewer labels.
- Query-by-Committee
- A method that treats how much the predictions of several models (a committee) disagree as uncertainty, and picks samples with high disagreement first.
- TypiClust
- A diversity sampling technique that splits the data into clusters and picks the most typical sample from large unlabeled clusters, so that the data space is covered evenly.
- N10%
- The number of training samples needed to first get within 1.1 times the RMSE obtained when training on the full dataset. Smaller means better data efficiency.
Original
Authors: Minh Nghia Nguyen, Beate Paulus, Akhil S. Nair · Posted 2026-11-12 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.