Paper review · November 21, 2026
A statistical framework for auditing LLM training data
A Statistical Framework for Training Data Auditing of Large Language Models
The paper proposes training data auditing with controlled error rates at the document, collection and suspect-subset levels, by aggregating chunk-level Membership evidence into document scores with Cauchy combination and producing calibrated p-values.
Problem
Determining whether specific data was used to train an LLM matters for privacy, copyright and benchmark contamination detection. In long documents, however, Membership signals are concentrated in only a few segments, so simple averaging dilutes the signal. Existing dataset-level inference relies on independence assumptions, and error control was also lacking when listing suspect documents.
Method
Documents are split into chunks, and several Membership evidence statistics such as Perplexity and Min-K% are computed for each chunk. A logistic regression trained on a small set of labeled documents combines these statistics into a single chunk-level piece of evidence. Chunk evidence is aggregated into a document score with Cauchy combination. Empirical p-values are computed by comparison with calibration documents that were certainly not used in training. For collection-level decisions, an Energy permutation test compares the target distribution with the non-member distribution. For selecting suspect documents, the Benjamini–Hochberg procedure controls the FDR. In Fine-tuning audits, the model before Fine-tuning is used as the reference model.
Results
- The authors report proofs of finite-sample validity for document-level p-values under an exchangeability condition, Type I error control for both individual and global tests, and FDR control for subset selection. They also quantify the additional error that arises when the condition is violated.
- The authors report that the framework showed detection performance and reliability in four cases: pretraining audits (BookTection books, Pile–arXiv papers) and fine-tuning audits (Cosmopedia synthetic text, HealthCareMagic medical QA). The provided excerpt of the paper contains no specific figures.
Limitations
The theoretical guarantees depend on the exchangeability assumption. They require calibration documents drawn from the same distribution as the target documents but not used in training, as well as labeled documents for learning the combination rule. Such data may be hard to obtain in real audits. Access to token probabilities is assumed, and the Min-K%++ feature additionally requires the full next-token distribution. The models used in the evaluation are relatively small, such as Pythia and Llama-3.2-3B. The text cuts off partway through the method description, so the experimental figures could not be checked.
Why it matters
When copyright holders or data owners claim that "an AI model used our data," this lets them present a list of suspect documents, with the false attribution rate controlled, as statistical evidence. It is meaningful as an auditing tool that can be used in legal and regulatory review.
Key terms
- Membership Inference
- A technique for estimating whether specific data was included in a model's training set, based on the model's outputs (e.g., token probabilities).
- Cauchy combination
- A method for combining multiple mutually dependent p-values or pieces of evidence through a Cauchy distribution transform. It preserves the signal well even when only some of them carry a strong signal.
- FDR(False Discovery Rate)
- The expected proportion of items selected as positive that are actually negative. The BH procedure keeps this proportion at or below a preset level.
- Min-K%
- The average log probability of the bottom K% of tokens, the ones hardest to predict. A higher value is taken to mean the text is more likely to have been training data.
Original
Authors: Shengjie Niu, Yeheng Ge, Jian Huang · Posted 2026-11-19 · License arXiv 비독점 배포 라이선스 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.