Paper review · November 8, 2026

Embedding-space fingerprinting of LoRA fine-tuned models: inferring the training-data domain from the model alone

Embedding-Space Fingerprinting of LoRA Fine-Tunes: Recovering the Training-Data Domain from the Model Artifact Alone

AI security Language models Intermediate cs.AI

The authors extracted weight, activation and generated-text features from 36 LoRA fine-tuned models and trained a classifier on them. For models trained on unseen datasets, it identified the training domain with up to 64% accuracy, well above the 20% chance level.

Problem

LoRA adapters are small files that are easy to share. Can you tell what kind of data a model was trained on from a model artifact such as an adapter alone, without seeing the training data? This question matters for both provenance auditing and privacy. Existing membership and dataset inference methods assumed access to candidate datasets or known attributes.

Method

The authors used Qwen3.5-0.8B as the single base model and fine-tuned it on 18 public datasets across 5 domains (medical, code, math, non-English, narrative writing), with 2 seeds each and the same LoRA settings (rank 16, 1.5 million tokens), producing 36 models. Features were extracted at three access levels. White-box features are the norms, singular values and per-layer distribution of the LoRA weight updates. Gray-box features are per-layer hidden state statistics when 200 domain-neutral prompts are fed in. Black-box features are the entropy, length, token histograms and Sentence-BERT embedding centroids of text generated from the same prompts. The classifier is logistic regression, evaluated with a leave-one-dataset-out (LODO) protocol fixed before looking at the results.

Results

On predicting the domain of an unseen dataset (5-way, 20% chance), black-box output features reached 63.9%, gray-box activations 61.1%, and white-box weight statistics 41.7%. The dataset-level cluster bootstrap confidence interval for the white-box result (0.194–0.639) includes chance, and the authors state they do not regard it as an established result.
The pre-registered hypothesis that the signal would weaken in the order weights > activations > outputs was rejected. Activations and outputs carried more information than weight statistics.
Using only generated text, without token probabilities, black-box accuracy stayed at 63.9%, and MiniLM embedding centroids (384 dimensions) alone gave 61.1%. This means domain inference is possible with ordinary API access alone.
On re-identifying models retrained on the same dataset with a different seed (RQ1), weights and activations scored 100% and outputs 94.4% (chance 5.6%). Across 54 models, after adding a third run trained on independently resampled data, domain accuracy (40.7/64.8/63.0%) stayed nearly the same.
When transferred to rank 8 and rank 64 adapters, black-box predictions did not change (8 of 10 correct). For white-box, changing the rank altered the norms and spectra, and performance dropped to 3/10.

Limitations

As the authors note, they used only one 0.8B hybrid base model. With 36 models and 5 domains, the scale is small and the confidence intervals are wide. The two seeds of the same dataset share training data, so their predictions are not independent. The white-box features are hand-designed norm and spectral statistics that discard singular-vector direction information, which may have led to underestimating the weight signal. The rank and resampling robustness checks were added post hoc, and the rank experiment used only one dataset per domain, so it is indicative only. Confusion remains between medical and creative writing (English prose), and between code and math.

Why it matters

The paper shows that LoRA adapters uploaded to public repositories reveal their training-data domain through output text alone. This can serve as a tool for auditing undisclosed training data or verifying provenance. Conversely, it is a privacy risk for those who want to hide the nature of their training data. Reporting a pre-registered evaluation protocol together with dataset-level confidence intervals is also an evaluation practice worth following in this field.

Key terms

LoRA(Low-Rank Adaptation): A parameter-efficient fine-tuning method that freezes the base model's weights and trains only the product of two small low-rank matrices (B·A) to represent the weight update. The result is a small adapter file that is easy to share.
White-box/gray-box/black-box access: Levels describing how much access you have to a model. They refer, respectively, to seeing the full weights, the internal hidden states, or only the output text (or token probabilities) for a given input.
LODO(Leave-one-dataset-out): An evaluation method that holds out all models from one dataset, trains the classifier on the rest, and then predicts the held-out models. It checks whether the classifier generalizes at the domain level rather than memorizing datasets.
Model fingerprinting: A technique that identifies provenance information, such as a model's lineage or training data, from distinctive traces left in model artifacts.

Original

Authors: Shivesh Rane · Posted 2026-11-05 · License CC BY 4.0 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.