AI in manufacturing: visual defect inspection, predictive maintenance, digital twins

Why it was needed: defects are rare, and human eyes get tired

Quality inspection on manufacturing lines long relied on human eyes. The PaDiM paper paper raises this issue in its introduction. On a production line, defects are very rare events, so finding them one by one is tedious for people. Automation can keep inspection quality consistent even when attention lapses, and can reduce the burden on workers.

Yet it is also hard to collect many 'normal' and 'defective' images and do supervised learning (learning from labeled data), as in ordinary image classification. The same paper gives two reasons: defect examples are always scarce, and defects can appear in unexpected forms. Normal data is plentiful while defect data is extremely scarce, which is an extreme case of class imbalance.

The PatchCore paper paper calls this the 'cold-start' problem. Images of normal products are easy to get, but defining every possible future defect in advance is costly and complex. Defects also vary widely, from subtle ones like thin scratches to large structural flaws like missing parts. So the approach of 'learning only from normal data, then finding what differs from normal' became the starting point for industrial anomaly detection.

Core idea: know normal well, and anomalies stand out

An analogy is a skilled inspector. This inspector does not memorize a catalog of defects. Having seen thousands of normal products, they can tell that something is 'off' even with a defect they have never seen. This setup is called one-class learning. The training data contains only the normal class, and at test time samples that differ from normal are classified as anomalies.

Here, anomaly detection splits into two tasks: 'detection,' which gives a whole image an anomaly score, and 'localization,' which scores each pixel or small region (patch) to build an anomaly map (heatmap). The PaDiM paper paper explains that localization gives more precise and more interpretable results. For workers on site, knowing 'where' the problem is matters as much as the 'defective' verdict itself.

Both foundational papers rely on a pretrained CNN (convolutional neural network). They only extract features from a model already trained on large-scale general images, without heavily training a new network. 'What normal looks like' is represented not by the model's weights but by statistics or a collection of gathered normal features.

How it works: PaDiM and PatchCore

PaDiM paper works as follows. 1) Normal images are fed into a pretrained CNN. 2) Activations corresponding to each patch position in the image are taken from several layers and concatenated to form patch embeddings (numeric vectors). Several layers are mixed because shallow layers capture fine texture and deep layers capture more abstract meaning. 3) The mean and covariance of normal patches at the same position are computed, representing each position as a multivariate Gaussian distribution. 4) At inspection time, how far a new patch lies from the normal distribution at that position is measured with the Mahalanobis distance (a distance that also accounts for the spread of the distribution) and used as the anomaly score. The paper highlights as an advantage for industrial use that the time and space complexity at test time is low and independent of the training data size.

PatchCore paper takes a different route. 1) It uses patch features from the CNN's intermediate layers and groups them with their neighbors to preserve spatial context. Intermediate layers are used to reduce the bias of deep-layer features, which are trained for general image classification and do not match factory images well. 2) All normal patch features are gathered into a memory bank (a store of normal features). 3) Because similar features are redundant, greedy coreset subsampling (keeping only a subset that represents the whole well) shrinks the bank, lowering storage and inference time. 4) At inspection time, the distance between each patch and the nearest normal patch in the bank is used as the anomaly score. It takes advantage of the fact that a whole image can be considered anomalous if even one patch is anomalous.

The PatchCore paper reported image-level detection AUROC (a discrimination metric; closer to 1 is better) of up to 99.6% on the widely used MVTec AD benchmark. It also stated that it cut errors by more than half compared with the previous best method. Both methods are attractive in practice because they do not require retraining the network on the target data.

How the field developed: from reconstruction to feature comparison

The PaDiM paper paper groups existing methods into two lines. The first is reconstruction-based. An autoencoder or GAN is trained only on normal images, and regions that are not reconstructed well are treated as anomalies. This is intuitive and easy to interpret, but the paper notes that performance is limited because autoencoders sometimes reconstruct anomalous images well too.

The second is embedding similarity. Features from a pretrained model are compared against a normal reference. Among these, comparing against all normal patches with KNN (k-nearest neighbors) gave good localization performance. But it had to search the entire training set at test time, so computation grew linearly as data increased. PaDiM avoided this scalability problem by summarizing normal data as per-position Gaussian distributions. It also broadened the evaluation setup to include data where product positions are not aligned, closer to real inspection conditions.

PatchCore paper belongs to the same feature-comparison family. It identified as limitations that deep-layer features trained on general images do not match factory images well, and that little normal information is available at test time. As solutions it introduced intermediate-layer patch features, neighbor grouping and a coreset-compressed memory bank. It reported performance comparable to existing methods using only part of the normal data, showing promise for settings with few samples.

Where it is heading: few samples, synthetic defects, inspection that explains

The first line of work covers cases where only a few normal images of a new product are available. Anchor and Adapt paper is a prompt-learning method in the VLM family. It first learns reference points (anchors) for 'normal' and 'anomalous' from labeled data of other products and freezes them. It then adjusts only the normal side using a few normal samples of the target product. The aim is to reduce the burden of having people write separate defect descriptions for each product. It reported competitive detection and localization performance in 1-, 2- and 4-shot settings across MVTec-AD and VisA.

The second is synthetic defects. Given only normal images, FLASH paper uses guidance from a VLM and an image generation model to create a small number of defect images. It cuts out the defect patches, stores them, and repeatedly composites them onto new normal images ('generate once, composite many times'). The synthetic data is used for threshold calibration. On MVTec AD 2, it reported 78.1%, close to the upper bound obtained when calibrating with real defects (image-level F1 of 83.6%).

The third is inspection that explains and reasons about defects. Anomaly-LR paper observes that industrial anomaly detection is moving beyond detection and localization toward multimodal inspection that explains and reasons about defects. It proposed a method that refines defect evidence step by step within the image's visual representation space. For this, it built IAD-LR-22K, a training dataset of 22,228 image-question pairs drawn from 4,523 industrial images.

Predictive maintenance and digital twins with sensor time series

Predictive maintenance (PdM) uses data from sensors, machines and production systems to estimate when equipment will fail, so maintenance can be planned before the failure. A hybrid architecture benchmark study paper ran more than 700 experiments on six deep learning architectures. Transformers tracked stable, slowly changing processes well, but on chaotic data they tended to mistake sensor noise for meaningful signal. In contrast, the study reported that hybrids combining LSTM (a type of RNN) layers with transformer layers were more robust to factory noise. The interpretation is that the LSTM acts as a noise filter.

The prediction horizon also matters. TQRNN30d paper covered 72 machines across 9 manufacturing facilities. It split training, validation and test at the machine level into 43/14/15 machines, evaluating on machines not used in training. It reported an F1 of 79.97% for predictions 30 days ahead.

Transferring to other machines and conditions is another major problem. FreqCondNorm paper tried to bring signals with sampling frequencies ranging from 1Hz to about 100kHz into a single transformer foundation model. For fault diagnosis, it reported 99.2% accuracy on the CWRU data and 82.1% on the MFPT data, which it was not trained on. However, the authors themselves noted that it did not improve remaining useful life (RUL) prediction.

A digital twin is a virtual replica that mirrors the structure and state of real equipment in data. A digital twin study paper compared five time-series foundation models on RUL prediction with the C-MAPSS data. Multivariate architectures that look at multiple sensors together clearly outperformed univariate ones, and the gap was especially large when operating conditions changed. The study then proposed a fusion method that uses the equipment structure stored in the digital twin (which part connects to which) to limit the range attention can look at. It reported that pretrained weights, adaptation to the target task and digital twin information complement one another.

Data and operational issues and limits in real-world deployment

The most common warning is 'it works in the lab but not in the factory.' The hybrid architecture benchmark study paper cites as its motivation a 'reliability gap': predictive maintenance models that looked perfect in the lab can fail unexpectedly on real factory data. Vision inspection is no different. PaDiM paper extended its evaluation to non-aligned data because real inspection conditions are rougher than benchmarks. A score such as 99.6% on MVTec AD should not be read as the defect detection rate on a real line.

The second issue is data change. When lighting, camera position, raw material lots or equipment aging change, what 'normal' looks like changes too. A model that learned only normal data can easily misjudge the new normal as anomalous. Operational procedures are needed to monitor this data drift and rebuild the normal reference. As FreqCondNorm paper shows, transferring to different machines or conditions is still not easy.

The third is the decision threshold. Anomaly scores are continuous, so a cutoff must be chosen. But since real defect samples are scarce, setting the threshold is also hard. This is why FLASH paper uses synthetic defects for threshold calibration. Still, the synthetic result fell short of the upper bound set by real defects.

Finally, there is accountability. Missed defects can lead to safety problems for customers, and too many false alarms stop the line. A realistic setup is human-in-the-loop: the model serves as a tool that narrows down suspicious areas, while people make final shipping and maintenance decisions and changes to decision criteria. The sources for this article contain no regulations from any specific country or agency on manufacturing AI. Regulatory requirements should therefore be checked separately against official guidance for each industry and country.

Further study

For vision inspection, it is best to start with PaDiM paper. Its introduction and related work sections briefly summarize the difference between reconstruction-based and feature-comparison approaches. Reading PatchCore paper next makes it natural to understand why a memory bank and coreset are needed. Both papers use MVTec AD, so looking at the benchmark figures alongside them helps.

After that, choose by interest. If new products have few samples, read Anchor and Adapt paper; if setting a threshold is hard because there is no defect data, read FLASH paper. If you are curious about inspection that explains defects in words, read Anomaly-LR paper.

For predictive maintenance, it is good to first understand the noise problem through the hybrid architecture benchmark study paper. Then look at machine-level split evaluation and long-horizon prediction in TQRNN30d paper, and the problem of transferring to other machines in FreqCondNorm paper. For combining digital twins with time-series foundation models, the digital twin study paper is the starting point.