Self-supervised ViT for low-label industrial defect detection — IEEE Transactions 2025 Base Paper Project

This project builds a self-supervised Vision Transformer (ViT) system that inspects product images for surface defects while needing only a small labelled dataset, using masked autoencoder (MAE) pretraining on unlabelled images and fine-tuning on few labels. Manual visual inspection on a production line is slow, tiring and inconsistent: the same scratch can be passed by one inspector and rejected by another, and labelling thousands of defect photos to train a conventional supervised model is expensive for a student team. The system first learns what a normal product looks like from unlabelled images, then flags anything that deviates from that learned normal appearance, so a handful of labelled defect examples is enough to start. An attention heatmap shows where the model looked, and an uncertainty score decides whether a verdict can be trusted or should go to a human reviewer.

This is a built-to-order software project: the training code, inspection demo, report, presentation and viva Q&A are prepared fresh for you.

Base paper at a glance

Base Paper

• Paper: Self-Supervised and Interpretable Anomaly Detection Using Network Transformers
• Authors: D. L. Marino, C. S. Wickramasinghe, C. Rieger, and M. Manic (Marino et al.)
• Journal: IEEE Transactions on Industrial Informatics, vol. 21, no. 5, pp. 4252-4261, 2025
• DOI: https://doi.org/10.1109/TII.2025.3534443
• What is implemented from the paper: The paper's self-supervised learning idea and its interpretability principle — a transformer model trained without defect labels that learns normal behaviour, plus attention-based explanations that show which inputs drove an anomaly verdict.
• Honest transfer framing (read this first): The base paper is about self-supervised anomaly detection for network traffic and industrial control systems, NOT about product images. It never claimed image defect detection. This project TRANSFERS the paper's self-supervised plus interpretable framework to a new domain: product images, using a Vision Transformer (ViT) encoder and masked autoencoder (MAE) pretraining in place of the paper's network transformer. The transfer itself — proving that a framework built for network sequences can work on image patches — is the student's own contribution, and it is treated in the report as a hypothesis to be tested, not as something the paper already proved.

Own Features (the student's own contribution, distinct from the base paper)

• [Cross-modal transfer to images] The paper's framework is re-engineered for product images: image patches replace network-traffic tokens, a ViT encoder replaces the network transformer, and MAE reconstruction error becomes the anomaly signal. This transfer is designed, implemented and evaluated by the student team.
• [Low-label protocol with label count as a variable] Training is repeated at three labelled-set sizes (approximately 10, 50 and 200 labelled images per class, design protocol) and image-level AUROC is reported at each size, so the report can show exactly how much labelling the system actually needs.
• [MC-dropout uncertainty triage] At inspection time the model is run multiple times with dropout active; disagreement across runs becomes an uncertainty score, and samples above a configurable threshold are routed to a manual-review queue instead of receiving an automatic pass/fail verdict.
• [Defect localisation maps] Patch-level attention and reconstruction-error maps are rendered as heatmaps over the inspected image, so a reviewer can see where a scratch or crack was found rather than trusting a bare score.

This self-supervised ViT defect-detection system works in three phases: MAE pretraining, few-label fine-tuning, and uncertainty-aware inspection. First, product photos are split into a large unlabelled set of normal products and a small labelled set of normal and defective examples, all resized to 224 x 224 pixels and cut into patches. Pretraining masks most patches and teaches the model to reconstruct them from the visible ones, so it learns normal appearance without any labels. Fine-tuning then teaches the pretrained encoder to score defect likelihood from the small labelled set. At inspection time, each new photo is scored, localised with a heatmap, and checked for uncertainty — confident verdicts are automatic, unsure samples go to manual review, and every reviewed sample is logged for the report's error analysis, closing the loop between inspection and evidence.

1. Product photos are collected and split into a large unlabelled set of normal products and a small labelled set containing normal and defective examples.
2. Images are resized to a fixed input size (224 x 224 pixels, design setting), normalized, and split into non-overlapping patches for the ViT encoder.
3. In the MAE pretraining phase, a large fraction of patches (approximately 75%, design setting) is masked and the model learns to reconstruct them from the visible patches, using only unlabelled normal images.
4. In the fine-tuning phase, the pretrained encoder is trained on the small labelled set to score images for defect likelihood and, where labels allow, defect type.
5. At evaluation time, image-level AUROC and patch-level localisation quality are measured on a held-out test split at each of the three label-count settings, and the trend is recorded for the report.
6. In the inspection demo, a new photo is encoded, scored, and shown with its attention heatmap and MAE reconstruction; the MC-dropout uncertainty score decides whether the verdict is automatic or routed to manual review.
7. Reviewed samples are logged with their score, uncertainty and heatmap, so the review queue itself becomes evidence for the report's error-analysis section.

Tech stack:

• Python 3
• PyTorch
• Vision Transformer (ViT) encoder
• Masked autoencoder (MAE) pretraining
• OpenCV image preprocessing
• MVTec AD benchmark subset (evaluation reference)
• HTML · CSS · JavaScript inspection demo
• Jupyter Notebook (training & evaluation)

Dataset & model details

• Dataset: MVTec AD (MVTec Software GmbH) — a public industrial inspection benchmark of normal and defective product images across object and texture categories; a selected subset of categories is used for this implementation, plus a small custom set of product photos collected by the team where a category is unavailable. Normal images are used for MAE pretraining; the small labelled subset carries defect labels.
• Task: Unsupervised-pretrained anomaly detection with few-label fine-tuning; input = one product image (224 x 224 pixels), output = anomaly score, defect heatmap, and uncertainty score.
• Model: Vision Transformer encoder (patch size 16 x 16 pixels, design setting) pretrained with a masked autoencoder decoder on unlabelled normal images, then fine-tuned with a lightweight scoring head on the small labelled set; MC dropout is enabled at inference for uncertainty estimation.
• Metrics: Image-level AUROC, pixel-level AUROC for localisation, and uncertainty-vs-accuracy analysis (how often high-uncertainty samples are the wrong ones). No accuracy or AUROC figure is claimed as measured until the training runs for the order are executed; the report documents the actual achieved figures.

The key features of this self-supervised ViT defect-detection system are the pieces that make it work with few labels and stay explainable: MAE pretraining learns normal product appearance from unlabelled images, so only a small labelled set is needed for fine-tuning; every verdict carries an attention heatmap and a reconstruction view that show exactly where a suspected defect is; and MC-dropout uncertainty triage refuses to guess on unsure samples, routing them to manual review instead. Together with the low-label comparison study, which measures performance at three label counts, these features give the project both a working inspection pipeline and a genuine experiment an examiner can interrogate, rather than a bare classifier score — and the demo console makes each feature visible and testable in the browser, so an examiner can try the heatmap, the reconstruction view and the triage threshold directly.

• [MAE pretraining on unlabelled images] The ViT encoder is first pretrained by masking random image patches and reconstructing them, learning normal product appearance from unlabelled photos before any defect label is used.
• [Few-label fine-tuning] A small labelled set (defect vs normal, plus defect-type labels where available) fine-tunes the pretrained encoder, so the project is feasible without a large annotated dataset.
• [Attention heatmap for every verdict] Each inspection returns a patch-level heatmap over the product photo, showing which region drove the anomaly score — the interpretability feature carried over from the base paper's design.
• [MAE reconstruction view] The demo shows the model's reconstruction of each inspected image side by side with the original, so a defect appears as the region the model could not reconstruct.
• [MC-dropout uncertainty triage] Multiple stochastic forward passes produce an uncertainty score per sample, and a threshold slider routes uncertain samples to manual review in the demo queue.
• [Low-label comparison study] The same pipeline is evaluated at three labelled-set sizes and the AUROC trend is tabulated in the report, giving the viva a genuine experimental result to discuss.
• [Inspection demo console] A working web demo lets the user pick a product sample, toggle the heatmap and reconstruction views, adjust the triage threshold, and watch the review queue re-route samples live.

• Complete training code: MAE pretraining, few-label fine-tuning, and evaluation notebooks
• Inspection demo console (heatmap, reconstruction view, uncertainty triage queue)
• Project report PDF (base-paper analysis, transfer design, low-label study, results, limitations)
• Base-paper summary and literature survey section with the verified Marino et al. citation
• PPT presentation for final review
• Viva Q&A preparation document (ViT, MAE, self-supervision, uncertainty, transfer validity)
• Source code with setup instructions