Unsupervised Visual-to-Geometric Feature Reconstruction for Vision-Based Industrial Anomaly Detection
Industrial anomaly detection involves identifying abnormal regions in products and plays a crucial role in quality inspection. While 2D image-based anomaly detection has been extensively explored, combining two-dimensional (2D) images with three-dimensional (3D) point clouds remains less studied. Ex...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10820339/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841550881401602048 |
|---|---|
| author | Dinh-Cuong Hoang Phan Xuan Tan Anh-Nhat Nguyen Duc-Thanh Tran Van-Hiep Duong Anh-Truong Mai Duc-Long Pham Khanh-Toan Phan Minh-Quang Do Ta Huu Anh Duong Tuan-Minh Huynh Son-Anh Bui Duc-Manh Nguyen Viet-Anh Trinh Khanh-Duong Tran Thu-Uyen Nguyen |
| author_facet | Dinh-Cuong Hoang Phan Xuan Tan Anh-Nhat Nguyen Duc-Thanh Tran Van-Hiep Duong Anh-Truong Mai Duc-Long Pham Khanh-Toan Phan Minh-Quang Do Ta Huu Anh Duong Tuan-Minh Huynh Son-Anh Bui Duc-Manh Nguyen Viet-Anh Trinh Khanh-Duong Tran Thu-Uyen Nguyen |
| author_sort | Dinh-Cuong Hoang |
| collection | DOAJ |
| description | Industrial anomaly detection involves identifying abnormal regions in products and plays a crucial role in quality inspection. While 2D image-based anomaly detection has been extensively explored, combining two-dimensional (2D) images with three-dimensional (3D) point clouds remains less studied. Existing multimodal methods often combine features from different modalities, leading to feature interference and degraded performance. To overcome this, we propose a novel framework for unsupervised industrial anomaly detection that leverages both visual and geometric information. Specifically, we use pre-trained 2D and 3D models to extract visual features from color images and geometric features from 3D point clouds. Instead of directly fusing these features, we propose a geometric feature reconstruction network that predicts 3D geometric features from the 2D visual features. During training, we minimize the difference between the predicted geometric features and the extracted geometric features, enabling the model to learn how 2D appearance correlates with 3D structure in anomaly-free images. During inference, this learned relationship allows the model to detect anomalies: significant discrepancies between the reconstructed and actual geometric features indicate abnormal regions. Evaluated on the MVTec 3D-AD dataset, our method achieves state-of-the-art performance with an average image-level AUROC score of 0.968, surpassing previous approaches. Additionally, it provides fast inference at 8.2 frames per second with a memory footprint of only 1045 MB, making it highly efficient for industrial applications. |
| format | Article |
| id | doaj-art-e1b5e5dde7b04e33beffb0d8bea260e0 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-e1b5e5dde7b04e33beffb0d8bea260e02025-01-10T00:00:52ZengIEEEIEEE Access2169-35362025-01-01133667368210.1109/ACCESS.2025.352556710820339Unsupervised Visual-to-Geometric Feature Reconstruction for Vision-Based Industrial Anomaly DetectionDinh-Cuong Hoang0https://orcid.org/0000-0001-6058-2426Phan Xuan Tan1https://orcid.org/0000-0002-9592-0226Anh-Nhat Nguyen2https://orcid.org/0009-0009-4006-4352Duc-Thanh Tran3https://orcid.org/0009-0009-2314-0560Van-Hiep Duong4https://orcid.org/0009-0006-3800-3014Anh-Truong Mai5https://orcid.org/0009-0001-2469-0961Duc-Long Pham6https://orcid.org/0009-0000-7851-6064Khanh-Toan Phan7https://orcid.org/0009-0006-6378-8052Minh-Quang Do8Ta Huu Anh Duong9https://orcid.org/0009-0002-1952-5176Tuan-Minh Huynh10Son-Anh Bui11https://orcid.org/0009-0007-9451-5695Duc-Manh Nguyen12https://orcid.org/0009-0002-2373-4616Viet-Anh Trinh13https://orcid.org/0009-0000-6100-2403Khanh-Duong Tran14https://orcid.org/0009-0001-7312-1060Thu-Uyen Nguyen15https://orcid.org/0009-0001-0689-1292Greenwich Vietnam, FPT University, Hanoi, VietnamCollege of Engineering, Shibaura Institute of Technology, Tokyo, JapanIT Department, FPT University, Hanoi, VietnamIT Department, FPT University, Hanoi, VietnamIT Department, FPT University, Hanoi, VietnamIT Department, FPT University, Hanoi, VietnamIT Department, FPT University, Hanoi, VietnamIT Department, FPT University, Hanoi, VietnamIT Department, FPT University, Hanoi, VietnamGreenwich Vietnam, FPT University, Hanoi, VietnamGreenwich Vietnam, FPT University, Hanoi, VietnamGreenwich Vietnam, FPT University, Hanoi, VietnamGreenwich Vietnam, FPT University, Hanoi, VietnamGreenwich Vietnam, FPT University, Hanoi, VietnamGreenwich Vietnam, FPT University, Hanoi, VietnamGreenwich Vietnam, FPT University, Hanoi, VietnamIndustrial anomaly detection involves identifying abnormal regions in products and plays a crucial role in quality inspection. While 2D image-based anomaly detection has been extensively explored, combining two-dimensional (2D) images with three-dimensional (3D) point clouds remains less studied. Existing multimodal methods often combine features from different modalities, leading to feature interference and degraded performance. To overcome this, we propose a novel framework for unsupervised industrial anomaly detection that leverages both visual and geometric information. Specifically, we use pre-trained 2D and 3D models to extract visual features from color images and geometric features from 3D point clouds. Instead of directly fusing these features, we propose a geometric feature reconstruction network that predicts 3D geometric features from the 2D visual features. During training, we minimize the difference between the predicted geometric features and the extracted geometric features, enabling the model to learn how 2D appearance correlates with 3D structure in anomaly-free images. During inference, this learned relationship allows the model to detect anomalies: significant discrepancies between the reconstructed and actual geometric features indicate abnormal regions. Evaluated on the MVTec 3D-AD dataset, our method achieves state-of-the-art performance with an average image-level AUROC score of 0.968, surpassing previous approaches. Additionally, it provides fast inference at 8.2 frames per second with a memory footprint of only 1045 MB, making it highly efficient for industrial applications.https://ieeexplore.ieee.org/document/10820339/Industrial anomaly detectionimage-based anomaly detection3D point clouds |
| spellingShingle | Dinh-Cuong Hoang Phan Xuan Tan Anh-Nhat Nguyen Duc-Thanh Tran Van-Hiep Duong Anh-Truong Mai Duc-Long Pham Khanh-Toan Phan Minh-Quang Do Ta Huu Anh Duong Tuan-Minh Huynh Son-Anh Bui Duc-Manh Nguyen Viet-Anh Trinh Khanh-Duong Tran Thu-Uyen Nguyen Unsupervised Visual-to-Geometric Feature Reconstruction for Vision-Based Industrial Anomaly Detection IEEE Access Industrial anomaly detection image-based anomaly detection 3D point clouds |
| title | Unsupervised Visual-to-Geometric Feature Reconstruction for Vision-Based Industrial Anomaly Detection |
| title_full | Unsupervised Visual-to-Geometric Feature Reconstruction for Vision-Based Industrial Anomaly Detection |
| title_fullStr | Unsupervised Visual-to-Geometric Feature Reconstruction for Vision-Based Industrial Anomaly Detection |
| title_full_unstemmed | Unsupervised Visual-to-Geometric Feature Reconstruction for Vision-Based Industrial Anomaly Detection |
| title_short | Unsupervised Visual-to-Geometric Feature Reconstruction for Vision-Based Industrial Anomaly Detection |
| title_sort | unsupervised visual to geometric feature reconstruction for vision based industrial anomaly detection |
| topic | Industrial anomaly detection image-based anomaly detection 3D point clouds |
| url | https://ieeexplore.ieee.org/document/10820339/ |
| work_keys_str_mv | AT dinhcuonghoang unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT phanxuantan unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT anhnhatnguyen unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT ducthanhtran unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT vanhiepduong unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT anhtruongmai unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT duclongpham unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT khanhtoanphan unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT minhquangdo unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT tahuuanhduong unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT tuanminhhuynh unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT sonanhbui unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT ducmanhnguyen unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT vietanhtrinh unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT khanhduongtran unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection AT thuuyennguyen unsupervisedvisualtogeometricfeaturereconstructionforvisionbasedindustrialanomalydetection |