Article

Natural Synthetic Anomalies for Self-supervised Anomaly Detection and Localization

Authors:

Hannah M. Schlüter,

Benjamin Hou, and

Bernhard KainzAuthors Info & Claims

Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXI

October 2022

Pages 474 - 489

https://doi.org/10.1007/978-3-031-19821-2_27

Published: 23 October 2022 Publication History

Abstract

We introduce a simple and intuitive self-supervision task, Natural Synthetic Anomalies (NSA), for training an end-to-end model for anomaly detection and localization using only normal training data. NSA integrates Poisson image editing to seamlessly blend scaled patches of various sizes from separate images. This creates a wide range of synthetic anomalies which are more similar to natural sub-image irregularities than previous data-augmentation strategies for self-supervised anomaly detection. We evaluate the proposed method using natural and medical images. Our experiments with the MVTec AD dataset show that a model trained to localize NSA anomalies generalizes well to detecting real-world a priori unknown types of manufacturing defects. Our method achieves an overall detection AUROC of 97.2 outperforming all previous methods that learn without the use of additional datasets. Code available at https://github.com/hmsch/natural-synthetic-anomalies.

References

[1]

Abati, D., Porrello, A., Calderara, S., Cucchiara, R.: Latent space autoregression for novelty detection. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, 16–20 June 2019, pp. 481–490. Computer Vision Foundation/IEEE (2019)

[2]

Bergmann P, Batzner K, Fauser M, Sattlegger D, and Steger C The MVTec anomaly detection dataset: a comprehensive real-world dataset for unsupervised anomaly detection Int. J. Comput. Vision 2021 129 4 1038-1059

[3]

Bradski, G.: The OpenCV library. Dr. Dobb’s J. Softw. Tools (2000)

[4]

Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., Vedaldi, A.: Describing textures in the wild. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3606–3613 (2014)

[5]

Cohen, N., Hoshen, Y.: Sub-image anomaly detection with deep pyramid correspondences (2021)

[6]

Defard T, Setkov A, Loesch A, Audigier R, et al. Del Bimbo A et al. PaDiM: a patch distribution modeling framework for anomaly detection and localization Pattern Recognition. ICPR International Workshops and Challenges 2021 Cham Springer 475-489

[7]

Doersch, C., Gupta, A., Efros, A.A.: Unsupervised visual representation learning by context prediction. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1422–1430 (2015)

[8]

Gidaris, S., Singh, P., Komodakis, N.: Unsupervised representation learning by predicting image rotations. In: International Conference on Learning Representations (2018)

[9]

Golan, I., El-Yaniv, R.: Deep anomaly detection using geometric transformations. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS 2018, pp. 9781–9791. Curran Associates Inc., Red Hook (2018)

[10]

Grathwohl, W., Chen, R.T.Q., Bettencourt, J., Sutskever, I., Duvenaud, D.: FFJORD: free-form continuous dynamics for scalable reversible generative models. In: 7th International Conference on Learning Representations, ICLR 2019, 6–9 May 2019, New Orleans, LA, USA (2019). OpenReview.net

[11]

Gudovskiy, D.A., Ishizaka, S., Kozuka, K.: CFLOW-AD: real-time unsupervised anomaly detection with localization via conditional normalizing flows. In: 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 1819–1828 (2022)

[12]

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)

[13]

Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015)

[14]

Li, C.L., Sohn, K., Yoon, J., Pfister, T.: CutPaste: self-supervised learning for anomaly detection and localization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9664–9674, June 2021.

[15]

Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with warm restarts. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, 24–26 April 2017, Conference Track Proceedings (2017). OpenReview.net

[16]

Mack, A., Rock, I., et al.: Inattentional Blindness. MIT Press (1998)

[17]

Marimont, S.N., Tarroni, G.: Anomaly detection through latent space restoration using vector quantized variational autoencoders. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 1764–1767 (2021)

[18]

Nalisnick, E.T., Matsukawa, A., Teh, Y.W., Görür, D., Lakshminarayanan, B.: Do deep generative models know what they don’t know? In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, 6–9 May 2019 (2019). OpenReview.net

[19]

Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, vol. 32, pp. 8024–8035. Curran Associates, Inc. (2019)

[20]

Pawlowski, N., et al.: Unsupervised lesion detection in brain CT using Bayesian convolutional autoencoders. In: OpenReview (2018)

[21]

Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. In: ACM SIGGRAPH 2003 Papers, SIGGRAPH 2003, pp. 313–318. Association for Computing Machinery, New York (2003)

[22]

Raghu, M., Zhang, C., Kleinberg, J., Bengio, S.: Transfusion: understanding transfer learning for medical imaging. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’ Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32. Curran Associates, Inc. (2019)

[23]

Rippel, O., Mertens, P., Merhof, D.: Modeling the distribution of normal data in pre-trained deep features for anomaly detection. In: 25th International Conference on Pattern Recognition, ICPR 2020, Virtual Event/Milan, Italy, 10–15 January 2021, pp. 6726–6733. IEEE (2020)

[24]

Roth, K., Pemula, L., Zepeda, J., Schölkopf, B., Brox, T., Gehler, P.: Towards total recall in industrial anomaly detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14318–14328, June 2022

[25]

Ruff, L., et al.: A unifying review of deep and shallow anomaly detection. Proc. IEEE 109, 756–795 (2021)

[26]

Ruff, L., et al.: Deep one-class classification. In: Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 4393–4402. PMLR (2018)

[27]

Schlegl T, Seeböck P, Waldstein SM, Langs G, and Schmidt-Erfurth U f-AnoGAN: fast unsupervised anomaly detection with generative adversarial networks Med. Image Anal. 2019 54 30-44

[28]

Tan, J., Hou, B., Batten, J., Qiu, H., Kainz, B.: Detecting outliers with foreign patch interpolation. Mach. Learn. Biomed. Imaging 2022, 1–27 (2022). https://melba-journal.org/papers/2022:013.html

[29]

Tan J, Hou B, Day T, Simpson J, Rueckert D, Kainz B, et al. de Bruijne M et al. Detecting outliers with poisson image interpolation Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 2021 Cham Springer 581-591

[30]

Tan, M., Le, Q.: EfficientNet: Rethinking model scaling for convolutional neural networks. In: Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 97, pp. 6105–6114. PMLR (06 2019)

[31]

Tang, Y.X., et al.: Automated abnormality classification of chest radiographs using deep convolutional neural networks. NPJ Digit. Med. 3(1), 1–8 (2020)

[32]

Wang, L., Zhang, D., Guo, J., Han, Y.: Image anomaly detection using normal data only by latent space resampling. Appl. Sci. 10(23) (2020)

[33]

Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: ChestX-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2097–2106 (2017).

[34]

Yi, J., Yoon, S.: Patch SVDD: patch-level SVDD for anomaly detection and segmentation. In: Proceedings of the Asian Conference on Computer Vision (ACCV) (2020)

[35]

Yu, J., et al.: FastFlow: unsupervised anomaly detection and localization via 2d normalizing flows. CoRR abs/2111.07677 (2021)

[36]

Zavrtanik, V., Kristan, M., Skocaj, D.: DRAEM-a discriminatively trained reconstruction embedding for surface anomaly detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8330–8339 (2021).

[37]

Zhang, L., Goldstein, M., Ranganath, R.: Understanding failures in out-of-distribution detection with deep generative models. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 139, pp. 12427–12436. PMLR, 18–24 July 2021

[38]

Zimmerer D, Isensee F, Petersen J, Kohl S, Maier-Hein K, et al. Shen D et al. Unsupervised anomaly localization using variational auto-encoders Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 2019 Cham Springer 289-297

Cited By

Lyu SMo DWong W(2024)REBKnowledge-Based Systems10.1016/j.knosys.2024.111563290:COnline publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1016/j.knosys.2024.111563
Song YCheng J(2023)Self-supervised anomaly detection of medical images based on dual-module discrepancyProceedings of the 5th ACM International Conference on Multimedia in Asia10.1145/3595916.3626388(1-7)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3595916.3626388
Jiang YCao YShen W(2023)A masked reverse knowledge distillation method incorporating global and local information for image anomaly detectionKnowledge-Based Systems10.1016/j.knosys.2023.110982280:COnline publication date: 25-Nov-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110982
Show More Cited By

Index Terms

Natural Synthetic Anomalies for Self-supervised Anomaly Detection and Localization
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Machine learning
    1. Learning paradigms

Index terms have been assigned to the content through auto-classification.

Recommendations

Deep Weakly-supervised Anomaly Detection
KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Recent semi-supervised anomaly detection methods that are trained using small labeled anomaly examples and large unlabeled data (mostly normal data) have shown largely improved performance over unsupervised methods. However, these methods often focus on ...
Read More
Self-supervised Augmented Patches Segmentation for Anomaly Detection
Computer Vision – ACCV 2022
Abstract
In this paper, our goal is to detect unknown defects in high-resolution images in the absence of anomalous data. Anomaly detection is usually performed at image-level or pixel-level. Considering that pixel-level anomaly classification achieves ...
Read More
Self-supervised anomaly detection in computer vision and beyond: A survey and outlook
Abstract
Anomaly detection (AD) plays a crucial role in various domains, including cybersecurity, finance, and healthcare, by identifying patterns or events that deviate from normal behavior. In recent years, significant progress has been made in this ...
Highlights
- We present a cohesive overview of self-supervised methods for anomaly detection.
- We categorize existing self-supervised anomaly detection algorithms. their proxy tasks.
- We conduct a comprehensive performance comparison.
- We ...
Read More

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXI

Oct 2022

809 pages

ISBN:978-3-031-19820-5

DOI:10.1007/978-3-031-19821-2

Editors:
Shai Avidan
Tel Aviv University, Tel Aviv, Israel
,
Gabriel Brostow
University College London, London, UK
,
Moustapha Cissé
Google AI, Accra, Ghana
,
Giovanni Maria Farinella
University of Catania, Catania, Italy
,
Tal Hassner
Facebook (United States), Menlo Park, CA, USA

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 23 October 2022

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Lyu SMo DWong W(2024)REBKnowledge-Based Systems10.1016/j.knosys.2024.111563290:COnline publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1016/j.knosys.2024.111563
Song YCheng J(2023)Self-supervised anomaly detection of medical images based on dual-module discrepancyProceedings of the 5th ACM International Conference on Multimedia in Asia10.1145/3595916.3626388(1-7)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3595916.3626388
Jiang YCao YShen W(2023)A masked reverse knowledge distillation method incorporating global and local information for image anomaly detectionKnowledge-Based Systems10.1016/j.knosys.2023.110982280:COnline publication date: 25-Nov-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110982
Tong GLi QSong Y(2023)Two-stage reverse knowledge distillation incorporated and Self-Supervised Masking strategy for industrial anomaly detectionKnowledge-Based Systems10.1016/j.knosys.2023.110611273:COnline publication date: 3-Aug-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110611
Pei MLiu N(2023)A Simplified Student Network with Multi-teacher Feature Fusion for Industrial Defect DetectionPattern Recognition10.1007/978-3-031-47634-1_19(245-258)Online publication date: 5-Nov-2023
https://dl.acm.org/doi/10.1007/978-3-031-47634-1_19
Bozorgtabar BMahapatra DThiran J(2023)AMAE: Adaptation of Pre-trained Masked Autoencoder for Dual-Distribution Anomaly Detection in Chest X-RaysMedical Image Computing and Computer Assisted Intervention – MICCAI 202310.1007/978-3-031-43907-0_19(195-205)Online publication date: 8-Oct-2023
https://dl.acm.org/doi/10.1007/978-3-031-43907-0_19
Baugh MTan JMüller JDombrowski MBatten JKainz B(2023)Many Tasks Make Light Work: Learning to Localise Medical Anomalies from Multiple Synthetic TasksMedical Image Computing and Computer Assisted Intervention – MICCAI 202310.1007/978-3-031-43907-0_16(162-172)Online publication date: 8-Oct-2023
https://dl.acm.org/doi/10.1007/978-3-031-43907-0_16

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents