Out of Distribution Detection for Reliable Deployment of Machine Learning Models
Article Sidebar
Machine Learning,
Model Robustness,
Out-Of-Distribution Detection,
Uncertainty Estimation
Main Article Content
William Yeoh
Fadly Fadly
Maria Ulfa
Abstract
Background: Machine learning models can give confident predictions when they receive inputs that are different from the data used during training. These predictions may not always be reliable when the model encounters unfamiliar inputs. OOD detection can help identify such inputs and reduce the risk of incorrect predictions in real-world applications.
Aims: This study evaluates several OOD detection methods and examines their ability to reduce unreliable automatic predictions.
Methods: Several post hoc detection methods were evaluated using ResNet classifiers trained on CIFAR datasets. The methods included Maximum Softmax Probability, ODIN, Energy-Based Detection, Mahalanobis Distance, Deep k Nearest Neighbors, and ReAct. The evaluation covered near-OOD, far-OOD, and corrupted-in-distribution conditions. Detection performance was also examined together with prediction errors, review needs, processing time, and memory use.
Results: The methods showed different performance across the tested conditions. Deep k-Nearest Neighbors performed well for near OOD inputs, while ReAct showed strong performance for far OOD detection. The deployment simulation showed that ReAct-based gating reduced OOD leakage and improved the accuracy of predictions that were processed automatically. It also reduced incorrect predictions made without additional review. Energy-based detection provided competitive results with relatively low computational requirements.
Conclusion: OOD detection can improve the reliability of machine learning systems. When combined with abstention and fallback mechanisms, unfamiliar inputs can be identified and sent for further review. This approach reduces unreliable predictions while allowing familiar inputs to continue through the automated process.
Article Details
Copyright (c) 2026 William Yeoh, Fadly Fadly, Maria Ulfa
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
References
Berger, C., Paschali, M., Glocker, B., & Kamnitsas, K. (2021). Confidence-Based Out-of-Distribution Detection: A Comparative Study and Analysis. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12959 LNCS, 122–132. https://doi.org/10.1007/978-3-030-87735-4_12
Cai, F., & Koutsoukos, X. (2022). Real-time out-of-distribution detection in cyber-physical systems with learning-enabled components. IET Cyber-Physical Systems: Theory and Applications, 7(4), 212–234. https://doi.org/10.1049/cps2.12034
Chen, Q., Li, K., Chen, Z., Maul, T., & Yin, J. (2024). Exploring feature sparsity for out-of-distribution detection. Scientific Reports, 14(1), 1–14. https://doi.org/10.1038/s41598-024-79934-7
Guérin, J., Delmas, K., Ferreira, R., & Guiochet, J. (2023). Out-Of-Distribution Detection Is Not All You Need. Proceedings of the 37th AAAI Conference on Artificial Intelligence, AAAI 2023, 37, 14829–14837. https://doi.org/10.1609/aaai.v37i12.26732
Guille-Escuret, C., Noël, P. A., Mitliagkas, I., Vazquez, D., & Monteiro, J. (2023). Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection. Advances in Neural Information Processing Systems, 37. https://doi.org/10.52202/079017-4162
Gwon, K., & Yoo, J. (2023). Out-of-Distribution (OOD) Detection and Generalization Improved by Augmenting Adversarial Mixup Samples. Electronics (Switzerland), 12(6). https://doi.org/10.3390/electronics12061421
Han, T., & Li, Y. F. (2022). Out-of-distribution detection-assisted trustworthy machinery fault diagnosis approach with uncertainty-aware deep ensembles. Reliability Engineering & System Safety, 226, 108648. https://doi.org/10.1016/J.RESS.2022.108648
Hendrycks, D., & Dietterich, T. (2019). Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. 7th International Conference on Learning Representations, ICLR 2019. https://arxiv.org/pdf/1903.12261
Kaur, R., Jha, S., Roy, A., Park, S., Dobriban, E., Sokolsky, O., & Lee, I. (2022). iDECODe: In-Distribution Equivariance for Conformal Out-of-Distribution Detection. Proceedings of the 36th AAAI Conference on Artificial Intelligence, AAAI 2022, 36(iD), 7104–7114. https://doi.org/10.1609/aaai.v36i7.20670
Kim, B. C., Kim, B., & Hyun, Y. (2024). Investigation of out-of-distribution detection across various models and training methodologies. Neural Networks, 175(February), 106288. https://doi.org/10.1016/j.neunet.2024.106288
Lahoti, P., Gummadi, K., & Weikum, G. (2023). Responsible model deployment via model-agnostic uncertainty learning. In Machine Learning (Vol. 112, Number 3). Springer US. https://doi.org/10.1007/s10994-022-06248-y
Lazebnik, T. (2026). Introducing “Inside” out of distribution. International Journal of Data Science and Analytics, 21(1), 1–12. https://doi.org/10.1007/s41060-025-00947-0
Lee, K., Lee, K., Lee, H., & Shin, J. (2018). A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks. Advances in Neural Information Processing Systems, 2018-December, 7167–7177. https://arxiv.org/pdf/1807.03888
Liang, H., Peng, L., & Sun, J. (2024). Selective Classification Under Distribution Shifts. Transactions on Machine Learning Research, 2024. https://doi.org/10.48550/arXiv.2405.05160
Liang, S., Li, Y., & Srikant, R. (2017). Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks. 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings. https://arxiv.org/pdf/1706.02690
Liu, W., Wang, X., Owens, J. D., & Li, Y. (2020). Energy-based Out-of-distribution Detection. Advances in Neural Information Processing Systems, 33, 21464–21475. https://github.com/wetliu/
Lu, S., Wang, Y., Sheng, L., He, L., Zheng, A., & Liang, J. (2025a). Out-of-Distribution Detection : A Task-Oriented Survey of Recent Advances. ACM Computing Surveys, 58(2). https://doi.org/10.1145/3760390
Lu, S., Wang, Y., Sheng, L., He, L., Zheng, A., & Liang, J. (2025b). Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advances. ACM Computing Surveys, 58(2), 39. https://doi.org/10.1145/3760390
Luo, J., Luo, J., Nan, G., & Li, D. (2023). Fake review detection system for online E-commerce platforms: A supervised general mixed probability approach. Decision Support Systems, 175, 114045. https://doi.org/10.1016/J.DSS.2023.114045
Mouhammad, C. S., Shi, Z., & Alouini, M. S. (2026). Review on Modulation Formats and Channel Coding in Free Space Optical Communication: Selection Criteria, Trade-offs, and Emerging Trends. Journal of Lightwave Technology. https://doi.org/10.1109/JLT.2026.3664749
Nie, J., Luo, Y., Ye, S., Zhang, Y., Tian, X., & Fang, Z. (2025). Out-of-Distribution Detection with Virtual Outlier Smoothing. International Journal of Computer Vision, 133(2), 724–741. https://doi.org/10.1007/s11263-024-02210-8
Rao, A., Keller, A., Kalra, N., Steed, R., Kwegyir-Aggrey, K., Klyman, K., Staheli, D., Bergman, A., Rao, A., Keller, A., Kalra, N., Steed, R., Kwegyir-Aggrey, K., Klyman, K., Staheli, D., & Bergman, A. (2026). Challenges to the monitoring of deployed AI systems: Center for AI Standards and Innovation. https://doi.org/10.6028/NIST.AI.800-4
Sun, Y., Ming, Y., Zhu, X., & Li, Y. (2022). Out-of-Distribution Detection with Deep Nearest Neighbors. Proceedings of Machine Learning Research, 162, 20827–20840. https://arxiv.org/pdf/2204.06507
Suryawati, E., Pardede, H. F., Zilvan, V., Ramdan, A., Krisnandi, D., Heryana, A., Yuwana, R. S., Kusumo, R. B. S., Arisal, A., & Supianto, A. A. (2021). Unsupervised feature learning-based encoder and adversarial networks. Journal of Big Data 2021 8:1, 8(1), 118-. https://doi.org/10.1186/S40537-021-00508-9
Tosaki, T., Uchino, E., Kojima, R., Mineharu, Y., Okamoto, Y., Arita, M., Miyai, N., Tamada, Y., Mikami, T., Murashita, K., Nakaji, S., & Okuno, Y. (2025). Out-of-distribution reject option method for dataset shift problem in early disease onset prediction. Scientific Reports, 15(1), 1–14. https://doi.org/10.1038/s41598-025-01811-8
Touvron, H., Bojanowski, P., Caron, M., Cord, M., El-Nouby, A., Grave, E., Izacard, G., Joulin, A., Synnaeve, G., Verbeek, J., & Jegou, H. (2023). ResMLP: Feedforward Networks for Image Classification with Data-Efficient Training. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4), 5314–5321. https://doi.org/10.1109/TPAMI.2022.3206148
Wang, H., Vaze, S., & Han, K. (2025). Dissecting Out-of-Distribution Detection and Open-Set Recognition: A Critical Analysis of Methods and Benchmarks. International Journal of Computer Vision, 133(3), 1326–1351. https://doi.org/10.1007/s11263-024-02222-4
Wang, K., Ma, Q., Shen, C., & Lu, J. (2025). Application of Uncertainty to Out-of-Distribution Detection for Autonomous Driving Perception Safety. IEEE Transactions on Intelligent Transportation Systems, 26(8), 11276–11293. https://doi.org/10.1109/TITS.2025.3557996
Wu, B., Jiang, J., Ren, H., Du, Z., Wang, W., Li, Z., Cai, D., He, X., Lin, B., & Liu, W. (2023). Towards In-Distribution Compatible Out-of-Distribution Detection. The Thirty-Seventh AAAI Conference on Artificial Intelligence, 39(9), 10333–10341. https://doi.org/10.1609/aaai.v37i9.26230
Wu, Y. C., Lyu, S. H., Shang, H., Wang, X., & Qian, C. (2024). Confidence-aware Contrastive Learning for Selective Classification. Proceedings of Machine Learning Research, 235, 53706–53729. https://doi.org/10.48550/arXiv.2406.04745
Xia, G., & Bouganis, C. S. (2024). Augmenting the Softmax with Additional Confidence Scores for Improved Selective Classification with Out-of-Distribution Data. International Journal of Computer Vision, 132(9), 3714–3752. https://doi.org/10.1007/s11263-024-02029-3
Yang, J., Wang, P., Zou, D., Zhou, Z., Ding, K., Peng, W., Wang, H., Chen, G., Li, B., Sun, Y., Du, X., Zhou, K., Zhang, W., Hendrycks, D., Li, Y., & Liu, Z. (2022). OpenOOD: Benchmarking Generalized Out-of-Distribution Detection. Advances in Neural Information Processing Systems, 35. https://arxiv.org/pdf/2210.07242
Yang, J., Zhou, K., Li, Y., & Liu, Z. (2024). Generalized Out-of-Distribution Detection: A Survey. International Journal of Computer Vision, 132(12), 5635–5662. https://doi.org/10.1007/S11263-024-02117-4
Yu, Y., Shin, S., Ko, M., & Lee, K. (2024). Exploring using jigsaw puzzles for out-of-distribution detection. Computer Vision and Image Understanding, 241(February), 103968. https://doi.org/10.1016/j.cviu.2024.103968
Zhao, Z., & Cao, L. (2024). Weighting non-IID batches for out-of-distribution detection. Machine Learning, 113(10), 7371–7391. https://doi.org/10.1007/s10994-024-06605-z