AI-Driven Software Testing for a Test Center of Excellence: A Systematic Mapping and Evidence Synthesis

Ali Ahmad

doi:10.15408/jti.v19i2.53564

Authors

Ali Ahmad System and Information Technology Study Program, Faculty of Science and Technology, University of Darunnajah, Jakarta, Indonesia

DOI:

https://doi.org/10.15408/jti.v19i2.53564

Keywords:

Artificial Intelligence, Software Testing, Test Center of Excellence, Test Automation, Test Generation, Machine Learning

Abstract

Artificial intelligence (AI) is increasingly used to automate and optimize software testing, but the evidence is distributed across heterogeneous testing activities, AI techniques, and evaluation measures. This study maps AI-driven software-testing research and translates the evidence into capabilities relevant to a Test Center of Excellence (TCoE). A systematic evidence workflow was applied to a unified audit corpus of 2,072 records, followed by focused eligibility reconciliation and full-text verification. The final focused evidence workset contains 29 records: 15 primary empirical studies, 9 secondary studies, 1 scope exclusion, and 4 full-text/access exclusions. The primary evidence covers machine learning, reinforcement learning, evolutionary and search-based methods, deep learning, explainable AI, and large language models. The supported testing activities include test generation, selection and prioritization, minimization and optimization, fuzzing, validation, adaptive automation, and maintenance. The synthesis shows that AI should be treated in a TCoE as a governed capability layer rather than as a single automation tool. The proposed capability model links AI techniques to testing activities, measurable outcomes, reusable TCoE capabilities, and governance with human oversight. The study also identifies gaps in reliability, context awareness, test-oracle support, long-term maintenance, CI/CD integration, reproducibility, and standardized evaluation.

References

[1] A. A. Philip, R. Bhagwan, R. Kumar, C. S. Maddila, and N. Nagappan, “FastLane: Test minimization for rapidly deployed large-scale online services,” Proc 41st IEEEACM Int Conf Softw. Eng. ICSE, pp. 408–418, 2019, doi: 10.1109/ICSE.2019.00054.

[2] S. Gerasimou, H. F. Eniser, A. Sen, and A. Cakan, “Importance-driven deep learning system testing,” Proc 42nd IEEEACM Int Conf Softw. Eng. ICSE, pp. 1123–1134, 2020, doi: 10.1145/3377811.3380391.

[3] Q. Hu et al., “An empirical study on data distribution-aware test selection for deep learning enhancement,” ACM Trans Softw. Eng. Methodol., 2022, doi: 10.1145/3511598.

[4] J. Kim, R. Feldt, and S. Yoo, “Evaluating Surprise Adequacy for Deep Learning System Testing,” ACM Trans Softw. Eng. Methodol., 2023, doi: 10.1145/3546947.

[5] X. Dang, Y. Li, M. Papadakis, J. Klein, T. F. Bissyandé, and Y. Le Traon, “Test input prioritization for machine learning classifiers,” IEEE Trans Softw. Eng., pp. 413–442, 2024, doi: 10.1109/TSE.2024.3350019.

[6] M. A. Khan et al., “On the effectiveness of feature selection techniques in the context of ML-based regression test prioritization,” IEEE Access, pp. 131556–131575, 2024, doi: 10.1109/ACCESS.2024.3459656.

[7] D. Marijan, “Comparative study of machine learning test case prioritization for continuous integration testing,” Softw. Qual. J., 2023, doi: 10.1007/s11219-023-09646-0.

[8] M. Esnaashari and A. H. Damia, “Automation of software test data generation using genetic algorithm and reinforcement learning,” Expert Syst. Appl., 2021, doi: 10.1016/j.eswa.2021.115446.

[9] B. K. Konidena, A. R. Bairi, and T. Pichaimani, “Reinforcement learning-driven adaptive test case generation in Agile development,” Am. J. Data Sci. Artif. Intell. Innov., pp. 241–273, 2021.

[10] S. Ji et al., “TAEFuzz: Automatic fuzzing for image-based deep learning systems via transferable adversarial examples,” ACM Trans Softw. Eng. Methodol., 2025, doi: 10.1145/3714463.

[11] Y. Zhang et al., “CITYWALK: Enhancing LLM-based C++ unit test generation via project-dependency awareness and language-specific knowledge,” ACM Trans Softw. Eng. Methodol., 2026, doi: 10.1145/3763791.

[12] M. Deder, S. Sahin, M. Yilmazer, and M. Karaköse, “Hallucination-reduced and robust accuracy unit test generation,” Int. J. Manag. Data Anal., pp. 191–205, 2026.

[13] S. Senthil Murugan and M. Balamurugan, “An adaptive scriptless behavior-driven development automation framework with self-healing intelligence for evolving software applications,” J. Internet Serv. Inf. Secur., pp. 291–310, 2026, doi: 10.58346/JISIS.2026.I1.017.

[14] B. A. Kitchenham and S. Charters, “Guidelines for performing systematic literature reviews in software engineering,” 2007.

[15] K. Petersen, S. Vakkalank, and L. Kuzniarz, “Guidelines for conducting systematic mapping studies in software engineering: An update,” Inf. Softw. Technol., pp. 1–18, 2015, doi: 10.1016/j.infsof.2015.03.007.

[16] M. J. Page, J. E. McKenzie, P. M. Bossuyt, and et al., “The PRISMA 2020 statement: An updated guideline for reporting systematic reviews,” BMJ, 2021, doi: 10.1136/bmj.n71.

[17] D. S. Cruzes and T. Dybå, “Synthesizing evidence in software engineering research,” in Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, Bolzano-Bozen Italy: ACM, Sep. 2010, pp. 1–10. doi: 10.1145/1852786.1852788.

[18] A. Bala and R. S. Chhillar, “Automatic test cases generation using multistage-based genetic algorithm for object oriented testing,” Int. J. Recent Technol. Eng., pp. 561–573, 2019, doi: 10.35940/ijrte.b1099.0982s1019.

[19] X. Chen, M. Biagiola, V. Riccio, M. d’Amorim, and A. Stocco, “XMutant: XAI-based fuzzing for deep learning systems,” Empir. Softw. Eng., 2026, doi: 10.1007/s10664-025-10792-1.

[20] R. Wang, C. Artho, L. M. Kristensen, and V. Stolz, “Multi-objective search for model-based testing,” Proc 2020 IEEE 20th Int Conf Softw. Qual. Reliab. Secur. QRS, pp. 130–141, 2020, doi: 10.1109/QRS51102.2020.00029.

[21] R. Pan, T. A. Ghaleb, and L. C. Briand, “LTM: Scalable and Black-Box Similarity-Based Test Suite Minimization Based on Language Models,” IEEE Trans Softw. Eng., pp. 3053–3070, 2024, doi: 10.1109/TSE.2024.3469582.

[22] A. Escalante-Viteri and D. Mauricio, “Artificial intelligence in software testing: A systematic review of a decade of evolution and taxonomy,” Algorithms, 2025, doi: 10.3390/a18110717.

[23] A. Faraji and N. Pombo, “AI-driven software test automation: An AI4SE-oriented survey of techniques, tools, and challenges,” IEEE Access, pp. 183296–183313, 2025, doi: 10.1109/ACCESS.2025.3623944.

[24] F. Ricca, A. Marchetto, and A. Stocco, “A multi-year grey literature review on AI-assisted test automation,” Inf. Softw. Technol., 2025, doi: 10.1016/j.infsof.2025.107799.

[25] V. Garousi, N. Joy, and D. Taibi, “AI-powered software testing tools: Full autonomy remains a distant goal,” IEEE Softw., pp. 98–111, 2025, doi: 10.1109/MS.2025.3557895.

[26] M. Tasarsu, A. V. Tokmak, and C. Catal, “Test case generation using large language models: A systematic literature review,” Clust. Comput., 2026, doi: 10.1007/s10586-026-06021-z.

[27] M. Lahami and M. Krichen, “Architectural perspectives on GenAI-driven software testing systems: A systematic literature review,” Inf. Softw. Technol., 2026, doi: 10.1016/j.infsof.2026.108246.