Abstract

This paper tries to explore recent research developments in the application of Machine Learning in functional regression testing of GUIs, mainly focusing towards test case optimization scenarios. A brief literature study was conducted by exploring the available literature from top digital repositories mainly from years 2017-2022 and identifying the research gaps and challenges. Analysis reported certain important research gaps in the available literature and also challenges faced by researchers. This paper provides a quick overview for those who are interested in this area of research. Simplified description and presentation of the research literature provides clear mapping for further research scope.

References

• Ngah A., Munro M., AbdallahM,”An Overview of Regression Testing”, Journal of Telecommunication, Electronic and Computer Engineering 9, pp. 45-49, 2017.
• Durelli V.H., Durelli R.S., Borges S.D., Endo A.T., Eler M.M., Dias D.R., Guimarães M.P, “Machine Learning Applied to Software Testing: A Systematic Mapping Study”. IEEE Transactions on Reliability 68, pp 1189-1212, 2019.
• Nass M.D., Alégroth E., Feldt R, “Why many challenges with GUI test automation (will) remain”, Inf. Softw. Technol. 138, 106625, 2021.
• “Infer”, Available at: https://fbinfer.com/. Accessed on 20 February 2022.
• “Diffblue”, Available at: https://www.diffblue.com/.Accessed on 21 March 2022.
• “SmartBear” Available at: https://smartbear.com/. Accessed on 21 March 2022.
• Chittimalli P.K., Harrold M.J,”Recomputing coverage information to assist regression testing”. IEEE Transactions on Software Engineering 35(4): pp. 452–469, 2009.
• “softwaretestinghelp”, Available at: https://www.softwaretestinghelp.com/. Accessed on 26 March 2022.
• Arora P.K. and Bhatia R. 2018. Agent-Based Regression Test Case Generation using Class Diagram, Use cases and Activity Diagram. In Procedia Computer Science 125: pp 747-753. ISSN 1877-0509, https://doi.org/10.1016/j.procs.2017.12.096 2018.
• Wetzlmaier T. and Ramler R. 2017. Hybrid monkey testing: enhancing automated GUI tests with random test generation. In Proceedings of the 8th ACM SIGSOFT International Workshop on Automated Software Testing.
• Kamal M.M., Darwish S.M. and ElfatatryA. 2019.Enhancing the Automation of GUI Testing. In Proceedings of the 2019 8th International Conference on Software and Information Engineering.
• Granda M.F., Gonzalez O.P., and Alba-Sarango B.2021.Towards a Model-Driven Testing Framework for GUI Test Cases Generation from User Stories. In Proceedings of16th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2021), pages 453-460.
• Memon A.M., Soffa M.L,”Regression testing of GUIs”, SIGSOFT Softw. Eng. Notes 28, 5, pp 118–127, https://doi.org/10.1145/949952.940088, 2003.
• Pan M., Xu T., Pei Y., Li Z., Zhang T. and Li X,” GUI-Guided Test Script Repair for Mobile Apps” IEEE Transactions on Software Engineering, 2020.
• Gao Z., Chen Z., Zou Y. andMemon A.M,” SITAR: GUI Test Script Repair”, IEEE Transactions on Software Engineering 42, pp 170-186, 2016.
• Engström E.,Runeson P. and Skoglund M,” A systematic review on regression test selection techniques,” Inf. Softw. Technol 52, pp 14-30, 2010.
• Kazmi R., Jawawi D.N., Mohamad R. andGhani I,” Effective Regression Test Case Selection”, ACM Computing Surveys (CSUR) 50, pp- 1 – 32, 2017.
• Magalhães C., Mota A. andMomente L,” UI Test case prioritization on an industrial setting: A search for the best criteria.” Software Quality Journal 29, pp 381–403, https://doi.org/10.1007/s11219-021-09549-y 2021.
• Machalica M., Samylkin A., Porth M. and Chandra S. 2019.Predictive Test Selection. In proceedings of the IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSESEIP), pp 91-100.
• “launchableinc”. Available at: https://www.launchableinc.com/what-is-test-impact-analysis. Accessed on 10 March 2022.
• Maitrikul C. and Limpiyakorn Y., “GUI Test Case Prioritization using Social Network Analysis”, Journal of Physics. IOP Publishing. DOI: 10.1088/1742- 6596/1619/1/012020,2020.
• He Z.W. and Bai C.G. 2015. GUI Test Case Prioritization by State-Coverage Criterion. In 2015 IEEE/ACM 10th International Workshop on Automation of Software Test, pp 18-22, DOI: 10.1109/AST.2015.11.
• Sun W., Gao Z., Yang W., Fang C. and Chen Z. 2013 .Multi-objective test case prioritization for GUI applications In proceedings of the 28th Annual ACM Symposium on Applied Computing.
• Yu Z., Fahid F.M., Menzies T., Rothermel G., Patrick K. and Cherian S.2019.TERMINATOR: better automated UI test case prioritization.In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering.
• Coppola R., Morisio M. and Torchiano M. 2017. Scripted GUI Testing of Android Apps: A Study on Diffusion, Evolution and Fragility. In Proceedings of the 13th International Conference on Predictive Models and Data Analytics in Software Engineering.
• Nguyen V. and Le B,” RLTCP: A reinforcement learning approach to prioritizing automated user interface tests”, Information and Software Technology, Volume 136, https://doi.org/10.1016/j.infsof.2021.106574, 2021.
• McMaster S. and Memon A.M. 2005. Call stack coverage for test suite reduction. In proceedings of the 21st IEEE International Conference on Software Maintenance (ICSM'05), pp. 539-548,doi: 10.1109/ICSM.2005.29.
• Arlt S., Podelski A. andWehrle M. 2014. Reducing GUI test suites via program slicing. In proceedings of ISSTA.
• Chetouane N., Wotawa F., Felbinger H. and Nica M. 2020. On Using k-means Clustering for Test Suite Reduction. In proceedings of 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp 380-385.
• Saputra M.C. and Katayama T. 2020. Code Coverage Similarity Measurement Using Machine Learning for Test Cases Minimization. In proceedings of IEEE 9th Global Conference on Consumer Electronics (GCCE), pp 287-291, doi: 10.1109/GCCE50665.2020.9291990.
• Gove R.J. and Faytong J. 2011. Identifying Infeasible GUI Test Cases Using Support Vector Machines and Induced Grammars. In IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, pp 202-211.
• Cruciani E., Miranda B., Verdecchia R. and Bertolino A. 2019. Scalable Approaches for Test Suite Reduction. In proceedings of 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), pp: 419-429.
• Memon A.M., Gao Z., Nguyen B., Dhanda S., Nickell E., Siemborski R. and Micco J. 2017. Taming Google-Scale Continuous Testing. In IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP), pp 233-242.
• “Engineering at Meta”. Available at: https://engineering.fb.com/2020/12/10/developertools/probabilistic-flakiness/. Accessed on 20 February 2022.
• Parry O., Kapfhammer G.M., Hilton M.C. and McMinn P., “A Survey of Flaky Tests”, ACM Trans. Softw. Eng. Methodol. 31, 17:1-17:74, 2021.
• Dix A, 1990. Non-determinism as a paradigm for understanding the user interface. In E. H. Harrison, editors. Formal Methods in Human Computer Interaction. Cambridge University Press, pp 97-127.
• “Dropbox.Tech”. Available at: https://dropbox.tech/infrastructure/athena-our-automatedbuild-health-management-system. Accessed on 26 March 2022.
• Spotify “R&D Engineering”. Available at: https://engineering.atspotify.com/2019/11/testflakiness-methods-for-identifying-and-dealingwith-flaky-tests/. Accessed on 28 March 2022.
• Romano A., Song Z., Grandhi S., Yang W. and Wang W.2021. An Empirical Analysis of UI-Based Flaky Tests. In proceedings of the IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pp 1585-1597.
• Marshall K.P., Horton E., Heckman S. andStolee K. 2019. Wait, Wait. No, Tell Me. Analyzing Selenium Configuration Effects on Test Flakiness. In proceedings of the IEEE/ACM 14th International Workshop on Automation of Software Test (AST), pp 7-13, doi: 10.1109/AST.2019.000.
• Shi A., Lam W., Oei R., Xei T. and Marinov D. 2019.iFixFlakies: A framework for automatically fixing order dependent flaky tests. In Proceedings of the 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE ’19) New York, doi:https://doi.org/10.1145/3338906.3338925.
• Lam W., Godefroid P., Nath S., Santhiar A. and Thummalapenta S. 2019. Root Causing Flaky Tests in a Large-Scale Industrial Setting. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis.
• Alshammari A., Morris C., Hilton M. and Bell J. 2021. FlakeFlagger: Predicting Flakiness without Rerunning Tests.In Proceedings of the 43rdInternational Conference on Software Engineering (ICSE’21).