AI summaryⓘ
The authors studied different methods for automatically testing web apps by clicking around the interface to find bugs and cover code. They looked at three main approaches—model-based, reinforcement learning (RL)-based, and large language model (LLM)-based—and how the way pages with similar functions are grouped (called state abstraction) affects testing success. Their results show no single method is best at everything; rather, each has strengths in various areas like code coverage or finding bugs. They also found that how detailed or simple the state abstraction is matters differently depending on the method used. Finally, they highlight that just covering a lot of code doesn't always mean more bugs are found, suggesting testing should consider multiple factors.
Automated web GUI testingExploration strategiesModel-based testingReinforcement learningLarge language modelsState abstractionCode coverageFailure detectionHistory representation
Authors
Chenxu Liu, Wei Yang, Ying Zhang, Tao Xie
Abstract
Automated web GUI testing (AWGT) relies on exploration strategies that exercise web applications through GUI actions to maximize code coverage, spanning traditional model-based, reinforcement learning (RL)-based, and emerging large language model (LLM)-based approaches. State abstraction, which detects pages with the same functionality to avoid repeated testing, has long been recognized as critical to guiding exploration. However, how exploration strategies and state abstractions jointly affect testing effectiveness remains underexplored. We present an empirical study analyzing both factors from the perspectives of code coverage and failure revelation. We compare representative model-based, RL-based, and LLM-based approaches; investigate how six state abstractions influence model-based and RL-based approaches; examine LLM-based approaches under different history representations, which act as a form of state abstraction; and compare the failures exposed by different approaches. Our results show that no single strategy excels across all dimensions; instead, categories exhibit complementary strengths in code coverage, state coverage, and failure discovery. State abstraction is a key factor: strict, fine-grained abstractions favor model-based strategies, while compact ones better support RL-based strategies. History representation substantially affects LLM-based strategies, where concise, functionality-level context performs best. We also find that code coverage is weakly correlated with failure-revealing ability, underscoring the need for multi-dimensional evaluation. These findings offer practical guidance for selecting exploration strategies and designing effective state abstractions for AWGT.