Papers for

digital accessibility service providers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Interactive llm agents improve web accessibility error detection accuracy

InterA11y: Interactive Web Accessibility Error Detection with LLM Agents

Abstract: Large Language Models (LLMs) have shown potential to expand the coverage of web accessibility evaluation. However, current LLM-based checkers neglect much of the interactive and dynamic web contexts, resulting in limited coverage, high false discovery rates, and inconsistent performance across models. We present InterA11y, an agentic accessibility evaluation system that combines rubrics and interactive tools to support LLM agents in evaluating accessibility effectively. The rubrics decompose WCAG success criteria (SC) into testable aspects and guide agents in collecting and interpreting evidence. The interactive tools inspect or operate a webpage deterministically to collect evidence, such as focus appearances and error messages. We developed these rubrics and tools on 682 curated test cases grounded in established accessibility guidelines that target 10 commonly violated SC. We evaluated InterA11y on 197 expert-annotated cases sampled from 54 real webpages. Results show InterA11y achieves higher recall than the strongest evaluated LLM-driven baseline (79.0% vs. 44.4%), while maintaining comparable precision to best rule-based checkers (93.1% vs. 90.3-97.6%), with little variability between models. Further, when equipped with our interactive tools, general-purpose web agents detect more accessibility errors with fewer false positives and less token usage than without these tools.

Thu 29 OctHuman-Computer Interaction
The gist
Websites need to be accessible to everyone, but current tools that check accessibility often miss tricky interactive problems or give too many false alarms. The authors created InterA11y, a system that uses detailed guidelines and interactive tools to help large language models test websites more carefully. This approach helps the system find more real accessibility errors while keeping false alarms low. InterA11y works well across different models and uses less computing power when checking websites.
Open → 2610.39488v1