Papers for

ai infrastructure architects

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Agentic web technologies baseline shows task specific performance gains

A Baseline Assessment for the Agentic Web Technology Stack

Abstract: The fast pace in which protocols, frameworks, and technologies are being proposed for LLM-based agents results in a complex scenario for anyone aiming to create state-of-the-art AI solutions. Without an assessment on how different protocols, frameworks, and libraries work together to compose an Agentic Web solution, developers and organizations face different challenges (e.g., skilling, governance, infrastructure) even before becoming part of the Agentic Web ecosystem. In this context, we propose a system for performing Agentic Web evaluations called MASTIF (Multi-Agent System TestIng Framework) and present a baseline assessment of Agentic Web technologies, including 3 LLMs, 3 protocols, and 4 agent implementation frameworks, against 300 Mind2Web benchmark tasks. Results suggest that selecting a task-specific technology stack may result in gains of task completion rate, efficiency score, and social value stability scores; and reductions in verbosity and time to complete tasks. With the presented results, we expect to present a baseline in terms of metrics and results for this complex and fast evolving scenario for those aiming at designing, assessing, and developing multi-agent systems.

Fri 9 OctArtificial IntelligenceEmerging TechnologiesSoftware Engineering

The gist

There are many new tools and rules for making AI agents that can work together on the web, but it’s confusing to know which ones work best. The authors created a system to test these AI tools together on 300 different tasks. They found that picking the right combination of AI models, communication methods, and software frameworks helps complete tasks faster, with less extra talk and more stable results. This work gives a starting point for building and comparing complex AI systems that work together online.

Open → 2610.13703v1

Finnish speech models improve using confidence based filtering and model comparisons

From 2.1M Hours of Raw Broadcast Audio to Finnish Speech SSL: Confidence Filtering and Architectural Analysis at Scale

Abstract: Broadcast archives offer scale and diversity for self-supervised learning (SSL) but contain substantial noisy, non-speech, and multilingual content. Prior work typically relies on voice activity detection or language identification (LID) to extract speech, ensuring presence rather than quality. Whether and how to filter unlabeled speech by quality for SSL, particularly from noisy broadcasts, remains underexplored. We present a large-scale Finnish SSL study starting from 2.1M hours of unconstrained broadcast audio. We propose automatic speech recognition (ASR) confidence as a proxy for pretraining data quality and show 3-7% relative WER improvements over LID-based filtering despite a smaller training set. We further address the lack of controlled architectural comparisons in speech SSL by evaluating Transformer, Macaron-Net, Conformer, and E-Branchformer at matched parameter budgets. Our models improve upon prior Finnish SSL baselines, particularly at Base scale and out-of-domain.

Fri 9 OctComputation and LanguageSound

The gist

Working with millions of hours of Finnish broadcast audio, the authors studied how to pick good speech segments for teaching AI to understand spoken language better. Instead of just checking if speech is present, they used a tool that guesses how confident it is about the speech content to filter out poor-quality data. This approach led to clearer AI understanding with fewer errors. They also compared different computer designs for making these speech models, finding some worked better for Finnish speech.

Open → 2610.13102v1