Papers for

automated transcription services

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Multimodal models improve financial forecasting from earnings calls

MM-FinEval: A Multi-Task Multimodal Benchmark for Real-World Financial Forecasting

Abstract: Financial forecasting from earnings conference calls requires models to reason over complex corporate disclosures, market expectations, and subtle communication signals. However, existing financial benchmarks are often limited to unimodal inputs or single-task settings, making it difficult to evaluate whether multimodal large language models (LLMs) can support real-world financial analysis. In this paper, we introduce MM-FinEval, a novel benchmark designed to evaluate multimodal LLMs across multiple financial tasks. MM-FinEval spans a diverse timeline from 2019 to 2022. The entire proposed dataset contains 2,045 S\&P 500 conference earning calls as inputs and 12 financial task labels as outputs. Each input contains three modalities: a word-to-word text transcript of the earning call, the corresponding presentation slides used during the call, and the entire audio recording. To establish a rigorous evaluation framework, we analyze 19 baseline models across three distinct model categories: Image-Text, Audio-Text, and Any-to-Any configurations. We observe that small-size Any-to-Any models processing all three modalities achieve strong performance, even when compared against larger proprietary models restricted to two-modality inputs. This indicates that our tri-modal dataset design introduces useful, non-redundant information. These results validate that text, audio, and visual data serve as important, complementary signals that mimic the decision-making process of expert human analysts.

Tue 29 SeptMachine LearningArtificial Intelligence
The gist
Financial forecasting often relies on understanding complex information from company earnings calls, which include spoken words, presentation slides, and audio cues. The authors created a new benchmark called MM-FinEval that tests how well AI models can analyze all three types of information together for multiple financial prediction tasks. They found that smaller models using all three data types perform better than some larger models using fewer types. This shows that combining text, audio, and visuals adds helpful information that mimics human analysts.
Open → 2609.38523v1