Tefm reduces tokens and improves trust in structured data models
TEFM: Token-Efficient Faithful Modeling for Structured Data
Computation and LanguageMachine Learning
Summary
Using large language models with complex data can be expensive and hard to trust. The authors developed TEFM, which shrinks data into small, meaningful codes so models use far fewer tokens without losing much information. Their approach also helps explain predictions better by focusing on the most important features. Tests show TEFM keeps accuracy high while using only a tiny fraction of tokens, especially in health and security data.
What this means in practice
- •For clinical data teams: Analyze complex patient records with far fewer tokens while maintaining trustworthy explanations for diagnoses.
- •For security analysts: Process extensive security logs efficiently and identify key factors behind threat classifications with concise, faithful rationales.
Authors
Zhichao Hou, Lingdao Sha, Xueyu Mao, Yang Liu, Peijie Qiu, Rui Song
Abstract
In this paper, we solve two fundamental obstacles in applying LLMs to critical domains: token efficiency and faithfulness. To address both constraints jointly, we present TEFM (Token-Efficient Faithful Modeling), a framework designed for structured data analysis in critical domains. TEFM achieves token efficiency by compressing lengthy structured observations into compact Behavioral Code tokens, dramatically reducing token consumption with minimal information loss. Moreover, TEFM enables faithful rationalization through a dual-fidelity objective that jointly optimizes code-level reconstruction and prediction-level fidelity, identifying minimal sufficient feature subsets grounded in input data. Comprehensive experiments across various domain datasets and model backbones (Qwen3, Gemma-2, Phi-4) show that TEFM achieves competitive classification accuracy with dramatic token reduction (approximately 1\% token retention in clinical and 2\% in security domains) while producing faithful rationales.