SenseAgent improves motion sensor accuracy across users and devices
SenseAgent: An LLM Agent for Adaptive Cross-Domain IMU Sensing
Artificial Intelligence
Summary
Motion sensors in wearables and phones often work well when trained on one person or device but struggle when used by others or in different positions. The authors developed SenseAgent, a smart system that uses a large language model to help decide how to interpret sensor data for new users or conditions without needing many labeled examples. SenseAgent watches the sensor data, tries different specialized methods, and verifies if changes help or hurt accuracy. This adaptive approach helps sensing systems stay accurate across a variety of real-world situations.
What this means in practice
- •For mobile app developers: Create apps that maintain accurate motion tracking despite different users, devices, or sensor placements by integrating SenseAgent's adaptive sensing strategy.
- •For wearable device engineers: Improve activity recognition on fitness trackers and smartwatches by using SenseAgent to dynamically adjust sensing methods for new users and positions.
Authors
Tianya Zhao, Chuan Liu, Xuyu Wang
Abstract
Deep learning has improved inertial measurement unit (IMU) sensing for mobile and wearable applications. However, an IMU model trained in one domain often becomes unreliable when it is used with a new user, device, or body position. Existing methods usually treat this problem as a static model-design task: they pretrain a stronger representation, add data augmentation, or select one adaptation method before deployment. In practice, the target domain is only gradually observed, labels are scarce, and different domain shifts require different sensing actions. This paper presents SenseAgent, an LLM-guided sensing agent for cross-domain IMU activity recognition. Instead of asking an LLM to classify raw IMU signals, SenseAgent uses the LLM as a runtime planner over sensing tools, source-domain experience memory, online target memory, and verifiers. The agent builds a label-free diagnosis report from the target stream and uses it to decide whether to keep raw inference or invoke specialized tools, including gravity-aware sensing, prototype transfer, and style normalization. Verifiers check source calibration, target-memory reliability, and no-harm criteria before accepting high-risk tool decisions. SenseAgent also supports scarce feedback without retraining the backbone or replacing the label-free route. This design converts cross-domain IMU sensing from a fixed inference pipeline into a closed-loop sensing process that diagnoses target shifts, selects suitable sensing actions, and rejects unsafe adaptations. We evaluate SenseAgent across multiple IMU datasets and deployment shifts. Results show that its verified route selection improves cross-domain sensing, especially under harder placement and compound shifts, and further benefits from limited user feedback.