TEE-X: TEE-aware Acceleration Framework for Large Vision Models at the Edge
2026-08-24 • Cryptography and Security
Cryptography and SecurityArtificial Intelligence
AI summaryⓘ
The authors address security challenges in running large vision machine learning models on devices by using Trusted Execution Environments (TEEs), which protect the model and its execution from attackers. They focus on making TEEs fast enough for real-time use on edge devices, which usually struggle due to memory limits and slower processing. Their solution, called TEE-X, organizes the model computations thoughtfully and speeds up processing inside the TEE. They tested TEE-X on a specific secure environment setup and showed it runs vision models quickly without losing accuracy or security. This work helps make secure, private vision AI more practical on everyday devices.
Trusted Execution Environment (TEE)Vision Transformer (ViT)Edge computingModel confidentialityInference latencyOP-TEEArm TrustZoneVectorizationModularizationNVIDIA Jetson AGX Xavier
Authors
Kurt M Wilson, Mohaiminul Al Nahian, Abeer Matar A. Almalky, Sadat Shahriyar, Souvik Kundu, Zhishan Guo, Abdullah Al Arafat, Adnan Siraj Rakin
Abstract
Despite their remarkable success, machine learning models, particularly in vision applications, are alarmingly vulnerable to a range of security threats. One key factor in the attack landscape is the distinction between white-box and black-box threat models, as the latter poses challenges that limit attack effectiveness when access to model information is limited. As a result, using Trusted Execution Environments (TEEs) enhances security for machine learning applications by protecting model confidentiality and execution integrity, effectively shifting the execution environment from the white-box to the black-box side of the threat model spectrum. While adopting TEEs for large vision models, e.g., Vision Transformers (ViTs), is crucial for enhancing security and privacy, significant challenges related to memory constraints and increased computational latency must be addressed, especially in time-sensitive edge applications where safety and privacy are paramount. The objective of this work is to enable large vision models to be fully hosted within TEEs, achieving GPU-level inference latency for time-sensitive edge vision applications while maintaining performance. To this end, we propose TEE-X, a TEE-aware acceleration framework that introduces a sensitivity-aware modularization technique and enables vectorization in TEE inference. This design is validated on OP-TEE for Arm TrustZone, configured to optimize performance on the NVIDIA Jetson AGX Xavier for efficient edge vision applications using ViT models. The findings reveal that TEE-X delivers an effective TEE-aware acceleration framework that achieves minimal accuracy-latency trade-offs while ensuring fast and secure edge inference for vision models.