Live GPU hidden states reveal real time AI model behavior
From Bytes to Behavior: Request-Level Attribution from Live GPU Hidden States
Cryptography and SecurityArtificial Intelligence
Summary
Most tools that check how AI models work focus on static parts like saved settings or past inputs, missing what happens during actual thinking steps when the model is running. The authors developed a way to peek inside a running AI model's brain on a graphics card without changing anything or needing to know its setup. They tested this on many different models and found the data they grabbed matched perfectly what the models were using internally. Using this data, they could tell if the model was agreeing or refusing certain prompts, helping understand its behavior as it happens.
What this means in practice
- •For ai operations teams: Monitor and analyze live AI model behaviors without interrupting service using recovered internal GPU states.
- •For security analysts: Investigate AI model compliance and refusal actions with forensic access to transient hidden states during inference.
Authors
Jonathan Pan, Yidi Yuan, Xin Wei Chia, Xinhui Su, Vincent Tan, Brenda Ong
Abstract
Existing memory forensics for machine learning systems typically targets static artifacts, such as model weights or prompt logs, rather than the transient computational states of a live inference process. This gap limits an examiner's ability to analyze the specific behavior of a running large language model (LLM) service that cannot be restarted or modified. We present a method for acquiring per-layer residual-stream activations from a live GPU process using a non-modifying debugger, without requiring prior knowledge of the model's architecture parameters or weights. The approach employs a general six-stage process with platform-specific adapters, utilizing either a PyTorch allocator clock or a kernel-launch clock to identify and read tensor allocations from video memory. We validate this acquisition on 33 open-source decoder-only models across nine architecture classes, demonstrating that the recovered bytes are byte-exact and usable for downstream analysis. As a use case, we apply linear probes trained on standard framework activations to the recovered states, successfully distinguishing refusal from compliance with an AUROC of 0.88 on held-out prompts. This work establishes that live, observational access to intermediate hidden states is feasible on named inference platforms, providing a new source of evidence for examining the operational behavior of deployed AI systems.