ChaosProbe: A Neurochaotic Lens on Frozen Transformer Input-Embedding Spaces
2026-08-03 • Machine Learning
Machine LearningNeural and Evolutionary Computing
AI summaryⓘ
The authors propose a new method called ChaosProbe that studies transformer models by looking at how their input embeddings react to a special deterministic test before any learning happens. ChaosProbe treats the embeddings like a chaotic system and creates unique response signatures for each model. In their study with four popular transformer models, these signatures helped group models from the same family together accurately. The work shows that such response-based fingerprints can reveal meaningful similarities among frozen transformer embeddings.
Transformer modelsinput embeddingsChaosProbeneurochaosFiring RateEntropyPearson correlationSpearman correlationcosine similarityfrozen embeddings
Authors
Kunal Kumar Pant, Nithin Nagaraj
Abstract
Transformer models are most often understood through what they do: their benchmark performance, generation quality, or behavior on downstream tasks. Yet frozen transformer input-embedding spaces may also be examined through their responses to a controlled deterministic probe before contextual computation or task-specific adaptation. Guided by this response-based view, we introduce \emph{ChaosProbe}, a deterministic neurochaos-inspired method for constructing response-based fingerprints of frozen transformer input-embedding spaces. For each prompt-level embedding matrix, ChaosProbe applies a chaotic trajectory-based transformation and summarizes its Firing Rate and Entropy channel responses with complementary representation-level measures, producing a fixed-length signature for each model. In a bounded proof-of-concept study of $80$ neutral prompts and four pretrained models---GPT-2, DistilGPT2, BERT-base-uncased, and RoBERTa-base---Pearson correlation, Spearman correlation, and cosine similarity each recover all four same-family nearest-neighbor assignments and both expected mutual family pairs. Euclidean distance recovers three of the four assignments and one of the two mutual family pairs. Paired bootstrap resampling supports the stability of the Pearson and Spearman pairings over the observed prompt set, and signature-validity checks show that constant or collapsed responses do not dominate the reported fingerprints. These results provide a cohort-dependent proof of concept that deterministic neurochaotic response signatures can expose broad structure among frozen transformer input-embedding spaces.