Computer security researchers from Singapore’s Home Team Science and Technology Agency published a preprint on November 13, 2026, demonstrating an observational memory-forensics technique that captures transient GPU hidden states from live large language model serving processes without modifying weights, restarting services, or injecting invasive application-level hooks. The methodology introduces non-intrusive request-level attribution to running neural inference workloads, reshaping operational auditing, multi-tenant privacy boundaries, and research computing security protocols.
Investigating live inference without modifying server state
Forensic examination of machine learning infrastructure has historically relied on static disk artifacts, such as saved model weights, architecture configuration files, or high-level application access logs. However, when an autonomous agent or language model exhibits anomalous or adversarial behavior in production, static weights fail to reveal what transpired during a specific, ephemeral inference pass. Application-level prompt logs can be disabled, truncated, or forged by compromised host processes, creating an evidence gap for forensic examiners.
In the preprint titled From Bytes to Behavior: Request-Level Attribution from Live GPU Hidden States (arXiv:2611.18272), co-authors Jonathan Pan, Yidi Yuan, Xin Wei Chia, Xinhui Su, Vincent Tan, and Brenda Ong establish that an observer with operating system debugger privileges can extract byte-exact intermediate hidden states directly from video memory (VRAM). Their technique avoids traditional application instrumentation such as PyTorch forward hooks or dynamic library preloading, which alter process memory footprints and compromise the evidentiary neutrality required during a formal model audit.
The core innovation intercepts tensor allocations at the hardware abstraction layer rather than the Python interpreter. By synchronizing non-stopping debugger events with memory allocator clocks, the research team isolated residual-stream tensor activations across individual inference passes, allowing examiners to reconstruct internal computational trajectories without prior knowledge of model architecture parameters.
Six-stage attribution architecture and allocator clocks
The proposed framework operates through a standardized six-stage pipeline designed to maintain forensic soundess across diverse serving stacks. Rather than modifying running binaries, the acquisition harness attaches to the target process using standard system tracing interfaces through the NVIDIA CUDA Debugger (cuda-gdb). The six sequential stages proceed as follows:
- Process attachment: A non-modifying debugger attaches via standard ptrace interfaces to the live server process without altering memory contents or executable segments.
- Clock generation: The system synthesizes an event stream that marks when transformer layers execute by tapping internal memory allocation cycles.
- Blind identification: Residual-stream buffers are isolated dynamically from video memory pools using temporal heuristics rather than static address offsets or proprietary layer definitions.
- Precise VRAM capture: The acquisition tool reads raw bytes from device memory at the exact instant the allocator clock confirms the layer activations are fully materialized.
- Offline verification: Recovered activations are validated against reference calibration models in isolated environments to confirm byte-for-byte fidelity without executing checking code on the live server.
- Downstream behavioral inference: The extracted tensors are piped to analytical classifiers, such as linear representation probes, to assess operational behaviors.
To establish temporal execution boundaries, the authors developed two platform-specific synchronization adapters. The primary adapter leverages the internal mechanics of the PyTorch CUDA caching allocator. Because PyTorch routes tensor memory management through centralized code paths, placing non-halting breakpoints on native allocation routines allows examiners to track the birth, memory address, and deallocation of intermediate buffers. For inference engines where PyTorch memory management is bypassed, the authors introduced an alternate kernel-launch clock that keys on normalization kernels such as RMSNorm, timestamping transformer block boundaries directly from GPU kernel dispatches.
Empirical evaluation across architectures and safety benchmarks
The research team evaluated the memory extraction methodology across 33 open-source decoder-only language models spanning nine distinct architectural families, including Qwen, Llama, and Mistral derivatives. The testbed paired an NVIDIA RTX GPU running CUDA 13.0 and PyTorch 2.13.0 with modern open-source serving frameworks, specifically vLLM serving engine (version 0.32.0) and SGLang (version 0.5.23).
Across all 33 evaluated models, the blind two-step selector recovered intermediate residual-stream activations with byte-exact fidelity when benchmarked against synchronized in-process reference hooks in controlled laboratory environments. The authors categorized the resulting forensic evidence into three hierarchical tiers:
| Evidence Tier | Extracted Information | Fidelity and Reliability | Forensic Application |
|---|---|---|---|
| Level 1: Structural | Token counts, hidden dimension sizes, and layer depths | Deterministic (exact match across platforms) | Validating actual running architecture against registered deployment baselines |
| Level 2: Computational | Raw per-layer residual-stream activation tensors | Byte-exact in laboratory benchmarks; within 2% on vLLM | Reconstructing internal mathematical representations of specific inference passes |
| Level 3: Behavioral | Safety stance classification (refusal vs. compliance) | Probabilistic (0.88 AUROC with linear probes) | Detecting policy violations, jailbreak execution, or unauthorized agent task completion |
To demonstrate practical utility for behavioral auditing, the authors deployed linear probes trained on standard model representations against the recovered hidden states. Testing on 142 held-out safety evaluation prompts drawn from standardized benchmarks, including the HarmBench evaluation framework and XSTest, the linear probes classified whether the model complied with or refused a sensitive instruction with an Area Under the Receiver Operating Characteristic (AUROC) curve of 0.88. This proves that transient VRAM tensors retain sufficient semantic structure to verify model compliance post-execution, even when output text streams are intercepted or withheld.
Auditing utility versus privacy risks in shared clusters
The demonstration of request-level memory forensics reveals a critical duality for enterprise and academic high-performance computing environments. On one hand, the capability bolsters incident response workflows. Guidelines established in NIST Special Publication 800-86 emphasize acquiring volatile digital evidence without altering target system states. Live GPU inspection enables compliance officers to audit recalcitrant model instances, identify model extraction attempts, and establish definitive data provenance during automated scientific discoveries without needing cooperative application-level logging.
On the other hand, reconstructing intermediate hidden states from live GPU hardware introduces severe privacy vulnerabilities on shared, multi-tenant research clusters. If a containerized workload or sidecar process gains sufficient host privileges to execute tracing primitives, malicious actors could observe intermediate activations of neighboring tenants. Because intermediate hidden states encode semantic representations of input prompts and internal reasoning traces, unauthorized memory snooping could expose sensitive research data, proprietary prompts, or confidential patient records processed on shared accelerator nodes.
Furthermore, the forensic method imposes a severe operational penalty. Halting GPU execution at allocation breakpoints causes serving performance to degrade by factors ranging from 46x to 192x compared to native throughput. This latency overhead restricts live debugger acquisition to targeted, for-cause forensic investigations rather than persistent production telemetry.
Implications for academic research computing and security officers
For university Chief Information Security Officers (CISOs), research computing directors, and administrators overseeing high-performance computing (HPC) facilities, the preprint highlights urgent infrastructure governance priorities. High-density GPU clusters hosting multi-user LLM workloads frequently rely on standard Linux containerization to isolate user jobs. However, conventional namespace boundaries must be reinforced to prevent cross-process memory snooping.
Research facilities must align their shared computing infrastructure with stringent ISO 27001 research data security controls and national research security guidelines. Institutional technical teams should implement several immediate defensive configurations:
- Restrict ptrace and debugging capabilities: Ensure that container runtimes explicitly drop
CAP_SYS_PTRACEprivileges and configure kernel security modules to block unprivileged process tracing across user namespaces. - Isolate multi-tenant accelerator hardware: Disallow co-locating untrusted multi-tenant workloads on the same physical GPU unless hardware-enforced partitioning, such as NVIDIA Multi-Instance GPU or Confidential Computing with memory encryption, is actively verified.
- Harden workflow orchestration: Integrate memory isolation checks into institutional reproducibility infrastructure and container isolation policies to prevent sidecar inspection during automated research pipelines.
- Establish forensic acquisition protocols: Define explicit legal and operational boundaries for when high-overhead memory inspection may be initiated during suspected research misconduct or export-control investigations.
Limitations and research roadmap
While the findings establish a foundational proof of concept for live GPU forensics, the authors emphasize clear technical boundaries. The current implementation was validated on single-GPU hardware with batch sizes of one and short prompt sequences under 100 tokens. In enterprise production environments, modern serving stacks deploy dynamic batching, paged attention pools, and multi-node tensor parallelism, which disrupt the predictable allocation intervals required by the PyTorch allocator clock.
Subsequent research will focus on developing platform-independent identification heuristics capable of tracking tensor ownership across continuous batch streams and exploring hardware-assisted memory telemetry. Until then, the Singapore HTX preprint provides research institutions with both a novel tool for rigorous AI auditing and a clear imperative to re-examine accelerator memory isolation in academic computing centers.
Sources
- Pan, J., Yuan, Y., Chia, X. W., Su, X., Tan, V., & Ong, B. (2026-11-13). From Bytes to Behavior: Request-Level Attribution from Live GPU Hidden States. arXiv. https://arxiv.org/abs/2611.18272
- National Institute of Standards and Technology. (2006-08). Guide to Integrating Forensic Techniques into Incident Response (NIST Special Publication 800-86). U.S. Department of Commerce. https://csrc.nist.gov/publications/detail/sp/800-86/final
- PyTorch Foundation. (2026). CUDA Semantics and Memory Management Documentation. PyTorch Open Source Project. https://pytorch.org/docs/stable/notes/cuda.html
- NVIDIA Corporation. (2026). CUDA-GDB: NVIDIA CUDA Debugger User Guide. NVIDIA Developer Zone. https://docs.nvidia.com/cuda/cuda-gdb/index.html
- vLLM Project Team. (2026). vLLM Documentation: High-Throughput and Memory-Efficient Inference Engine for LLMs. vLLM Project. https://docs.vllm.ai/en/latest/
- Mazeika, M., Phan, L., Yin, X., Zou, A., Wang, Z., Mu, N., Sakhaee, E., Li, N., Basart, S., Li, B., Forsyth, D., & Hendrycks, D. (2024). HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robustness. Center for AI Safety. https://www.harmbench.org/







