EBPF tools extended to monitor secure and custom execution environments
Extending eBPF observability to Non-standard execution environments
Operating Systems
Summary
Standard Linux monitoring tools struggle to observe special computing areas like trusted enclaves or custom Linux environments because these areas handle errors and memory in unusual ways. The authors created new methods that let powerful monitoring tools called eBPF safely access and track these special environments. They tested their methods on Intel SGX enclaves and lightweight operating systems, showing these can trace activity and measure performance with only a small delay. This work helps bring the same kind of deep monitoring that Linux tools have, into these harder-to-watch computing areas.
What this means in practice
- •For cloud infrastructure teams: Monitor and analyze code execution inside secure enclaves to detect performance bottlenecks and security issues using extended eBPF probes.
- •For security operations teams: Trace and debug software running in custom isolated environments like LibOSes to improve incident response capabilities.
Authors
Pamenas Kariuk, André Martin, Christof Fetzer
Abstract
eBPF observability of non-standard execution environments (NEEs) like TEEs or LibOSes is hindered by their unconventional exception-handling and memory-access mechanisms that limit standard Linux tooling. This work introduces two mechanisms that enable eBPF-based observability for NEEs: (1) kernel memory extensions for safely accessing NEE memory from eBPF programs, and (2) a lightweight flexible probe performance measurement unit (LWFP PMU) that provides flexible and generic probing, for NEEs, through the following LWFP: simple (SLWFP), enclave (ELWFP) and extended (ExLWFP) probes. We demonstrate the practicality of these extensions by developing tooling for tracing, stack sampling with Flame Graphs, dynamic instrumentation, timing analysis, and USDT support for Intel SGX enclaves and LibOSes. Performance measurements show that SLWFP probes achieve a latency of 394 ns, outperforming uprobes, which exhibit 25% higher latency, while ELWFP probes incur a latency ~2.8 microseconds, which is practical for enclave observability. The addition of SLWFP introduces negligible overhead to existing uprobe performance. Taken together, this work lays the foundation for closing the long-standing gap between NEE tooling and Linux observability tooling by enabling generic and reusable eBPF tooling for diverse NEE hardware and software architectures.