ProfInfer: An eBPF-based Fine-Grained LLM Inference Profiler#

Bohua Zou, Debayan Roy, Dhimankumar Airao, Weihao Xu, Binqi Sun, Yutao Liu, Haibo Chen (affiliations not listed in the program)

• Why: eBPF is in your core stack, and this paper shows its reach extending beyond networking into runtime-level observability of LLM inference — with under 4% overhead and zero source modification. The pattern (dynamic probes on runtime functions, rich derived views) transfers to other "black-box runtime, need operator-level visibility" problems.
• What: A fine-grained, non-intrusive profiling framework for LLM inference engines, focused on resource-constrained edge devices and exemplified by llama.cpp, exposing operator-level time, memory-vs-compute boundedness, MoE routing behavior, and operator offloading, rendered as operator graphs, timelines, and hardware-counter trends (from abstract only).
• How: eBPF probes dynamically attached to runtime functions across multiple layers without recompilation; collected traces are transformed into visualizations of operators, graphs, and counter trends.
• Where applicable: llama.cpp-class CPU/edge inference runtimes on Linux — this is not a GPU-cluster profiler, and the fidelity claims are tied to that runtime architecture. The eBPF technique itself is broadly portable.