Replicability: Challenging the Myth of Randomness Patterns in Encrypted Traffic#

Authors: Timofey Shpakov (ETH Zürich), Martin Burkhart (armasuisse Science and Technology), Laurent Vanbever (ETH Zürich)

• Why: A methodological correction with real consequences: several recent studies (most prominently the often-cited ET-BERT) claim encrypted traffic exhibits detectable randomness deficiencies usable for application classification. If those evidentiary foundations are artifacts, a chunk of the encrypted-traffic-classification literature — and vendor claims built on it — rests on sand.
• What: A systematic replication of the randomness experiments, re-implemented with the NIST Statistical Test Suite on encrypted traffic generated under controlled conditions. The reported statistical signals do not provide sufficient evidence of non-randomness beyond what statistical hypothesis testing and finite-sample effects would produce. Observed test failures and discriminative patterns are consistent with methodological artifacts: improper aggregation, p-value misinterpretation, and train/test information leakage. (from abstract only)
• How: Controlled-condition re-implementation of prior experiments, examining the evidentiary step — the claim that detectable patterns exist in ciphertext — rather than the downstream classifiers.
• Where applicable: Anyone evaluating encrypted-traffic classification research or products, or building on randomness-based classification themselves. The authors are careful to scope the critique: it says nothing about methods exploiting metadata, packet sizes, timing, or other side channels.