Keeping the Cloud Afloat with Deterministic Simulation Testing#

Marcus Hodgson, Antithesis & Marek Siarkowicz, Google

• Why: etcd's v3.5 consistency errors are a known scar in the ecosystem, and the honest admission here is that the robustness testing framework built in response demands unsustainable effort and expertise to maintain. The proposed replacement is a technique, not more labour.
• What: Deterministic simulation testing as a way to validate entire distributed systems, now applied to etcd through a CNCF-sponsored collaboration with Antithesis.
• How: Deterministic execution plus fault injection, which together reproduce elusive bugs exactly rather than probabilistically.
• Where applicable: General technique for distributed systems. The etcd deployment is the proof point; the reason to watch it is whether this becomes the default expectation for critical CNCF infrastructure.