Filtered Nonlinear Cryptanalysis of Reduced-Round Serpent, and the Wrong-Key Randomization Hypothesis
We present a deterministic algorithm to find nonlinear S-box approximations, and a new nonlinear cryptanalytic technique; the "filtered" nonlinear attack, which achieves the lowest data complexity of any known-plaintext attack on reduced-round Serpent so far. We demonstrate that the Wrong-Key Randomization Hypothesis is not entirely valid for attacks on reduced-round Serpent which rely on linear cryptanalysis or a variant thereof, and survey the effects of this on existing attacks including existing nonlinear attacks on 11 and 12-round Serpent.