Interpolation-aware models for train-test consistency in mixup

Mixup is a learning principle that trains a neural network on convex combinations of pairs of examples and their labels. Despite of its good performance, there is an inherent inconsistency between training and testing in mixup, which makes theoretical understanding difficult and hurts the performance in some cases. In this work, we propose λ-mixup to alleviate this inconsistency. Specifically, λ-mixup reformulates the model to take the interpolation coefficient (𝜆) as input as well, so that a class of models indexed by 𝜆 is learned and we can select one specific coefficient or multiple coefficients for ensembles depending on the testing distribution. We theoretically demonstrate that, with enough data and model capacity, λ-mixup can recover the original conditional distribution. Moreover, we conduct image classification tasks on multiple datasets, including CIFAR-10, CIFAR-100 and Tiny-Imagenet, showing that comparing with mixup, λ-mixup exhibits better generalization, calibration and robustness to adversarial attacks and out-of-distribution transformations.