Real Low-Resolution (LR) face images, especially the extremely small ones, often contain degradations (e.g. noise, blur, compression artefacts) which are too varied and complex to be captured by the standard bicubic downsampling model often employed in training Single Image Super-Resolution (SISR) networks. Some recent works attempt to address this by modelling such degradations using a Generative Adversarial Network (GAN) that, by leveraging a real-world LR facial images dataset, learns to inflict various degradations conditioned on random vectors. However, even though effective against real degradations, these works: (a) often fail to maintain a consistent output under varying degradations and (b) do not maximize the challenge for the downstream SISR networks.

To address the first problem, we fashioned our SISR network as an autoencoder and put explicit consistency constraint on the encoder. By using Entropy Regularized Wasserstein Divergence on the encoder, we encouraged the decoder to rely only on those features which are smooth (invariant) under degradations. Using a judicious interpolation strategy, we achieved a good trade-off between correspondence and robustness.

To address the second problem, we proposed an adversarial attack that is faster than Projected Gradient Descent (PGD) and more effective than the Fast Gradient Sign Method (FGSM). Our method leverages the parameterizable property of the Mean Squared Error (MSE) loss surfaces of an SISR network (trained with learned degradation modules) and finds adversarial samples without multiple gradient-ascent steps.