Employing Discrete Fourier Transform in Representational Learning

Image Representation learning via input reconstruction is a common technique in machine learning for generating representations that can be effectively utilized by arbitrary downstream tasks. A well-established approach is using autoencoders to extract latent representations at the network's compression point. These representations are valuable because they retain essential information necessary for reconstructing the original input from the compressed latent space. In this paper, we propose an alternative learning objective. Instead of using the raw input as the reconstruction target, we employ the Discrete Fourier Transform (DFT) of the input. The DFT provides meaningful global information at each frequency level, making individual frequency components useful as separate learning targets. When dealing with multidimensional input data, the DFT offers remarkable flexibility by enabling selective transformation across specific dimensions while preserving others in the computation. Moreover, certain types of input exhibit distinct patterns in their frequency distributions, where specific frequency components consistently contain most of the magnitude, allowing us to focus on a subset of frequencies rather than the entire spectrum. These characteristics position the DFT as a viable learning objective for representation learning and we validate our approach by achieving 52.8% top-1 accuracy on CIFAR-10 with ResNet-50 and outperforming the traditional autoencoder by 12.8 points under identical architectural configurations. Additionally, we demonstrate that training on only the lower-frequency components - those with the highest magnitudes yields results comparable to using the full frequency spectrum, with only minimal reductions in accuracy.