How Deep is Your Art: An Experimental Study on the Limits of Artistic Understanding in a Single-Task, Single-Modality Neural Network

Mathematical modeling and aesthetic rule extraction of works of art are complex activities. This is because art is a multidimensional, subjective discipline. Perception and interpretation of art are, to many extents, relative and open-ended rather than measurable. Following the explainable Artificial Intelligence paradigm, this paper investigated in a human-understandable fashion the limits to which a single-task, single-modality benchmark computer vision model performs in classifying contemporary 2D visual arts. It is important to point out that this work does not introduce an interpreting method to open the black box of Deep Neural Networks, instead it uses existing evaluating metrics derived from the confusion matrix to try to uncover the mechanism with which Deep Neural Networks understand art. To achieve so, VGG-11, pre-trained on ImageNet and discriminatively fine-tuned, was used on handcrafted small-data datasets designed from real-world photography gallery shows. We demonstrated that the artwork's Exhibited Properties or formal factors such as shape and color, rather than Non-Exhibited Properties or content factors such as history and intention, have much higher potential to be the determinant when art pieces have very similar Exhibited Properties. We also showed that a single-task and single-modality model's understanding of art is inadequate as it largely ignores Non-Exhibited Properties.