We report on an extensive study of the current benefits and limitations of deep learning approaches to robot vision and introduce a novel dataset used for our investigation. To avoid the biases in currently available datasets, we consider a human-robot interaction setting to design a data-acquisition protocol for visual object recognition on the iCub humanoid robot. Considering the performance of off-the-shelf models trained on off-line large-scale image retrieval datasets, we show the necessity for knowledge transfer. Indeed, we analyze different ways in which this last step can be done, and identify the major bottlenecks in robotics scenarios. By studying both object categorization and identification tasks, we highlight the key differences between object recognition in robotics and in image retrieval tasks, for which the considered deep learning approaches have been originally designed. In a nutshell, our results confirm also in the considered setting the remarkable improvements yield by deep learning, while pointing to specific open challenges that need to be addressed for seamless deployment in robotics.