Deep Neural Networks (DNNs) are widely used to perform machine learning tasks in speech, image, and natural language processing. The high computation and storage demands of DNNs have led to a need for energy-efficient implementations. Resistive crossbar systems have emerged as promising candidates due to their ability to compactly and efficiently realize the primitive DNN operation, viz., vector-matrix multiplication. However, in practice, the functionality of resistive crossbars may deviate considerably from the ideal abstraction due to the device and circuit level non-idealities such as driver resistance, sensing resistance, sneak paths, and interconnect parasitics. Although DNNs are somewhat tolerant to errors in their computations, it is still essential to evaluate the impact of errors introduced due to crossbar non-idealities on DNN accuracy. Unfortunately, device and circuit-level models are not feasible to use in the context of large-scale DNNs with 2.6-15.5 billion synaptic connections. In this work, we present a fast and accurate simulation framework to enable training and evaluation of large-scale DNNs on resistive crossbar based hardware fabrics. We propose a Fast Crossbar Model (FCM) that accurately captures the errors arising due to non-idealities while being five orders of magnitude faster than circuit simulation. We develop Rx-Caffe, an enhanced version of the popular Caffe machine learning software framework to train and evaluate DNNs on crossbars. We use Rx-Caffe to evaluate large-scale image recognition DNNs designed for the ImageNet dataset. Our experiments reveal that crossbar non-idealities can significantly degrade DNN accuracy by 9.6%-32%. To the best of our knowledge, this work is the first evaluation of the accuracy for large-scale DNNs on resistive crossbars and highlights the need for further efforts to address the impact of non-idealities.