Optical coherence tomography (OCT) and its angiography (OCTA) have several advantages for the early detection and diagnosis of diabetic retinopathy (DR). However, automated, complete DR classification frameworks based on both OCT and OCTA data have not been proposed. In this study, a densely and continuously connected neural network with adaptive rate dropout (DcardNet) is proposed to fulfill a DR classification framework using en face OCT and OCTA. The proposed network outputs three separate classification depths on each case based on the International Clinical Diabetic Retinopathy scale. At the highest level the network classifies scans as referable or non-referable for DR. The second depth classifies the eye as non-DR, non-proliferative DR (NPDR), or proliferative DR (PDR). The last depth classifies the case as no DR, mild and moderate NPDR, severe NPDR, and PDR. We used 10-fold cross-validation with 10% of the data to assess the performance of our network. The overall classification accuracies of the three depths were 95.7%, 85.0%, and 71.0% respectively.