Retinal image quality assessment is an essential task in the diagnosis of retinal diseases. Recently, there are emerging deep models to grade quality of retinal images. Current state-of-the-arts either directly transfer classification networks originally designed for natural images to quality classification of retinal image or introduce extra image quality priors via multiple CNN branches or independent CNNs. This paper proposes a dark and bright prior guided deep network for retinal image quality assessment called GuidedNet. Specifically, the dark and bright channel priors are embedded into the start layer of network to improve the discriminate ability of deep features. Experimental results on retinal image quality dataset Eye-Quality demonstrate the effectiveness of the proposed GuidedNet.