Diabetic Retinopathy (DR) is one of the leading causes of blindness worldwide. Detecting DR and grading its severity is essential for disease treatment. Convolutional neural networks (CNNs) have achieved state-of-the-art performance in many different visual classification tasks. In this paper, we propose to combine CNNs with dictionary based approaches, which incorporates pathology specific image representation into the learning framework, for improved DR severity classification. Specifically, we construct discriminative and generative pathology histograms and combine them with feature representations extracted from fully connected CNN layers. Our experimental results indicate that the proposed method shows improvement in quadratic kappa score (κ2 = 0.86) compared to the state-of-the-art CNN based method (κ2 = 0.81).