Retinal fundus images are mainly used by ophthalmologists to diagnose and monitor the development of retinal and systemic diseases. A number of computer-aided diagnosis (CAD) systems have been developed aimed at automation of mass screening and diagnosis of retinal diseases. Eye type (left or right eye) of a given retinal image is an important meta data information for a CAD. At present, eye type is graded manually which is time consuming and error prone. This article presents an automatic method for eye type detection, which can be integrated into existing retinal CAD systems to make them more faster and accurate. Our method combines transfer learning and anatomical prior knowledge based features to maximize the classification accuracy. We evaluate the proposed method on a retinal image set containing 5000 images. Our method shows a classification accuracy of 94% (area under the receiver operating characteristics curve (AUC) = 0.990).