In hospitals, physicians diagnose brain-related disorders such as epilepsy by analyzing electroencephalograms (EEG). However, manual analysis of EEG data requires highly trained clinicians or neurophysiologists and is a procedure that is known to have relatively low inter-rater agreement (IRA). Moreover, the volume of the data and rate at which new data is acquired makes interpretation a time-consuming, resource hungry, and expensive process. In contrast, automated analysis offers the potential to improve the quality of patient care by shortening the time to diagnosis, reducing manual error, and automatically detecting debilitating events. In this paper, we focus on one of the early decisions made in this process which is identifying whether an EEG session is normal or abnormal. Unlike previous approaches, we do not extract hand-engineered features but employ deep neural networks that automatically learn meaningful representations. We undertake a holistic study by exploring various pre-processing techniques and machine learning algorithms for addressing this problem and compare their performance. We have used the recently released 'TUH Abnormal EEG Corpus' dataset for evaluating the performance of these algorithms. We show that modern deep gated recurrent neural networks achieve 3.47% better performance than previously reported results.