Despite the large number of patients in Electronic Health Records (EHRs), the subset of usable data for modeling outcomes of specific phenotypes are often imbalanced and of modest size. This can be attributed to the uneven coverage of medical concepts in EHRs. We propose OMTL, an Ontology-driven Multi-Task Learning framework, that is designed to overcome such data limitations.The key contribution of our work is the effective use of knowledge from a predefined well-established medical relationship graph (ontology) to construct a novel deep learning network architecture that mirrors this ontology. This enables common representations to be shared across related phenotypes, and was found to improve the learning performance. The proposed OMTL naturally allows for multi-task learning of different phenotypes on distinct predictive tasks. These phenotypes are tied together by their semantic relationship according to the external medical ontology. Using the publicly available MIMIC-III database, we evaluate OMTL and demonstrate its efficacy on several real patient outcome predictions over state-of-the-art multi-task learning schemes. The results of evaluating the proposed approach on six experiments show improvement in the area under ROC curve by 9% and by 8% in the area under precision-recall curve.