Advances in deep learning have yielded simplified solutions in many challenging medical imaging problems. Despite their capability, these learning approaches often fail to produce accurate and reliable segmentation in echocardiography image sequences due to varying amounts of speckle noise accompanied with ill-defined and missing boundaries. For this, we propose a combination of deep learning and a shape-driven deformable model in the form of level set. The proposed method uses an end-to-end trained fully convolutional network that acts as a prior to drive the level set-based deformable model. Further, we define a new energy formulation within the level set framework that accounts for characteristics of the desired structure. By employing the proposed method, we achieve accurate and reliable segmentation of cardiac structures (left ventricles) both qualitatively and quantitatively.