Despite considerable progress in the development of rapid evaluation methods for physics-based reservoir model simulators there still exists a significant gap in acceleration and accuracy needed to enable complex optimization methods, including Monte Carlo and Reinforcement Learning. The latter techniques bear a great potential to improve existing workflows and create new ones for a variety of applications, including field development planning. Building on latest developments in modern deep learning technology, this paper describes an end-to-end deep surrogate model capable of modeling field and individual-well production rates given arbitrary sequences of actions (schedules) including varying well lo-cations, controls and completions. We focus on generalization properties of the surrogate model which is trained given a certain number of simulations. We study its spatial and time interpolation and extrapolation properties using the SPE9 case, followed by a validation on a large-scale real field. Our results indicate that the surrogate model achieves acceleration rates of about 15000x and 40000x for the SPE9 and the real field, respectively, incurring relative error ranging between 2% and 4% in the interpolation case, and between 5% and 12% in the various spacial and time extrapolation cases. These results provide concrete measures of the efficacy of the deep surrogate model as an enabling technology for the development of optimization techniques previously out of reach due to computational complexity.