In many real-world applications, prediction problems are used to model forecast inputs for downstream optimization problems and it often suffices to check the performance of the final task-based objective, instead of intermediate task objectives, such as prediction error. The difficulty in end-to-end learning lies in differentiating through the optimization problem. Therefore, we propose a neural network architecture that can learn to approximately solve these linear programs, particularly ensuring its output satisfies the feasibility constraints. We further apply this to a multi-location newsvendor problem with cross fulfillment. We also analyze this problem with explicit fulfillment rules, and show the end-to-end problem can be solved with the exact derivative, without the need for approximations. We show that both these methods out-perform the predict following by optimize approach.