We propose a new task, MultiDoc2Dial, for modeling goal-oriented dialogues grounded in multiple documents. Most previous works treat document-grounded dialogue as machine reading comprehension task based on a single given document or passage. We aim to address a more realistic scenario where a goal-oriented information-seeking conversation involves multiple topics, and hence is grounded on different documents. To facilitate such a task, we introduce a new dataset that contains dialogues grounded in multiple documents from four different domains. We also explore modeling the dialogue-based and document-based contexts in the dataset. We present strong baseline approaches and various experimental results, aiming to support further research efforts on such a task.