Many irregular domains such as social networks, financial transactions, neuronconnections, and natural language structures are represented as graphs. A varietyof graph neural networks (GNNs) have been successfully applied for representa-tion learning and prediction on such graphs. However, in many of the applica-tions, the underlying graph changes over time and existing GNNs are inadequatefor handling such time varying graphs. In this paper we propose a novel techniquefor learning embeddings of time varying graphs based on a tensor framework. Themethod extends the popular graph convolutional network (GCN) for learning rep-resentations of time varying graphs using the recently proposed tensor M-producttechnique. Numerical experiments on four real datasets demonstrate that our pro-posed method outperforms a baseline method when used for edge classification.