Drug side-effects (SEs) may cause unexpected and adverse reactions in some patients. To better predict SEs, machine learning (ML) methods are more and more used. However, many existing ML methods can only be used to identify pair-wise associations between drug substructures and SEs, we propose to use a novel method called GraphSE to learning for patterns among SEs, among drug sub-structures, and between multiple drug substructures and the SEs. GraphSE performs its tasks by first computing an association measure to determine the significance of co-occurrence of each drug substructure and each specific SE. Each SE can then be characterized by attributes represented by these significant substructures. Based on it, an attributed graph can be constructed for each SE by defining a measure of molecular similarity based on a low-rank approximation scheme. Given the attributed graphs, we can discover in them a set of subgraphs that can be explainable and can be used to predict if a drug may lead to a certain SE using a Bayesian approach. Extensive experiments using real-world data show that GraphSE can be potentially very useful.