Digitization of educational data and processes has enabled widespread development of technologies to support personalized learning. A key requirement in any personalized learning setup is to be able to accurately estimate students' weaknesses so they can be addressed appropriately during personalization. In this paper, we describe our work toward identifying K-12 students at risk of poor academic performance, with a special focus on 1) identifying specific components of varying granularity in the curriculum (such as subjects, topics, and concepts) that a student is finding difficult and 2) determining how early we can accurately estimate the risks. Such predictions could help teachers in planning effective personalized interventions for at-risk students and hence could help in achieving a long-term goal of minimal grade-level retentions and school dropouts. To predict performance risks, we use statistical models that utilize historical student data to learn patterns in their longitudinal journeys that correspond to performance risks. We describe in detail the risk prediction system we developed and its evaluation using data from one of the largest school districts in the United States. The experimental results demonstrate the ability of our system to make accurate risk predictions for subject-specific outcomes of varying granularity across different grade levels, early in a student's K-12 journey.