Ontology evolution continues to be an important problem that needs further research. Key challenges in ontology evolution and creation of a highly consumable ontology include accomodating: (a) the subtle changes in the meaning of a model element over time, (b) the changing relevance of various parts of the model to the user, and (c) the complexity in representing time-varying semantics of model elements in a dynamic domain. In this work, we address the challenge of evolving an on- tology to keep up with the domain changes while focusing on the utility of its content for relevance and imposing constraints for performance. We propose a novel evidence accumulation framework as a principled approach for ontology evolution, which is sufficiently expressive and semantically clear. Our approach classifies model elements (e.g., concepts) into three categories: definitely relevant (that must be included in the ontology), potentially relevant (that can be kept as backup), and irrelevant (that should be removed). Further, our approach dynamically re-classifies models based on external triggers like evidence or internal triggers, like the age of a model in the ontology. As a result, users will have an ontology which is both effective and efficient. We evalu- ate our approach based on two measures - ontology concept retention and ontology concept placement. This comprehen- sive evaluation in a single framework is novel and we show that our approach yields promising results. Copyright © 2013 ACM.