Cognitive storage
Big advances in storage technologies and in storage management have enabled storage administrators to keep up with the exponential data growth over the decades, and to keep the ever increasing storage system complexity manageable so far. However, the question arises whether these advances in storage technologies will be sufficient to accommodate the future data growth and to effectively handle the ever increasing system complexity.
Another question is whether the future storage capacity growth will fall behind data growth rates, meaning that the standard model of storing all data forever will no longer be sustainable due to a shortage of available storage resources. In other words, we must decide whether we even need or want to store all the data that is being generated today and will be in the future.
What we do under the umbrella of cognitive storage is to make an attempt to understand the value and relevance of the data and, based on this data-specific knowledge, determine where, with how much redundancy, and for how long to store the data. Depending on the distribution and evolution of the relevance of the data, there appears to be a large potential for significant storage capacity savings.
Many challenges have to be addressed on the way to a fully-fledged cognitive storage system, most prominently whether and how the data value can actually be defined in a systematic way, whether and how it can be obtained automatically, how it will change over time, whether a future storage system should be able to extract relevant information from data autonomously and store it in a modified compact form, and many more.
Contact the expert
Mark Lantz
IBM Research Scientist