Color Image Semantics for Digital Libraries

Abstract

Color is the most frequently used feature in image analysis and retrieval. Over the years color features have been significantly improved, advancing from simple histogram-based descriptors to address more complex features such as dominant colors, important regions and spatial relationships. However, no matter how sophisticated, most of the proposed representations are still unable to cross the well-know semantic gap and when applied to diverse image collections seldom produce satisfactory results. This work analyzes the shortcomings of "traditional" color retrieval approach, and provides a new framework for "capturing" the elementary aspects of image semantics. Our approach is based on the observation that the words we use to describe images and their content represent a true link between the pictorial (low-level) representation and higher-level semantics derived by observers. Following previous experimental findings on the role of color in the perception of basic semantic templates (people, nature, indoor/outdoor, etc.), we propose a new feature representation based on color names. We also address several important steps needed to construct such representation: design of a color-name vocabulary and syntax, design of computational model and color naming metric, and the application to complex images. We also include some results that provide a good match to human judgments, thus validating the use color names to develop semantic descriptors.