About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Publication
CIVR 2007
Conference paper
Semantics reinforcement and fusion learning for multimedia streams
Abstract
Fusion of multimedia streams for enhanced performance is a critical problem for retrieval. However, fusion performance tends to easily overt the hillclimb set used to learn fusion rules. In this paper, we perform fusion learning for multimedia streams using a greedy performance driven algorithm. In our fusion learning paradigm, fused output is a linear combination of multiple classifers or ranked streams. The algorithm is inspired from Ensemble Learning [2] but takes that idea further for improving generalization capability. A key application of our fusion learning algorithm, described in this work, is semantics reinforcement using an ensemble of classifers built using the same training dataset but groundtruth corresponding to different concepts. We expect that classifers built for semantically close concepts should reinforce each other's performance and fusion learning is an excellent post-classifcation way to reinforce semantics and performance. Fusion learning experiments have been performed on TRECVID 2005 test set. Experiments using the well established retrieval efectiveness measure of mean average precision reveal that our proposed algorithm improves over the best classifer (oracle) by 3.8%. We also present and discuss some interesting and intuitive semantic reinforcement trends observed during fusion learning. Copyright 2007 ACM.