Abstract
In a number of applications including surveillance, there is a need to reliably retrieve an action-depicting segment in a video. This is an enormously difficult problem due to the variability in an action's appearance when seen at different times. It requires reliable object and action segmentation, and robust methods for indexing the action content in a video. In this paper, we present a novel approach to action retrieval that extracts salient action events in query and database videos. These events serve as anchor points to initiate action recognition. Actions are recognized by forming a spatio-temporal shape for an action called the action cylinder. Robust recognition is achieved by recovering the viewpoint transformation and time correspondence between a query action and a given action segment in the video. We demonstrate the versatility of our method for the retrieving of complex actions within videos.