Segmenting actions in velocity curve space

Reliable segmentation of actions performed by objects, is critical to the understanding of semantic content in a video. Deciding which portion of object motion sequence is a distinct action unit, however, is often difficult, even for humans. We seem to hierarchically compose actions by noting characteristic changes in the nature of motion. This paper present an action segmentation algorithm that mimics these characteristics of action perception. Specifically, we show that significant curvature changes in the spatiotemporal curve formed from average velocity of the object, represent action boundaries. A hierarchical description of action is generated using a scale-space representation of this velocity curve. A method of automatic scale selection is also proposed to enable an optimal data-driven action segmentation. The technique is extensively evaluated on several different action types and comparison with human judgment of segments is reported. © 2002 IEEE.