Image classification and querying using composite region templates

The tremendous growth in digital imagery is driving the need for more sophisticated methods for automatic image analysis, cataloging, and searching. We present a method for classifying and querying images based on the spatial orderings of regions or objects using composite region templates (CRTs). The CRTs capture the spatial information statistically and provide a robust way to measure similarity in the presence of region insertions, deletions, substitutions, replications, and relocations. The CRTs can be used for classifying and annotating images by assigning symbols to the regions or objects and by extracting symbol strings from spatial scans of the images. The symbol strings can be decoded using a library of annotated CRTs to automatically label and classify the images. The CRTs can also be used for searching by sketch or example by measuring image similarity based on relative counts of the CRTs.