Interactive Image Segmentation Guided by Visual Prompting

Abstract

We present an image segmentation framework based on a fast feature matching algorithm designed to leverage powerful pretrained vision foundation models. Our framework is embedded into a computer vision annotation tool which enables an interactive image segmentation workflow for precise segmentation guided by visual prompting. Users begin by uploading an image and marking objects of interest with scribbles directly on the interface. These scribbles serve as annotations to guide the segmentation process. Upon initiating segmentation, the system leverages pretrained models and our efficient matching algorithms to generate segmentation masks in real-time. Users can then review the initial results and iteratively refine them by adding or deleting scribbles as needed. This interactive workflow enables rapid feedback loops, facilitating the iterative refinement of annotations tailored to the requirements of the particular visual domain at hand. By customizing annotations iteratively, users can swiftly optimize segmentation accuracy, resulting in a comprehensive, data-driven solution for achieving precise segmentation across diverse technical visual domains.