Inspecto – Large Vision Model Inspection Service

Increasing speed and quality of enterprise visual inspection, leveraging AI and domain-specific Large Vision Models

Overview

General AI for computer vision is experiencing a surge of innovation fueled by the advent of vision transformers and Large Vision Models (LVMs). At IBM Research, we extended this technology to work for enterprise visual inspection, such as for infrastructure, automotive, manufacturing lines, quality control, and other domains where defects are often small and rare. We designed new algorithms and pipelines to make AI work on such applications, with high-resolution images and a limited amount of ground truth data.

Inspecto is an industry-research SaaS where this technology is prototyped and validated in collaboration with clients, before graduating into IBM products. Inspecto combines the use of LVMs, with advanced computer vision tools to enable engineers to perform complex inspection tasks. Inspecto aims to extend IBM’s Maximo for Civil Infrastructure workflow, to allow clients to conveniently navigate, explore, manage, and review hundreds of thousands of images and defects, and produce fully digitalized inspection reports, including measures and assessment scores.

Inspecto applied to the concrete surface inspection of the Stenungsöbron bridge in Sweden. This inspection was performed in a project for Trafikverket.

What are Large Vision Models?

Large Vision Models (LVMs) are foundation models trained on a large volume of data. They are designed to achieve high performance on downstream computer vision tasks, such as defect detection or object localization, with less labelled data.

Domain-specific Large Vision Models are LVMs that are fine-tuned on customers’ proprietary datasets of images from a specific business domain. Such models learn the most technical features of the specific domain and deliver higher accuracy on difficult inspection tasks.

At IBM Research we build domain-specific LVMs for technical domains and enterprise inspection applications, leveraging our hierarchical training pipeline that minimizes the need for annotated data.

‌Inspecto1-large.PNG — A diagram describing our hierarchical training pipeline for LVMs. Top left side: we fine-tune publicly available LVMs on images from user domains, and then we only use a minimal amount of annotations to build the desired detection task. Bottom left side: we leverage our pre-trained industry-ready LVMs to offer a solid starting point for critical infrastructure applications, such as concrete surface inspection (and soon also for rail). Right side: when users do not have enough data to start building an LVM, we leverage Visual Prompting to solve the cold start problem and build a first detection model from just a few images.

Industry-Ready Concrete Surface Inspection LVM

Detecting cracks in civil infrastructure, such as bridges, roads, and airport runways is crucial to prevent bigger problems and enhance maintenance routines. The 2020 American Road & Transportation Builders Association (ARTBA) report report says that more than 46,000 US bridges are “structurally deficient” and are in poor condition — those bridges are crossed 178 million times a day.

To address this issue, IBM Research has developed an AI model that uses computer vision to detect tiny cracks in high-resolution images collected by drones. It is a Foundation Model trained on more than 200k high-resolution images of concrete structures, and specialized in high-performance detection and localization of six critical defects: cracks, net-cracks, cracks with precipitations, rust, spalling, and algae.

Example of defects detected by our concrete surface inspection LVM.

The model was built with support and validation from domain experts and infrastructure partners. The model features state-of-the-art AI technology, such as vision transformers. It also includes innovation developed by IBM Research specifically designed for this application, which is not available from any other public model or vendor.

A qualitative comparison of traditional AI vs our IBM LVM. Our model needs only a third of the original annotated data to detect many more defects, generating a more precise segmentation mask at the same time.

Success Stories

In the past few years, we applied Inspecto and our LVM to many civil infrastructure projects, including the inspection of the Stenungsöbron bridge in Sweden with Trafikverket, as well as the inspection of the Great Belt Link in Denmark with Sund & Baelt:

Why Smarter Roads, Bridges and Tunnels are Good for Economies and Societies.

Last year, we successfully inspected the Dubendorf Air Base, near Zurich in collaboration with Canton of Zurich and drone company Pixmap.

Innovation Sandbox - What role will AI play in infrastructure maintenance?

Check out our interactive exploration of the runway, from aerial view to grass-level visualizations of tiny cracks in a few clicks.

Interactive Inspection of the Dubendorf Airbase.

Get in Touch

Our team at IBM Research is looking for prospective customers who wish to try our LVM for concrete inspection and learn more about our work. Submit your request here and we will contact you.