How to foster deliberate innovation with open-source software

Open source is at the heart of much of what we do at IBM Research. From accelerating materials discovery and advancing open-source scientific research, to building the simplest way to run AI pipelines on the cloud and by collaborating across open-source communities like Ray, PyTorch, Knative, and Kubernetes, open-source is so fundamental to how we’re envisioning the future of computing. It’s imperative to the development and hosting of IBM's large-scale, flexible foundation models, and to how we’re securing the hybrid cloud.

As researchers, this open innovation provides a forum for us to discuss the problems and challenges with a broader community. By default, this accelerates the pace of innovation when less time is spent having to duplicate proprietary technologies and ideas are shared across organizations. When we collaborate, we can converge on standard approaches across industries. It's essential to the way we work in IBM Research. 

With all our open-source efforts, our goal is to achieve deliberate innovation, which we see as the union of technical innovation and actual demand, driven by some need. This could be something like the requirements of strategically important emerging AI workloads from industry partners. This creates an opportunity for us to collaborate and progress. 

Enterprises have accelerated their use of cloud-native technology built on Kubernetes, driven primarily by a need to digitize every aspect of their business. For many, this is an opportunity to innovate, as well as add flexibility and scalability into their businesses. But innovating with open-source software is complicated by the availability of potentially viable open-source projects to solve any particular challenge.

It’s often hard to find open-source projects, but even more difficult to know which the right open-source projects are to invest in. The only way to determine whether an open-source project is right for your needs is to assess your requirements against what the project supports, experiment with it, and stand up a small proof of the technology to vet your use case. This is easier said than done, particularly when there is a lack of readily available skills to go through this type of evaluation process. 

This was what led IBM Research to create the Open-Source Incubator (OSI). We wanted to foster the right culture for enterprises invested in open-source innovation to work closely with us to solve their unsolved problems.

While we cover many different open-source projects in our research, we focused on three things when considering what projects to start with for the Incubator. We wanted open-source projects that forward our hybrid multi-cloud vision while making it simpler for clients to adopt the cloud. We also wanted to include products where we are either co-founders or we have strong leadership positions in the community to help guide these projects into the future. Finally, we wanted projects that are mature enough to solve real problems and that are ready for major client engagements.

The Open-Source Incubator helps enterprises explore open-source projects, prove they work for their problems, and evolve these projects to meet their needs. We want to work closely with organizations, to help them move from pilot engagements to pre-production, and ultimately, production readiness. As open-source projects evolve and become more mature and mainstream, our Incubator clients will be able to immediately benefit from them, rather than experiment and find all the gaps later on down the road.

In some cases, our clients are already active in open source. The OSI is an opportunity for us to join forces — we know that moving the direction of large open-source communities is not easy. It takes time and shared vision to convince the community. A notable example is Kubernetes: IBM Research is continuing to evolve the capabilities of Kubernetes to meet new workload needs, such as AI and machine learning. 

To learn how you can become part of the Open-Source Incubator, click here.