Software has eaten the world. What now?

Generative AI is changing the creative process, not just for writers and artists but for software developers, too. Because coding languages evolve, software developers are in constant demand to update applications to meet modern expectations. As chief scientist at IBM Research, Ruchir Puri leads IBM’s efforts to bring coding assistants, powered by generative AI, to code modernization.

IBM recently made its new watsonx Code Assistant generally available. It can selectively translate COBOL applications to high-quality Java code optimized for IBM Z and the hybrid cloud. It can also build IT automations for RedHat Ansible in plain English. We caught up with Puri to discuss the AI-for-code landscape and the future of software.

What’s the issue with legacy code?

The issue with legacy code is not that it’s legacy; legacy is not a bad word. The issue is that as programming languages and preferences evolve, the skill set for the legacy programming languages becomes smaller and smaller. Over time, it becomes a bottleneck for enterprises that have built significant code base in that language. You're not able to find enough skills to continue to update that mission-critical application. In finance, COBOL-based applications are running multitrillion dollar transactions on a regular basis; COBOL isn’t a bad language. There just aren’t enough skilled COBOL programmers to introduce new features and offerings. You want to bring the broader application into a modern environment.

Modernization!

Exactly. I would add that modernization is a continuous process because whatever is modern today will be legacy tomorrow. If your application is going to live for a long time, there’s no such thing as one-time modernization. IBM mainframes and the IBM Z platform have endured since the 1960s. Banks, insurance companies, governments, and others have built systems that operate with mission-critical SLAs (service-level agreements) and need to last. To modernize, you need to keep running those systems with the transactional performance, security, and resilience that's expected while updating the underlying language. To do that, you need to understand how to refactor the system into smaller components, or microservices, and wherever possible, move into a modern language. Some components may stay in COBOL, because of the transactional performance, but others will move into a modern enterprise language like Java.

What makes modernization so difficult?

Understanding legacy mission-critical applications is hard. If they are written in a legacy language like COBOL, it’s even more difficult. Most of the world’s remaining COBOL programmers are busy incrementally maintaining the system, which may have hundreds to millions of lines of code. Modernization is a manually intensive, drawn-out process that can take a decade or more and cost up to a billion dollars. Banks and governments want to move to a hybrid cloud environment and run Java on IBM mainframes. That’s where generative AI can assist. It can help them understand the complex application, refactor it, transform it, then validate the system. Generative AI can accelerate the transformation and reduce the cost by orders of magnitude.

At what point did IBM see generative AI as a solution for modernization?

Long before ChatGPT, it was clear to us that the world of AI and software would collide. This was our vision. Since data is the foundation of AI, we started our AI for Code effort by creating a new dataset called CodeNet, which Google DeepMind also used to create AlphaCode. We then moved on to generating Ansible code for one of our most popular Red Hat products, Red Hat Ansible Automation Platform (AAP). Then we moved on to modernizing legacy COBOL code for IBM Z, IBM’s transactional processing platform that is the backbone for much of world’s financial transactions. Along the way, we learned what it takes to roll out generative AI as part of enterprise developer tooling and developed watsonx Code Assistant. Both watsonx Code Assistant for Red Hat Ansible and IBM Z are now generally available.

For IBM watsonx Code Assistant for Z, the goal is to ease the pain points our customers are facing. Number one: close the skills gap. Number two: reduce the time to modernize these applications. And number three: continue to run those applications with SLAs, under very tight constraints. This is a transformational product that’s going to change how software is written.

How did you train IBM’s Granite models to “speak” COBOL?

Let’s say you want to teach your child Shakespeare. They first need to learn English and the nuances of the language. These models learn to understand coding as a language. In this case, IBM’s 20-billion parameter Granite model was trained on a large code base: 115 programming language, and 1.63 trillion tokens (which are words and parts of words).

This Granite 20B model is trained on a 32,000 token context window, which is the number of tokens the model can process at one time. If you’re generating code from a natural language prompt, the context window can be potentially smaller since the prompt length is limited. However, if you’re trying to translate an application written in COBOL with millions of lines of code, you need a much larger context window.

Another defining feature of our solution is its knowledge of COBOL, which is what’s known as a low resource language. There are probably more than 200 billion lines of COBOL code out there, but none of it’s on the internet. It’s behind firewalls. If you train a model on general, public data, you're going to come out with zero COBOL performance.

So where did you get the COBOL data?

IBM has worked with COBOL since its inception. It’s fair to say we know something about it. We have hundreds of experts who deal with that language every day. We spent an enormous amount of time curating a dataset to teach our model how to translate from COBOL to Java, not just generic COBOL patterns, but coding patterns for enterprise COBOL applications, like CICS, Db2, VSAM, and IMS. We created thousands of paired Java-COBOL programs. Since generative AI learns code with human patterns, the code it produces is also human-like. Some providers claim to have a system that translates COBOL to Java. Unfortunately, those systems rely on fixed rules-based translation. They tend to produce ‘JOBOL’, which is neither COBOL nor Java. It’s Java syntax written in COBOL style!

What’s next?

I've been working in the field of automation for more than three decades and this is probably the most exciting juncture I’ve seen. Marc Andreessen said in 2011 ‘software is eating the world.’ We can all agree software has eaten the world. Every enterprise is a software enterprise. The demand for software developers is through the roof. There isn’t a skills gap so much as an availability gap.

But AI can now transform how software is developed, generated, and tested. I call it software 2.0. Most software so far has been human generated. In this new automation-driven era, most code will be machine generated. That has been the goal since the word artificial intelligence was coined: for machines to be able to generate code for themselves. My hypothesis is it will be a decade-long journey, that we are just beginning. Human language will become the next interface to programming languages. The history of computer science has been all about abstractions. We used to program in a language called Assembly, which was close to a machine language. From there, we went to Fortran, COBOL, Pascal, then object-oriented languages like C++, Java, then scripting languages like Python, JavaScript, Go. The next abstraction level will be human language and that will bridge that gap to modernization, too.

Will there still be software engineers?

Definitely. I never have any doubts about the ingenuity and creativity of humans. It’s my personal belief that when machines and humans join forces, the world is a better place. Software developers will write a lot more software because they've got assistants. An AI agent can explain the code to them and help them debug and test it. That can make developers more productive, and enterprises more productive and secure.