Technical debt can slow IT to a crawl: Dr. Bill Curtis, Cast Software

Businesses today are facing extremely expensive glitches owing to accumulated technical debt across global IT systems, says Dr. Bill Curtis, Chief Scientist at Cast Software and Director of the Consortium for IT Software Quality (CISQ).

Dr. Bill Curtis is best known for championing the spread of software process improvement globally as a Director of the Software Engineering Institute at Carnegie Mellon University. 

He is currently the Director of the Consortium for IT Software Quality (CISQ) and Chief Scientist of Cast Software. In an exclusive interaction with IDG Media, Dr. Curtis explained what's wrong with the prevalent software processes that are causing multi-million dollar glitches for companies.

Edited Excerpts

You were one of the pioneers of software intelligence and software process improvement. Can you throw some light on the notion of software intelligence?

The fact is businesses today are at risk now that everything is going digital and you see these 100 million dollar glitches in the system. 

Executives have no idea on how to measure risk on the business side. The notion of software intelligence is that we can provide hard data, based on the software at different levels and give an estimate of risk so developers can know the major defects in the system. 

 

... There is a lot of pressure on IT teams from the business side. Developers are made to work too hard and they get tired, they make more mistakes. The bottom line is that business executives need to understand that there is a limit at which they can deploy new software safely and with trust.
Dr. Bill Curtis
Chief Scientist at Cast Software and Director, CISQ

You have talked about technical debt in the past. Can you explain what happens when technical debt gets accumulated in IT systems?

Let’s define technical debt. The IT management has to make sub-optimal decisions to get the software out fast, which means they must come back and fix those sub-optimal pieces of code. This is technical debt. 

We at Consortium for IT Software Quality (CISQ) have quality metrics at the source-code level that are based on reliability, security, performance efficiency and maintainability.

We count the number of technical weaknesses to estimate technical debt and how long it will take to fix it. Now, if technical debt gets accumulated, it will affect the performance of the system and slow it down. Developers will make mistakes because the code become hard to figure out.

What can businesses do about technical debt?

It has to be a matter of policy that IT teams have protected time to go back and fix sub-optimal code. 

“To address technical debt in legacy systems, there has to be rigorous analysis of the COBOL code to extract the code design so it can be redone, and this is where artificial intelligence (AI) can also help.”

It has to be executive policy that protects that time. Business will always demand more in less time because of competition and customer demand. The business always wins, but developers must find time to fix technical debt first otherwise the business will not be agile. If technical debt accumulates, the code will become hard to work with and prevent new software functionality. 

If you look at legacy systems such as in the banking sector, do you see technical debt there and how can it be addressed?

In the analysis of legacy systems, you will see massive amounts of code written in COBOL. The code was written in a very complex manner and that's technical debt as it is unnecessarily complex. 

As the initial code was not documented mostly, companies don't want to mess up while improving the code, and that is a huge challenge. 

Today, what you see is developers creating wrappers around legacy systems and writing new code on top using JAVA to create modern functions, so the banking systems can interact with ATM machines and mobiles. 

To address technical debt in legacy systems, there has to be rigorous analysis of the COBOL code to extract the code design so it can be redone, and this is where artificial intelligence (AI) can also help. Until we do rigorous analysis and translate it right, there is going to be fear in re-writing the systems. 

There has been exponential innovation in software over the last 10 years. Has it made IT systems too complex and filled with technical debt? 

I have heard major analysts come to me and say they have seen massive growth in technical debt and it may bring a lot of systems to crawl. At the same time, it does not have to be that way. 

If you are a high maturity organization, you shouldn't be creating much technical debt. And even if you are going to create technical debt, you must have dedicated resources to eliminate it. If today IT teams are creating a lot of technical debt and they don't devote protected time to refactoring it, then it's going to be out of control later on.

How do you see open source platforms in the context of software development? Are they making IT systems more complex?

There are benefits to using open source in the sense that you don't have to build code from scratch. More and more developers, at least in the applications world, are picking pieces of software and mashing them together. 

The drawback is that you need to ensure that piece of software is a high quality component and doesn't put risk into the business application. The IT team must take responsibility to identify weaknesses in the code. 

Do you think there is less discipline in software engineering because businesses today are too focused on operational agility and competition?

If people are going to have highly iterative programming using DevOps chain to put things into operation, then that iteration better be disciplined because otherwise you will be shooting bad code down the chain and building more technical debt.”

There is a lot of pressure on IT teams from the business side. Developers are made to work too hard and as they get tired, they make more mistakes. The bottom line is that business executives need to understand that there is a limit at which they can deploy new software safely and with trust. 

As you get more automated tools to help developers write code, you can certainly increase the speed at which code is written. But if you overdo and give IT teams too much in not enough time, then you are going to crash and burn. 

What are your thoughts on AI playing an important role in software development?

We have been thinking that for decades. Back in the 1980s, we believed that AI was the way to do software development. The problem is that it is something very hard to do. It is hard to automate some of those really tricky decisions. 

But, that doesn't mean that there aren't roles for AI/ML in software development. While it has been challenging to use AI for complex processes, we certainly can build code generators for less complex processes. 

To sum it up, what according to you is the best practice for software development?

Here's what is going on - industrialization. While DevOps movement is exploding, developers are working as hard as they can to automate routine work to help improve business productivity.

If people are going to have highly iterative programming using DevOps chain to put things into operation, then that iteration better be disciplined because otherwise you will be shooting bad code down the chain and building more technical debt. 

The IT needs to embrace data and use it to continually improve the overall software development. It takes discipline, standardized processes and the need for trustworthy software.