What is technical debt (and how to reduce it)

May 28, 2013 Robert Hyde

While watching news reports about the US government and its struggle to bridge the gap between spending and generated tax revenue, I’ve discovered there are striking similarities between this issue and software development.

Financial debt is similar to technical debt – and both can be hard to deal with.

The problem with incurring a large debt (financial or technical) is that the cost of carrying it usually grows over time. Financial debt is usually caused by a shortfall in revenues due to changing economic conditions or the need for extra expenditure. Rather than dealing directly with the root cause of the problem, organizations sometimes choose to borrow – deferring the problem and thinking they can pay it down later. But borrowing has a cost, reflected as interest on debt. This interest compounds, which means it becomes harder and harder to fix as time passes.

As I see it, this same vicious cycle also occurs in the world of software development. A common issue we find when working with different organizations is that, over time, it takes longer for development teams to implement new changes into a system. And as more technical debt accumulates over time, it becomes harder to address the problem.

What is technical debt?

Ward Cunningham first used the Debt Metaphor in 1992 to describe the cumulative consequences of cutting corners throughout a software project’s design and development.[1] These consequences are manifested as a heavy backlog of technical inefficiencies that slow the pace of change and require more effort and cost.

This debt can build up and get in the way of efforts to add or change functionality within a system. When a system is newly built, a tweak or code change takes little effort and is completed close to the pace that customers or the business expects. Over time however, the pace slows and changes cost more, and on the surface it can be hard to understand why. Often, the reason is that shortsighted, “cost-conscious” decisions have incurred technical debt, and along the way organizations start to pay interest on this debt. Gartner estimates that worldwide technical debt is in the range of $500 billion dollars.[2]

What is the root cause of technical debt?

There are a variety of reasons why a system starts to incur technical debt including the following:

  1. Ambiguity. A lack of clarity of what’s truly needed can result in incorrect requirements, and issues with the application that’s produced.
  2. Shortcuts. Minimal budget, enhanced urgency or aggressive time to market requirements can result in shortcuts. “We don’t have the time to add the functional test now, the business just needs this done”, or, “The customer isn’t willing to pay for this, do without.”
  3. Weak engineering practices. The team can incur technical debt because they don’t practice disciplined software engineering, often without even realizing there’s an issue until much later.
  4. Weak refactoring. A subset of all three of the above, weak refactoring can make a code base brittle and hard to change. Refactoring is the practice of making changes throughout the codebase as a result of changing business requirements, architectural changes, technology changes, etc. It should be a regular, often daily activity in small doses – constant cleanup. In the worst examples of weak or nonexistent refactoring, code contains chunks of unsupported technology, cut and paste duplication, and other serious issues.
  5. Weak test automation. We define a legacy system as one without extensive automated tests. These systems are costly to maintain, and resist proper refactoring (because change is expensive and risky).

How does an organization deal with technical debt?

As Cunningham points out, incurring technical debt is not always a bad thing.[3] In the same way that credit cards allow us to obtain something sooner, we may incur technical debt as a way of getting an application to market faster. The problem really occurs when we don’t acknowledge this debt and/or put in place a plan for reducing it.

Eliminating technical debt isn’t easy, but I believe there are a few steps that can significantly reduce the problem. Many of these are training and mindset issues for the development team:

  • Identify technical debt explicitly. Call it out, size it if you can, and raise conversations with the product owners about the costs. Ensure there are trade-off conversations about spending time now to reduce the technical debt so that more product features can be included later, faster. This is a “pay it now or pay it later” conversation that needs to happen.
  • Implement engineering practices. Test-driven development (TDD) and continuous integration help ensure that code always reflects the current understanding of the business and is in a well-formed state.
  • Strive for low-dependency architecture. Reducing coupling (and therefore complexity) of interactions between modules and subsystems and striving towards a low-dependency architecture.
  • Readable code. By ensuring that code reflects current state and is readable by others makes it more maintainable and reduces the likelihood of creating additional technical debt.
  • Refactor aggressively and automate testing. Enhance the testability of code, implement programmatic (rather than scripted) automated testing, and fix the problems the developers know are there. Make use of automation tools where possible so that more focus and effort can be applied to refactoring and other debt pay-down activities.
  • Employ team-based approaches. Use cross-functional teams and pair programming where appropriate. This will better align developers and build clarity on requirements and implementation of those requirements into code, and tends to reduce the debt created.

Technical debt pay-down is a worthwhile commitment

The existence of technical debt is usually apparent to developers working with code on a daily basis, but is often hard to quantify for the business. Teams need to articulate the impact this debt has on the ability to deliver additional capabilities fast. In turn, the business needs to sponsor tasks that tackle technical debt.

Organizations interested in longer-term performance and competitive advantage should commit to strategies to pay-down technical debt in the software applications they build. With proper training, discipline and mindset, and an awareness of the principles of technical debt and its accumulation in applications, they can derive better benefits like cost reduction and faster time to market.


[2] http://blog.techdebt.org/resources-links/67/ward-cunningham-interview-about-technical-debt-sqale-agile

[3] http://c2.com/cgi/wiki?WardExplainsDebtMetaphor

 

As one of Architech’s Principal Consultants, Robert is responsible for business development, solution architecture, and leading teams in delivering Agile development projects. His role also involves the assessment and review of clients’ technology strategies. Connect with Robert on Linkedin or Twitter.







Previous Article
Toronto custom software development firm named #192 on the PROFIT 500
Toronto custom software development firm named #192 on the PROFIT 500

Architech lands on the PROFIT 500 for the second time in as many years...

Next Article
Organizational Agile first steps: Creating a vision
Organizational Agile first steps: Creating a vision

Somehow, somewhere, somebody decided Agile was the right thing for your...

We help our clients get their products to market 87% faster.

Find out how