Software wear versus technical debt

During software development, there inevitably comes a time when the architecture of a software starts to erode. This process usually creeps in secretly and only when a critical mass is reached or exceeded do we as software developers consciously notice the problem. Carola Lilienthal attributes the causes of software erosion to the following four phenomena [1]:

1. The "anyone can program" phenomenon
2. Architectural erosion increases unnoticed
3. The complexity and size of software systems
4. Management's and customers' lack of understanding of software development.

The first phenomenon occurs when inexperienced software developers implement local architectural or design ideas without considering the overall architecture. Unnoticed architecture erosion (phenomenon 2) occurs when the planned architecture no longer meets increasing requirements, but developers fail to make appropriate adjustments for reasons of time or cost. The complexity and size of software systems increases with the complexity of the application problem (see phenomenon 3). Usually, developers try to find simple solutions to complex problems, but this is not always successful. The last phenomenon arises from the problem that developers have to communicate software erosion to their customers and justify time as well as money for architectural adaptations. This article is dedicated to this phenomenon.

Once developers have recognised software erosion, they would like to refactor the software. Because as developers, we know that without refactorings, changes and extensions to the software will be difficult and expensive. If a development team ignores refactorings over a longer period of time, the maintenance and upkeep of a software takes steadily more time. The development team has less and less time to implement new requirements.

Figure 1: Increasing maintenance costs of software [1].

Motivating refactorings, but how?

Before this can happen, however, developers and project managers must convince their customers that refactorings are sensible measures that lead to an improvement in the software. This task is not easy, because refactorings initially cause costs and usually do not generate any technical added value. Customers only feel the benefits of refactorings very indirectly, for example, when they are able to realise more technical requirements in the long term. These reasons mean that refactorings are difficult candidates to place.

In order to describe the benefits of refactorings more in the language of the customers, the term technical debt was coined in software engineering [2]. According to Cunningham, minor technical debt can briefly speed up development, for example when developers incorporate a shortcut (or hack) to solve a task faster. As long as the shortcut is promptly removed through refactorings and the technical debt is thus paid off, there is nothing to be said against this approach. It becomes dangerous when debts are not repaid and developers implement new requirements on an eroded code basis. Then interest-interest effects arise that hinder further development or even make it impossible [2]. Every client should therefore understand that technical debt needs to be paid off in the short rather than medium term and that this task is imperative.

The problematic thing about debt is the term itself

What is difficult about technical debt is the psychological effect it has. Project managers can once or twice credibly convey to the client that the team has taken on technical debt. The third or fourth time at the latest, things get more difficult. Because in everyday life, everyone tries to avoid unnecessary debt by acting in a business-smart way.

Now the question arises, what impression does a customer get of a development team that repeatedly comes up with technical debt? The term debt alone negatively implies that a technically skilled development team could have, or even should have, avoided architectural erosion by acting correctly in software terms. Debts, whether technical or business-related, have a negative connotation and are therefore difficult to communicate. In business, they are usually consciously accepted in order to achieve added value in the medium or long term with the associated acquisition. In software development, technical debt is usually taken on unconsciously and rarely achieves added value even in the long term.

The term technical debt has a negative connotation. How can we nevertheless communicate the sense and benefit of refactorings to the customer in a simple and understandable way? At the same time, we as developers do not want to have to justify ourselves for having worked in a software-technically unclean way.

The term software wear and tear as a means of better communication

From our point of view, the term software wear and tear hits the nail on the head better. Wear and tear is a natural and unstoppable process that is well known in every manufacturing industry. Even in the most modern plants with the best skilled personnel, wear inevitably occurs and causes wear costs. The higher the production is driven in a plant, the higher the wear costs are. And those who drive on wear and tear in the long term will have to purchase a new plant after a certain period of time. In technical plants, maintenance is therefore part of everyday life; its technical and economic benefit is undisputed.

In software development, the term software wear can help simplify communication between customers and developers. Developers and project managers can make customers understand that the implementation of many technical requirements (possibly in a short time) causes the software to wear out over time. This leads to wear and tear costs that increase over time. Regular maintenance (refactoring) can reduce the wear and tear costs.

The advantage of the term software wear is that it keeps the blame off the developers. The wear and tear is not due to the lack of software skills of the developers, but to the extensive effort of the team and the consumption of resources. Even if there is no mechanical stress on components in software, wear and tear manifests itself in declining software quality. Software wear is a natural and unavoidable process. Unlike a debt, it is not consciously accepted, but occurs regularly in the best teams. Good teams are able to recognise software wear and tear and implement appropriate maintenance measures. Worse teams, or those that are forced to do so, develop software on wear and tear and have to realise at some point that, in the worst case, they will not be able to further develop their software.

Monetise the benefits of refactorings

In manufacturing industries, both the costs of wear and the costs and benefits of maintenance can be monetised and evaluated. In software development, on the other hand, it is very difficult for us to evaluate the economic benefit or the risks of missing refactorings. We usually try to convince the customer, present obvious arguments that everyone should be able to see, or point out horror scenarios if refactorings are not carried out.

Gernot Starke, Carola Lilienthal and Peter Hruschka presented an approach in Java Magazin [3] on how they evaluate the economic benefit of refactorings. Similar to the estimation of requirements, refactorings can also be broken down into small parts and evaluated. The basis for the evaluation can be measurable results such as execution time, memory consumption or assumptions. The latter allow the uncertainty of an estimate to be classified in an economic range. The higher the breadth of an estimate, the greater the uncertainty about the probability of risks occurring. For example, if a customer learns that the wear and tear of a software component causes monthly costs between 10,000 and 50,000 EUR, he can independently assess the risk to his business and make appropriate decisions.

While the estimation of requirements is established in software development, the estimation of wear costs is not yet common practice. The methods for this are comparable to those used in requirements elicitation. We just need more practice at it.

Example of estimating wear costs

We will use a client/server application as an example. The architecture of the client-side software does not allow remote services to be exchanged for local dummy services. Consequently, during UI development, the application must wait at startup for a remote service to fetch data from the server and visualise it in the client. The following measurements and assumptions are possible in our example:

• The measurable time of data transfer at application launch is 45 seconds per launch.
• Two developers work in pairs at a daily rate of 800 EUR/person.
• We assume that the application is launched 150 times per day to check the UI layout
• the development of the UI components is estimated to take eight weeks.

This results in the following costs:

• 800 EUR daily rate = 0.028 EUR/sec.
• 2 developers * 45 sec * 150 application launches/day * 0.028 EUR/sec = 378 EUR/day
• 40 days * 378 EUR = 15,120 EUR

The wear and tear costs will add up to 15,120 EUR in the eight weeks of development time. This includes only the pure development time. It is estimated that 70% of the costs of a software are incurred after delivery, in the maintenance phase. We also have to factor in these costs over the lifetime of a software. However, it is much more difficult to make assumptions about this, so we will stick with the 15,120 EUR. These can now be compared with the effort required for refactoring and evaluated in monetary terms. If the refactoring effort is significantly lower than the wear and tear costs, no customer will turn down refactoring.

References

[1] Gernot Starke in the preface to Carola Lilienthal: Langlebige Software-Architekturen, p. VI. dpunkt.verlag 2015

[2] Ward Cunningham: The WyCash Portfolio Management System. Experience Report, OOPSLA 1992

[3] Gernot Starke, Carola Lilienthal, Peter Hruschka: Against the Dark Force - Improve, but do it right. Java Magazine 8/15