Software wear and tear versus technical debt

During software development, there inevitably comes a time when the architecture of a software starts to erode. This process usually creeps in secretly and only when a critical mass is reached or exceeded do we as software developers consciously realise the problem. Carola Lilienthal attributes the causes of software erosion to the following four phenomena [1]:

1. The "anyone can programme" phenomenon
2. Architecture erosion is increasing unnoticed
3. Complexity and size of software systems
4. The lack of understanding of software development on the part of management and customers

The first phenomenon occurs when inexperienced software developers implement local architecture or design ideas without considering the overall architecture. Unnoticed architecture erosion (phenomenon 2) occurs when the planned architecture no longer meets the increasing requirements, but developers fail to make the necessary adjustments for reasons of time or cost. The complexity and size of software systems increases with the complexity of the application problem (see phenomenon 3). Developers usually try to find simple solutions for complex problems, but this is not always successful. The last phenomenon arises from the problem that developers have to communicate software erosion to their customers and justify time and money for architecture adaptations. This article is dedicated to this phenomenon.

Once developers have recognised software erosion, they would like to refactor the software. As developers, we know that without refactorings, changes and extensions to the software become difficult and expensive. If a development team ignores refactorings over a longer period of time, the maintenance and upkeep of the software takes up more and more time. The development team has less and less time to implement new requirements.

Figure 1: Rising software maintenance costs [1]

Motivating refactorings, but how?

Before this happens, however, developers and project managers want and need to convince their customers that refactorings are sensible measures that lead to an improvement in the software. This is not an easy task, as refactorings initially incur costs and generally do not generate any technical added value. Customers only feel the benefits of refactorings very indirectly, for example when they are able to realise more technical requirements in the long term. These reasons mean that refactorings are difficult candidates to sell.

In order to describe the benefits of refactorings more in the language of customers, the term technical debt was coined in software engineering [2]. According to Cunningham, minor technical debt can speed up development for a short time, e.g. when developers add a shortcut (or hack) to solve a task faster. As long as the shortcut is promptly removed again through refactorings and the technical debt is thus repaid, there is nothing to be said against this approach. It becomes dangerous when debts are not repaid and developers implement new requirements on an eroded code base. This creates interest-rate effects that hinder further development or even make it impossible [2]. Every customer should therefore understand that technical debt must be reduced in the short rather than the medium term and that this task is imperative.

The problem with debt is the term itself

The difficult thing about technical debt is the psychological effect it has. Project managers can credibly convey to the customer once or twice that the team has taken on technical debt. The third or fourth time at the latest, things become more difficult. Because in everyday life, everyone tries to avoid unnecessary debt by acting in a business-smart way.

Now the question arises as to what impression a customer gets of a development team that has recurring technical debts. The term debt alone implies negatively that a technically skilled development team could or even should have avoided architectural erosion if it had acted correctly in terms of software technology. Debts, whether technical or commercial, have a negative connotation and are therefore difficult to communicate. In business, they are usually consciously accepted in order to achieve added value in the medium or long term with the associated purchase. In software development, technical debt is usually taken on unconsciously and rarely achieves added value in the long term.

The term technical debt has a negative connotation. How can we nevertheless communicate the sense and benefits of refactorings to the customer in a simple and understandable way? As developers, we don't want to have to justify ourselves for having worked uncleanly in terms of software technology.

The term software wear and tear as a means of better communication

From our point of view, the term software wear and tear hits the nail on the head. Wear and tear is a natural and unstoppable process that is familiar in every manufacturing industry. Even in the most modern systems with the best specialised personnel, wear is unavoidable and causes wear costs. The higher the production in a plant, the higher the wear costs. And anyone who runs on wear and tear in the long term will have to purchase a new system after a certain time. Maintenance is therefore part of everyday life in technical systems, and its technical and economic benefits are undisputed.

In software development, the concept of software wear and tear can help to simplify communication between customers and developers. Developers and project managers can make customers understand that the realisation of many technical requirements (possibly in a short time) causes the software to wear out over time. This leads to wear and tear costs that increase over time. Regular maintenance (refactoring) can reduce wear and tear costs.

The advantage of the term software wear and tear is that it keeps the question of blame away from the developers. The wear and tear is not due to the developers' lack of software skills, but to the team's extensive performance and the consumption of resources. Even if there is no mechanical stress on components in the software, wear and tear manifests itself in declining software quality. Software wear and tear is a natural and unavoidable process. In contrast to a debt, it is not consciously accepted, but occurs regularly in the best teams. Good teams are able to recognise software wear and tear and carry out appropriate maintenance measures. Poorer teams, or those that are forced to do so, develop software for wear and tear and have to realise at some point that, in the worst case scenario, they will not be able to further develop their software.

Monetise the benefits of refactoring

In manufacturing industries, both the wear and tear costs and the costs and benefits of maintenance can be recognised and evaluated in monetary terms. In software development, on the other hand, it is very difficult for us to assess the economic benefits or risks of missing refactorings. We usually try to do this by making a good case to the customer, presenting obvious arguments that everyone should be able to understand or pointing out horror scenarios if refactorings are not carried out.

Gernot Starke, Carola Lilienthal and Peter Hruschka have presented an approach in Java Magazin [3] on how they assess the economic benefits of refactorings. Similar to the estimation of requirements, refactorings can also be broken down into small parts and evaluated. The basis for the evaluation can be measurable results such as execution time, memory consumption or assumptions. The latter allow the uncertainty of an estimate to be categorised into an economic range. The higher the range of an estimate, the greater the uncertainty about the probability of risks occurring. For example, if a customer learns that the wear and tear of a software component will cause monthly costs of between EUR 10,000 and EUR 50,000, they can independently assess the risk to their business and make appropriate decisions.

While the estimation of requirements is well established in software development, the estimation of wear and tear costs is not yet common practice. The methods for this are comparable to the methods used in requirements engineering. We just need more practice.

Example of an estimation of wear and tear costs

We will use a client/server application as an example. The architecture of the client-side software does not allow remote services to be exchanged for local dummy services. Consequently, during UI development, the application must wait at startup for a remote service to retrieve data from the server and visualise it in the client. The following measurements and assumptions are possible in our example:

• The measurable time of data transfer when starting the application is 45 seconds per start.
• Two developers work in pairs at a daily rate of 800 EUR/person
• We assume that the application is started 150 times per day to check the UI layout
• the development of the UI components will take an estimated eight weeks.

This results in the following costs:

• 800 EUR daily rate = 0.028 EUR/sec.
• 2 developers * 45 sec * 150 application starts/day * 0.028 EUR/sec = 378 EUR/day
• 40 days * 378 EUR = 15,120 EUR

The wear and tear costs will add up to 15,120 EUR in the eight weeks of development time. This only includes the pure development time. It is estimated that 70% of the costs of software only arise after delivery, during the maintenance phase. These costs must also be factored in over the lifetime of the software. However, it is much more difficult to make assumptions about this, so we stick with the EUR 15,120. This can now be compared with the cost of refactoring and evaluated in monetary terms. If the refactoring effort is significantly lower than the wear and tear costs, no customer will turn down refactoring.

References

[1] Gernot Starke in the foreword to Carola Lilienthal: Langlebige Software-Architekturen, p. VI. dpunkt.verlag 2015

[2] Ward Cunningham: The WyCash Portfolio Management System. Experience Report, OOPSLA 1992

[3] Gernot Starke, Carola Lilienthal, Peter Hruschka: Gegen die dunkle Macht - verbessern, aber richtig. Java Magazine 8/15