Posted on by

Focus on Delivery

Year by year, the general quality of software worldwide, in every industry, declines. Software testing specialists lament the low quality of applications they use in their regular lives as well as those they are engaged to test.

It has become a sort of game to take photos of crashed billboards, kiosks, train and bus signage, and other software failures and post them online. The examples are called Kevlin Henneys, after a software engineer who started posting such photos several years ago. They’re amusing, but also lead one to wonder why software failures are so common, and how we came to accept bad software as normal.

I don’t claim to know all the causes of the phenomenon. One cause may be the sheer number of people involved with developing software. It’s been said that the number of programmers in the world has doubled every five years since about 1970. That may or may not be absolutely accurate, but in any case the key observation is that half the people writing the software products we depend on in our lives have less than five years experience.

Many of them never learn software development as an engineering discipline. They are graduates of rapid learning programs such as “bootcamps,” aimed at enabling a person to generate a mostly-boilerplate mobile app or web app quickly. Intuitively, this looks like it could be one cause of poor software quality.

But I’m exploring another possible cause in this post: The obsessive, almost manic preoccupation with delivery.

What’s wrong with delivery?

Since the early days of software, people have struggled to understand the nature of it. A number of metaphors are commonly used to describe software and software development.

It seems to me most or all of these assume a software product is basically the same as any other sort of product. It’s a “thing” that someone “makes” and then “delivers” to a “customer” who then “uses” it. The thing might wear out or get broken and require “maintenance.”

For what it’s worth, I don’t think that’s accurate. The nature of software is such that there is no clear dividing line between “under development” and “in use.” This is particularly true when contemporary methods of development are applied, when customers get their hands on a very early version of the product and provide feedback to developers for ongoing enhancement.

Considering that lifecycle for a software product, it seems obvious that the product is “in production” practically from Day One of its development, and is not “finished” until the last user shuts it down for the last time.

So, what does “delivery” mean? Does it mean anything at all, in the context of software? Are there any implications for the proper role of a software team? Is there really any point where they can say their work is “done,” as long as even one user remains active?

Is there really an obsessive focus on delivery?

Is it just me, or is the focus on delivery real?

At the risk of hearing the grating noise of someone demanding statistics to quantify the phrase, “vast majority,” I will use that phrase in the way human beings use it in everyday conversation, and share my observation that the vast majority of software developers and development managers hold the view their work is finished the moment the product becomes “shippable” or “potentially shippable” or is deployed to “production.” After that, it is SEP – Someone Else’s Problem.

There has also been an emphasis on speed of delivery since at least around 2000. Software development teams have tried to go faster, faster, faster. They want to get the work off their plates as quickly as possible. Shipping fast became more important than shipping quality. And it’s been that way going on 25 years now, with half the population of developers diligently applying their less-than-five-years experience to the effort.

Now, I understand you’re disinclined to believe me. So don’t. Those with the obsession to deliver will tell you themselves, proudly.

Deliver. Ship. Release. Deliver. Deliver. Ship. Ship. Deploy. Production-ready. Deliver. Ship. Ship. Ship. Cheeseburger, cheeseburger, ship, ship, ship! (Cultural literacy reference for those who need it: The Olympia Restaurant, SNL).

There’s nothing wrong with trying to deliver results to our customers without needless delay. But the nature of this concept of “delivery” strikes me as tantamount to delivering an unwanted baby to the steps of the orphanage. The baby is the initial form of the human. What about the rest of its life? SEP?

It’s the same with software. A “solution increment” that has been “delivered” is the initial form of a software feature. What about the rest of the product’s life? SEP?

Definition of Done

The Definition of Done, or DoD, is a popular concept on many software teams. It’s easy to find definitions, descriptions, and examples online. I’m going to describe the concept in a way its proponents probably would not.

Actually, that’s not quite true. I’m going to describe the concept in a way its proponents definitely would not.

There are several interesting quotes in this article that underscore the focus on delivery, in the sense of delivering the work product somewhere else besides the software team’s lap.

The Definition of Done is a “short, measurable checklist that mirrors usable and (sic) results in (sic) no further work required to ship your product.”

My take: “Dear orphanage, please take care of this baby. The mother squirted it out in accordance with her Definition of Done. Her sprint (i.e. her pregnancy) is closed. Now the baby is SEP. Thanks.”

“If you can’t ship working software at least every 30 days then by its very definition, you are not yet doing Scrum.”

My take: “Doing Scrum” is the purpose of the team. Getting the baby to the orphanage witout delay is the purpose of Scrum.

“Without a Defenition (sic) of Done we don’t understand what working software means, and without working software we can’t have predictable delivery.”

My take: For discussion purposes, let’s set aside the question of whether we can really understand what “working” means before users have engaged with the product in their real-world context. This statement says (to me): Delivery is the most important thing, and it needs to be predictable – that is, never-ending. Deliver, deliver, deliver. Cheeseburger, cheeseburger, cheeseburger…even if the product doesn’t really need another cheeseburger.

The DoD is “a list of criteria that must be met before a product increment ‘often a user story’ is considered ‘done’. Failure to meet these criteria at the end of a sprint normally implies that the work should not be counted toward that sprint’s velocity.”

My take: We did our part, and now we’re done. The baby has all the attributes you agreed to before you had a clear idea of how it would really turn out. If the child is misbehaving, contact the orphanage. We’ve got our velocity score, and you can’t take that away. We delivered.

“The Definition of Done is an agreed-upon set of items that must be completed before a project or user story can be considered complete. It is applied consistently and serves as an official gate separating things from being ‘in progress’ to ‘done.'”

My take: There is a hard boundary line between “in progress” and “done.” Once the work crosses that boundary, the development team has no further responsibility for it. (Except, as noted above, that is not the nature of software.)

Appearance of Activity

There’s another insidious factor related to the focus on delivery. Many people want to see activity on a software product, or they assume the product is “dead.” With respect to Open Source projects, for example, one consideration people apply when choosing a product to use is whether the code is updated frequently. If the code is not updated frequently, the assumption is that no one is paying attention to the product, and it would be risky to use it.

But many products are complete enough that they don’t really need more features. Additional features only add bloat and make the product harder for people to use. Once you’ve delivered the unwanted baby to the orphanage, there’s no value in continually adding more arms and legs and eyeballs to it.

The push to add more and more and more features as rapidly as possible only degrades the general quality of the product. As people come to rely on Open Source for more and more of their needs, they are increasingly affected by this steady degradation in quality.

In a kind of vicious circle, the demand for “activity” on Open Source project leads the maintainers of the project to push new features as rapidly as they can. In some cases, they make up “features” that aren’t necessary at all, so the product has an active commit history.

CapEx and OpEx

In my opinion, the main reason people persist in believing – or trying to force the issue – that there is a meaningful difference between software “in development” and software “in production” is that standard tax accounting rules in the United States allow the cost of software “in development” to be treated as a capital expenditure.

If we reason directly from the inherent characteristics of software, no such distinction is apparent. In contrast with most other kinds of products, software is always simultaneously in development and in use throughout its lifetime. On the merits, all costs associated with software ought to be expensed.

In August, 1985, the Financial Accounting Standards Board (FASB) issued Statement of Financial Accounting Standards No. 86: Accounting for the Costs of Computer Software to Be Sold, Leased, or Otherwise, Marketed.

It’s desirable for companies to capitalize product development costs because it enables them to defer tax liability. The rationale is that revenue from the product will not be realized until the product is in the market and generating sales, so it’s unfair to tax the development costs prior to that time.

For products that are not software, that makes sense to me. For instance, consider a company that designs and builds cars. The process of designing and figuring out how to mass-produce a new car model might take a couple of years. During that time, the company will earn exactly zero profit from the new model. It’s fair for them to be able to defer tax liability until the car is on the market and generating revenue.

But the nature of software is quite different. For a software product that is shrink-wrapped and sold, or that is leased, I can see the argument in favor of capitalizing the costs of initial development. But what about software for a company’s own use, for internal operations? It seems to be development of such a product never ceases as long as the software remains in use.

Companies requested further guidance regarding development costs for software intended for internal use. FASB issued ASC 350-40: Internal-Use Software Accounting & Capitalization in 2015.

Additional clarifications and guidance has been forthcoming to help companies account for development costs of cloud-based software and work done on a contract basis by remote workers. The underlying assumption is that software is “just like” other products, and the intent of the guidance is to help companies find some way to characterize software development as a capitalizable cost.

The advent of iterative software development methods and incremental software product delivery made this very challenging. With methods that have us performing “new” development and enhancement/support work simultaneously, or rapidly switching between the two on a time scale of minutes, it’s hard to draw the hard boundary line that people wish existed between “in development” and “in use.”

Paul Argiry, then CFO of the consultancy Leading Agile, spent considerable time trying to understand the implications of “agile” methods on the accounting side of software development, particularly for internal-use software. He has a short video summarizing his findings and suggestions, on Leading Agile’s site.

My time at Leading Agile overlapped Paul’s, and I asked him to teach me about software accounting rules so I could better understand our company’s position and advice to clients. Although I think I understood everything he explained, my concept of the nature of software did not allow me to accept the idea of capitalizing internal software development costs. It seemed like a game to me, to minimize the company’s tax liability any way they possibly could.

The distinction between “in development” and “in use” was just made up. But it happens to have been made up by people with the power of law behind them, and so it stands. Tracking the tiny bits of time that may or may not qualify as CapEx “feels” like thrashing to me; but as far as I know it may be financially beneficial, provided the tax savings are sufficient.

In the context of this post, my interest is not in whether there are tax benefits, but rather whether the practice affects software quality negatively. I think it does.

Most large companies separate the people who do “new development” from those who do “enhancement” or “maintenance” or “production support” specifically so that they can capitalize the costs of the “new development.” But as a practical reality, there’s no difference in those forms of work.

What makes sense, based on the nature of software, is that a team owns responsibility for a software product or a suite of products. Whatever needs to be done with respect to that product falls to its team. And the cost of the team is a steady expense.

The organizational separation of activities to enable capitalization has led practitioners of software development and support to assume the separation is meaningful, natural, and desirable. Thus, developers believe their work is truly done once the product is in the hands of users. But in my view this is not so, because software for internal use is continually under development.

When companies apply methods that are designed for product development, like Scrum, they experience difficulty mixing planned work on new features with unplanned work that arises in the normal course of operations. Those difficulties, in turn, lead development teams to try and define a boundary line beyond which they are no longer responsible for the baby and they can turn their attention to the next set of planned features. Their focus is on maintaining a steady stream of delivery.


There’s another kind of pressure on software teams in these companies. They need some way to show that they are actually performing work. Otherwise, they will be released during the next annual Reduction in Force Festival. Hence, you see many teams emphasizing “velocity” as a measure of “productivity.” To “earn” points toward velocity, they must deliver. Deliver what, exactly? Well, that’s not really the priority. It can be anything. And there goes quality.

Conclusion

It puzzles me when people say they don’t perceive software to have generally low quality, and they don’t really think there’s a problem. It seems pretty clear to me.

The question is, What causes low quality software? How did we get here, to a world heavily dependent on software and yet operated by software that barely functions and is often all but unusable?

We’re making the problem worse by the day, with our focus on delivering feature after feature, adding software to products that don’t need it, like refrigerators and toilet seats and desktop fans, plummeting headlong into a world dominated by AI that doesn’t even work but gives people the superficial impression of self-awareness.

I suspect there are multiple causes, and I think a disproportionate emphasis on delivery for the sake of delivery is probably one of them.

The next question becomes, What causes the disproportionate emphasis on delivery?

I think the answer to that question comes down to the assumption that there is a meaningful distinction between “in development” and “in use.” Can we move beyond that assumption? Time will tell.