The Cost Of Software

August 1, 2023

Big thanks to Rick Mackenbach, Pratibimb Dwivedi, Nicole Tan , Erwin Hoogerwoord and members of my family for their feedback and contributions to this article.

Why does software development cost companies up to 4.3 trillion USD annually?

This figure tells me that it is very inaccessible to problem solvers world wide. As a powerful tool for building solutions, it is worthwhile exploring why it costs so much.

Why Software Is Eating The World

Software is eating the world, said Marc Andreesen in 2011. Today in the year 2023, almost every organisation relies on automation and data, enabled by software systems. We use phones, laptops, smart fridges, Toyotas with parking sensors, security cameras, digital door locks, coffee machines and self ordering terminals at McDonald’s.

Software eats the world because it allows organisations to run on auto-pilot. Fanuc, the Japanese robot manufacturer, once ran their factory without human intervention for close to 60 days. It created data for the Fanuc team to tell them what they should change, so that they can improve their target metrics.

Any team looking to solve problems today, relies on software development as a crucial capability. Whether they are solving food waste, healthcare cost or recycling. However, there is a major problem that makes that hard, if not impossible for them.

Automation lowers operational expenses, something new businesses often need to be viable at all. Systems bring self-service to millions of people, allowing them to access essential services that would not be possible if we only relied on phone or paperwork.

Automated systems allow services to run with low error rates and have 24/7 availability. Something not possible when relying on a workforce alone. This is why organisations are willing to spend so much.

The Problem With Software.

Software costs companies globally a combined $4.3 trillion U.S dollars each year . This figure is from 2022, and is projected to be $4.6T in 2023.

This spending is split across developer salaries, supporting staff, cloud hosting and infrastructure.

It makes software almost completely inaccessible to many problem solvers around the world. People who want to make things better, likely can’t get started at all. This is a form of inequality where a small group of people with access to capital are able to fund problem solving, while many are left out.

This is why the cost of software is a problem worth studying.

It is about ideas that survive.

This cost will lead problem solvers to try and raise funding to enable them to take a shot. Startups track their runway, the number of months they can last on their current bank balance.

The cost of software significantly shortens their runway, and threatens the survival of new ideas.

How does this problem manifest?

An example of this problem is shown in a story about a city council that got a 100m GBP bill for a software project. What was supposed to be a straightforward migration, resulted in delays and a fourfold increase of their initial budget. Source: The Register + Birmingham City Council report.

The council delegated problem solving, and did not understand the process of software development. They attempted to lower the risk by adding additional requirements. Consultants were more than happy to help, and bill extra hours for additional requirements.

The system they used was not flexible, leading to longer times for changes, and higher financial cost for each change. Team members became afraid to ask for solutions, as any change meant that the bill kept running up. Well, maybe they were not afraid to ask for changes, considering the 100M invoice that landed on the CFO’s desk.

When you are already deep out of the pocket, it feels like you "can't just start over", or "the system works like this now we can't just change it".

The slow pace and financial cost of changes, meant that experimentation became prohibitively expensive in this model.

Experimentation means to iterate, and is a key method in keeping costs low. You do not attempt to get a correct specification, but rather test reality to discover what a problem really is.

Yes I am serious, this process can be applied to ERP systems too. A usual belief is that ‘big’ and ‘serious’ systems should require ‘serious’ planning and risk management. I will touch upon how traditional management styles can have counterintuitive results as it comes to systems.

So what is the problem really?

Raising large amounts of cash is not a possibility for many founders. And tech salaries don’t fit their budget, meaning they can’t use methods like iteration, experimentation and trial and error to test ideas. Most teams likely lack the skill to develop software themselves, based on the figure that there are only 26 million software developers in the world today.

Some ideas require traction to raise capital, but gaining traction requires some kind of working prototype, which requires capital. That’s a frustrating chicken and egg problem for new ventures.

1. Aren’t there proven methods to lower the risk of software development?
2. And, can’t you “just find a good hacker”?

1 -> The methods to lower risk of software projects require additional workforce and time. That is mostly only accessible to well funded startups and enterprises.

2 -> “Good hackers” are in fact rare. The opportunity cost, stability and life circumstances of engineers, means they’re not readily available to jump into new ventures.

So why is software development hard?

How software systems are built today is a complicated mess.

At the root of it, are multiple people sharing a logical understanding of a problem. Their shared understanding is not complete at the start of a project. They need a model in their head of the same logic, to be able to collaborate effectively.

A team is made up of UX Researchers, Product Managers, Designers, Developers, Testers and maybe a Business Analyst or Manager. They use tools like Miro, Jira, Clickup, Figma, Sketch, Github, Visual Studio Code, Postman, Google Analytics and maybe Tableau.

The cost of translation between people and tools.

The steps taken to build a system are fragmented between people and the tools they use. The team bears the cost of translating a user story on Jira, written by a product manager, into a design on Figma, by a designer. Each step, between each tool is adding time, to have one person like a developer, understand what a designer or product manager intends to achieve. All these costs add up to go from problem to idea, to user story, to persona to design to code, to testing to release, deployment and analytics.

Changes to the system result in unexpected outcomes that become issues to be resolved through the same pipeline of

1. -> Researcher
2. -> Product Manager
3. -> Designer
4. -> Developer
5. -> Tester
6. -> Analyst

In practice, companies often take shortcuts from the ‘proper’ process, because having enough time and cash is rare to fund entire teams. Shortcuts could mean skipping the research step, or going straight from user stories to development.

Software can be a black box in the value chain.

The step of writing code appears like an opaque, black box that nobody understands except developers. So the step of ‘touching the code’ seems like a nuclear risk requiring a hazmat suit to those inexperienced.

Trying to lower the risk of software development.

Because of the high cost of software development, additional fears are imposed on how developers work. Managers try to lower risk by introducing more processes, to make sure it’s all 100% clear what is going to be built. But those attempts to reduce the risk of software development, which are driven by the fear of running into delays and budget overruns, can themselves slow down projects and increase risk.

The problem with giving requirements to software developers.

This problem is magnified when working with freelance developers or external agencies. Because they need to balance the risk of payment, with delivering the requirements.

In my 10 years in software, having full agreement on what the requirements are is often not possible. That means most projects start with an incomplete understanding of the requirements. Unlike having a wall painted, a marketing poster designed, there is less flexibility with software development. What appears like a small insignificant change to a manager, may add an unacceptable cost to a developer.

When a contractor delivers a software project, from the customer's point of view, it is almost never what they expected, based on their understanding of the requirements.

This problem is unique to engineering, because it depends on a shared understanding of logic, and “how the world works” between 2 parties, who often don’t share the same understanding. The outsized cost to the engineering party for making changes to a system means that they resist.

Furthermore, with the attempt to introduce deadlines on software projects, everyone sets themselves up for disappointment. Fixing a living room light is easy to predict, painting a wall too, but solving a software problem is not predictable, and especially not if it has to follow a set of requirements that are put into a contract.

Video game studios use a method where they don’t announce a release date until they have achieved some level of quality. Although even then it doesn’t mean they’ll always meet that self-imposed-deadline.

If you want to test my thesis here, go to a few software developers and ask the following questions:

1. Ask them to guarantee that they can deliver the requirements
2. Ask them for an estimate on how much time it will take
3. Then ask them for a guarantee that they will deliver in that amount of time

You will find that getting a guaranteed delivery date, means getting an estimate amount of time that you do not feel comfortable with. You ask if it can be less or you ask another software developers to do it. This drives developers, employees, contractors on a downward spiral to promise delivery on timelines that are not possible.

As long as we rely on individuals who write the code on systems that no one else is allowed to touch, we will remain stuck in this compromise that leads to budgets that overrun by 75 million GBP.

The unpredictability of problem solving.

When solving new problems, new information emerges as you go along. It requires building a model of how the world works, and how your solution will work with that.

People do not always comprehend the complete nature of a problem, they are not able to build that model in one sitting. It takes testing, experimentation, research and observation.

Because of the unknown-unknowns it is not possible to predict when or how a problem will be solved. The scarcity of this skill set means the price per hour is high.

Failed attempts at countering the financial risk of software development.

The unpredictable nature of problem solving means that the costs related to it are not predictable either. To any corporation, including startups, that is financial risk. It depends on how you feel about taking those risks. If you feel bad about this risk, you might be induced to introduce measures to limit that risk.

But with systems development, traditional risk-management methods can increase risk instead of reducing it. Adding process means lowering experimentation. Adding checks, balances and reporting means removing creativity.

Risk management can suck the oxygen out of a problem solving process. If the process of problem solving is not understood by those attempting to manage risk. This can make it more unpredictable, instead of less.

Conclusion.

In the world of software development, a problem should only have to be solved once. In reality, this is not shared between all people.

Software development applies to robotics, healthcare, websites, ecommerce, payments and much more. Software development is problem solving, which means to analyse the who/why/what/when of how the world really works. It leads us to take a much closer look at the world than we normally do, and our intuition is not our only guiding light that we can rely on.

One of the biggest costs is reaching agreement about how the world works and what a solution might be. Reaching that agreement means to make various translation steps between skills, backgrounds and tools.

Talent is rare and time is finite, leading to high salaries and a short supply of people. Sub-par problem solving means the process suffers and the costs are often higher than expected.

I haven’t fully explored the problem with this article, and plan to gather inputs from my readers to continue on my next investigation.

So you want to solve problems?

My best advice so far:

1. Divide and conquer
2. Lower complexity at each step
3. Choose to solve less problems
4. Find out where the level of requirements can be lowered
5. Keep your team small
6. Invest more cash into iterations, less in big one-off projects
7. Focus on problems, not solutions
8. Keep reviewing requirements to find which to reduce or cut out
9. Double time spent with customers and testers