“Oh no, why is my software broken again?” A scary IT story

Today I am going to leave the technical stuff on the side and take a wider look at the Software industry. By reading the news you might get the impression that everything is nice and well in IT Land. Between success stories of startups worth millions or even billions of dollars and computer guys making a six-figures just by typing on a keyboard, it might seem like Software is the New Eldorado. Coding bootcamps and other programming schools are on the rise, promising that you will become a Web Developer in under 3 months! and live the millionaire life in just double that time.

Well, today I want to focus on the other side of the coin, and talk about a scary, yet very real aspect of our industry: project failures. If you type ‘software project failure rate‘ in your favourite search engine, you’ll find a lot of interesting statistics that seem to repeat over the last 10 to 15 years. Here are some of the crispy bits that I found:

  • only 39% of all projects succeed (delivered on time, on budget, and with required features and functions).
  • One in six IT projects have an average cost overrun of 200%.
  • 75% of IT executives believe their projects are ‘doomed from the start‘.
  • Only 2.5% of companies successfully complete 100% of their projects.
  • The United States economy loses $50-$150 billion per year due to failed IT projects.

That doesn’t look good at all. if the sight of those statistics makes your legs weak and palms sweaty, perfect. That was the desired effect. If not, you might want to read those once again. The message is pretty clear: Software development is hard. Anyone telling you otherwise is lying. It requires a lot to get it right. It takes very little to get it wrong.

Tell us why, Johnny! Why is this so hard to make software?!. Rest assured, for the past decades Mankind has spent a tremendous amount of time, energy, and money trying to answer this question. So what are the usual suspects? First, Let me give you my personal take on the matter (caution, touches of satire ahead).

Business vs IT

Those two main actors of almost any significant Software project seem to find an endless source of joy in making each other’s life miserable. The business will often report production issues at 5pm on Fridays and make constant and meaningless requests for things they won’t be using after all. Meanwhile, IT will plan a full upgrade of all servers right in the middle of the month-end closing and devise vague error messages because… Well because they can.

How IT views Business…
and how Business views IT!

Methodologies

– My Scrum is better than your Kanban!
Nah, my Waterfall is stronger and it will kick your Agile right in the n…!

Software development methodologies have been seen by many as Silver Bullets, the Holy Grail of IT that would solve all our problems. In practice though, they do not always prevent projects from failing.

Management

This is a tough one. Where to start?
– Office politics and people undermining the project to serve their own interests? Check
– Favoring reporting work over doing work (time-sheet hell)? Check
– Decision makers completely out of touch with the realities of Software development? Check
– Refusal to invest in tools and better processes? Check
– Fear of change? Check
And the list could go on…

But more seriously though, let’s have a look at some other figures I extracted from this source:

Most Common Causes of Project Failure:
Changing priorities within organization – 40%
Inaccurate requirements – 38%
Change in project objectives – 35%
Undefined risks/opportunities – 30%
Poor communication – 30%
Undefined project goals – 30%
Inadequate sponsor support – 29%
Inadequate cost estimates – 29%
Inaccurate task time estimate – 27%
Resource dependency – 25%
Poor change management – 25%
Inadequate resource forecasting – 23%
Inexperienced project manager – 20%
Limited resources – 20%
Procrastination within team – 13%
Task dependency – 11%
Other – 9%

There are a few interesting observations we can make based on that list.

  • Project failures seem very rarely caused by technical problems! Isn’t it surprising, considering the very nature of Software development? Those damn developers watching cat videos during work hours might not be totally responsible for that disaster, after all.
  • Project management issues are omnipresent. Failure to plan, adapt, or communicate is all over the place. According to this article from Joseph Gulla, a former IT specialist at IBM, ‘54 percent of IT project failures can be attributed to project management, whereas only 3 percent are attributed to technical challenges’.
  • The third point is curiously something that people very often tend to forget about: the lack of discipline in capturing requirements. Look again at those items:
    • Inaccurate requirements – 38%
    • Change in project objectives – 35%
    • Undefined project goals – 30%
    • Poor change management – 25%

This last point is the most striking to me for two reasons. First, requirements are the foundations upon which a project is built up. You probably know what would be the expected consequences of building a house foundation in a swamp. The results are not much different in the Software world.

Secondly, it is astonishing that such a fundamental part of the project is so frequently overlooked, and sometimes even simply ignored. People keep repeating the same mistakes again and again.

Now, it would be very presumptuous of me to criticize without at least suggesting a few ways to improve the current state of things. This is why I will be following up shortly with another post dedicated to capturing requirements and helping your programmers get their Software right.

Did you also encounter IT project failures during your career? Don’t hesitate to share them with us in the comments below!

Till next time!

3 thoughts on ““Oh no, why is my software broken again?” A scary IT story

  1. Pingback: Help yourself and your programmers by writing better specs! – Youenn Bouglouan

  2. Pingback: Programmers want to have fun, too! – Youenn Bouglouan

  3. Pingback: Putting together the Software Dream-Team – Youenn Bouglouan

Leave a Reply

Your email address will not be published. Required fields are marked *