RTFM only works if documentation exists

Mark Cutting Analysis, Devops, Documentation, Management, Planning, Risk, Staff, Strategy, Technology 3 Comments

If there’s one thing that really frustrates me about my job, it’s documentation. It’s certainly not the case that I won’t commit any time to it, but after all my years in this industry, the process hasn’t become any easier – I’ve written thousands of pages of carefully constructed documentation over my time – in fact, I’d go as far as to say that figure is probably closer to a million by now without any exaggeration. My point here is that it’s a begrudging task. One with little reward for the author, but a potential goldmine to the audience that consumes it.

Compared to a simple paradigm, how important is this seemingly endless list of instructions ? Here’s an example. You go to a local furniture store, and choose a bookshelf, or perhaps a cabinet of some sort. These days, furniture tends to be flat packed for storage and logistical reasons, and is supposed to be simple to assemble. Depending on how competent a person you are in terms of traditional DIY really determines if you bother to even read (let alone follow) the assembly instructions provided. For all males reading this, we don’t need instructions, right ? Of course not. We’ll simply toss these aside, and get down to business. Quite what we’ll end up with afterwards can leave a lot to be desired – particularly when you have leftover parts and a pile of fixtures that don’t seem to fit anywhere.

It’s a male thing 🙂

We don’t do instructions – plain and simple.

Despite our assembly looking like something Frankenstein wouldn’t even confess to creating, the job is done, right ?

No. It isn’t. Can we apply the same ethic when it comes to rebuilding a complex system from scratch ? Definitely not – especially when it is usually someone else that has to rebuild that system whilst you’re sunning yourself on vacation, and what seems like Armageddon has now ensued with an alarming pace at work. When something stops working, you’d usually find yourself trawling through debug logs trying to gain an understanding of why – then simply head over to Google and look for the answer. Whilst this undoubtedly works for vendor supplied applications, it doesn’t work very well at all when you develop your own systems – and worse still, simply neglect the documentation altogether.

The importance of documentation

Arguably the worst offenders in the industry are developers (sorry guys, but it’s true). To those truly gifted individuals who create stunning open source applications for the world to make use of, my hat goes off to you. It’s a well known fact I’m an Open Source evangelist, and having written a dozen or so applications in my time, it’s a buzz to know that your code helped someone else. Did I document my creations ? What do you think ? Of course I didn’t. I knew enough to rectify issues as they became evident, and could respond with a bug fix when needed. However, what use was that to my counterparts and colleagues ? Effectively, I was the only one who knew the system, and if it failed or there was an issue, I’d fix it, right ? Wrong. Entirely wrong.

Assumption is a powerful beast, and one that cannot easily be tamed once it’s used to feeding on prime cuts of meat. By this, I mean that putting yourself in a position of being the only one who can recover a system when it fails is great from the job preservation perspective, but you are digging your own hole – and you will fall into it at some point. The problem here is that when the system breaks, people assume you will fix it – even if that means logging in from a Caribbean island somewhere, or being called at 2am by frustrated users in another country who now can’t perform their duties. The stress levels under such circumstances often reach boiling point unnecessarily. It’s nice to carry the torch of importance when it comes to a key system, but the real problem can be aligned to Atlas in Greek mythology – with the added twist that you are effectively punishing yourself in this instance – all because your system lacks the documentation to enable someone else to carry some of the weight. And more importantly, resolve an issue if you are unavailable – or in most cases, oversubscribed.

Why doesn’t documentation exist ?

One common argument for a lack of documentation is either several capable developers that could easily assume responsibility for somebody else’s code and remediate the issue, while another common trend is the so called “agile development” approach. The truth here is that development without documentation is anything but agile. It’s a common issue, and very frustrating when you ask for documentation, and find there isn’t anything. One major factor in the SDLC is that hiring requirements are often flexed in times of need. In this case, contractors are used to write major sections of code and then subsequently leave – no documentation in this case creates a potential minefield of issues that can easily become insurmountable, forcing organisations to either refactor existing code provided sufficient insight can be gained, or rewriting it altogether. Adhering to industry standards when writing code is one thing, but how many developers actually document each function or class they write to benefit someone else reviewing the code a few months down the line ?

Below is an info graphic supplied by Agile Modelling which shows how documentation should scale during the SDLC


There’s an interesting spin here too. Every organisation I’ve worked for over the years ranged from having comprehensive documentation around systems and procedures to having absolutely nothing. In the case of having nothing, you can align the experience to being blindfolded in a building you’ve never been in before, then told to navigate your way out. The same applies in most cases with developers. They typically request documentation around systems when they join a firm to understand how code intersects. Without that documentation, are they supposed to guess ? I’ve often heard the adage of driving a car in this case. Of course, nobody reads the vehicle’s manual when they purchase a new car – the fundamental principles of driving haven’t changed that much over the years that you’d need to read a manual before you can drive that vehicle. Sadly, internally developed software doesn’t work this way. It’s essentially built to fulfil a business need, or serve a particular function – one that is unique in most cases, and very bespoke.

Those organisations that choose to adopt software packages from established vendors are suitably covered in this area as documentation and knowledge bases tend to exist in abundance – this would also be part of the due diligence typically completed to ensure that the system you are purchasing is at least covered by an escrow agreement.

If there was no such agreement within any contract, my advice would be to steer clear. The software runs the potentially huge risk that it will be completely unsupported in the event that the vendor responsible for it goes into receivership. Whilst the system works, then this obviously isn’t an issue. However, with the never ending barrage of updates needing to be performed on systems to ensure security compliance on what seems to be a daily basis, how can you be sure that one of these patches does not have a negative impact on the application you are using ?  The obvious answer here is to test the newly patched system using established regression techniques, meaning you can be sure the application functions as desired post patching before you deploy the same patches in a production environment.

Undocumented system, unprecedented risk

Whilst this is an effective guard against rendering an application inoperable, it also exposes an element of risk in the sense that in house developed systems can become easily impacted by a change to .net runtime (for example) that may cause undesirable behaviour. Several developers I know use logging in their applications, although that same logging is typically generated by the stack itself rather than the application – hence the logs are of no real use to anyone outside of the development team as they aren’t exactly plain English. This means that anyone outside of the development loop is in a no win situation, where the only absolute certainty is they now cannot perform the functions needed to meet deadlines. Developer knowledge is also key in this particular issue. You may have one developer who’s knowledge of a programming language or a system they designed by far exceeds the capabilities of another despite them being in the same team. Should the usual developer be unable for an extended period of time, the recovery time in the event of failure is unnecessarily extended whilst additional resources inevitably need to be pooled in order to remediate any issues as quickly as possible.

Here’s some parallels that are similar in relevance

  • Historical explorers didn’t discover new continents and countries without documenting their experiences
  • Genealogists do not spend years on research yet record none of the events
  • Scientists do not make new discoveries and not document the necessary steps or formulae when finding a cure for a disease
  • Samuel Pepys used a diary as a record of his life between 1660 – 1669 (although not published until the 19th century), and to this day is still considered a major information source for the English Restoration period.

And yet, despite these major milestones, in the 21st century, we still have critical applications running in organisations with no documentation whatsoever 🙁

About the Author
Mark Cutting

Mark Cutting

Facebook Twitter Google+

Mark Cutting is the founder of Phenomlab.com and Inocul8r.net. He is a network, security and infrastructure expert with more than 27 years service in the Information Technology sector. Mark has a significant eye for detail, coupled with an extensive skill set. Having worked in numerous industries including trading, finance, hedge funds, marketing, manufacturing and distribution, he has been exposed to a wide variety of environments and technologies alike.

Leave a Reply

2 Comment threads
1 Thread replies
Most reacted comment
Hottest comment thread
2 Comment authors
Mark CuttingCody Weicker Recent comment authors
newest oldest most voted
Notify of
Cody Weicker
Cody Weicker

Awesome article. You make some great points here about documentation – something we all seem to either be lacking in completely, or it’s in short supply.