Misused Figures of DevOps

A major challenge of software development is that our work is by and large invisible. This makes our folklore essential in business matters. Some of our commonly used arguments and visualizations are digital urban legends rather than solid foundations for informed decisions. Here, we’ll go through a few examples and some measures to address our misconceptions.

As software engineers, much of our concrete work is done in front of computers. But many decisions are made based on the stories we have accepted or figures we draw on our whiteboards. I will show you three graphs that are commonly misused.

The shortcuts represented in the graphs are valuable as they stop us from reinventing the wheel over and over. These figures can either live in a single engineers head, in a team, company or the industry as a whole. They are powerful, but can be wielded for both good and bad outcomes.

In particular we end up in a bad place when our shortcuts are hiding information or are simply wrong.

Hide your burndown charts!

One chart I see all self-proclaimed Agile teams using is the burndown chart.

The purpose of the chart is to show whether the team is on track to delivering what it committed to. This is supposed to give the team a heads up, allowing them to inform stakeholders or reprioritize if need be.

However, when we put a target on the chart, it implies some loaded assumptions about how software development behaves. It assumes that work can be performed in a predictable and smooth fashion, which is wrong. But that’s not necessarily a problem. It could serve as an early warning to the team that something is not as expected, and allow the team to take appropriate actions.

More commonly, the burndown chart is wielded as a management whip. A project manager or malign scrum master might swoop in to give the team a shakedown mid-sprint because the burndown looks bad to them. But it doesn’t make sense to discuss whether a burndown chart is good or bad. The chart simply reflects the current reality.

We must remember that the burndown chart exists for the team. It doesn’t tell us anything valuable or concrete in itself. If our burndown surprises or scares us we should figure out what exactly is surprising us and how we can address this as a team.

When the burndown chart is misused, it not only fails to bring value but it also actively frustrates engineers trying to solve complex problems they have not seen before.

Handling late projects

Projects running late from time to time is completely unavoidable. Every team over commits at some point. They may be pressured to do so, or they simply miss their target.

Hopefully we notice that our ambition is unlikely to be fulfilled early in the process, so we have time to act. But the default action in the case of a probable missed deadline is broken. Unfortunately, when we become aware of a deadline sliding, our intuitive reaction only amplifies the problem.

The default action is this. Typically, companies throw in more resources and more engineers to sort the problem out. The expectation or hope here is that when we have more hands and heads working on our problem we will get there faster.

The reality, however, means that on-boarding people to a project will cause an immediate dip in productivity. The time needed for spreading domain knowledge, reviewing work, and the challenges of scaling a team will start slowing the already-late team down.

Until enough time passes and the new team members are up to speed, the additional resources will only have made it less likely that we hit our deadline. As The mythical Man -Month recommends, the only way to handle a deadline that seems to be missed is to descope or postpone the deadline.

There is also a hidden cost to adding more resources. In all likelihood, when the resources are needed to save a project that is running behind schedule, the saviours will be found internally. This impacts not only the projects these people were originally assigned to, but it also creates an unstable team structure, making continuous improvement difficult if not impossible.

This would not be a problem if it happened rarely, but unfortunately for many enterprises, it’s the norm: moving employees around from project to project in a disorienting dance of shifting priorities.

But everything is not hopeless, even though I ended up painting a bleak view of missed deadlines. There are things we can do.

Rip off the bandaid and descope

The most important step is to consider whether a project will run late upfront. Often the best solution is to descope, and be aware of how you can descope to a meaningful increment.

Descoping can be demotivating, or even unacceptable in some cases, but then we need to invest in making it easier to add resources to our projects, should the need arise.

When the deadline is looming and you notice you will miss your target, it’s difficult to amend the situation. There are however some steps that you can take to make it easier to add resources to a project as needed.

Top tips for easing on-boarding

I recommend you treat all software projects as open source. What this means is that there must be a focus on making it easy to contribute.

There must be documentation both for use, testing and developing the project. Testing must be automated and the quality gate can’t be a senior engineer, rather it should be a pedantic robot. The tools that store our source code have been developed either to support open source workflows or are heavily inspired by them, so take advantage of the native features of these platforms.

At the very least make sure that you have the following in place:

• A README, with good accessible documentation on how to build, use and test the product
• Links to pipelines, coding styles, external documentation are present
• A workflow which is explicitly described
• Any key personnel is mentioned, included how to reach them

The hard thing about these measures is that they function as insurance. It can be hard to remember how important they are when you don’t feel you need them. But the day you need it, you can’t afford not to have taken those steps.

The surprising truth about automation

As a DevOps person, I have a not entirely healthy bias towards automation. We know that automation is a prerequisite for scaling superlinearly with team size. It also has all sorts of benefits in terms of quality, auditing, delivery speed and confidence.

Our intuition is that as we invest in automation, the amount of manual work will go down. But this is actually contradicted by Accelerate DevOps Research. They have found that deployment automation actually creates more manual work. This is due to the fact that there is now a ton of changes to test and review.

From the 2018 State of DevOps report by DORA

We should be aware, though, that we have removed only the one type of manual work. The trick here is to make sure you measure the right thing, or your automation effort will be deemed a failure, even though you have successfully automated a part of your workflow.

This increase in manual work is why many automation efforts are put on hold or deemed unsuccessful. Not only do we now have to do more manual work, we also have to maintain our automation.

In organizations that are in the early stages of automating their processes, or have limited experience with automation, there is a tendency to mistrust automation. This leads to redundant work done by both robots and humans.

To reap the benefits of automation we also need a shift in mindset. Being aware of this bump in manual work before starting an automation process can hopefully avoid a nasty surprise. In this blogpost I’ve shown some commonly accepted figures. You can read about an enterprise level misconception in the agile alignment trap.

I had the pleasure of attending DevOpsDays Zurich 2019 and sharing my thoughts from the stage with the talk: Misused Figures of DevOps.