How DevOps Failed
…. and can rise again
lessons from the OODA Journey
“The side who can gather information and make better decisions faster will generate a rapid tempo of operations and gain a decided advantage”
–Joint Publication 1: Doctrine for the Armed Forces of the United States
I was thrilled when the concept of DevOps first took off. At the time I had just successfully built a whole new way of delivering services at Yahoo! that allowed new products and changes to be built, deployed and managed at scale that mirrored much of the early intent of the movement. It had been a hard slog, building on many of the hard lessons about the importance of situational awareness, responsiveness, delivery safety, and collaboration that many of us in the Valley had learned during the ’90s dotcom boom. When the book Continuous Delivery came out a little while later, I was hopeful that the fights of old would finally be put to rest and we could now ensure that we as an industry safely and effectively delivered services that enabled users to measurably achieve their desired outcomes.
Boy, was I wrong.
While people still often talk about many of the same ideas, the term “DevOps” has been hijacked by various groups to sell all sorts of tools, technologies, processes, and even job roles. Not only do these seem to do little to further what I had envisioned, they have watered down the term so much that it risks becoming a meaningless buzzword like “Agile”.
Seeing the industry fall back into the siloed output-centered ways of old has been incredibly discouraging. Even now, I see the pattern beginning anew with the “platform engineering” narrative.
Does this mean that DevOps has failed?
We’re far from the only industry that has managed to screw up a good thing this way. It reminds me of a similar journey made by a fighter pilot named John Boyd. He sought to build the ultimate model for ensuring military air superiority. He followed a similar path before discovering the OODA loop.
Boyd’s Journey
Boyd entered the US Air Force in time to participate in the Korean War. The US dominated the airspace over Korea, impressively with the US F-86 shooting down eight enemy aircraft for every US loss. Boyd was not only interested in understanding why but wanted to know if this “secret sauce” could be expanded and built upon.
The Cold War was just beginning, and interest in maintaining a lead over the Soviets was high. So, Boyd got to work.
“It’s the Process!”
Boyd began his journey with the idea that maybe it was the superior processes and fighting tactics of the US pilots that made the difference in combat.
To test his theory, Boyd became an instructor at the USAF Fighter Weapons School (FWS). This gave him the ideal place to map out and document all dogfighting tactics and countermeasures. His constant practice and experimentation made him so adept at dogfighting that he was dubbed “Forty Second Boyd” for his ability to begin from a position of disadvantage and then defeat any opposing pilot in air combat maneuvering in less than 40 seconds.
The end product of Boyd’s efforts came in the form of the “Aerial Attack Study”, which has become the seminal manual for dogfighting. This manual was a big hit. It is still widely used by air forces from around the world as a tool to train pilots.
However, Boyd was not satisfied. While his work was indeed useful, he knew from his own dogfighting success at FWS that a pilot’s success was dependent on far more than just blindly applying some pre-determined series of maneuvers and countermeasures.
If you have been through an Agile, DevOps, SRE, or ITIL transformation that was entirely focused on following a process, you can probably relate. Like Boyd, it isn’t that the processes themselves are wrong or bad. The problem is that success doesn’t come from performing the process, but from the process directly contributing to helping those use it to achieve the outcome being pursued.
“It’s the tools!”
After realizing that success could not be secured by just having superior processes, Boyd started to look to see if the real differentiator was with the aircraft itself. It was true that the US (F-86) flew different aircraft from the North (MiG-15). He figured that the reason must come down to superior equipment on the US side.
Does this sound familiar?
Boyd went to work trying to come up with a formula that could quantitatively model and compare aircraft. This would allow the US to maintain its edge. He went back to school to study engineering to help him with his work. With long hours and lots of mainframe time, he came up with the Energy-Maneuverability (EM) theory.
The formula was revolutionary, and was quickly adopted for use to guide design decisions for the F-15, F-16, F/A-18 and A-10. Even now it is still used by engineers to compare trade-offs between different capabilities and designs.
Similar useful measures have been created in DevOps. DORA metrics is one that comes to mind for giving you an idea of organizational reliable service delivery agility.
Unfortunately, Boyd quickly discovered that his theory that winning comes from having the best tools and technology was hopelessly flawed. When he compared the EM values for the US F-86 to the Soviet MiG-15, the MiG-15 always came on top. These calculations were verified when the US managed to acquire and test several of the Soviet airframes.
Boyd found that such results were hardly unusual. Armies as far back as Alexander the Great handily defeated far better equipped enemies.
This also holds true in IT. I regularly see lots of companies that acquire a vast array of hyped DevOps technologies believing that having them will somehow give them a leg up on the competition, regardless of whether they are being used effectively to meet their customers’ needs.
Once Again to the Drawing Board
After seeing that success could not be guaranteed solely by process or tool, Boyd looked back on his own career and his time at the FWS for answers. He knew that much of his dogfighting success came from out-thinking his opponent. He wondered whether military success came from the decision-making process itself.
He went back to look at military history for clues to see what made one side’s decision-making better than another. Alexander the Great and Genghis Khan relied heavily on maintaining knowledge mismatches between themselves (who often had loyal troops with local knowledge) and the enemy (who were often flooded with misinformation about the size and ruthlessness of the invading forces). Napoleon’s success came from the flexibility he gave to the troops on the field to adjust to changing conditions.
He reached out to interview legendary commanders of the Second World War, along with their subordinates, to observe how they worked. He noticed a pattern. Those who were the most successful were the ones who had clear outcome objectives but ensured that there was sufficient trust and flexibility for those closest to the action to freely decide and act. This balance of trust, situational awareness, and orientation to a shared outcome is what became the OODA Loop that most people know today.
We too can succeed if we can take a step back from concentrating so much on the tech and the processes around it and instead look at what drives our own decisions. Ask yourself these questions:
- Is it clear as to what the target outcomes are from the work we are doing? Can our progress toward those outcomes be measured in ways meaningful to the customer, and not in outputs like feature counts, lines of code, uptime, or our ability to release things quickly that do not provide customer value?
- How well do we understand how our work will interact with the ecosystem around it? Do we value that understanding, or do we trust that the work that comes our way and its acceptance criteria are the only things we need to be concerned about?
- How easily can we stay aligned with our teammates, other teams, and the customers themselves? Do we maintain an easy alignment ourselves, or do we count entirely on managers to sort that out for us?
- How much safety do we have to make mistakes and learn? Are mistakes things to fear and be punished for?
- How much do we rely upon technology and processes to “do the right thing”? Do you know how well they help deliver the outcomes desired?
Conclusion
To me, DevOps and service delivery are on this same journey. I have seen some promising signs as teams start to move beyond the hype to better understand their service delivery ecosystem and the flow of information within them. I am hoping that we can move away from disconnected lists of requirements and acceptance criteria and instead allow delivery teams to apply their skills toward achieving the measurable outcomes that are desired.