Signs you might not be doing a DevOps (even if you think you do)
2022-09-12
DevOps was conceived to solve fundamental challenges with software delivery, and like many of the practices that have spurred forth over the last few decades, it's one of the many that aim to bring the team closer to the customer and embrace autonomy. It's also starting to show its age. Middle management is discovering that the way they applied DevOps at their previous large enterprise job doesn't work for the 200-strong, budding tech company.
Historically, many didn't understand what DevOps was. It has constantly faced an uphill battle of misunderstandings (as do a lot of new ideas in tech). It's fallen victim to a sort of "telephone game" where middle and upper management barely understood it and didn't want to admit that. "Oh, yEa. I kNoW aBoUt tHe DeVoPs!" you could hear them exclaim from the uncomfortable, swivelling chair of their open-plan office. However, the clearest of descriptions isn't going to cut it if the leap in understanding is simply too great. I think we're in a better place now, but there are still ways to get this wrong (I've done it—got it wrong, that is. I mean, I might look perfect...).
What is DevOps?
There must be plenty of great descriptions out there, so instead of reinventing the wheel, let's take a brief look at some of those provided by our self-proclaimed industry technology leaders.
Let's take Amazon:
DevOps is the combination of cultural philosophies, practices, and tools that increases an organization's ability to deliver applications and services at high velocity, evolving and improving products at a faster pace than organisations using traditional software development and infrastructure management processes.
Dear God... what a mouthful. They mentioned tools—a good start. I hate the word "processes", though. Don't get me wrong, I think its meaning is fine in this context, I just hate the way it looks. How about Atlassian:
DevOps is a set of practices, tools, and a cultural philosophy that automate and integrate the processes between software development and IT teams. It emphasises team empowerment, cross-team communication and collaboration, and technology automation.
A little bit better, I think. At least I don't feel like it's trying to sell me something. Still, I can't help but feel that it kind of doesn't quite get to the heart of it. How do you apply that?
Save us, Wikipedia!
DevOps is a set of practices that combines software development (Dev) and IT operations (Ops). It aims to shorten the systems development life cycle and provide continuous delivery with high software quality. DevOps is complementary with Agile software development; several DevOps aspects came from the Agile methodology.
It's good, but I wish it would also mention the reduction in cognitive burden to release software, the opposite of which is a common anti-pattern. Relating it to Agile probably makes the most sense, but you need to intuitively understand Agile—which is yet another commonly bastardised term in the industry.
Why would you want "DevOps"?
Let's take a step back and appreciate DevOps for what it was originally intended to be — a symbiotic relationship between DEVelopment and IT OPerations. Incepted to solve the largely dysfunctional relationship between those that build and those that run applications in enterprise organisations. Over time it's basically lost all meaning, but the traditional intention was to have developers care about some of the things that IT operations care about, and for IT operations to make available to developers ways for them to care about it. It is a way of working—a culture. (Another perspective is, traditionally you'd get IT operations folk to write self-service APIs for their infrastructure, devs then use that to run their services but it's not just about the tech capability, it's about ownership).
I'm not going to try and write the ultimate description for DevOps, that's been
done attempted before. Instead I'm going to look at some of the common
anti-patterns that companies and teams deploy that aren't actually in the spirit
of DevOps but are commonly thought to be. Red flags if you will.
Your position description for "DevOps engineer" could be the same for a "Cloud Engineer"
"DevOps Engineer" is commonly (and incorrectly) used as a synonym for "Cloud Engineer". Someone who knows your IAC tool of choice, has working knowledge of your chosen cloud vendor, and can build out infrastructure for deployments of specific software. Unfortunately, being in the cloud doesn't imply that you are doing DevOps and neither does having engineers with the title DevOps Engineer.
DevOps engineers (Or as I like to refer to them, engineers), depending on the scale of your organisation, would more appropriately be engineers that are focused on building engineering operational capability. Some engineers will be better suited to this type of work, but for the most part, operational capability is just another product that a team is delivering. The only difference being that they are building it for other teams within the company as their customer, not typically for your external customers. They are force-multipliers.
Your "DevOps Engineers" can't write code
It's common for System Administrators to jump into DevOps roles—I don't blame them, and I certainly don't want to say that that path shouldn't be made available to them. However, a DevOps Engineer that can't write code is generally antithetical to the point of any engineering role, DevOps included, and more often than not SysAdmins come with little to no ability to write code let alone follow good engineering practices.
Your stream-aligned teams struggle to deliver work without involving someone from the "DevOps team"
If your stream-aligned teams are unable to deliver work without your DevOps engineers, then it's likely that you don't have autonomous, cross-functional teams. In this struggling model DevOps becomes a team of specialists that aren't accountable for estimating work within the context of a team's sprint and thus aren't accountable for the goals of that team and often become a blocking dependency.
In an ideal world, teams are autonomous and have all of the capability they require to deliver change into production and subsequently be able to monitor that change. If you have engineers in another team that are constantly required to get the other team's work into production, then they should either simply be a part of the cross-functional team (so that they have the capability that they need within the unit), or the team that they depend on should be instead focusing on building self-service capability.
Your teams get blocked frequently because "DevOps hasn't fixed the CI/CD pipeline" that is broken
This is especially egregious if the team can't tell you if it's a problem with the application or if it's a problem with the automation platform you might be using. A problem with the application would imply that there is something that the team has done that has broken the pipeline, and the pipeline infrastructure is probably fine.
"DevOps engineers" are often tasked with building and fixing pipelines. The underlying infrastructure? Possibly. The red/green status of a particular team's CI/CD pipeline? No. If a team is unable to build and maintain the status of their application pipeline they need training or people who do know added to their team.
You think that you are doing DevOps, but your teams have this other problem where they neither share nor have reusable patterns.
Shared capability doesn't often come from thin air, and you're unbelievably lucky if it does. If you have teams rebuilding the same thing but in different ways then it's possible that your team responsible for building or guiding this shared capability are not enabled to do so.
Coordinating shared capabilities is a big effort. It's as much effort as building out a product for an external customer when done well. User research for internal capability should be as rigorous as that taken out on your external customers. It takes a strong tech-product focus to be able to understand where and what needs to be built, as well as coordination with very senior technical people (like architects, managers, and/or principals) to understand where shared capability can have the most positive impact.
Your DevOps engineers are always saying that they are too busy or that they are constantly context switching
If your DevOps team is saying they are way too busy, this is probably because they are being pulled left-and-right by the teams each demanding their attention.
This is usually a strong sign that your DevOps team is simply being used as a buffer for capacity constraints and failures in planning. Again, are your teams autonomous? Are you actively managing the WIP Load?
You can't easily tell if you're meeting your SLAs
If you can't, at a glance, see the current state of the business and understand how the technology is performing, then your DevOps team is not being enabled to provide this crucial capability.
On top of that, if each team is rebuilding this capability, or one team has this but another doesn't (and it's been a minute), something has gone wrong.
Teams often need someone with deep knowledge of your cloud vendor to assist them
Again, this is a problem wherein the teams might not be as autonomous or cross-functional as you think they are. You're probably bottle-necked by the capacity of your Cloud Engineers which, in a dysfunctional model, is a skill monopolised by your DevOps team.
It's not like the late noughties where virtual machines had to be provisioned manually in VmWare on a specific server with enough reserved capacity for the Dev team to then install their software. We actually have a lot of the abstractions around infrastructure for self-service now in the shape of Cloud Vendors like AWS, GCP, and Azure. It's got a lot better, but it's still complex as hell.
All of your software engineers must be experts in cloud infrastructure.
Maybe your front-end engineers need to know how to deploy a VPC and correctly configure the CIDR ranges to peer into other networks in your Cloud Infrastructure. Or maybe you have mobile engineers SSH-ing into EC2 instances to connect a debugger to a running process.
Not every developer should need to know how to do everything and we have more surface area and more types of engineering niches today than we have ever had. A DevOps team should be abstracting away a significant amount of cognitive burden especially as it relates to your cloud vendor. That's why we have APIs, so that we can hide the logic of something behind a pretty little interface that allows callers to ignore the underlying implementation.
DevOps has the keys to the kingdom (admin everywhere!)
Commonly, DevOps have administrative access and ownership of all the vendor products, systems, and production infrastructure. This is a lot of pressure, and I'm not sure about you but I'd be losing my sleep (and hair) worrying if my 32 character long master password is long enough.