Get the highlights in your inbox every week.
8 principles to achieve DevOps at scale
8 principles to achieve DevOps at scale
If your DevOps processes aren't meeting your expectations, try these principles to build an initiative that fits your organization.
Since you clicked on this article, you may be wondering why you aren't achieving the level of quality, efficiency, and satisfaction you expect from your DevOps processes. Maybe you think other organizations are achieving more than you are. If so, you might be trying to do what everyone else is doing, rather than thinking independently and building a DevOps initiative that fits your organization.
Back to basics
Take a moment to think about what you want to achieve with DevOps and why. Ultimately, we all want to build and develop software faster. If we practice any agile method (scrum, kanban, SAFe, etc.), it's because we believe it will help us achieve this goal.
How? By constantly inspecting and adapting.
Everything is about moving faster to achieve your goals. The goals you and your team or company are made for. These may not be the goals you planned six, 12, or 18 months ago.
A strong DevOps organization has the ability to adapt and self-heal. It has a few simple rules that allows a self-organized system to emerge where decisions are made for the good of the whole organization, without requiring a central authority
To achieve DevOps at scale, you need a management strategy; you can't just focus on a technical implementation like continuous integration/continuous delivery (CI/CD) or microservices. The following eight principles guide how we at La Poste achieve DevOps at scale.
1. Build a task force, not a team
In its simplest form, DevOps can be seen as an extension of agile for the delivery process. It's a way of working to break silos between your dev and ops teams. A DevOps team finds its place between the other teams, adding a "DevOps silo" between the dev and ops silos. In contrast, a DevOps task force blurs the lines between dev and ops to bring them together in one silo. This is not just about renaming "team" to "task force"; it's a totally different strategic approach that scales your DevOps practices.
Consider your task force as a multi-competence center. Its purpose is to join a project and help the dev and ops teams with everything that can speed up their delivery process—not only CI/CD implementation.
For example, technical debt can drain your velocity in the following ways:
- Not adopting the latest performance updates can slow down your product and may lead to custom code workarounds.
- Not implementing the latest security updates may require extra support from vendors (in other words, money), but many organizations won't deal with it until there is a security breach. This can cause a lot of damage to a company—plus taking care of the damage means using resources that could have been invested in expanding the business.
- Migration planning for software and tools can take six to 18 months. Waiting for them to reach their end of life (EOL) before beginning migration planning creates significant impacts and costs to the business and slows the product teams.
A task force is known for its highly skilled and diverse members that can help other projects and departments in ways including:
- Identifying slow processes (e.g., onboarding interviews, live teams working during multiple sprints)
- Automating slow processes (e.g., on-demand staging environment, app testing, platform testing, infrastructure testing, virtual machine templating, container build process, ChatOps)
- Helping people improve their performance (e.g., training labs, new-hire helpers, brown bag lunches, meetups)
- Ensuring teams can continue and maintain their work
2. Create a self-organized system
When you watch a flock of birds flying, you may notice no one bird is orchestrating their movement. Simple rules allow a self-organized system to emerge that benefits the group as a whole.
In a DevOps-compliant company, teams need to be able to sync with each other with little to no overhead. For example, a development team can interact with a service provider without having to sync with internal teams. They must achieve this sync-less ability internally.
Below are my three simple rules (find ones that fit your organization):
- Give full insight on teams' availability (e.g., dev–build–services statuses, people planning)
- Give full access to your ongoing work (e.g., kill emails and politics and instead use online boards for everything: projects, tasks, meeting notes, links, discussions)
- Work on one task at a time to prevent context switching
To comply with my rules, I open my email app twice a day. I disabled Slack notifications and look at it once per hour. All the waste is gone, real stuff appears, and people call or come to my desk if I'm urgently needed. I also use the mind-mapping app FreeMind and a Pomodoro-based timer app on my phone to help me focus.
You'll have to constantly inspect and adapt so teams can find their own way of implementing and dealing with the rules. Three things to note:
- I believe the first iteration will not be perfect because adoption is slow and comes with iterations
- You'll have to accept slowdowns before you gain speed
- Helping teams expose insights on their internals is part of the DevOps task force
3. Work closely with agile coaches
DevOps is an extension of agile, so you need to work with an agile coach. Hire one if you need to, because you'll need an advocate. Your so-called "digital transformation" depends on building your vision collectively with others; you must build consensus because you do not always have the right answer.
In today's organizations, everyone has an idea about how to achieve DevOps. If they don't believe you're doing the right thing, they'll slow the whole system down (on purpose or not). People need to understand what you're doing and why. Once consensus is reached, let the advocate do their job. This is the gentle inception needed to continue to make progress.
4. Train teams on soft skills
I recommend offering short, highly focused, and practical training on soft skills—the tools DevOps teams use on a daily basis without having to become advanced users. In my experience, everybody is satisfied with this approach, and adoption immediately increases.
Some of the soft skills I've trained teams on include:
- GitLab: what are the differences between bronze/starter or gold/ultimate? what feature can we add?
- How and why we sign our Git commits
- Bash advanced usage
5. Empower teams
Raise engagement by giving teams some space. Yes, trust has to be built and taken very seriously. Clarify the limits on their responsibilities and what you want them to achieve.
Know what they can (and will) break within the these limits. Still, let them decide their own go/no go.
The most important thing is that when something breaks, you know and can contain the outage area, and this is more likely to happen when you've empowered the team.
6. Uncertainty is your friend
Here's a simple loop:
- Engage a small group of engineers on real, end-user key performance indicators (KPIs)
- Let them decide how to achieve them
- Manage how to handle any failure
Help the group with KPI reviews. Work like business angels work with startups: bid on many (remember this is about scale), control failure, and expand and collect successes.
Remember that failure remains failure if nothing is learned from it; otherwise, it becomes experience. And experience is what drives quality.
As a manager, don't focus on failure. Instead, focus on blameless post-mortem: How do we prevent it from happening again? Participate in the post-mortem, if you can.
7. This is not (just) about technical implementation
DevOps is about helping to extend agile and lean development principles to production. This is often viewed as automating the delivery process, but it is so much more.
Site Reliability Engineering (SRE), which Google introduced long before DevOps appeared, implements DevOps principles. DevOps (and agile generally) can be seen as an evolutionary mechanism: constantly failing, fixing, and trying to adapt.
Some best practices (such as CI/CD pipelines) and technical architectures (such as container orchestration and microservices) are first-class citizens in our lives because they, by design, tend to address how fast and how stealthy we can recover from failures. This is both agile and lean by definition.
8. Get up to date
Agile and DevOps advocates try to help companies improve their adaptability. A lot of legacy and large companies seem to believe DevOps is just about hype. In my experience, it's about survival.