8 CI/CD best practices to set you up for success | Opensource.com

8 CI/CD best practices to set you up for success

Using best practices in your continuous integration and continuous delivery pipelines will make you more successful.

Brain on a computer screen
Image by : 

opensource.com

x

Subscribe now

Get the highlights in your inbox every week.

Continuous integration (CI) and continuous delivery (CD) are usually associated with DevOps, DevSecOps, artificial intelligence for IT operations (AIOps), GitOps, and more. It's not enough to just say you're doing CI and CD; there are certain best practices that, if used well and consistently, will make your CI/CD pipelines more successful.

The phrase "best practices" suggests the steps, processes, ways, iterations, etc. that should be implemented or executed to get the best results out of something like software or product delivery. In CI/CD, it also includes the way monitoring is configured to support design and deployment.

A deployment pipeline is a good example of this. It includes the steps to build, test, and deploy:

  • Application software
  • Systems, servers, and devices
  • Configuration updates (e.g., firewalls)
  • Anything else

Its purpose is to automate deployment by:

  1. Using a tool to automate individual tasks, resulting in a string of automated tasks (a toolchain)
  2. Using a tool to orchestrate the whole process or toolchain
  3. Continually expanding and improving the toolchain

The following are examples of different deployment pipelines.

Applications deployment pipeline

Server configuration deployment pipeline

Firewall configuration deployment pipeline

Another example of CI enabling pipeline deployment and continuous delivery is database CI; database changes tend to complicate CI, but this can be simplified with database continuous integration (DBCI) using an agile cross-functional team that includes a database administrator, using "database as code," and refactoring databases using object-oriented (OO) principles.

There are a variety of best practices in CI/CD that will improve your workflow and outcomes. In this article, I will explore some of them.

Pipelines and tools

CI/CD best practices involve the combination of an effective, efficient deployment pipeline and orchestration tools, such as Jenkins, GitLab, and Azure Pipelines. The benefits of these include increased velocity, reduced risk of human error, and enabling self-service.

Version control

Using shared version control is a best practice that provides a single source of truth for all teams (e.g., development, quality assurance [QA], information security, operations, etc.) and puts all artifacts in one repository. Version control must also be secure, utilizing standards such as encrypted repositories, signed check-ins, and separation of duties (e.g., two people required to commit). Some things you could expect in version control include:

QA and testing

CI/CD best practices extend to quality management, as well. Deployment pipelines must test, test, test, and test some more to mitigate defects with a sense of urgency and an eye on speed to recovery. Put your focus on quality-related principles, such as:

  1. "Shift left" testing
  2. Small batches, frequent releases, learning from escaped defects
  3. Five layers of security testing
  4. Software vs. infrastructure testing

Best practices demand that testing be automated to detect issues faster—and mitigate them even faster. Some tests include:

testing.png

Testing pyramid

(Taz Brown, CC BY-SA 4.0)

Testing should include code, particularly code analysis and automated testing tools.

Common code-analysis tools include:

  • Language-specific
    – Python: Pylint & Pyflakes
    – Java: PMD, Checkstyle
    – C++: Cppcheck, Clang-Tidy
    – PHP: PHP_CodeSniffer, PHPMD
    – Ruby: RuboCop, Reek
  • Non-language specific
    – SonarQube: 20+ languages
    – Security scanners
    – Complexity analyzers

Common automated testing tools include:

  • Unit testing
    – JUnit, NUnit
  • Functional or performance testing
    – HP ALM
    – Cucumber
    – Selenium
  • Security testing
    – Penetration test tools
    – Active analysis tools

Testing your own processes

Teams should be persistent in doing CI, and you can check how well your team is doing by answering the following questions:

  • Do developers check into the trunk at least once a day?
  • Does every check-in trigger an automated build and testing (both unit and regression)?
  • If the build is broken or tests fail, is the problem solved in a few minutes?

Metrics, monitoring, alerting

All of the following should be automated and continuous in a CI/CD pipeline:

  • Metrics are data that can be captured and are useful for detecting events (changes in the state of something relevant) and understanding the current state and history.
  • Monitoring involves watching metrics to identify events that are of interest.
  • Alerting notifies people when an event requires their action.

Incorporating telemetry, a "highly automated communications process by which measurements are made and other data collected at remote or inaccessible points and transmitted to receiving equipment for monitoring, display, and recording," is one option to build continuous metrics, monitoring, and alerting into your CI/CD pipelines.

Value stream and value-stream mapping

Identifying value streams and then mapping them can help an organization and teams when trying to solve problems. It also helps establish value for customers through activities that take a product or service from its beginning all the way through to the customer. Value stream examples include:

  • Software development
  • User support
  • Incident management

Infrastructure as code

Treating infrastructure as code (IaC) is a best practice that has many benefits for CI and CD, such as:

  1. Providing visibility about application-infrastructure dependencies
  2. Allowing testing and staging in production-like environments earlier in dev projects that enhance agile teams' definition of "done"
  3. Understanding that infrastructure is easier to build than to repair
    – Physical systems: Wipe and reconfigure
    – Virtual and cloud: Destroy and recreate (e.g., Netflix's Doctor Monkey)
  4. Complying with the ideal continuity plan, which according to Adam Jacob, CTO at Chef, "enable[s] the reconstruction of the business from nothing but a source code repository, an application data backup, and bare-metal resources"

Amplify feedback

Building in the ability to amplify feedback is another CI/CD best practice. One system to work safely within complex systems is:

  1. Manage in a way that reveals errors in design and operation
  2. Swarm and resolve problems resulting in quick construction of new knowledge
  3. Make new local knowledge explicit globally throughout the organization
  4. Create leaders who continue these conditions

Response to failure

Best practices around responding to failure should be built into a build, test, deploy, analysis, and design system. This is not only about considering which failure-response systems should be designed, built, and deployed; the ways organizations or teams will respond should also be well thought out and executed.

The ideal culture considers failures to be features and an opportunity to learn, rather than a way to place blame. Blameless retrospectives or postmortems are a best practice that allows teams to focus on facts, consider many perspectives, assign action items, and learn.

Creating a culture of learning and innovation around continuous integration and delivery is another best practice that will result in continuous experimentation and improvement in the short term and sustainability, reliability, and stability in the long run.

Leaders can create the necessary conditions for this cultural shift in a number of ways, including:

  • Make postmortems blameless
  • Value learning and problem-solving
  • Set strategic improvement goals
  • Teach and coach team members in:
    – Problem-solving skills
    – Experimentation techniques
    – Practicing failures

Best practices involve winning and losing, failing fast, removing barriers, fine-tuning systems, breaking complex software by using popular tools, and, more importantly, driving people to respond in ways that solve problems fast, are decisive, and are creative and innovative.

It's about creating a culture where people expect more. It's about shifting your mindset to deliver on what's promised and deliver value in ways that delight.

Summary

Building out a CI/CD pipeline is a test of resiliency, persistence, drive, and fortitude to automate when it's called for and pull back when it's not. It's about being able to make a determination because it's in the best interest of your systems, dependencies, teams, customers, and organizations.

arrows cycle symbol for failing faster

Continuous integration (CI) and continuous delivery (CD) are extremely common terms in software production. But do you know what they really mean?
Women in tech boardroom

Agile and DevOps have many differences, yet they both seek to address complexity, improve quality, and innovate around software design.
CICD with gears

An easy guide to the top open source continuous integration, continuous delivery, and continuous deployment tools.

Topics

About the author

Taz Brown - Taz Brown, was a Red Hat Sr. Ansible Automation Consultant  at Red Hat but has moved on and is currently in the position of Sr. Scrum Master at the largest public utility company in Missouri working with DevSecOps team.  She is an avid writer and speaker with a diverse background in Scrum, Agile, Linux systems engineering, management and deployment of DevOps solutions.  She now focuses on making software development teams high performing through the practice of Scrum and other agile...