Choosing The Right CI/CD Tools

CI/CD, standing for Continuous Integration and Continuous Delivery or Development, is a development paradigm that is becoming extremely popular, especially for SaaS and web applications. Software delivery is a spectrum with monolithic releases falling at one end and true continuous delivery at the other – many vendors now push new code several times a day!

What is CI/CD

With CI/CD, every time you refactor your code or develop a new feature you push it to the production environment via your testing or staging environment. This means that your production codebase is constantly evolving. CI/CD brings several benefits over monolithic delivery:

• Increased speed of delivery. This means your customers benefit from new features instantly and also means that each release is generally only adding a single feature.
• Reduction in bugs. Releasing features incrementally should reduce the number of bugs. It also means the scope of the bugs is much narrower – you know that the most recent code triggered the bug and so diagnosing the cause is that much easier.
• Reduced deployment risk. With monolithic releases, you run the risk of unexpected problems like library version incompatibilities when you come to deploy.
• Streamlined development. Building and releasing constantly means there’s less risk of incompatible code from different developers on the team. Rather than working against an old code branch to develop their new code, developers are always able to work on the latest version.

However, to be able to use CI/CD, you need to create or find a suitable tool to add to your development toolchain. In the remainder of this blog, I will go through the various factors you need to consider when choosing a CI/CD tool.

Open Source, Proprietary or Roll-your-own

The first thing to decide is whether you want to choose an open-source tool, a proprietary off-the-shelf tool or whether you need to develop your own custom solution. As always there’s a trade-off to be made here.

Open-source software is free to use (though you may need to pay for hosting – see point 2 below). However, you always face the risk that the code may stop being actively developed, or that the developers may suddenly completely change everything. Also, unless it’s a popular tool, open source generally means a lack of support (see point 4).  

By contrast, most proprietary software will come with good support, often including hands-on support to deal with issues. Some vendors may even offer the option to customize their software for your particular requirements. OTOH proprietary software will cost you money (see point 3). Also, if the company goes bust you may be left high and dry! 

For most projects, you should be able to find a suitable open source or proprietary tool. Only the most arcane of projects will need you to develop your own, and even then, you should be able to assemble the pieces from existing tools.

Hosting: Cloud, On-Premise or SaaS

The next point to consider is which hosting model you need to use. Do you have a requirement that all your codebase remains on-premises? If so, you will need a solution that you can deploy on your own servers. This, in turn, will mean the solution you choose will need to be able to run on your existing servers (unless you are prepared to invest in new infrastructure). Alternatively, you may already be using virtual infrastructure for your development, in which case you can add the CI/CD tool to that. In both cases, your DevOps Engineer(s) will need to have the skills to maintain the setup (see point 4).

Some CI/CD tools operate on the SaaS model, meaning you are effectively buying them as a service and the actual tool runs on the vendor’s infrastructure in the cloud. In this case, you can offload a lot of the support costs to the vendor, but OTOH you have to be aware of issues such as where they are hosting the service (point 6).Billing

Models and Costs

Adding a CI/CD tool to your development toolchain will always come at a cost. Even if you have chosen an open-source tool and are running it on your own infrastructure, you will be faced with costs which you need to take into account. These include the time needed to setup and support the tool and the loss in productivity as your developers learn the new toolchain. Proprietary and SaaS tools will additionally have costs based on the level of usage. Factors you need to consider before you start looking are:

• How many builds will I be doing? On larger projects, you may end up running builds many times a day. Some vendors may impose limits on this or may charge based on this.
• Do you need to be building concurrently? If you have several development teams they may all want access to the build toolchain at the same time. This means you may need more infrastructure or more resources from a SaaS vendor.
• How many users do you need? Many vendors will charge you more the more users you have.

Often you will find that there’s no clear answer as to which solution offers the best value. Not least because you will be trading off the costs of CI/CD against the savings in streamlining your development and release cycle.

Usability, Learning Curve and Community Support

As you all know, some software tools do a great job, but their poor UI makes them notoriously hard to learn and use (not mentioning any tools that rhyme with mirror…). It’s important to check that there’s a good UI for any tool you choose. Whilst many developers love to use the commandline for everything, your CI/CD tool needs to be accessible to everyone on the team including managers.

One key thing you have to factor into your decision is the potential short-term loss in productivity as you move your team over to using the new CI/CD release pipeline. The important things to look at are whether the tool is intuitive and easy to lean, what training materials are available and whether there is active support and help from the vendors or developers of the tool. 

Another consideration is community support. Some tools have a great community built up around them that can solve any issues you may hit, while other tools may be so obscure that there is no community available to support you. In the latter case, you may find yourself left to figure out issues all by yourself.

Integration with Other Tools

Clearly, you really want a tool that integrates with your existing setup. This means you need to have a good feel for what your existing toolchain uses. Are you already using Jenkins for instance? Do you use a repository service such as Bitbucket or GitHub to store your code? Do your build and production environments run on Kubernetes and Docker? If so you need to ensure there are suitable images available for the new tool. Do you need to be able to run the tool on AWS, Azure or Google Cloud? What compiler toolchain(s) are you using? What underlying languages are you using for your codebase? Do you need to integrate with external tools such as Jira or Confluence?. Do you want to receive automatic build notifications via Slack?

Security, Data Storage and Retention

If you are opting to use a tool that runs on virtual infrastructure or as a SaaS service, then you need to consider security for both your data and your access keys. Almost invariably, a CI/CD tool will need access to your SSH keys for your code repository. It will potentially also need your AWS or Azure credentials and may need many more such secrets. Clearly, you need to be satisfied that these are suitably secure. You also need to give consideration to the security of your actual codebase. For many companies, protecting source code is seen as critical. So, you need to be looking at what sort of protections are offered if your code is being exported from your own servers or repos.

Another important factor is data retention. Often for audit and quality assurance, you need to be able to look back at test results and the like many months later. This is a particular factor to consider when you choose a SaaS vendor. You need to satisfy yourself that their data retention policy is suitable. On a similar note, you also need to look at exactly where your data is being stored. For instance, a US may have issues if their data is being stored on servers in China.

Conclusions

There’s a plethora of CI/CD tools to choose from. Choosing the right tool for your particular project is essential – there’s no one-size-fits-all solution. Hopefully, the advice above will make it easier for you to reach a decision as to which tool is right for you. If nothing else, it should give you some idea of the sort of things you have to consider when making your decision.