Fear of Deployments

This article covers:

  • Rushed features — Larger releases means less frequent releases. Teams might find themselves rushing a feature to ensure they can make it for the next release cycle.
  • Crushing change control — When you have larger releases, they become riskier. The more risk, the bigger the impact of failure. The larger the impact of failure, the tighter the process becomes with additional layers of approvals and change control heaped on top.

The layers of a deployment

When I think of a deployment, I think of it in layers. Once you start looking at it in terms of multiple pieces of a deployment, you can begin to see that there are multiple places where you can make a deployment and a rollback easier. In many organizations, particularly large ones, the thought is that there are a series of separate deployments that happen, but these different parts are all part of the same deployment! A deployment of code does you no good if the database change hasn’t been applied.

  • Fleet deployment is the process of getting the artifact deployment done across many servers.
  • Artifact deployment is the process of getting new code installed on a single server.
  • Database deployment is the process of any changes that need to happen to the database as part of the new code deployment.
Figure 1. The layers of deployments in an application.

Making deployments routine affairs

The deployment process details the steps necessary to get your code into a usable state by your customer. This involves moving code from some development or staging environment onto the production servers where customers interact with it. It can also include database level activities to ensure that the code and the database schema are compatible with each other. Up until now I’ve described the code and the features they provide as one in the same. Deploying new code means deploying new features, but in this article the delivery of new code can be decoupled from the delivery of new features In much the same way that a database schema change can be performed in phases versus a single big bang approach. One of the ways you make an activity routine is to do it in a controlled environment beforehand. The more you can make the environment like the real thing, the better and more comfortable you become. This all starts with our pre-production environments.

Accurate pre-production environments

Accuracy in pre-production environments is key to the confidence they inspire in subsequent steps throughout the process. Let me start by stating my definition of a preproduction environment. It is any application environment which isn’t production. Not all pre-production environments are created equal. You might have a pre-production environment where the nodes are extremely small, and the database is a smaller subset of production data, but the way the environment is configured should be the same, minus any performance specific tuning. Things that might differ are the number of database connection threads between production and pre-production, because the size of your database servers will likely be radically different. If you’re running Apache HTTPD 2.4.3 in production, you should be running that in pre-production as well. If you’re forcing all traffic to be TLS based in production, you should force all traffic to be TLS based in pre-production.

Docker and Kubernetes, new tech, same problems

You’ve probably at least heard of Docker and Kubernetes by now. Docker is the most popular container in use today. But that begs, what are containers? Containers allow developers to package all the necessary components of an application into a single artifact (the container). Inside the container are all the libraries, files, applications and other dependencies necessary to run the application. Containers use two key features of the Linux kernel cgroups and namespaces. These features allow you to completely isolate a container running on a host machine from other containers. Compared to virtual machines, containers are much lighter from a resource perspective because they share the kernel of the host operating system. This eliminates a lot of duplication of resources that exist in virtual machines. Containers also launch a lot faster than virtual machines, making them attractive for large deployments as well as local development environments.

Staging is never exactly like production

In a lot of organizations there’s a ton of energy and effort put into making the staging environment behave exactly like production. But the unspoken truth is that production and staging are almost never alike. The biggest thing missing from staging environments is the thing that often causes the most problems in a system, users and concurrency.

Follow Manning Publications on Medium for free content and exclusive discounts.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store