APM The Mutability of Containerized Immutable Infrastructure
February 28, 2018

Christopher Tozzi
Fixate IO

"Containers" and "immutable infrastructure" are words that you frequently hear used in the same sentence. That's unsurprising — One of the features that have helped make containers so popular since Docker's release in 2013 is the ability of containers to create application images that are immutable.

Yet in many senses, container-based environments are not as immutable as they may first appear. When planning a strategy for deploying and managing containers, it's essential to understand what aspects of containers are immutable, and what aspects are not. This blog post explains.

What Is Immutable Infrastructure?

Immutable infrastructure is a software management strategy that involves completely destroying and replacing each instance of an application or service when the application or service needs to be updated.

This approach is different from traditional software management because under the immutable infrastructure model, changes are not made to software that is live and running. Immutable infrastructure helps to lower the risk of introducing accidental configuration changes that could lead to security problems. It also minimizes "configuration drift," or the tendency of configurations of similar platforms to diverge over time, and therefore become harder to manage.

Finally, it reduces the chance that a configuration change to a running instance will cause the instance to fail and lead to service disruptions. With immutable infrastructure, changes can be tested thoroughly before they are brought live through a new instance. It is also easy to revert to an earlier configuration if something goes wrong when a new instance goes live.

Immutable Infrastructure and Containers

Containers are a popular building block for immutable infrastructure because containers make it possible to create multiple instances of an application or service based on container images. When updates need to be applied to the application or service, the container image is updated and new container instances are spun up in place of the old ones.

There is no need to reconfigure the application or service instances while they are running, and updates can be thoroughly vetted before the new instance is launched.

The Limits of Immutability

In this respect, containers are a great way to deploy applications without having to worry that unforeseen configuration challenges will disrupt a running application or create security vulnerabilities.

However, it's important to recognize that not all elements of a container environment are immutable. Container images are immutable, but a variety of other important parts and characteristics of the environment are not:

Host infrastructure configurations can change. Most host servers for container environments are built using conventional operating systems, whose configuration changes constantly. Although in theory changes to the host should not impact applications that are isolated inside containers, in practice they can. Host infrastructure modifications might disrupt storage services on which containers rely, for example. Networking resources could also be changed in ways that cause problems for containers.

The scale of the container environment changes. The number of application and service instances fluctuates in response to shifts in demand and as new services are added to the container environment. Data storage requirements can also change. As scale fluctuates, so does the ability of other tools within the environment to work properly. For example, you may not need to worry much about load balancing when your environment is small. But when you have thousands of containers running, an improperly configured load balancer can lead to severe application performance problems.

Endpoints change. In most container environments, changes to network endpoints should not cause problems because the changes will be detected automatically through service discovery. However, any seasoned container admin will tell you that service discovery can sometimes fail. Updates to endpoint configurations can cause unexpected performance problems if they are not properly handled.

Dependencies change. Updates to container images may result in application changes that break dependencies. For example, a container-based microservice that depends on another container-based microservice to function may stop working properly if the container image on which the microservice is based is updated. This is true whether or not the update is rolled out through immutable container images. In other words, immutable container images do not prevent the risk of software updates that break the complex stack on which your application relies in order to function properly.

Orchestration tools change. One of the advantages of containers is that they make it easy to move applications from one host environment to another. Sometimes, migrations require you not just to change host servers, but also to change orchestration tools. For example, AWS' ECS container service runs a built-in orchestrator that is not available on other platforms. Moving a container environment to or from ECS may therefore require you to change orchestrators.

Staying ahead of potential performance, stability and security that could result from changes to any of the factors listed above requires constant monitoring of the container environment.

Detecting changes in a containerized environment based on these factors is especially important because the containerized applications are so dynamic by their nature. If you can't distinguish normal changes within the environment from ones that may be linked to a problem, you lack visibility into the environment and will be slow to respond to critical issues.


Although the immutable nature of container images offers advantages for deploying and updating containerized applications, it is not a cure-all. Many other components of a container environment change rapidly. They require a sophisticated, container-aware approach to monitoring in order to guarantee visibility into fluctuating container environments and detection of problems before they become serious.

To learn more about monitoring approaches for container environments, join the webcast on March 7: Improving Performance in Constantly Changing Applications.

Christopher Tozzi is Senior Editor of Content and DevOps Analyst at Fixate IO
Share this