You can always tell that a technology is moving into the mainstream when the discussion shifts from reasons to adopt to challenges and obstacles.
Docker is one such technology.
Thankfully, most folks get the value of Docker containers; they're convinced about the business value of a platform that facilitates development independence and rapid deployment of software services.
No argument there, but being empowered and fast means zilch if all that microservice goodness is fragile, sickly and becomes an operational nightmare.
Monitoring containers and microservices is challenging because of their defining characteristics. IT operations has always had a ton of stuff to keep watch over, but now the ephemeral nature of containers puts more strain on the monitoring function. There's a massive increase in objects, more moving parts and dependencies, which together with API-centric messaging means an exponential increase in system checks, metrics, logs, events and alarms. Moreover, DevOps and cross-functional accountability for quality means the purview of monitoring has extended beyond operations.
This combination of complexity and increased scope makes for a complex challenge. Not only do solutions have to cater for increased componentry, load and noise, they must also present information to many more stakeholders and address a variety of use-cases. Horses for courses shall we say.
So make no mistake, while Docker containers and microservices are a veritable tech taste explosion, they're slippery little suckers to manage. Or as Benjamin Wootton so aptly describes them in his excellent article – they're not a free lunch.
At CA Technologies, we get that Docker Monitoring is, well, different. That's why we've incorporated advanced functionality into our Application Performance Management solution that caters for all the container nuances, while also addressing the needs of many different teams – be that the app team triaging problems, the site reliability engineering squad looking to increase architectural resilience, or a development group needing to better understand how code quality impacts performance.
Here are just three things we've designed to address the slipperiness.
A single microservice can be composed of many containers, each running a different process. In these architectures a failed container can quickly be replaced with another, so what's more important than instrumenting individual containers is understanding overall service – be that at a cluster or application level.
This is why CA APM has been engineered to deliver an agentless service for Docker; monitoring the health and performance of hosts and clusters through the Docker Daemon and also extending to applications running inside the containers – with deeper instrumentation when required. This approach also means less hassle configuring monitoring – freeing up time to dedicate on more important container activities – like optimizing services and increasing architectural resilience.
Monitoring used to be focused on the objectives of IT operations managing the data center, but that's all changed. Because developers are now more accountable for applications once in production (aka - you code it, you run it), they need increased visibility into application performance across the entire lifecycle. Increasingly, new DevOps roles are becoming the full stack managers and responsible for optimizing performance and providing continuous performance feedback to many more teams.
This DevOps dynamic has also guided the design of CA APM. By utilizing an advanced multi-dimensional, attribute-driven data model, CA APM helps different teams visualize microservice relationships/dependencies and consume metrics in ways that support their specific roles and responsibilities. Rather than incorporate discreet monitoring dashboards into practices and workflows, teams are presented performance information in context. But while teams can review performance from a different perspective, data model and visualization consistency facilitates increased collaboration and information sharing – again critical for Docker monitoring, but also a DevOps stimulant.
When compared to more traditional application stacks, containers present some thorny data management problems. However simple a deployment, there's always going to be more containers per host to monitor, plus the number of metrics per host increases too. This can all exponentially increase alerts and alarms; an issue described in superb detail Sara Wells' excellent presentation Alert Overload – How to Adopt a Microservices Architecture without Being Overwhelmed by Noise .
With CA APM we've addressed this problem through the use of analytics. One such service is called Differential Analysis, which by leveraging proven statistical models can distinguish small nuisance alerts and false-positives from anomalous trends worthy of attention. For example, one container going down and being replaced might not warrant action. However, if the pattern happens consistently over a longer period of time then differential analysis would detect the anomaly and service degradation - automatically. Once again, looking at microservices as a collective unit provides valuable application insight.
In many ways Docker containers and microservices are analogous to fast food – simple in concept, repeatable and quick to deliver. But always ensure convenience is coupled with the optimum performance and availability that modern Docker Monitoring delivers. That way you'll ensure the tech always delivers on its promise and doesn't end up being half-eaten by the business or leaving your customers with an insipid experience.