As organizations deploy multi-cloud environments and adopt more containers, microservices, and cloud-native technologies, they are facing increasingly distributed systems. As a result, the use of observability is becoming mainstream — with organizations turning to observability solutions to provide an understanding of critical interdependencies across application workloads and infrastructure and address the ever-growing complexity that goes along with modernizing your business.
VMware's 2022 State of Observability found that companies are still grappling with increasing complexity and a lack of visibility — with 97% reporting challenges in their ability to monitor cloud application environments. In this blog, I'll outline key takeaways, including how cloud-native apps have rapidly grown in complexity, why the perception of observability has shifted, and what challenges still need to be addressed.
Cloud Native Apps Growing in Complexity
Enterprise operations are overwhelmingly hybrid and multi-cloud, with 89% of respondents agreeing that today's cloud applications are significantly more complex than they were five years ago. Development teams are also moving toward microservice architectures and picking up the pace of code delivery, leading to an increased need for observability tools.
In addition, a vast amount of organizations are releasing or making updates to critical cloud applications multiple times a week or more. With less than 19% of respondents stating they push updates less than once a month. Even with the best processes in place, organizations cannot expect all changes to go smoothly when these changes are happening frequently.
The Shift in Perception of Observability
A clear value sign of how essential observability tools are becoming is the sentiments shared by those using the tools, with 41% of survey respondents claiming it is not just valuable but necessary for daily operations. For those who have already implemented an observability solution, it is clear that it doesn't just benefit one part of the team but the entire team. Almost nine out of ten respondents (88%) agree that cloud services and applications would have better availability and performance if stakeholders — DevOps, developers, SREs, architects, etc. — had visibility into necessary infrastructure and application behavior metrics.
In the past year, there has been a 32% jump in those just using observability shifting their beliefs to acknowledge the necessity of observability tools. All this to say that the perceived value of observability is at an all-time high. In terms of most valued observability capabilities, the top four were: easy integration with existing tooling including open source (51%), monitoring multi-sourced data at scale including metrics, histograms, traces, and span logs (49%), performing both synthetic and real-user monitoring (48%), and easily correlate customer and business metrics to application and infrastructure performance (48%).
Some Challenges Still Remain
While the need for organizations to adopt an observability solution is becoming business imperative, it doesn't come without its challenges. As organizations add observability tools, almost half are using more than five separate tools and have no clear consensus on how to rationalize toolsets. This year, two-thirds of respondents (67%) agreed that it is a mistake to monitor applications and infrastructure separately in a cloud environment, up significantly from 57% who felt that way last year. Showing that having separate tools for infrastructure and application monitoring, with no way to correlate metrics across domains, is insufficient in helping troubleshoot accidents.
While it can be argued that there is value in having individual software tools specifically suited to specific needs, every tool requires energy and effort to deploy and maintain. The group was divided when respondents were asked what they believed was the best way to improve their existing monitoring tools. 48% preferred identifying gaps in the current toolset and acquiring additional capabilities as needed, (38%) preferred integrating existing or new tools to reduce their overall toolset. Lastly, 14% thought evaluating new needs and starting from scratch would be best. Signifying that even though the road to observability isn't always easy, it's necessary.
What Does All This Mean
Observability has gained significant momentum in the last year. With 70% of those surveyed either using, implementing, or evaluating observability tools, observability has reached a tipping point and is set to enter mainstream adoption.
In today's complex IT landscape, monitoring can tell you and your teams when something is wrong, but observability is the key to figuring out why and how to fix it. Observability provides visibility across the full IT stack and helps resolve cloud complexities, allowing teams to identify problems in complex cloud applications easily. Organizations that are using observability report significant benefits, with 87% rating the technology as either necessary (41%) or very valuable (46%). While there is still a need to simplify monitoring tools ( 46% using five or more tools), the best way to unify those tools is not yet crystallized.
Complexity as organizations transform their business will only continue to grow. Observability is quickly becoming recognized as a mission-critical component to reducing complexity, delivering business outcomes, and innovating faster.
