In today's digital economy, monitoring is a must. Your customers must be able to access your website and your apps, interact, purchase — and monitoring is one way to make sure this keeps happening.
Monitoring is the means to an end: digital performance. We don't monitor just to monitor. We are looking for the details, the insight, the actionable information that will help us ensure and improve digital performance.
Dr. Terry Critchley, author of The IT Glossary: Information Technology Concepts and Topics Defined and Explained, says: "If you can't measure it, you can't manage it. If you can't manage it, you are heading for failure."
John Rakowski, Director of Technology Strategy at AppDynamics, elaborates: "In the digital business, the focus shouldn't only be on what could be monitored but rather how the information is used to automatically respond. Monitoring technical or business data is only useful if it drives action, whether that is to fix an emerging performance issue quickly or to optimize digital services."
But the first question has to be: What should be monitored? With this in mind, APMdigest asked experts from across the IT industry — from analysts and consultants to users and the top vendors — for their opinions on what IT departments should be monitoring to ensure digital performance.
Many of the experts who provided recommendations for this list believe you need to monitor a vast variety of devices, components, transactions, to ensure digital performance. In fact, it was difficult to keep them all from simply saying "Monitor everything." Almost everyone thinks you should monitor everything. But that being said, what is everything? What are the many pieces that make up everything? That is what this list attempts to answer.
There is no way this list can be totally comprehensive. In this format, it would takes weeks to list all the aspects of IT that you should be monitoring. But hopefully this list will represent an in-depth cross-section of the many options. The purpose of the list is to enlighten you by offering a few suggestions from our experts that can help you make your monitoring even more complete, more robust, and ultimately more effective.
Larry Dragich, Technology Executive and Founder of the APM Strategies Group on LinkedIn, explains: "It's not so much what to monitor but how you monitor. Organizations need a solution to collect, assimilate, and correlate all the events in the environment that outlines the dependencies between the application and infrastructure elements. If you only monitor each component of the infrastructure, your vision becomes myopic and tracking the user experience will be difficult. If you only capture the availability statistics for the application, you'll miss the end-user-experience. The need to ensure digital performance comes down to using a converged application and infrastructure performance monitoring solution."
Taking a look at the big picture is critical, and this list can provide some snapshots that make up that big picture.
The list of "What You Should Be Monitoring to Ensure Digital Performance" will be posted in 5 parts over the next week. Part 1 starts with a high level view of the end-user and customer.
REAL USER PERFORMANCE
Real-user monitoring (RUM) is a critical in ensuring digital performance.
Hayden James
Linux Systems Analyst, haydenjames.io
To reduce bounced sessions and improve engagement, especially for your mobile users, it's important to use real user performance data across 100% of your site traffic to understand the difference between when the user perceives that your application is ready to use, and when the application is actually interactive. If there is a gap or a delay between visually ready and time to interactive, this can be the trigger for end user frustration that leads to bounced sessions, reduced engagement, or even rage clicks. If you're optimizing for page load time only, you could be inadvertently contributing to this user frustration by deferring certain page resources. For example, heavy JavaScript files could be taking more time to fully load on mobile devices even after the page has painted, and blocking the end user from clicking, scrolling, touching, or swiping the page.
Anthony Larkin
Senior Product Marketing Manager, Akamai
END-USER EXPERIENCE
While there are many elements relevant to optimizing digital performance, once performance is more closely linked to business outcomes, as per digital transformation, the most salient arena in my opinion would be full bore insights on end user (and customer) experience. This would include application performance (no matter what type of application, cloud resident or not), as well as insights into usability, effectiveness in supporting business outcomes, usage (and hence relevance), and even insights on application design. The data that can be collected where the 'rubber hits the road' as consumers consume digital services is for me the single most important departure point for both digital performance and digital transformation.
Dennis Drogseth
VP of Research, Enterprise Management Associates (EMA)
At the end of the day, all digital performance operations converge at one metric — user experience. From analyzing application transactions to the SQL queries in the DBs, the DevOps processes are so complex and too much data can often be confusing. From a global business perspective, active/synthetic monitoring is extremely crucial to understand the performance of your apps across geographies and subsequently the insights into quality of experience being delivered to the users can significantly help the Marketing, DevOps and Quality assurance teams internally. Come to think of it, the more the gaps between such inter-org teams are bridged, the better the development and delivery of services usually will be.
Anugraha Benjamin
Market Analyst, ITOM-Applications Manager, ManageEngine
One cannot measure everything, so identifying the key applications and transactions will be the first step. Once these are identified measuring the experience of the user as an early warning indicator, and measuring transactions from end to end will help ensure faster diagnoses and repair.
Jonah Kowall
VP of Market Development and Insights, AppDynamics
Your customer is at the center of your digital business, which means you need to monitor and measure the customer experience to ensure that a consistently good user experience happens every time on any device.
Mark Levy
Director of Strategy, Micro Focus
In an increasing digital world, it is the end user that matters more than anything. Without true endpoint-based end user experience monitoring, how do you really know if your end users are getting the smooth digital experience they expect? Especially when you know they're still logging tickets with the helpdesk even when all your other IT monitoring systems are green?
Amena Siddiqi
Product Marketing Director, Riverbed APM
Organizations must establish an understanding of an application's "health" to include monitoring of the application itself as well as resource dependencies and end user experience. The latter has become a significant focus for organizations and has prompted changes in technology used to collect relevant telemetry. Leveraging technology that proactively monitors application behavior, such as responsiveness from the perspective of the end user, ensures operations and developers can identify and predict bottlenecks and other performance issues that impact the application and influence customer satisfaction.
David Ishmael
Director of IT Operations Analytics, Trace3
QUALITY OF EXPERIENCE
Digital transformation initiatives touch many parts of an IT organization — from the front end design through to fulfillment and support. The single component that impacts every client, partner and employee is responsiveness. So, monitoring Quality of Experience (QoE) policies with active datasets to debug and fix the issues plays a crucial role in successful digital deployments and expansions globally.
Mark Milinkovich
Director of Product Marketing, LiveAction
CUSTOMER SATISFACTION
Monitor how end-users experience the applications and services they are receiving. Businesses often get too hung up on speeds and feeds and how everything looks on a dashboard. But all the green happy charts on a dashboard don't mean a thing if you're end-users are unhappy with the performance and functionality of the application.
Jim Rapoza
Research Director, Aberdeen
User satisfaction is the most important metric to monitor via digital performance. Offering your users a secure infrastructure along with an intuitive platform will not only improve your overall sales but will also reduce the risk of downtime and possible security threats while reducing costs and optimizing resources.
Otis Gospodnetić
Founder, Sematext
When we're talking about Digital Performance the key is to ask why are organizations so focused on digital in the first place? If we assume that it's about delivering more value to customers, then this should be the most critical part of any measurement strategy. Customer satisfaction (CSAT), NPS, customer growth, customer lifetime value, etc. there are many examples of metrics to choose from and some will be more relevant than others depending on the exact situation.
Mike Hughes
Principal Platform Evangelist, OutSystems
SERVICE LEVELS
For each service your team owns (whether technology or process), find the metric that really matters to your customer (external or internal) — this is your service level indicator (SLI). Managing to a committed service level is transparent and data-driven, and ultimately leads to improved operational strategy and decision-making.
Peter Fry
VP of Engineering, Platform & Security, Mavenlink
UTILIZATION
Regardless of how your organization monitors things, don't focus only on the IT/ops aspects. Think business, maximizing resource use, elasticity, and lowering costs. With the availability of containers, orchestrations tools, auto-scaling groups, and similar services, it is possible to automate and optimize your resource utilization. To do this well, monitor the utilization of your infrastructure and feed utilization metrics back into the system, so the system can react to demand in real-time and without manual interventions. Doing this will benefit both the IT (more automation!) and business (optimized costs!).
Otis Gospodnetić
Founder, Sematext
Real-time monitoring of activity is the most important aspect for any organization providing a product or service that is "always on." If users are victim to issues, big or small, such as an outage or suspicious activity, it can instantaneously ruin the external perception of a product's quality and the company's digital performance and overall brand.
Geeta Schmidt
CEO, Humio
BUSINESS RESULTS
While monitoring technical performance data is no doubt important, to ensure digital performance it's vital that enterprises are able to monitor and correlate this data with business outcome metrics. For an eCommerce application these are metrics such as conversion rate percentage (the percentage of visits which result in e-commerce transactions - sales) and the amount of revenue generated by an application during a specified time period.
John Rakowski
Director of Technology Strategy, AppDynamics
Our end users are the reason why we even have a system in the first place, so it's very important to focus on the human element. In IT that translates to tracking the entire user journey (e.g took our clients 25 seconds from search to buy) and tying these metrics back to business conversions. Business metrics can speak just as loud as system metrics if not louder sometimes. Just make sure you're listening to the right ones.
Ravi Lachhman
Evangelist, AppDynamics
NET PROMOTER SCORE
From a business perspective, monitoring net promoter score (NPS) feedback ensures that continuous delivery efforts are driving the business in the right direction.
Mike Mallo
Offering Management Lead - Hybrid Cloud DevOps, IBM Cloud Unit, IBM Corporation
Large companies in particular are often peppered with many IT questions. Adopting a multi-dimensional set of measures for help desk efficiencies in the form of NPS scores of the user population to watch long-term trends on productivity is useful. This will measure the hits and misses to determine proactiveness and clarity in answers. Simultaneously, it can also pick up on any trending issues well ahead of when they may turn into fires, which will substantially increase IT effectiveness.
Ian Pitt
CIO, LogMeIn
EMPLOYEE EXPERIENCE
Digital performance is determined not just by the technology, but most importantly by those using it. Monitoring the employee experience is crucial, because it enables the organization to identify clunky software, confusing user interfaces, user-related errors, system errors impacting employee productivity, inefficient workflows, process bottlenecks and general usability issues that can hinder digital performance. If you don't know how your employees are interacting with the business applications they use all day every day, then you don't know how your company is performing.
Brian Berns
CEO, Knoa Software
Read What You Should Be Monitoring to Ensure Digital Performance - Part 2, covering key metrics like availability and response time.
