Flying Blind — The 2013 IT Operations Quotient Report
June 11, 2013

Sasha Gilenson
Evolven

Share this

IT Operations is now overwhelmed — by the volume, velocity and variety of change and configuration data, lacking insight or actionable information, all making change and configuration problems a chronic pain.

As shown by recent surveys at the Gartner Data Center Summit and ServiceNow Knowledge13 conferences, where Evolven surveyed over 300 IT Operations professionals asking questions critical to IT operations management, 84% of IT professionals said that they want to significantly improve their IT operations management.

The 2013 IT OQ (Operations Quotient) Report provides a good indication to IT executives as to whether IT ops investments have yielded desired results, using the IT Operations Quotient (OQ), a metric for evaluating operational ability to support existing business services and incoming business requirements.

When an Incident Occurs, Can You Quickly Know What Changed?

Only 7% of the professionals surveys indicated that, using their current IT management tools, they could quickly identify what changed in order to respond to problems and incidents.

The first question IT operations asks themselves when an incident occurs is "what changed?" Due to the complexity and dynamics taking place in the modern data center, with overwhelming configuration data and frequent changes, this question has become quite formidable.

Between applications, environments, and individual instances, mistakes and unauthorized changes happen, demanding that IT ops spend significant amounts of time managing configuration values.

Traditional IT management tools were not designed to deal with the complexity and dynamics of the modern data center. These tools have not been automated to collect data down to granular details, analyzing all changes and consolidating information to extract meaningful information from the sea of raw change and configuration data.

Without systems to manage and organize this growth, IT will drown in its own data.

Can You Automatically Validate that Your Release Deployed Accurately?

Only 8% of the participants surveyed agreed that they could currently automatically validate the accuracy of their deployments. Available release management tools are unprepared for one-off changes or changes that do not follow policy.

IT organizations regularly transition changes to production environments, checking changes throughout a set of pre-production environments.

Now IT is under even more pressure. To meet business requirements, application deployments have accelerated and software deployment schedules have driven up high-paced change activity. The increasingly agile nature of application and infrastructure change requests, leaves IT operations at a loss as they are inundated by change requests that run the gamut from the critical and high priority to the minor and unimportant.

With a typical environment having thousands of different system configuration parameters, any little change can impact performance. So it’s not surprising to see many companies going through painful stabilization periods after a release, as well as production outages.

Even when using automated tools for deployment, the lack of detailed visibility into the release means IT ops can’t ensure accurate, error-free deployments.

Can You Quickly Identify the Incident’s Root Cause?

As shown in this survey, the vast majority of IT professionals surveyed concurred that they lack the capabilities to quickly identify an incident’s root cause. IT organizations find themselves challenged when assessing system failure and tracking down the root cause, such as if a patch wasn't deployed or a server failed.

Any minute misconfiguration or omission of a single configuration parameter can quickly lead to an incident with high impact. With an infinite number of these configuration parameters in play when an environment incident hits, finding the root cause consumes both precious time and manpower, making MTTR woefully high in most organizations.

The root cause of downtime and incidents often start at the most granular level of configuration changes where today's configuration management and change management tools don't provide visibility. The different groups in organizations, like IT Development, Support, and Operations, tend to point the finger of blame for issues, and fail to diagnose or deal with the root cause of the problem.

After a major incident, root cause analysis should focus on root cause of the failure in order to not only resolve the incident but to head off a recurrence. Even when IT teams manage to suppress a failure, and operations can return to "normal", the true root cause may still remain unresolved, leaving the organization exposed to further chaos.

Can You Automatically Verify the Consistency of Your Environments?

From our survey, only 5% of the respondents felt that currently they can automatically verify the consistency of their environments, where they need to go into the fine, granular details and identify the make-up of even minor changes, having to process the enormous amounts of configuration data, for verifying the consistency between servers and environments.

As IT organizations regularly transition changes to production environments, IT teams need to check changes throughout a set of pre-production environments that can include system test, performance test, UAT, staging, etc (changes are also mirrored in a Disaster Recovery environment). IT has sought to diversify their workloads, spreading deployments over multiple IT environments to mitigate risk, yet also doubling complexity.

The high volumes of changes means that not all changes consistently make their way to all environments (pre-prod, prod, DR). The configuration parameters must be validated for consistency in real-time.

IT Operations Analytics Helps

With performance at risk from any disruptions to stability, IT teams need to know exactly what has changed in an environment.

Managing IT environments with intelligent automated analytics will drive more sophisticated proactive processes like comparing environment states, validating releases, and verifying consistency of changes,helping to prevent or identify critical issues. So rather than continue to feed bloated system tools, IT Operations should strive to simplify and implement configuration management based on IT Operations Analytics, and turn the situation around from what can’t be managed to being what can be done about performance and availability.

Sasha Gilenson is the Founder and CEO of Evolven Software.

Share this

The Latest

April 15, 2021

A growing need for process automation as a result of the confluence of digital transformation initiatives with the remote/hybrid work policies brought on by the pandemic was uncovered by an independent survey of over 500 IT Operations, DevOps, and Site Reliability Engineering (SRE) professionals commissioned by Transposit for its inaugural State of DevOps Automation Report ...

April 14, 2021

As the Covid-19 pandemic forces a global reset of how we gather and work, 60% of organizations are looking forward to increased spending in 2021 to deploy new technologies, according to the 14th annual State of the Network global study of enterprise networking and security challenges released by VIAVI Solutions ...

April 13, 2021

Complexity breaks correlation. Intelligence brings cohesion. This simple principle is what makes real-time asset intelligence a must-have for AIOps that is meant to diffuse complexity. To further create a context for the user, it is critical to understand service dependencies and correlate alerts across the stack to resolve incidents ...

April 12, 2021

We're all familiar with the process of QA within the software development cycle. Developers build a product and send it to QA engineers, who test and bless it before pushing it into the world. After release, a different team of SREs with their own toolset then monitor for issues and bugs. Now, a new level of customer expectations for speed and reliability have pushed businesses further toward delivering rapid product iterations and innovations to keep up with customer demands. This leaves little time to run the traditional development process ...

April 08, 2021

On Wednesday January 27, 2021, Microsoft Office 365 experienced an outage affected a number of its services with a prolonged outage affecting Exchange Online. Despite Microsoft indicating that it was just Exchange Online affected during this outage, some monitoring tools detected that Azure Active Directory and dependent services like SharePoint and OneDrive were also affected at the time. The outage information indicated a rollout and rollback but we wouldn't expect to see such a widescale outage and slowdown just affecting some of the schema unless everything had to be taken offline ...

April 07, 2021

Application availability depends on the availability of other elements in a system, for example, network, server, operating system and so on, which support the application. Concentrating solely on the availability of any one block will not produce optimum availability of the application for the end user ...

April 06, 2021

A hybrid work environment will persist after the pandemic recedes, with over 80% stating that they expect over a quarter of workers to remain remote, and over two-thirds desiring flexibility between on-premises and remote deployments according to the 2021 State of the WAN report released by Aryaka ...

April 05, 2021

As vaccinations rise and businesses plan for a post-covid future, more than 80% of knowledge workers in the US would like their long-term work environment to include some element of remote work ...

April 01, 2021

With so many of us working from home, IT leaders and executives are now more than ever interested in ensuring that the cloud services their team relies on are available. But instead of accessing popular business-critical applications such as Salesforce, G Suite, Office 365, Microsoft 365, and so on through the company's data center, employees now get these services directly from the Internet. Experience and productivity at each location vary by internet, ISP, gateway, proxy, etc. ...

March 31, 2021

Integration challenges continue to be a major roadblock for digital transformation initiatives, according to MuleSoft’s 2021 Connectivity Benchmark Report. As digital initiatives accelerate, integration has emerged as a critical factor in determining the success and speed of digital transformation across industries ...