Hybrid cloud architecture is breaking the backs of network engineering and operations teams. These teams are more successful when their companies go all-in with the cloud or stay out of it entirely. When companies maintain hybrid infrastructure, with applications and data residing across data centers and public cloud services, the network team struggles.
This insight emerged in the newly published 2024 edition of Enterprise Management Associates' (EMA) Network Management Megatrends research, our biennial network operations benchmarking study based on a survey of 406 enterprise IT professionals. The survey found that nearly 51% of respondents were supporting a hybrid cloud environment. More than 26% were 100% in the public cloud, while more than 23% were relying entirely on private data center infrastructure.
EMA has long observed that network operations teams are challenged by the public cloud. Network engineers tell us that they lose control over architecture and struggle to gain visibility into cloud networks. They also struggle with skills gaps and collaboration challenges. However, this most recent finding suggests that it isn't just the cloud itself that necessarily causes these problems. Instead, it's the hybridization of infrastructure, with resources spanning public and private resources, that is causing significant pain. Network teams are finding it difficult to manage across the two worlds. They lack tools and processes for architecting and operationalizing an effective hybrid cloud network. To illustrate this point, the Megatrends research asked respondents to identify the typical root cause of complex service issues that pull together war room response teams with stakeholders from multiple IT groups. In hybrid cloud enterprises, network infrastructure was more likely to be the root cause of these challenging problems.
When one thinks about it, the challenge of a hybrid environment makes perfect sense. In a company that uses only private infrastructure, network teams have long had the tools, processes, and skills required to manage such networks. Those that adopt a 100% cloud environment can adapt by embracing cloud-native technologies for building, managing and observing networks. There are countless vendors that have developed solutions that excel at supporting purely cloud networks, and cloud providers themselves have built out an ecosystem of tools to enable networks within their environments.
It's much harder to build and manage networks that span these two worlds because they are fundamentally different. In private infrastructure, the network team has administrative access and control over the underlying infrastructure, making it easier to manage networks granularly and to extract monitoring data. In the public cloud, network teams must work with APIs and abstracted services because they have no administrative access to the underlying infrastructure. Deploying tools and developing processes that span these very different architectures is fundamentally complex.
Addressing Hybrid Cloud Complexity
The Megatrends research revealed that network teams that support a hybrid cloud architecture are more likely to be investing in new network performance management tools, suggesting that they've identified gaps in how their existing tools provide end-to-end visibility into hybrid cloud networks. They are looking for tools that can provide end to end visibility. In fact, 53% of network teams who support a hybrid cloud or multi-cloud environment reported that they are trying to build out end-to-end monitoring and troubleshooting capabilities. Network teams have a variety of options here, from their legacy on-premises tools and cloud-native observability tools, to tools that focus more on digital experience monitoring than infrastructure monitoring. It remains unclear which approach is best.
"I see a lack of holistic visibility into the cloud. It's hard to have a single pane of glass when you are looking at various systems," as a network security architect at a Fortune 500 cybersecurity company recently told EMA. "We've been relying on logs. But there are not tools that are designed to give us visibility across different [public and private] clouds."
Network teams with hybrid cloud environments are also more likely to be investing in new network security solutions, suggesting that they are trying to impose consistent and effective security controls across public and private infrastructure. These network teams were also more likely to share their network management and monitoring tools with their counterparts in cybersecurity teams, suggesting that the complexity of hybrid cloud security forces more collaboration between network and security teams.
Hybrid cloud network teams are also more likely to make new investments in multi-cloud networking solutions, software-defined WAN, and data center network infrastructure, indicating that hybrid environments are demanding modernization of local and wide-area network connectivity.
In other words, hybrid cloud infrastructure simply demands more of network teams than pure private and pure cloud environments. The complexity doesn't end there. EMA also observed that companies with hybrid environments are more likely to have a mix of on-premises data centers and colocation providers comprising their private infrastructure. Research participants told us that they typically incorporate colocation data centers into their networks to support efforts to reduce their on-premises data center footprints, to enable disaster recovery and resiliency, and to deploy applications and data closer to end users and customers.
Finally, all this complexity creates a need for operational efficiency. One key enabler of that efficiency is network automation. Unfortunately, enterprises with hybrid environments were more likely than others to struggle to hire IT personnel with network automation skills. Network teams will need vendors to support them as they try to mitigate these skills gaps.
Overall, just 39% of network pros who support a hybrid cloud and/or multi-cloud architecture believe that their efforts to impose end-to-end operations have been completely successful. EMA observed two potential paths toward success:
First, network teams that have improved their ability to hire and retain skilled personnel do better with these efforts. This will require IT leadership that is willing to go the extra mile, with improved recruiting efforts, competitive compensation, and opportunities for advancement and growth.
Second, network teams do better when they minimize their reliance on tools provided by individual cloud providers to operationalize hybrid networks. Instead, they need to push their traditional tool vendors to innovate.