As any network administrator can tell you, network traffic doesn't stand still. It is constantly changing and increasing in complexity. Networks have fundamentally changed, and the demands put on them by new technology, customers, mobility, and other factors are forcing IT to develop networks that are more agile and dynamic than ever before. While it seems like IT departments are bombarded with new challenges, there are three major trends that are making it difficult to gain visibility into networks: the increased adoption of virtualized infrastructure, enterprise mobility and the rise in encrypted traffic.

Virtualization and associated software-defined networking (SDN) approaches have created tremendous change in the data center, while mobility and encryption have created blind spots in infrastructure that traditional monitoring tools do not recognize. Compounding this problem is the fact that network administrators have been compelled to meet the needs of an organization's cybersecurity initiatives – which requires that they have full visibility into their infrastructure – and it's clear how difficult the problem they are facing is. Simply put, network administrators need to be able to see every packet to guarantee the performance and security of their networks, but the accelerated rate of change, and the complexities that has wrought, have made it nearly impossible.

Since networks and infrastructure are constantly changing, the methods that are used to gain visibility into them cannot afford to be static. When done well, visibility shines light on blind spots, enables detection of anomalous behavior and gives administrators the power to fix network and application issues proactively before they become problems for end users. But, giving administrators the power to be proactive is not enough in today's complex environment. It is no longer enough to simply point to a network bottleneck or send an alert for a spike in bandwidth demand – visibility must be automated so that the information is shared instantly. Manual intervention is a point of failure for network operations and security operations teams, and can be eliminated if the tools we use for visibility are designed to take action.

To automate visibility, we must architect visibility as a critical layer of infrastructure. Once designed in this fashion, an administrator is empowered with the ability to intelligently deliver any portion of network traffic to as many appliances and tools that need to monitor and analyze it. The administrator can use policies to select specific traffic that needs to be delivered to each of these tools. Such an architectural approach to visibility has the additional benefit of abstracting the operational tools needed to secure and manage a network from the specifics of the underlying network. Once such a layer is created, all security and operational tools can get access to critical network traffic from anywhere in the infrastructure. Further, when the intelligence derived from visibility is united with the rest of the network and security infrastructure, it is possible to automate policy management so that the tools can programmatically control the information they receive from the Visibility Fabric. Such automation improves responsiveness and effectiveness, simplifies tasks and establishes a framework for continuous monitoring and analytics of the infrastructure.

Technology will continue to be transformative – in the data center and beyond. No one can afford to sit still in this environment, least of all IT departments. Automating visibility is a critical step in getting control of the dramatic changes affecting infrastructure, and one that should be taken sooner rather than later – the next big challenge is likely right around the corner.

Ananda Rajagopal is VP of Product Management at Gigamon.