The Pros & Cons of Flow & Packet Data - Part 1
February 22, 2022

Jay Botelho

Share this

Designing and maintaining a network that delivers uninterrupted performance is a crucial function of most NetOps teams. But with new technology challenges around cloud and software defined architectures, many struggle to optimize and troubleshoot the high-performance networks of today.

According to a recent survey from LiveAction, 20% of NetOps teams are focused on improving application performance across the network, 19% are focused on improving network monitoring, and 15% are focused on improving performance at remote sites. Doing this effectively requires visibility into flow and packet data. When aggregated and analyzed properly, NetOps teams can gain valuable insights and operate more predictable, high-performing networks.

NetOps teams traditionally rely on network performance monitoring solutions to collect this data, but what are the pros and cons of flow and packet data and how is it used to troubleshoot networks?

First, let's quickly define flow and packet data. The goal of network flow monitoring is to tally, log, and analyze all network traffic as it passes through routers and other network devices, essentially creating a summary model of network usage. Deep Packet Inspection (DPI) is a process commonly used to inspect the payload content of each packet to make determinations about whether to act on that packet by rejecting it or allowing it to pass through the network. DPI can also be used to passively collect the traffic traversing the network to add visibility and troubleshooting capabilities into network monitoring solutions.

Packet capture is also used to store a mirror copy of network packets for detailed network analysis, using forensic search and filtering. The stored mirror copy can later be examined for a particular time frame, when new performance, security, or forensic incidents arise. When network messages are packetized (broken into pieces), they are then routed over the internet to other connections to be reassembled at their destination. Each packet is generally organized into three segments regardless of size — the header, payload and footer. As packets flow through the network routers, their headers are read and "fingerprinted" based on five to seven packet header attributes.

Today, most routers have some brand of xFlow export feature that allows flow data to be sent from the router to a collector and analyzer. Netflow is the de facto industry flow protocol (originating from Cisco), but other popular protocols include IPFIX, J-Flow, and sFlow. Source and Destination addresses tell who the originator and receiver of the traffic are. Ports and Class of Service tell what applications are in use and their traffic priority. Device interfaces tell how devices are utilizing traffic. By tallying packets, the total traffic flow amount can be determined. Timestamps are useful for placing flows in time and determining their rates. And finally, Application and Network Latency provide measurements about how long each transaction takes.

What are the pros of flow and packet data?

First, flow data is simple to set up. Most routers and switches come standard with the xFlow protocol feature. This means you get vendor-agnostic visibility across just about every network segment. Capturing flow data also requires no extra cabling or equipment, and in most cases no extra licensing, providing excellent network visibility essentially "for free." It also has low network bandwidth overhead since flow data approximates only 0.5% of network traffic, and no clients are necessary on end systems.

For Packet data, it's valuable because it contains every bit of information for every transaction on the network. It allows NetOps to understand bandwidth usage by analyzing details of application and user behavior.

Excessive bandwidth utilization often occurs over very small time periods, typically referred to as "microbursts" since these event happen over microseconds to milliseconds. These events are hidden by the typical reporting rates of xFlow data, but are easily exposed by packet data.

Packet data is also ideal for detailed monitoring and troubleshooting on critical applications, servers and connections. This helps with answering critical questions, like whether the network or the application is the root cause of a problem. Packet data provide specific, interpacket timing, and can expose critical data in payloads that provide proof of application problems. Packet data also offer significant name discovery, such as application names, file names, website URLs, and hostnames, which can be used for both detailed troubleshooting and reporting on custom, web-based applications.

Go to: The Pros and Cons of Flow and Packet Data - Part 2

Jay Botelho is Director of Engineering at LiveAction
Share this

The Latest

June 23, 2022

Digital businesses don't invest in monitoring for monitoring's sake. They do it to make the business run better. Every dollar spent on observability — every hour your team spends using monitoring tools or responding to what they reveal — should tie back directly to business outcomes: conversions, revenues, brand equity. If they don't? You might be missing the forest for the trees ...

June 22, 2022

Every day, companies are missing customer experience (CX) "red flags" because they don't have the tools to observe CX processes or metrics. Even basic errors or defects in automated customer interactions are left undetected for days, weeks or months, leading to widespread customer dissatisfaction. In fact, poor CX and digital technology investments are costing enterprises billions of dollars in lost potential revenue ...

June 21, 2022

Organizations are moving to microservices and cloud native architectures at an increasing pace. The primary incentive for these transformation projects is typically to increase the agility and velocity of software release and product innovation. These dynamic systems, however, are far more complex to manage and monitor, and they generate far higher data volumes ...

June 16, 2022

Global IT teams adapted to remote work in 2021, resolving employee tickets 23% faster than the year before as overall resolution time for IT tickets went down by 7 hours, according to the Freshservice Service Management Benchmark Report from Freshworks ...

June 15, 2022

Once upon a time data lived in the data center. Now data lives everywhere. All this signals the need for a new approach to data management, a next-gen solution ...

June 14, 2022

Findings from the 2022 State of Edge Messaging Report from Ably and Coleman Parkes Research show that most organizations (65%) that have built edge messaging capabilities in house have experienced an outage or significant downtime in the last 12-18 months. Most of the current in-house real-time messaging services aren't cutting it ...

June 13, 2022
Today's users want a complete digital experience when dealing with a software product or system. They are not content with the page load speeds or features alone but want the software to perform optimally in an omnichannel environment comprising multiple platforms, browsers, devices, and networks. This calls into question the role of load testing services to check whether the given software under testing can perform optimally when subjected to peak load ...
June 09, 2022

Networks need to be up and running for businesses to continue operating and sustaining customer-facing services. Streamlining and automating network administration tasks enable routine business processes to continue without disruption, eliminating any network downtime caused by human error or other system flaws ...

June 08, 2022

Enterprises have had access to various Project and Portfolio Management (PPM) tools for quite a few years, to guide in their project selection and execution lifecycle. Yet, in spite of the digital evolution of management software, many organizations still fail to construct an effective PPM plan or utilize cutting-edge management tools ...

June 07, 2022

It has become increasingly difficult for DevOps and SRE teams to minimize the impact of issues and ensure high-quality end-user experiences. In this blog, I'm going to propose a new approach to support real-time use cases — edge observability — that enables you to detect issues as they occur and resolve them in minutes ...