The Pros & Cons of Flow & Packet Data - Part 1
February 22, 2022

Jay Botelho

Share this

Designing and maintaining a network that delivers uninterrupted performance is a crucial function of most NetOps teams. But with new technology challenges around cloud and software defined architectures, many struggle to optimize and troubleshoot the high-performance networks of today.

According to a recent survey from LiveAction, 20% of NetOps teams are focused on improving application performance across the network, 19% are focused on improving network monitoring, and 15% are focused on improving performance at remote sites. Doing this effectively requires visibility into flow and packet data. When aggregated and analyzed properly, NetOps teams can gain valuable insights and operate more predictable, high-performing networks.

NetOps teams traditionally rely on network performance monitoring solutions to collect this data, but what are the pros and cons of flow and packet data and how is it used to troubleshoot networks?

First, let's quickly define flow and packet data. The goal of network flow monitoring is to tally, log, and analyze all network traffic as it passes through routers and other network devices, essentially creating a summary model of network usage. Deep Packet Inspection (DPI) is a process commonly used to inspect the payload content of each packet to make determinations about whether to act on that packet by rejecting it or allowing it to pass through the network. DPI can also be used to passively collect the traffic traversing the network to add visibility and troubleshooting capabilities into network monitoring solutions.

Packet capture is also used to store a mirror copy of network packets for detailed network analysis, using forensic search and filtering. The stored mirror copy can later be examined for a particular time frame, when new performance, security, or forensic incidents arise. When network messages are packetized (broken into pieces), they are then routed over the internet to other connections to be reassembled at their destination. Each packet is generally organized into three segments regardless of size — the header, payload and footer. As packets flow through the network routers, their headers are read and "fingerprinted" based on five to seven packet header attributes.

Today, most routers have some brand of xFlow export feature that allows flow data to be sent from the router to a collector and analyzer. Netflow is the de facto industry flow protocol (originating from Cisco), but other popular protocols include IPFIX, J-Flow, and sFlow. Source and Destination addresses tell who the originator and receiver of the traffic are. Ports and Class of Service tell what applications are in use and their traffic priority. Device interfaces tell how devices are utilizing traffic. By tallying packets, the total traffic flow amount can be determined. Timestamps are useful for placing flows in time and determining their rates. And finally, Application and Network Latency provide measurements about how long each transaction takes.

What are the pros of flow and packet data?

First, flow data is simple to set up. Most routers and switches come standard with the xFlow protocol feature. This means you get vendor-agnostic visibility across just about every network segment. Capturing flow data also requires no extra cabling or equipment, and in most cases no extra licensing, providing excellent network visibility essentially "for free." It also has low network bandwidth overhead since flow data approximates only 0.5% of network traffic, and no clients are necessary on end systems.

For Packet data, it's valuable because it contains every bit of information for every transaction on the network. It allows NetOps to understand bandwidth usage by analyzing details of application and user behavior.

Excessive bandwidth utilization often occurs over very small time periods, typically referred to as "microbursts" since these event happen over microseconds to milliseconds. These events are hidden by the typical reporting rates of xFlow data, but are easily exposed by packet data.

Packet data is also ideal for detailed monitoring and troubleshooting on critical applications, servers and connections. This helps with answering critical questions, like whether the network or the application is the root cause of a problem. Packet data provide specific, interpacket timing, and can expose critical data in payloads that provide proof of application problems. Packet data also offer significant name discovery, such as application names, file names, website URLs, and hostnames, which can be used for both detailed troubleshooting and reporting on custom, web-based applications.

Go to: The Pros and Cons of Flow and Packet Data - Part 2

Jay Botelho is Senior Director of Product Management at LiveAction
Share this

The Latest

January 26, 2023

As enterprises work to implement or improve their observability practices, tool sprawl is a very real phenomenon ... Tool sprawl can and does happen all across the organization. In this post, though, we'll focus specifically on how and why observability efforts often result in tool sprawl, some of the possible negative consequences of that sprawl, and we'll offer some advice on how to reduce or even avoid sprawl ...

January 25, 2023

As companies generate more data across their network footprints, they need network observability tools to help find meaning in that data for better decision-making and problem solving. It seems many companies believe that adding more tools leads to better and faster insights ... And yet, observability tools aren't meeting many companies' needs. In fact, adding more tools introduces new challenges ...

January 24, 2023

Driven by the need to create scalable, faster, and more agile systems, businesses are adopting cloud native approaches. But cloud native environments also come with an explosion of data and complexity that makes it harder for businesses to detect and remediate issues before everything comes to a screeching halt. Observability, if done right, can make it easier to mitigate these challenges and remediate incidents before they become major customer-impacting problems ...

January 23, 2023

The spiraling cost of energy is forcing public cloud providers to raise their prices significantly. A recent report by Canalys predicted that public cloud prices will jump by around 20% in the US and more than 30% in Europe in 2023. These steep price increases will test the conventional wisdom that moving to the cloud is a cheap computing alternative ...

January 19, 2023

Despite strong interest over the past decade, the actual investment in DX has been recent. While 100% of enterprises are now engaged with DX in some way, most (77%) have begun their DX journey within the past two years. And most are early stage, with a fourth (24%) at the discussion stage and half (49%) currently transforming. Only 27% say they have finished their DX efforts ...

January 18, 2023

While most thought that distraction and motivation would be the main contributors to low productivity in a work-from-home environment, many organizations discovered that it was gaps in their IT systems that created some of the most significant challenges ...

January 17, 2023
The US aviation sector was struggling to return to normal following a nationwide ground stop imposed by Federal Aviation Administration (FAA) early Wednesday over a computer issue ...
January 13, 2023

APMdigest and leading IT research firm Enterprise Management Associates (EMA) are teaming up on the EMA-APMdigest Podcast, a new podcast focused on the latest technologies impacting IT Operations. In Episode 1, Dan Twing, President and COO of EMA, discusses Observability and Automation with Will Schoeppner, Research Director covering Application Performance Management and Business Intelligence at EMA ...

January 12, 2023

APMdigest is following up our list of 2023 Application Performance Management Predictions with predictions from industry experts about how the cloud will evolve in 2023 ...

January 11, 2023

As demand for digital services increases and distributed systems become more complex, organizations must collect and process a growing amount of observability data (logs, metrics, and traces). Site reliability engineers (SREs), developers, and security engineers use observability data to learn how their applications and environments are performing so they can successfully respond to issues and mitigate risk ...