Mastering Observability: Navigating Costs and Complexity with eBPF Innovation
September 27, 2023

Aviv Zohari
groundcover

Share this

A colleague of mine recently embarked on a journey to explore the capabilities of a well-known legacy observability platform within his Kubernetes environment. He dedicated a week to familiarize himself with the platform, primarily testing out the different features for traces, logs, and infrastructure monitoring. However, his focus shifted when a critical feature needed an early release, diverting his attention away from the observability tool. Unfortunately, without any prior notification or warning, there was no rate limitation to the platform logs collection mechanism. One line of YAML configuration file meant all logs were collected, ingested and stored — with no mention of the projected cost.

Fast forward to the following week, a member of the billing department barged into his office, demanding an explanation for an astronomical observability bill totaling $33,000 for a single month, a staggering contrast to the anticipated $1,700.

This series of events left my work buddy struggling with the size of his mistake, and me questioning whether it really was entirely his fault.

The Complex Landscape of Observability Pricing

Navigating observability pricing models can be compared to solving a perplexing puzzle which includes financial variables and contractual intricacies. Predicting all potential costs in advance becomes an elusive endeavor, exemplified by a recent eye-popping $65 million observability bill.

Avoiding miscalculations as the one that happened to my friend requires continuous monitoring of the monitoring solution. This practice slows down day-to-day operations and long-term growth efforts.

The Challenge of Affordability in Observability

The escalating costs associated with observability represent a vast challenge which is confronting many organizations currently. Particularly in the age of cloud computing, IT leaders and even top executives have come to realize the imperative of reining in their infrastructure budgets, which often spiral out of control.

The proliferation of microservices and distributed architectures has ushered in a flood of data that demands observability. Traditionally, more data translates into higher expenses, accompanied by substantial resource consumption, leading not only to increased costs but also inefficiencies.

Regrettably, most observability tools employ pricing models that defy prediction. While applications generate large amounts of log data, instead of an advantage, this abundance has become a cause for concern. In response, best practices now advocate monitoring "only what you need" or limiting the retention period for collected data to a minimum. This raises two questions: how can you know in advance what you will need, and will limiting the retention period to a minimum make it impossible to correlate with out-of-range historical data.

Enter eBPF: A Game-Changer

eBPF (extended Berkeley Packet Filter) has recently emerged as a revolutionary technology that has significantly impacted the Linux kernell. eBPF operates at specific hook points within the kernel, extracting data with minimal overhead, safeguarding the application's resources from excessive consumption. It observes every packet entering or exiting the host, mapping them to processes or containers running on the host, thereby offering granular insights into network traffic.

Moreover, eBPF-powered agents operate independently of the primary application being monitored, ensuring minimal impact on microservice resources.

The combination of visibility depth and stability has made eBPF a groundbreaking technology for cybersecurity companies, and is predicted to have the same effect on observability, for exactly the same reasons.

Hassle-Free Observability

Observability should empower engineers, not bury them in a load of unexpected overheads, data volume surges, and huge subscription bills. The goal of observability platforms should be to guarantee complete protection against such surprises, offering immunity against sudden spikes in data volume and shielding engineers from unfortunate encounters with the billing department.

In conclusion, the journey to achieving efficient and cost-effective observability is full of challenges, but with the right tools and strategies, IT and DevOps leaders can help their organizations emerge from financial uncertainty and empower their engineers to become true observability heroes.

Aviv Zohari is the Founding Engineer of groundcover
Share this

The Latest

November 29, 2023
The past few years have presented numerous challenges for businesses: a pandemic, rising interest rates, supply chain disruptions, and geopolitical conflict that sent shockwaves across the global economy. But change may finally be on the horizon. According to a recent report by Endava ... a majority of executives confirmed they are feeling optimistic about the current business climate, and as a result, are forecasting larger IT budgets, increased technology funding and rollout, and prioritized innovation in the coming year ...
November 28, 2023

Incident management processes are not keeping pace with the demands of modern operations teams, failing to meet the needs of SREs as well as platform and ops teams. Results from the State of DevOps Automation and AI Survey, commissioned by Transposit, point to an incident management paradox. Despite nearly 60% of ITOps and DevOps professionals reporting they have a defined incident management process that's fully documented in one place and over 70% saying they have a level of automation that meets their needs, teams are unable to quickly resolve incidents ...

November 27, 2023

Today, in the world of enterprise technology, the challenges posed by legacy Virtual Desktop Infrastructure (VDI) systems have long been a source of concern for IT departments. In many instances, this promising solution has become an organizational burden, hindering progress, depleting resources, and taking a psychological and operational toll on employees ...

November 22, 2023

Within retail organizations across the world, IT teams will be bracing themselves for a hectic holiday season ... While this is an exciting opportunity for retailers to boost sales, it also intensifies severe risk. Any application performance slipup will cause consumers to turn their back on brands, possibly forever. Online shoppers will be completely unforgiving to any retailer who doesn't deliver a seamless digital experience ...

November 21, 2023

Black Friday is a time when consumers can cash in on some of the biggest deals retailers offer all year long ... Nearly two-thirds of consumers utilize a retailer's web and mobile app for holiday shopping, raising the stakes for competitors to provide the best online experience to retain customer loyalty. Perforce's 2023 Black Friday survey sheds light on consumers' expectations this time of year and how developers can properly prepare their applications for increased online traffic ...

November 20, 2023

This holiday shopping season, the stakes for online retailers couldn't be higher ... Even an hour or two of downtime for a digital storefront during this critical period can cost millions in lost revenue and has the potential to damage brand credibility. Savvy retailers are increasingly investing in observability to help ensure a seamless, omnichannel customer experience. Just ahead of the holiday season, New Relic released its State of Observability for Retail report, which offers insight and analysis on the adoption and business value of observability for the global retail/consumer industry ...

November 16, 2023

As organizations struggle to find and retain the talent they need to manage complex cloud implementations, many are leaning toward hybrid cloud as a solution ... While it's true that using the cloud is not a "one size fits all" proposition, it is clear that both large and small companies prefer a hybrid cloud model ...

November 15, 2023

In the same way a city is a sum of its districts and neighborhoods, complex IT systems are made of many components that continually interact. Observability requires a comprehensive and connected view of all aspects of the system, including even some that don't directly relate to its technological innards ...

November 14, 2023

Multicasting in this context refers to the process of directing data streams to two or more destinations. This might look like sending the same telemetry data to both an on-premises storage system and a cloud-based observability platform concurrently. The two principal benefits of this strategy are cost savings and service redundancy ...

November 13, 2023

In today's rapidly evolving business environment, Chief Information Officers (CIOs) and Chief Technology Officers (CTOs) are grappling with the challenge of regaining control over their IT roadmap. The constant evolution and introduction of new technology releases, combined with the pressure to deliver innovation on shrinking budgets, has added layers of complexity for executives who must transform the perception of the role of the IT leader from cost managers and maintainers to strategic enablers of growth and profitability ...