APM is More Than a Software Product
March 10, 2014

Oliver Cole
OC Systems

Share this

I have been in the performance engineering business for about 30 years. I get involved early on during system development and move off as the systems move to maintenance. Then, I move onto the next one.

I sometimes get called in to troubleshoot a performance crisis. I have sometimes been compared to a locust moving onto the next greenfield and, at other times, compared to Kwai Chang Caine (of Kung Fu fame), wandering the earth sowing justice, peace and performance.

Whichever you choose, I have noticed some similarities between the various projects that I have worked on.

1. Speed to Meet the Need

There are two aspects to performance engineering:

- To speed up the system to meet the required response times

- To talk the customer into lowering their response time requirements

It continually amazes me how high level system people will start off with unnecessarily fast response requirements. One customer wanted sub-second browser response time, even though the user would stare at the resulting web page for many minutes. Really, it was ok if it took a couple of seconds to fetch the web page.

So don't feel afraid to talk your customers into a lower performance criteria.

2. Keep Track of Your History

Performance requirements are always patchy. Yes, they get the big ones (user response time), but it is totally typical for the subcomponents to not have requirements on them, making it unclear where and how to make the end user response times. Complicating this, generally, all of the discussions leading to the system designs are lost. There was a big debate as to how large the system should be for how many users. Someone chose that 256 CPU server monster, but what was their thinking? Half way through the project, when the response requirements have been relaxed and the number of users has been increased, it sure would be nice to look though those emails and see what people were thinking so you can estimate what system changes need to be made to accommodate the business changes.

So during the early phases, have an archive account to which people can cc "important" emails. Later, these can be reviewed and all that wonderful history and knowledge will make the system change and maintenance a lot easier.

3. Dev vs. Ops

The development people do not really address the maintenance phase of the project. The development people are in a heated rush to meet performance goals and deadlines while not over running costs. The project managers are very focused on the deadline of "going live".

Then the system goes live, everyone throws a party and then they realize that they have not planned well for this lesser period of effort: maintenance. What artifacts did the developers leave the maintainers? Did they leave a workable load test? Did they leave a well organized set of production diagnostics implemented as part of the application? Did they leave proper alert monitoring? Did they leave proper daily reporting?

Ideally, the last stage of development is to set up the system for long term, cost effective maintenance, not the first stage of maintenance. This task is properly done when the development team is still available, not after the brightest are gone to their next adventure. As much as possible the maintenance tasks (such as daily reports, regression tests and load tests) should be automated and that may require changes to the system, changes that the development team can do easily, but will be much more difficult for the whittled down maintenance team.

4. An APM Solution Itself is not the Solution

Management often feels that buying a tool will solve a problem. "We have performance problems so we will buy an APM solution." This belief often results in shelfware: management has underestimated the effort to implement the solution.

Yes, an APM solution is good. But you need to have enough labor to implement it and look at and understand the results. Plan on it up front. Don't focus on the capital cost to your organization, focus on how it will integrate into your organization and what will be done on an ongoing basis.

5. How Important is Performance?

How important is your project? Everyone feels that their project is important, but some are more so than others. Some systems really are not that performance intensive. So, you can save money by not worrying so much about performance; a high performing system is more expensive to monitor and maintain than one that requires less performance.

Make the decision early on as to how important performance is. If it is important, make sure that it is considered important by everyone. If it is not so important, embrace that too and make clear goals for performance. Even if performance is not important, clearly indicate the goals. Even easy to reach goals should be articulated.

I have seen systems where the performance goal is to wait for users to complain, then address it. Sometimes that works fine.

Oliver Cole is President of OC Systems.

Share this

The Latest

March 27, 2024

Nearly all (99%) globa IT decision makers, regardless of region or industry, recognize generative AI's (GenAI) transformative potential to influence change within their organizations, according to The Elastic Generative AI Report ...

March 27, 2024

Agent-based approaches to real user monitoring (RUM) simply do not work. If you are pitched to install an "agent" in your mobile or web environments, you should run for the hills ...

March 26, 2024

The world is now all about end-users. This paradigm of focusing on the end-user was simply not true a few years ago, as backend metrics generally revolved around uptime, SLAs, latency, and the like. DevOps teams always pitched and presented the metrics they thought were the most correlated to the end-user experience. But let's be blunt: Unless there was an egregious fire, the correlated metrics were super loose or entirely false ...

March 25, 2024

This year, New Relic published the State of Observability for Financial Services and Insurance Report to share insights derived from the 2023 Observability Forecast on the adoption and business value of observability across the financial services industry (FSI) and insurance sectors. Here are seven key takeaways from the report ...

March 22, 2024

In MEAN TIME TO INSIGHT Episode 4 - Part 2, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at Enterprise Management Associates (EMA) discusses artificial intelligence and AIOps ...

March 21, 2024

In the course of EMA research over the last twelve years, the message for IT organizations looking to pursue a forward path in AIOps adoption is overall a strongly positive one. The benefits achieved are growing in diversity and value ...

March 20, 2024

Today, as enterprises transcend into a new era of work, surpassing the revolution, they must shift their focus and strategies to thrive in this environment. Here are five key areas that organizations should prioritize to strengthen their foundation and steer themselves through the ever-changing digital world ...

March 19, 2024

If there's one thing we should tame in today's data-driven marketing landscape, this would be data debt, a silent menace threatening to undermine all the trust you've put in the data-driven decisions that guide your strategies. This blog aims to explore the true costs of data debt in marketing operations, offering four actionable strategies to mitigate them through enhanced marketing observability ...

March 18, 2024

Gartner has highlighted the top trends that will impact technology providers in 2024: Generative AI (GenAI) is dominating the technical and product agenda of nearly every tech provider ...

March 15, 2024

In MEAN TIME TO INSIGHT Episode 4 - Part 1, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at Enterprise Management Associates (EMA) discusses artificial intelligence and network management ...