In BSMdigest's exclusive interview, Mark Petroff, President and CEO of Marketing Associates, and Andy Frey, CTO, talk about Application Performance Management and how it impacts the end-user experience.

BSM: Tell me about Marketing Associates.

MP: We are a technology-enabled marketing services firm. Our business started 44 years ago as a traditional marketing company, and now we are managing mission-critical sales and marketing applications and websites for our clients to help them process sales transactions and marketing programs. So we have grown into a back office technology services firm that supports sales and marketing programs for our customers.

BSM: What are your mission-critical applications?

MP: A number of off-the-shelf products; one proprietary digital asset management software product that we developed for our clients; and many customized applications to help our clients process incentives, rewards and sweepstakes.

BSM: Who are the end-users of your applications?

MP: In some cases the end-user is the consumer who is trying to buy a product or leverage an online offer. For example, for one of our clients we literally have hundreds of thousands of end-users coming to one of our platforms within a few minutes. But our business is also focused on dealers, distributors and franchisees who are using the application provided by their franchisor or OEM to make their jobs easier.

BSM: How do you define a positive end-user experience?

MP: When we develop or manage an application for our clients, we always set up SLAs associated with it. And those SLAs tie right back into the end-user's experience with the application. We have SLAs that govern uptime and availability, and some that tie back to response time. We measure it and provide the dashboards to our clients and monthly updates to show how we are doing relative to the SLAs.

BSM: How are those SLAs chosen?

MP: They vary from client to client, application to application, and transaction to transaction. During the development of the applications, in the test environment, we will simulate actions that will be taking place, and we benchmark performance times.

AF: For example, in our pricing analytics tool we have a 10 second response time for processing many of the actions. That SLA is great in that world, but for a regular website 10 seconds would be terrible. It just depends on the application.

BSM: Do you find that customers sometimes want SLAs with higher performance than necessary – performance levels that would not even impact the user experience – but they don't understand that?

AF: That is absolutely correct. When you say “10 seconds” that sounds horrible. It doesn't sound like a great response time. But we have one database application with 100 million rows versus 20 million rows, and all the data is actually in memory. We can't even get the response time down to 10 seconds because of all the analytical processes that have to happen in real-time. But when you compare it to what the organization used to do, where it used to take months to get the job done, and now it is a few hours, that is a substantial increase in productivity.

MP: It becomes a cost-benefit equation. We can optimize performance, but the current performance may be considered reasonable when considering the cost associated with the improvements.

BSM: Is application performance the main factor in providing a good end-user experience?

MP: I think that Application Performance Management is the tool for us to define a good end-user experience. And it qualifies it, either in terms of number of packets delivered, response time to a query, or completion of the transaction.

BSM: How do you know if you are delivering a positive user experience?

MP: We measure customer happiness in terms of adoption of the application. For an internal application, we also get phone calls saying the application is running slow. By having the performance monitoring, we can see if the perception is correct, and then troubleshoot to see where the slowness is coming from.

BSM: What about with external end-users, your customers' customers? Do you also get direct feedback?

AF: Yes, on one of our programs we get direct feedback from consumers on Facebook five minutes after a promotion runs, and we can see reactions to what actually happened with that one program at that one specific time.

BSM: What are the biggest challenges that you face in terms of maintaining or improving the end-user experience?

MP: One of the challenges we struggle with is the implementation of the monitoring tools across all of our applications. You don't just plug it in and start monitoring. It is relatively labor-intensive to get an application instrumented so that you can monitor. It is a huge challenge for us because our infrastructure team has lots of other jobs to do in terms of getting applications up and running, and attaching the performance management tools to those applications is a time-consuming task.

BSM: What features or functionality do you require from an APM vendor?

AF: One of the more important requirements is the ability to analyze end-user performance. That is one of the reasons why we chose Gomez – it is the only combination of server monitoring, application monitoring and end-user experience monitoring. For our customers' internal applications, we can place agents on desktops and see exactly what the end-user response time is. But even without the agent on the desktop, we can use the process of elimination to figure out end-user response time.

BSM: How do you measure the user experience on customer facing web applications or websites?

AF: We implement the APM on the application and it breaks down the web transaction to show where the main pain points are, where most of the process wait times are. APM gives us the ability to look into the application and see where we can make changes to improve the performance. By being able to look at each of the individual transactions which happen within the complete application, we are able to find the place where most of the cycles are being processed, and pinpoint the bottleneck in the application.

BSM: Are there any additional features or capabilities that you would like to see in the market, in terms of improving the end-user experience?

AF: We would like an easier way to figure out what the real problem is with the end-user. I wish there was an easier way to identify what the actual problem is over the Internet. Sometimes we don't understand why the network is taking longer for a particular group. We like to have more insight into that. The best way is to put an agent on the end-user's machine but you can't always do that.

BSM: What is the secret to providing a great end-user experience?

MP: The secret is that we are able to demonstrate the end-user experience with facts, figures, charts and diagrams, to show our customers what is happening with their customers when they go to our applications.