Apple App Store Turns 11: Benchmarks Mobile Teams Should Adopt to Improve Stability and Experiences
July 17, 2019

Eric Futoran
Embrace

Share this

The 11th anniversary of the Apple App Store frames a momentous time period in how we interact with each other and the services upon which we have come to rely. Even so, we continue to have our in-app mobile experiences marred by poor performance and instability. Apple has done little to help, and other tools provide little to no visibility and benchmarks on which to prioritize our efforts outside of crashes.


11 years ago, Apple opened their App Store. While other stores existed, none was tied to what would become our primary medium of interacting with the world - the iPhone.

Previously, all of us were using browsers to interact, which had its limitations, namely being tied to a stationary or bulky computer.

With the advent of the App Store and the fact of now having a hardware device in our pockets, the opportunity for higher fidelity experiences emerged. First, games and social experiences released, similar to those on a browser. While not an innovative jump from browser-based games and social networks, people readily adopted them because these apps were what we were already using, but more importantly, now readily available to us at all times of the day. Then those experiences improved rapidly to take advantage of the capabilities of the phone itself. Games improved, social experiences improved, and the way we interacted with the world changed.

Humans are drawn to the lowest friction experiences. Conversations became text and chat, marketing became push notifications and SMS, purchasing became instantaneous, and short form video clips became streaming augmented always-on experiences. Even food ordering, hailing a cab, and checking in at work changed because it was just so much easier to conduct on a phone and “seemingly” a better experience.

The behavioral change of where we spent our time happened hyper fast, but the tools available to developers to build those streamlined experiences did not. Initially, the focus was on acquiring users and developing features. Stability was an afterthought. Fittingly, Apple released a single metric, crashes, and only limited capabilities to solve these crashes. Even today most developers don't know that Apple provides a crash reporting tool, and to be honest, those that do, dismiss it as unusable.

Crashes became the de facto and only benchmark of app quality. If an app has 99% crash-free sessions, it's relatively good. Apple did push another benchmark, ratings, but user ratings are very easily gamed through prompts and most users with unstable or poor experiences walk without rating the app.

We all have issues with apps. It's not uncommon to find someone swearing at their phone or attempting to restart it to solve a problem. (Reminds me a lot of conducting a silent prayer and restarting an old desktop to solve a problem.) To help developers adapt and improve all of our experiences, below are some of the new benchmarks that apps should follow to actually provide a stable experience. In fact, Apple has been rumored to use or will soon use many of these benchmarks to determine app rankings in the store, while Google has already started to do so. Please note that only monitoring tools track these benchmarks and less than 1% of apps use a dedicated monitoring tool — although adoption is growing rapidly.


How to Read these Benchmarks: These benchmarks indicate the percentage of clean sessions for each app category by error type. The overall app health is based on a combination of all these metrics, not included here for simplicity. For example, Force Terminations are defined as when a user leaves an app and immediately terminates the app from memory while the app is still running. CPU Pegging is when the processor is fully maxed out and the app is unable to run.

Key Findings:

■ Crash session scores across all verticals are over 99%, which is expected since it has been the primary focus of developers to date.

■ Social Media apps have the worst Force Termination score, which may indicate a larger issue that is going unreported, such as a broken registration or frozen startup.

■ Shopping apps also resulted in a poor Force Termination score, which could be attributed to users frustrated with a feature not working in the app, like a purchase flow failing or a frozen item page.

■ Entertainment and Education apps perform the worst across several categories; an indication of instability, complexity of app, and immaturity of app type.

Now that we are on the 11th anniversary, it's time for us as app developers and teams to shift to the new benchmarks to provide our users with the experiences that they deserve, and Apple as a user-focused high-fidelity brand that users expect.

Methodology: Embrace created a data set from top 100 mobile iOS apps. User experiences were measured based on actual impact to a user, including freezes and abrupt closes.  The latter includes but is not limited to crashes. All data was collected for the time period between June 1 and 30, 2019.

Eric Futoran is CEO of Embrace
Share this

The Latest

April 15, 2024

Organizations recognize the value of observability, but only 10% of them are actually practicing full observability of their applications and infrastructure. This is among the key findings from the recently completed Logz.io 2024 Observability Pulse Survey and Report ...

April 11, 2024

Businesses must adopt a comprehensive Internet Performance Monitoring (IPM) strategy, says Enterprise Management Associates (EMA), a leading IT analyst research firm. This strategy is crucial to bridge the significant observability gap within today's complex IT infrastructures. The recommendation is particularly timely, given that 99% of enterprises are expanding their use of the Internet as a primary connectivity conduit while facing challenges due to the inefficiency of multiple, disjointed monitoring tools, according to Modern Enterprises Must Boost Observability with Internet Performance Monitoring, a new report from EMA and Catchpoint ...

April 10, 2024

Choosing the right approach is critical with cloud monitoring in hybrid environments. Otherwise, you may drive up costs with features you don’t need and risk diminishing the visibility of your on-premises IT ...

April 09, 2024

Consumers ranked the marketing strategies and missteps that most significantly impact brand trust, which 73% say is their biggest motivator to share first-party data, according to The Rules of the Marketing Game, a 2023 report from Pantheon ...

April 08, 2024

Digital experience monitoring is the practice of monitoring and analyzing the complete digital user journey of your applications, websites, APIs, and other digital services. It involves tracking the performance of your web application from the perspective of the end user, providing detailed insights on user experience, app performance, and customer satisfaction ...