Don't Be the Next Instapaper
February 21, 2017

Michelle McLean
ScaleArc

Share this

Instapaper, a "read later" tool for saving web pages to read on other devices or offline, suffered an extensive outage 2 weeks ago. The site was unavailable for a day and a half, and even after restoring service, the company had to explain that its archives would be impacted for another full week. Ultimately, it was able to restore the archives sooner, but the outage garnered extensive press and social media coverage.

The cause of the outage was that an indexing file Instapaper relies on for reaching all stored links exceeded the max file size supported on the older instance of Amazon Web Services the site was first built on. You can read if you want more details .

While Instapaper hit a unique problem — a file size limitation — its experience speaks to a much larger problem: scaling a database is difficult, and never quick. That basic fact explains why outages like the one Instapaper suffered are surprisingly common.

Engineering a scaled database — and then performing the application changes needed to take advantage of that scaled out database — is tough coding work indeed. We encounter companies with full control of their source code who are petrified to make the changes needed to scale database capacity. Perhaps it's an ecommerce app, and it's too close to Black Friday. Or maybe it's just a case of attrition: the folks who really understand that code base are long gone, and the current engineers don't dare mess with the interworkings of the app.

These kinds of meltdowns are common during surge events, like the one ESPN suffered with the launch of Fantasy Football or the one Macy's suffered last Black Friday. Sometimes customers can see these events coming (e.g., they're expecting a major traffic surge on Black Friday) and sometimes they simply don't (e.g., their product gets a nod from a celebrity and all of a sudden they're swamped).

When a traffic surge takes down your site, it usually means the data tier was already fragile. Scaling the web infrastructure is pretty easy, as is scaling internet capacity. But scaling the data tier itself is where the challenges lie.

The Instapaper crisis also illustrates how the cloud alone doesn't solve the challenge of scaling the data tier. While elasticity is a hallmark of cloud services, the physics around having an application talk to multiple instances of a database remains a challenge. We've seen some customers suffer from an inflated sense of confidence that running in the cloud takes away these difficulties.

Don't wait for disaster to strike. Whether you're running on prem or in the cloud, keep a close eye on all metrics that reveal how "hot" your systems are running. Ensure your disaster recovery plan is robust — and recently tested. Better yet, don't rely on disaster recovery. Instead, run in active/active mode, where you've got multiple instances of all critical systems running in different locales, with the systems able to take on the full load if one portion fails.

Take steps now to scale your data tier and avoid these kinds of catastrophic outages. Those "Here's why we failed" engineering blog entries are no fun to write.

Michelle McLean is VP of Marketing at ScaleArc.

Share this

The Latest

September 19, 2017

IT professionals tend to go above and beyond the scope of their core responsibilities as the changing business landscape demands more of their attention, both inside and outside of the office, according to the Little-Known Facts survey conducted by SolarWinds in honor of IT Professionals Day ...

September 18, 2017

Digital video consumption is viral and, according to a new study released by IBM and International Broadcasting Convention (IBC), more than half of the 21,000 consumers surveyed are using mobiles every day to watch streaming videos, and that number is expected to grow 45 percent in the next three years ...

September 15, 2017

No technology that touches more than one IT stakeholder, no matter how good and how transformative, can deliver its potential without attention to leadership, process considerations and dialog. In this blog, I'd like to share effective strategies for AIA adoption ...

September 14, 2017

Enterprise IT environments are becoming more heterogeneous and complex, with fragmentation permeating cloud infrastructure, tooling and culture, according to a survey recently conducted by IOD Cloud Technologies Research in partnership with Cloudify ...

September 12, 2017

One area that enables enterprises to reduce complexity and streamline operations is their virtual desktop infrastructure (VDI). Virtualization is a linchpin of digital transformation and effectively optimizing an enterprise's VDI is essential to moving forward with digital technologies. Delivering the best possible VDI performance means taking a fresh look at what "desktop" means today. The endpoint, or desktop, now can be a physical thin client, a software-defined thin client, a traditional laptop, a phone or tablet. To reduce operational waste and achieve better performance across the desktop environment, consider these five actions ...

September 11, 2017

In incident management, we often overlook the simple things in favor of trying to do too much, too soon. Why not make sure we've done the fundamentals properly? ...

September 08, 2017
For our Advanced IT Analytics (AIA) Buyer's Guide, we interviewed more than 20 deployments to help us better assess vendor strengths and limitations. So given the abundance of riches to work with, I've decided to illustrate several of the more prominent AIA benefit categories with actual real-world comments ...
September 07, 2017

The Input/Output Operations per Second (I/O) capabilities of modern computer systems are truly a modern wonder. Yet no matter how powerful the processors, no matter how many cores, how perfectly formed the bus architecture, or how many flash modules are added, somehow it never seems to be enough ...

September 06, 2017

By taking advantage of performance monitoring, IT and business decision makers can gain better visibility into their cloud and application performance. Dedicated performance monitoring has become essential for providing visibility into all areas of application performance and keeping the business running optimally ...

September 01, 2017

There may be no more critical emerging technology for IT organizations in the digital age than advanced IT analytics (AIA) — most commonly called “operational analytics.” It is with this in mind that EMA is launching what we believe is the first ever buyer's guide for AIA adoption ...