CTRL+ALT+DELETE: 5 Tips for Avoiding Data Disasters
June 18, 2018

Yaniv Yehuda
DBmaestro

Share this

It's every system administrator's worse nightmare. An attempt to restore a database results in empty files, and there is no way to get the data back, ever.

Despite the fear and panic created by data loss, more often than not it's due to simple things that are under our control and can be prevented. Studies have shown that the single largest cause for data outages is human error. Regardless of how much you try, there are still going to be mistakes and you have to account for them in the way database changes are managed.

Here are five simple tips for keeping things running smoothly and minimizing risk.

1. Define roles and responsibilities

Safeguards need to be put in place to ensure that only authorized people have access to the production database.

The level of access shouldn't be determined only by an employee's position but also by the level of seniority. A famous story made the rounds last year when a developer shared that while following instructions in a new employee manual, he accidentally deleted the production database. To make things worse, the backup was 6 hours old and took all too long to locate. You might be shaking your head in disapproval right now over how the company could have been so irresponsible to let this happen, but it turns out … it's really not uncommon (check out the comments on this tweet).

To prevent unauthorized changes in the database that can result in utter disaster, it is essential to define, assign, and enforce distinct roles for all employees. If you need to, set roles and permissions per project to avoid any accidental spillover.

2. Confirm back up procedures

You need a well-planned backup strategy to protect databases against data loss caused by different types of hardware, software, and human errors.

You'd be surprised by how often backups simply aren't happening. In one case a sys admin complained that bringing hard drives home with backed up data was inconvenient, so the company invested in an expensive remote system; the same sys admin never got around to creating the new procedure, so the latest version of the backed-up data was 3 months old.

Another employee discovered at his new job there hadn't been a single back-up for the past three years.

Knowing the back-ups are happening isn't enough. You also need to also check to make sure they are usable and include all the data that's needed. It's worth restoring and then checking that the restored database is an exact match to the production data. A check such as "Is the most recent backup size within x bytes of the previous one" is a simple solution to make sure the restored database matches the production database.

3. Adopt version control best practices

Version control practices have long since been adopted in other code development environments, ensuring the integrity of code as only one person can work on a segment at any given time.

Version control provides the ability to identify which changes have been made, when, and by whom. It protects the integrity of the database by labeling each piece of code, so a history of changes can be kept and developers can revert to a previous version.

Bringing these practices into the database is crucial for data loss prevention, especially in today's high-paced environment with increasingly shorter product release cycles. By tracking database changes across all development groups you are facilitating seamless collaboration, while enabling DevOps teams to build and ship better products faster.

4. Implement change policies

Databases are code repositories, so they need the same safeguards when changes are made. It's crucial to have clear policies on which changes are allowed and how they are administered and tracked.

Is the action of dropping an index in a database allowed? How about a table? Do you prohibit production database deployments during daytime hours? All of these policies should not only be practiced by participating teams, but enforced on the database level, too. Keep track of all the changes and attempted changes made. A detailed audit can help detect problems and potential security issues.

5. Automate releases

By taking advantage of comprehensive automated tools, DBAs and developers can move versions effortlessly from one environment to the next. Database development solutions allow DBAs to implement consistent, repeatable processes while becoming more agile to keep pace with fast changing business environments.

Automation also enables DBAs to focus instead on the broader activities that require human input and can deliver value to the business, such as database design, capacity planning, performance monitoring and problem resolution.

Databases often hold the backbone of an organization, a priceless container for the transactions, customers, employee info and financial data of both the company and its customers. All this information needs to be protected by following clear procedures for managing database changes. Reducing the likelihood of data loss due to human error can help everyone sleep better at night.

Yaniv Yehuda is CTO and Co-Founder at DBmaestro
Share this

The Latest

September 21, 2018

The performance gap between customer experience leaders and runners-up is widening, with those on top being disproportionately rewarded. Gartner said organizations must ignore three myths in order to achieve a superior customer experience ...

September 19, 2018

This summer marked three years since Microsoft announced Windows 10, its first "Windows as a service" Operating System (OS). Windows 10 brought with it a new Software-as-a-Service-like approach to updates, moving Microsoft and the millions of environments that depend on it, more frequent, bundled updates. Whether you believe the shift was for better or worse, one thing is certain, this "as a service" model is a natural progression for today's operating systems. That is why Windows 10 is changing not only how frequently updates are pushed out, but inherently how technology is purchased, how people consume it, and perhaps most importantly, how IT is run. Let's take a look at how Windows 10 has impacted these three key areas over the past three years ...

September 18, 2018

To celebrate IT Professionals Day 2018 (this year on September 18), the SolarWinds IT Pro Day 2018: A World Powered by Tech Pros survey explores a "Tech PROactive" world where technology professionals have the time, resources, and ability to use their technology prowess to do absolutely anything ...

September 17, 2018

Are digital war rooms obsolete because they're just a place for managers of siloed business units to find someone else to blame for a critical IT event such as a security breach? Far from it. Enterprises find these emergency response teams just as important, if not more important, than ever ...

September 14, 2018

The goal of EMA's latest research was to look at how advanced IT analytics (AIA) — EMA's term for primarily what today is best known as "AIOps" — is being deployed. Here are the remaining four of my seven personal takeaways ...

September 13, 2018

OK, the data is in! The goal of EMA's latest research was to look at how advanced IT analytics (AIA) — or EMA's term for primarily what today is best known as "AIOps" — is being deployed. Here are seven of my own personal takeaways ...

September 12, 2018

By maximizing the knowledge of end-to-end quality of service (QoS) using virtualized network functions (VNFs), the SD-WAN (edge) gateway establishes a suitable connection with minimal latency and maximum performance so that entire organizations can make the most of the Office 365 application suite ...

September 11, 2018

Market exuberance for Office 365 has inspired business mandates to adopt the cloud-hosted collaboration and productivity suite without regards to the underlying chaos. While multi-location organizations are virtualizing, operating models haven’t necessarily changed. This partial transformation that excludes automation and simplification of the network puts Office 365 deployments (and other software-as-a-service offerings) in danger of failing ...

September 10, 2018

Most organizations are undergoing a digital transformation that directly impacts how they do business, yet 70 percent of employees have not mastered the skills they need for their jobs today, and 80 percent of employees do not have the skills needed for their current and future roles, according to Gartner ...

September 06, 2018

In a survey within the VMware User Group community, Blue Medora took a closer look at how various metric collection strategies and access to Dimensional Data impacts IT success. We started with one question: How important is your monitoring integration strategy? ...