subscribe to stayonline blog

The Hidden Costs of a Sudden Data Center Shutdown

Best case scenario, a sudden data center shutdown results in a painfully expensive problem. Worst case scenario—it sinks your business. Most data center managers and business leaders see the immediate impact of an outage; the iceberg in the water, so to speak. But they make a titanic mistake by failing to consider the massive threats waiting just below the surface. Hidden costs make up the majority of the total cost of a data center shutdown.

Ponemon Institute releases an industry report once every three years to highlight extensive research on the true cost of a data center outage. Their latest research was published in 2016 (we covered the key findings in this blog). Today, we’re diving a little deeper into the topic and discussing the cost factors that lie just below your plane of sight but cause the most damage.

The Scope of The Problem

The average cost of an outage in 2016 was between $740,357 and $2,409,991. This cost is expected to continue rising. Complete unplanned data center outages lasted an average of 103 minutes and costed an average of $7,003 per minute. In other words, hundreds of thousands, or even millions of dollars for every hour that a data center is down.

Sourcing The Costs

In every outage case, organizations must take action to detect the source(s) of an outage, contain the damage, fix and recover from the outage, and prevent future disruptions. Every action costs money. More specifically, there are three cost categories: direct costs (which are typically the most anticipated expenses), indirect costs, and opportunity costs. As you can see in the chart below, Indirect and opportunity costs represent the majority of the total financial impact. These are the “hidden” costs that frequently get overlooked.

data-center-shutdown[Image credit: Ponemon Institute Study]

Direct Costs (36% of the total financial impact)

Any expense that can be directly attributed to a specific item or action needed for the prevention of and response to a data center outage. For example:

  • Management and detection software
  • Backup systems
  • Equipment repairs or replacements
  • Specialized cables and connectors to prevent outages
  • Cost of labor

Indirect Costs (51% of the total financial impact)

Expenses related to the time, effort and resources of an organization, incurred because of the outage or an attempt to resolve the outage. For example:

  • Lost employee productivity
  • Lost end-user productivity
  • Third-party support (consultants, auditors, etc.)

Opportunity Costs (12% of the total financial impact)

The cost of lost business and potential business that resulted from the disruption. For example:

  • Lost potential/existing customers
  • Missed sales and revenue
  • Damaged reputation

How to Estimate Indirect Costs[1]

As the largest contributor to the total cost of an outage, it is imperative that data center managers and business leaders be able to estimate indirect and opportunity costs as soon as an outage is contained. Understanding the full scope of the incident will help inform your rebound strategy. Below are formulas that you can use to reach approximations:

Formula: Productivity Cost = E x % x C x H

  • E = number of employees affected
  • % = percentage they are affected
  • C = average cost of employees per hour
  • H = number of downtime hours

Formula: Revenue Loss = (CR/TH) x I x H

  • GR = gross annual revenue
  • TH = total annual business hours
  • % = percentage impact
  • H = hours of downtime

Minimizing The Cost of an Outage

Beyond having the tools to identify the full scope of the damage, your best defense is to prevent an outage from happening in the first place.

1. Know Your Greatest Risk Factors

The top three causes of a sudden data center shutdown are:

  • Uninterruptible Power Supply (UPS) Failure
  • Human Error
  • Cybercrime (which also represents the fastest growing cause)

2. Configure The Right Electrical Hookups

Your UPS may require a specialized power cord and wall outlet capable of handling the ampacity and voltage of your power circuit. This is not the place to take shortcuts, install knockoff power cords, or make assumptions. Work with a professional manufacturer to ensure you get all the ratings and gauges correct. A trusted power cord manufacturer will also help validate that all output receptacles will accommodate the power cords associated with your servers and other relevant equipment. Additionally, don’t daisy-chain surge suppressors and extension cords to your UPS, as this may violate your local fire code and increase your risk of damage. Instead, use a properly wired and grounded 3-prong power outlet that directly connects to the UPS.

3. Have a Thorough Disaster Recovery Plan

Your disaster recovery plan is an iterative strategy that must be revisited regularly, tested periodically, and kept up-to-date with the latest regulations and technologies.

  • Use quality backup solutions
  • Remotely monitor your data center’s power
  • Implement a cyber security strategy
  • Have redundant power sources and multiple power paths
  • Use strong transfer switches capable of shifting to a secondary power source in milliseconds
  • Detail the exact protocol for systematically conducting a planned power shutdown (e.g., for scheduled electrical maintenance)

4. Follow power cord best practices

A clean, well organized electrical environment is the unsung hero of outage prevention and recovery.

  • Color code power cords to prevent accidental disconnects and to make sourcing issues easy and fast.
  • Triple-check that your power cord gauges are large enough to handle the ampacity of the circuit.
  • Use shielded cords to prevent Electromagnetic Interference (EMI), which can impact cord performance and result in equipment failures.
  • Use angled connectors to keep cables organized in tight spaces and allow for proper airflow.

Interested in learning more about how to improve your data center’s electrical environment? Check out this article: 6 Best Practices for Data Center Power Cords.

[1] Data Foundry