Preventing Outages During the Holiday Shopping Season

Michael Butt
Learn more about BigPanda

The most destructive root cause of 75 percent of outages during big online events like Black Friday and Cyber Monday are unplanned configuration changes to a system – when IT and Ops teams find something they think might cause a problem and try to fix it immediately, unintentionally creating a much bigger issue for the web or mobile site.


The following are BigPanda's top recommendations for preventing outages during throughout the entire holiday shopping season:

- Identify the systems that are mission critical to your business. Many companies don't and try to treat their entire system as business critical – and this is a mistake. 

- Have a bulletproof plan for your critical services. Once you've identified what your critical services are, know how to keep them up with a bulletproof plan for them. For instance, if Amazon checkout goes down – you need a disaster and recovery plan for this. But if the Recommendation Engine has problems, this is not at the same level of criticality. 

- Tier your services. Having 3-5 tiers makes prioritization and response much easier, quicker and more effective when there is a problem. And make sure you have a backup and failover plan for the highest tier of your services. 

- You don't need failover for everything. IT and Ops teams who try to have failover for everything often discover that they don't have it ready for anything. 

- Don't become overly focused on the components of infrastructure. Make sure you are spending more time and focus on your services. 

- Make sure you have planned for load capacity. Not planning for the sheer volume of people visiting your web or mobile site accounts for 25 percent of outages during big online events. 

- Use a tool that allows you to consolidate your IT data. Implementing an alert correlation platform allows IT and Ops teams to separate signal from noise and focus more on the customer experience by providing a consolidated view of their IT alert data. This allows them to stop being reactive firefighters and become proactive before an issue has the chance to affect the customer.

Michael Butt is Director of Product Marketing at BigPanda.

Related Links

Hot Topics

Related Vendors

The Latest

Your Observability Stack Has a Telemetry Pipeline Problem

Ask any senior SRE or platform engineer what keeps them up at night, and the answer probably isn't the monitoring tool — it's the data feeding it. The proliferation of APM, observability, and AIOps platforms has created a telemetry sprawl problem that most teams manage reactively rather than architect proactively. Metrics are going to one platform. Traces routed somewhere else. Logs duplicated across multiple backends because nobody wants to be caught without them when something breaks. Every redundant stream costs money ...

Operator to Orchestrator: 80% of IT Pros See Shift in Role as AI Permeates Workflows

80% of respondents agree that the IT role is shifting from operators to orchestrators, according to the 2026 IT Trends Report: The Human Side of Autonomous IT from SolarWinds ...

Gartner: 40% of Organizations Deploying AI Will Use AI Observability to Monitor Model Performance by 2028

40% of organizations deploying AI will implement dedicated AI observability tools by 2028 to monitor model performance, bias and outputs, according to Gartner ...

Almost Half of AI-Generated Code Fails in Production

Until AI-powered engineering tools have live visibility of how code behaves at runtime, they cannot be trusted to autonomously ensure reliable systems, according to the State of AI-Powered Engineering Report 2026 report from Lightrun. The report reveals that a major volume of manual work is required when AI-generated code is deployed: 43% of AI-generated code requires manual debugging in production, even after passing QA or staging tests. Furthermore, an average of three manual redeploy cycles are required to verify a single AI-suggested code fix in production ...

AI Requires Design, Not Edicts

Many organizations describe AI as strategic, but they do not manage it strategically. When AI plans are disconnected from strategy, detached from organizational learning, and protected from serious assumptions testing, the problem is no longer technical immaturity; it is a failure of management discipline ... Executives too often tell organizations to "use AI" before they define what AI is supposed to change. The problem deepens in organizations where strategy isn't well articulated in the first place ...

How Intelligent Orchestration Enables Enterprises to Move from AI POC to AI Production

Across the enterprise technology landscape, a quiet crisis is playing out. Organizations have run hundreds, sometimes thousands, of generative AI pilots. Leadership has celebrated the proof of concept (POCs) ... Industry experience points to a sobering reality: only 5-10% of AI POCs that progress to the pilot stage successfully reach scaled production. The remaining 90% fail because the enterprise environment around them was never ready to absorb them, not the AI models ...

Reliability Is the New Bottleneck of Innovation

Today's modern systems are not what they once were. Organizations now rely on distributed systems, event-driven workflows, hybrid and multi-cloud environments and continuous delivery pipelines. While each adds flexibility, it also introduces new, often invisible failures. Development speed is no longer the primary bottleneck of innovation. Reliability is ...

5 Takeaways from the Observability Forecast for Retail and eCommerce

Seeing is believing, or in this case, seeing is understanding, according to New Relic's 2025 Observability Forecast for Retail and eCommerce report. Retailers who want to provide exceptional customer experiences while improving IT operations efficiency are leaning on observability ... Here are five key takeaways from the report ...

Maximizing Impact Amid Constraints: The Role of Automation and Orchestration in Federal IT Modernization

Technology leaders across the federal landscape are facing, and will continue to face, an uphill battle when it comes to fortifying their digital environments against hostile and persistent threat actors. On one hand, they are being asked to push digital transformation ... On the other hand, they are facing the fiscal uncertainty of continuing resolutions (CR) and government shutdowns looming near and far. In the face of these challenges, CIOs, CTOs, and CISOs must figure out how to modernize legacy systems and infrastructure while doing more with less and still defending against external and internal threats ...

The SRE Report 2026: Reliability Is Being Redefined

Reliability is no longer proven by uptime alone, according to the The SRE Report 2026 from LogicMonitor. In the AI era, it is experienced through speed, consistency, and user trust, and increasingly judged by business impact. As digital services grow more complex and AI systems move into production, traditional monitoring approaches are struggling to keep pace, increasing the need for AI-first observability that spans applications, infrastructure, and the Internet ...

Preventing Outages During the Holiday Shopping Season

Michael Butt
Learn more about BigPanda

The most destructive root cause of 75 percent of outages during big online events like Black Friday and Cyber Monday are unplanned configuration changes to a system – when IT and Ops teams find something they think might cause a problem and try to fix it immediately, unintentionally creating a much bigger issue for the web or mobile site.


The following are BigPanda's top recommendations for preventing outages during throughout the entire holiday shopping season:

- Identify the systems that are mission critical to your business. Many companies don't and try to treat their entire system as business critical – and this is a mistake. 

- Have a bulletproof plan for your critical services. Once you've identified what your critical services are, know how to keep them up with a bulletproof plan for them. For instance, if Amazon checkout goes down – you need a disaster and recovery plan for this. But if the Recommendation Engine has problems, this is not at the same level of criticality. 

- Tier your services. Having 3-5 tiers makes prioritization and response much easier, quicker and more effective when there is a problem. And make sure you have a backup and failover plan for the highest tier of your services. 

- You don't need failover for everything. IT and Ops teams who try to have failover for everything often discover that they don't have it ready for anything. 

- Don't become overly focused on the components of infrastructure. Make sure you are spending more time and focus on your services. 

- Make sure you have planned for load capacity. Not planning for the sheer volume of people visiting your web or mobile site accounts for 25 percent of outages during big online events. 

- Use a tool that allows you to consolidate your IT data. Implementing an alert correlation platform allows IT and Ops teams to separate signal from noise and focus more on the customer experience by providing a consolidated view of their IT alert data. This allows them to stop being reactive firefighters and become proactive before an issue has the chance to affect the customer.

Michael Butt is Director of Product Marketing at BigPanda.

Related Links

Hot Topics

Related Vendors

The Latest

Your Observability Stack Has a Telemetry Pipeline Problem

Ask any senior SRE or platform engineer what keeps them up at night, and the answer probably isn't the monitoring tool — it's the data feeding it. The proliferation of APM, observability, and AIOps platforms has created a telemetry sprawl problem that most teams manage reactively rather than architect proactively. Metrics are going to one platform. Traces routed somewhere else. Logs duplicated across multiple backends because nobody wants to be caught without them when something breaks. Every redundant stream costs money ...

Operator to Orchestrator: 80% of IT Pros See Shift in Role as AI Permeates Workflows

80% of respondents agree that the IT role is shifting from operators to orchestrators, according to the 2026 IT Trends Report: The Human Side of Autonomous IT from SolarWinds ...

Gartner: 40% of Organizations Deploying AI Will Use AI Observability to Monitor Model Performance by 2028

40% of organizations deploying AI will implement dedicated AI observability tools by 2028 to monitor model performance, bias and outputs, according to Gartner ...

Almost Half of AI-Generated Code Fails in Production

Until AI-powered engineering tools have live visibility of how code behaves at runtime, they cannot be trusted to autonomously ensure reliable systems, according to the State of AI-Powered Engineering Report 2026 report from Lightrun. The report reveals that a major volume of manual work is required when AI-generated code is deployed: 43% of AI-generated code requires manual debugging in production, even after passing QA or staging tests. Furthermore, an average of three manual redeploy cycles are required to verify a single AI-suggested code fix in production ...

AI Requires Design, Not Edicts

Many organizations describe AI as strategic, but they do not manage it strategically. When AI plans are disconnected from strategy, detached from organizational learning, and protected from serious assumptions testing, the problem is no longer technical immaturity; it is a failure of management discipline ... Executives too often tell organizations to "use AI" before they define what AI is supposed to change. The problem deepens in organizations where strategy isn't well articulated in the first place ...

How Intelligent Orchestration Enables Enterprises to Move from AI POC to AI Production

Across the enterprise technology landscape, a quiet crisis is playing out. Organizations have run hundreds, sometimes thousands, of generative AI pilots. Leadership has celebrated the proof of concept (POCs) ... Industry experience points to a sobering reality: only 5-10% of AI POCs that progress to the pilot stage successfully reach scaled production. The remaining 90% fail because the enterprise environment around them was never ready to absorb them, not the AI models ...

Reliability Is the New Bottleneck of Innovation

Today's modern systems are not what they once were. Organizations now rely on distributed systems, event-driven workflows, hybrid and multi-cloud environments and continuous delivery pipelines. While each adds flexibility, it also introduces new, often invisible failures. Development speed is no longer the primary bottleneck of innovation. Reliability is ...

5 Takeaways from the Observability Forecast for Retail and eCommerce

Seeing is believing, or in this case, seeing is understanding, according to New Relic's 2025 Observability Forecast for Retail and eCommerce report. Retailers who want to provide exceptional customer experiences while improving IT operations efficiency are leaning on observability ... Here are five key takeaways from the report ...

Maximizing Impact Amid Constraints: The Role of Automation and Orchestration in Federal IT Modernization

Technology leaders across the federal landscape are facing, and will continue to face, an uphill battle when it comes to fortifying their digital environments against hostile and persistent threat actors. On one hand, they are being asked to push digital transformation ... On the other hand, they are facing the fiscal uncertainty of continuing resolutions (CR) and government shutdowns looming near and far. In the face of these challenges, CIOs, CTOs, and CISOs must figure out how to modernize legacy systems and infrastructure while doing more with less and still defending against external and internal threats ...

The SRE Report 2026: Reliability Is Being Redefined

Reliability is no longer proven by uptime alone, according to the The SRE Report 2026 from LogicMonitor. In the AI era, it is experienced through speed, consistency, and user trust, and increasingly judged by business impact. As digital services grow more complex and AI systems move into production, traditional monitoring approaches are struggling to keep pace, increasing the need for AI-first observability that spans applications, infrastructure, and the Internet ...