Skip to main content

The Data Center: IT's Halloween Fear Factory

Kong Yang

I was working in the data center, late one night
When my eyes beheld an eerie sight
For my infrastructure began to screech
And suddenly there was a breach …

 
Hey, it's Halloween, how could I not take advantage of the opportunity for a little IT-style "Monster Mash?"

The spookiest day of the year is here, and Halloween is actually a good reminder that trouble in the data center is always lurking just beneath the application surface, ready to wreak havoc at any moment.

So, in the spirit of Halloween, SolarWinds recently asked our THWACK community of IT professionals their deepest, darkest IT fears. While some are definitely good for a chuckle, and you'll probably just nod your head in agreement at others, the responses simply allude to the fact that in IT, if things can go wrong, they usually will, and how important it is to be prepared to address the many challenges that exist.

Here are a handful of the responses we got:

■ Stupidity. Yes, my number one fear is stupidity. Not mine, mind you, but others'. For example, I recently walked into a client's site and found a ton of power strips laying on the floor behind their telecom racks. It would have been so easy for someone to have simply tripped over one, unplugging it in the process. Doing so would have caused an outage to an entire manufacturing plant.

■ My greatest IT fear (and fear is general) is clowns on backhoes digging all around our facility looking for that undocumented fiber or twisted pair. (Shudder.)

■ My greatest fear is upper management not understanding the importance of redundancy. Our ERP system is at headquarters and all plants communicate with it almost nonstop for labels and shipping information to get product out. Having headquarters' WAN connection die and then needing to wait for hardware to arrive to replace it would bring business to a halt.

■ My leading fear is our monitoring software either taking an unscheduled dirt nap for some reason or otherwise becoming unavailable. Flying blind in this day and age is a scary proposition. It would indeed be a dark day.

■ Human error can be the worst nightmare of all. I've seen overly enthusiastic electricians and phone technicians cut lines they weren't supposed to.

■ Aside from my technological nightmares, my biggest fear is a server fan eating my beard. There are some devices out there (NexSAN SATABeast, anyone?) that have massive fans, and I've come close to being eaten alive a few times. Aside from the immediate pain, the call to support to get a replacement fan would be quite awkward...

■ My biggest fear? That a "new" and as of yet undetected vulnerability is wreaking havoc in my environment, letting bad guys take whatever data they want even as I write this … and that it then ends up on every news channel with our company logo big and bold.

■ Natural disasters are especially frightening to me as an IT professional, whether it be hurricanes, tornadoes or particularly earthquakes. I'm both curious (and not interested) in finding out the technological ramifications an earthquake would have in our data center and the subsequent ripple (no pun intended … OK, pun intended) effect throughout the organization. Not just server racks, but the smaller stuff, too, like all the creative ways an earthquake would kill spinning hard drives. (Though, if racks are just falling over, hard drives would probably be the least of our worries.)

■ I'm deathly afraid that there's ransomware living on some of our critical production and backup data without us knowing it, and then someone decides to pull the trigger and poof! All of our production and backup data are encrypted.

■ Weather in general has me always freaked out! Water and data centers don't mix well.

While data center-destroying clowns may not be so likely, some of these fears are definitely legitimate. Another fear many systems administrators have is staying relevant today and into the future, especially considering the continual and rapid rate of change we see in the data center (think hyperconvergence, the cloud and hybrid IT, microservices, containers, DevOps, serverless architecture, etc.). So, in closing, I thought I'd provide some advice I think may help:

Develop an application-centric mindset

What matters to the business most is that applications are working well all the time, because every business, and every component of every business, is now dependent on applications. The modern systems administrator needs to think about application uptime and performance first and foremost — end user experience metrics are now part of the CIO's SLA.

Use monitoring with discipline to be the "silent hero"

Given the importance of application uptime and performance, systems and application monitoring needs to become second nature. Systems administrators must implement and manage comprehensive monitoring solutions in order to optimize application performance, realign resources, identify early warning signs of problems and take proactive action. By finding and solving a problem before any end users even know there is a problem, the systems administrator becomes the "silent hero."

Embrace the role of strategic adviser rather than simply remaining a problem fixer

Thanks to the consumerization of technology, the control of many technology decisions has shifted from systems administrator to the end user. This means systems administrators should look to provide insight and advice to all parts of the business to help end users and department leaders make intelligent choices, rather than just responding to tickets.

Learn how to make the right technology decisions for the business

There is a myriad new technologies available to IT: from those mentioned above to IoT to big data. Systems administrators must be smart about choosing the technologies that can truly add value to the business and be able to integrate them when they reach the right level of maturity.

Always keep security top of mind

Whatever a systems administrator does, security needs to be a top priority. The sophistication of attacks is increasing and evolving just as quickly as organizations can prepare for them, sometimes faster. Exacerbating the issue is how much sensitive information companies are storing in today's era of Big Data. And the weakest link remain the end users. Today's systems administrators must continually take steps to ensure the security of their organizations' digital infrastructure.

Kong Yang is a Head Geek at SolarWinds.

The Latest

Performance bottlenecks aren't uncommon when it comes to rolling out new technology, regardless of how capable or game-changing that technology might be. Every generation of new tech has encountered roadblocks that had to be overcome before it was truly able to shine. Virtualization forced organizations to rethink resource allocation, cloud transformation had us shift our focus toward scalability and elasticity, and microservices introduced entirely new challenges around observability and distributed systems. There's something different about AI, however ...

Consider a single order represented across order-management, execution, and settlement systems. Each database, message broker, and application may be online and processing its own records correctly. Yet the workflow has failed if related events arrive on different clocks, rely on inconsistent state, or cannot be reconciled before an operational decision must be made ...

AI now exists in almost every IT workflow. In a recent survey of more than 800 IT service professionals, all respondents indicated the use of AI in some form within their organization. But there's a growing paradox: if dashboards are clearing faster and alerts are resolved at unprecedented speed, why aren't IT service desks reporting lighter workloads? The research found that 71% of IT teams said their actual workload has remained flat or increased since adopting AI. This reality appears to contradict what we’ve been told about AI ...

Two years ago, almost every customer conversation about AI started with the same questions: Which model should we use? What can it do? Is it ready for the enterprise? Today, those discussions have moved on. CIOs are far more interested in how to govern AI, integrate it with existing systems, prepare their workforce and make it part of everyday operations. The challenge is no longer to prove that AI can deliver value. It's instead about how to embed AI into the business in a way that's secure, scalable and delivers measurable outcomes ...

Two things happened to production incidents between 2023 and now, and they did not happen at the same speed. The first is that a class of dependency that barely existed three years ago now accounts for one incident in ten. Incidents disclosed by AI model and AI application providers rose from 1.7% of all disclosed unplanned incidents in 2023 to 10.7% in 2026 year to date, roughly a sixfold rise; that counts only incidents at AI companies themselves, so the true share is higher. The second is that the time to close an incident has not come down ...

When an AI assistant gives an incomplete or incorrect answer, teams often blame the model. They adjust prompts, switch models, increase context windows or test a new retrieval strategy. However the model may not be a problem. In many enterprise AI workflows, the problem begins inside the document-ingestion pipeline ...

If you talk to any security or observability teams right now, they're all fighting the same fire: their tooling was built to ingest X, but their sources are pumping Y and soon to be doing Z. The knee-jerk reaction is always the same: we need more platform. However, this reaction is wrong. Let me explain why, because the solution to this problem is foundational, not financial. Instead of hurling yet more money at the problem, make sure you've done what's needed upstream ...

Rapid AI adoption and the unique ways AI workloads operate is redefining the scope and structure of what these teams must deliver. This shift is forcing organizations to rethink how they manage scale, automation, and control, according to The State of SRE and Platform Engineering 2026, a new report from Dynatrace ...

AI is usually talked about as a software tool, but it also depends heavily on the network behind it. Whether a company is using AI for chatbots, automation, monitoring, analytics, or employee support, all of that information has to move across the network in a reliable and secure way. That means AI is not just an application decision. It is also an infrastructure decision. Before organizations rush into AI, they should ask a simple question: Is our network ready to support it? ...

Enterprise AI often lacks governed access to where business processes actually execute. Without that access, AI agents may be able to reason, but they cannot operate reliably across enterprise workflows. For AI agents to effectively carry out workflows, they will require integration-layer context and controls. Organizations can implement these prerequisites by providing AI with managed access to the middleware layer ...

The Data Center: IT's Halloween Fear Factory

Kong Yang

I was working in the data center, late one night
When my eyes beheld an eerie sight
For my infrastructure began to screech
And suddenly there was a breach …

 
Hey, it's Halloween, how could I not take advantage of the opportunity for a little IT-style "Monster Mash?"

The spookiest day of the year is here, and Halloween is actually a good reminder that trouble in the data center is always lurking just beneath the application surface, ready to wreak havoc at any moment.

So, in the spirit of Halloween, SolarWinds recently asked our THWACK community of IT professionals their deepest, darkest IT fears. While some are definitely good for a chuckle, and you'll probably just nod your head in agreement at others, the responses simply allude to the fact that in IT, if things can go wrong, they usually will, and how important it is to be prepared to address the many challenges that exist.

Here are a handful of the responses we got:

■ Stupidity. Yes, my number one fear is stupidity. Not mine, mind you, but others'. For example, I recently walked into a client's site and found a ton of power strips laying on the floor behind their telecom racks. It would have been so easy for someone to have simply tripped over one, unplugging it in the process. Doing so would have caused an outage to an entire manufacturing plant.

■ My greatest IT fear (and fear is general) is clowns on backhoes digging all around our facility looking for that undocumented fiber or twisted pair. (Shudder.)

■ My greatest fear is upper management not understanding the importance of redundancy. Our ERP system is at headquarters and all plants communicate with it almost nonstop for labels and shipping information to get product out. Having headquarters' WAN connection die and then needing to wait for hardware to arrive to replace it would bring business to a halt.

■ My leading fear is our monitoring software either taking an unscheduled dirt nap for some reason or otherwise becoming unavailable. Flying blind in this day and age is a scary proposition. It would indeed be a dark day.

■ Human error can be the worst nightmare of all. I've seen overly enthusiastic electricians and phone technicians cut lines they weren't supposed to.

■ Aside from my technological nightmares, my biggest fear is a server fan eating my beard. There are some devices out there (NexSAN SATABeast, anyone?) that have massive fans, and I've come close to being eaten alive a few times. Aside from the immediate pain, the call to support to get a replacement fan would be quite awkward...

■ My biggest fear? That a "new" and as of yet undetected vulnerability is wreaking havoc in my environment, letting bad guys take whatever data they want even as I write this … and that it then ends up on every news channel with our company logo big and bold.

■ Natural disasters are especially frightening to me as an IT professional, whether it be hurricanes, tornadoes or particularly earthquakes. I'm both curious (and not interested) in finding out the technological ramifications an earthquake would have in our data center and the subsequent ripple (no pun intended … OK, pun intended) effect throughout the organization. Not just server racks, but the smaller stuff, too, like all the creative ways an earthquake would kill spinning hard drives. (Though, if racks are just falling over, hard drives would probably be the least of our worries.)

■ I'm deathly afraid that there's ransomware living on some of our critical production and backup data without us knowing it, and then someone decides to pull the trigger and poof! All of our production and backup data are encrypted.

■ Weather in general has me always freaked out! Water and data centers don't mix well.

While data center-destroying clowns may not be so likely, some of these fears are definitely legitimate. Another fear many systems administrators have is staying relevant today and into the future, especially considering the continual and rapid rate of change we see in the data center (think hyperconvergence, the cloud and hybrid IT, microservices, containers, DevOps, serverless architecture, etc.). So, in closing, I thought I'd provide some advice I think may help:

Develop an application-centric mindset

What matters to the business most is that applications are working well all the time, because every business, and every component of every business, is now dependent on applications. The modern systems administrator needs to think about application uptime and performance first and foremost — end user experience metrics are now part of the CIO's SLA.

Use monitoring with discipline to be the "silent hero"

Given the importance of application uptime and performance, systems and application monitoring needs to become second nature. Systems administrators must implement and manage comprehensive monitoring solutions in order to optimize application performance, realign resources, identify early warning signs of problems and take proactive action. By finding and solving a problem before any end users even know there is a problem, the systems administrator becomes the "silent hero."

Embrace the role of strategic adviser rather than simply remaining a problem fixer

Thanks to the consumerization of technology, the control of many technology decisions has shifted from systems administrator to the end user. This means systems administrators should look to provide insight and advice to all parts of the business to help end users and department leaders make intelligent choices, rather than just responding to tickets.

Learn how to make the right technology decisions for the business

There is a myriad new technologies available to IT: from those mentioned above to IoT to big data. Systems administrators must be smart about choosing the technologies that can truly add value to the business and be able to integrate them when they reach the right level of maturity.

Always keep security top of mind

Whatever a systems administrator does, security needs to be a top priority. The sophistication of attacks is increasing and evolving just as quickly as organizations can prepare for them, sometimes faster. Exacerbating the issue is how much sensitive information companies are storing in today's era of Big Data. And the weakest link remain the end users. Today's systems administrators must continually take steps to ensure the security of their organizations' digital infrastructure.

Kong Yang is a Head Geek at SolarWinds.

The Latest

Performance bottlenecks aren't uncommon when it comes to rolling out new technology, regardless of how capable or game-changing that technology might be. Every generation of new tech has encountered roadblocks that had to be overcome before it was truly able to shine. Virtualization forced organizations to rethink resource allocation, cloud transformation had us shift our focus toward scalability and elasticity, and microservices introduced entirely new challenges around observability and distributed systems. There's something different about AI, however ...

Consider a single order represented across order-management, execution, and settlement systems. Each database, message broker, and application may be online and processing its own records correctly. Yet the workflow has failed if related events arrive on different clocks, rely on inconsistent state, or cannot be reconciled before an operational decision must be made ...

AI now exists in almost every IT workflow. In a recent survey of more than 800 IT service professionals, all respondents indicated the use of AI in some form within their organization. But there's a growing paradox: if dashboards are clearing faster and alerts are resolved at unprecedented speed, why aren't IT service desks reporting lighter workloads? The research found that 71% of IT teams said their actual workload has remained flat or increased since adopting AI. This reality appears to contradict what we’ve been told about AI ...

Two years ago, almost every customer conversation about AI started with the same questions: Which model should we use? What can it do? Is it ready for the enterprise? Today, those discussions have moved on. CIOs are far more interested in how to govern AI, integrate it with existing systems, prepare their workforce and make it part of everyday operations. The challenge is no longer to prove that AI can deliver value. It's instead about how to embed AI into the business in a way that's secure, scalable and delivers measurable outcomes ...

Two things happened to production incidents between 2023 and now, and they did not happen at the same speed. The first is that a class of dependency that barely existed three years ago now accounts for one incident in ten. Incidents disclosed by AI model and AI application providers rose from 1.7% of all disclosed unplanned incidents in 2023 to 10.7% in 2026 year to date, roughly a sixfold rise; that counts only incidents at AI companies themselves, so the true share is higher. The second is that the time to close an incident has not come down ...

When an AI assistant gives an incomplete or incorrect answer, teams often blame the model. They adjust prompts, switch models, increase context windows or test a new retrieval strategy. However the model may not be a problem. In many enterprise AI workflows, the problem begins inside the document-ingestion pipeline ...

If you talk to any security or observability teams right now, they're all fighting the same fire: their tooling was built to ingest X, but their sources are pumping Y and soon to be doing Z. The knee-jerk reaction is always the same: we need more platform. However, this reaction is wrong. Let me explain why, because the solution to this problem is foundational, not financial. Instead of hurling yet more money at the problem, make sure you've done what's needed upstream ...

Rapid AI adoption and the unique ways AI workloads operate is redefining the scope and structure of what these teams must deliver. This shift is forcing organizations to rethink how they manage scale, automation, and control, according to The State of SRE and Platform Engineering 2026, a new report from Dynatrace ...

AI is usually talked about as a software tool, but it also depends heavily on the network behind it. Whether a company is using AI for chatbots, automation, monitoring, analytics, or employee support, all of that information has to move across the network in a reliable and secure way. That means AI is not just an application decision. It is also an infrastructure decision. Before organizations rush into AI, they should ask a simple question: Is our network ready to support it? ...

Enterprise AI often lacks governed access to where business processes actually execute. Without that access, AI agents may be able to reason, but they cannot operate reliably across enterprise workflows. For AI agents to effectively carry out workflows, they will require integration-layer context and controls. Organizations can implement these prerequisites by providing AI with managed access to the middleware layer ...