Cloud outages are usually described as technical failures. When a service goes down, a dependency breaks, or a region has issues, the focus immediately shifts to infrastructure.
But if you look closely at how these incidents actually unfold, the root cause is rarely the technology itself. It is almost always tied to decisions made earlier, during design, implementation, or day-to-day operations. The system behaves the way it was built. The real question is how it was built.
The Tradeoffs That Don't Show Up Until Production
Every system starts with tradeoffs. You optimize for speed, simplify architecture to hit deadlines, or prioritize flexibility so teams can move faster. At that stage, those decisions are practical and often necessary.
Over time, those same choices begin to compound. A loosely defined service boundary leads to tighter coupling. A fast-moving pipeline becomes harder to validate. A system designed for iteration accumulates edge cases that no one fully tracks anymore.
These issues rarely surface immediately. They show up later, usually under load, during failure conditions, or when multiple systems interact in unexpected ways.
Google's Site Reliability Engineering work has consistently highlighted that outages are more often tied to system complexity, configuration, and operational decisions than to infrastructure limits.
In 2025, I worked with a major airline on the system design and production scaling of their airfare ticket sales platform. One of the central components was an ActiveMQ broker deployed on Amazon MQ, which served as the data broker between frontend client requests and the backend processing functions. Frontend clients connected directly to the broker, sent their messages, and closed the connection once the transaction completed. The design was straightforward and worked well throughout the development and testing cycle.
However, as peak season approached and the system began handling real production workloads, we noticed growing pressure on the brokers themselves. Specifically, the frontend clients were opening and closing large numbers of connections in very short windows, a pattern commonly known as connection churn. High connection churn is particularly problematic for ActiveMQ brokers and can lead to unstable broker behavior, including resource exhaustion and dropped sessions.
To address this, I worked with the client to build a mesh network across multiple ActiveMQ brokers and place a Network Load Balancer in front of the mesh. This allowed connections to be distributed evenly across nodes, while messages continued to flow freely through federated topics and queues for downstream processing.
Why Tools Don't Fix System Design
Modern cloud environments offer powerful tooling: observability platforms, automated scaling, orchestration layers, and managed services. Teams invest heavily in these tools, expecting them to improve reliability.
What tends to happen instead is that visibility improves while the underlying issues remain. You can see more, but you are still dealing with the same structural problems. Monitoring becomes more detailed, but dependencies are still fragile. Alerts become more precise, but failure modes do not change.
According to the Uptime Institute's 2025 Annual Outage Analysis, human factors such as misconfiguration, process breakdowns, and poor change management remain among the leading contributors to outages. The report notes that the majority of human-error incidents trace back to staff failing to follow procedures or to procedures that were inadequate to begin with. (This sounds a little strange since it is talking about human factor errors, not discussing tooling limitations that lead to production downtimes, should we replace or remove this paragraph? Tools are not equal to human factors).
Tools reflect the systems they observe. They do not replace the need for clear design, ownership, and reasoning about how the system behaves end to end.
One case I worked on captures this dynamic well. The client's business had grown rapidly over several years, to the point that live traffic was more than ten times the volume the original architecture had been designed to handle. They had deployed a RabbitMQ broker to decouple their upstream and downstream processing units, and they had followed the recommended best practices: publisher confirms, connection pooling, and a migration to quorum queues.
Despite all of that, the broker remained limited by the physical capacity of its underlying instance and frequently throttled throughput once live traffic surged. Since the client expected continued growth, a more elastic solution was needed.
That is where we transitioned the workload to a serverless messaging architecture. Serverless services such as Amazon SNS (for broadcast-style notifications) and Amazon SQS (for poll-based queues) fit this elasticity requirement well. The serverless setup natively distributes load across the cloud provider's underlying fleet, offers low-millisecond latency, and is backed by a 99.9% availability SLA, making it a strong fit for this client's growth trajectory.
Reliability Breaks at the Organizational Level
At a certain point, reliability stops being a purely technical concern. It becomes a function of how teams are structured and how they work together. Systems rarely fail in isolation. They fail across boundaries, between services, and through dependencies that span multiple teams.
When ownership is unclear, problems take longer to resolve. When communication is fragmented, teams operate with partial context. When systems grow faster than shared understanding, even small changes can produce unexpected outcomes.
This is why two organizations using similar cloud infrastructure can have very different reliability outcomes. The difference comes from how decisions are made, how assumptions are revisited, and how information flows across the system.
Improving reliability in this environment means treating it as a shared responsibility. It requires aligning teams around system behavior, making assumptions visible, and designing with failure as a normal condition rather than an exception. A team whose knowledge is strictly confined to its own scope, with little visibility into the upstream or downstream services it depends on, will almost always run into reliability issues in production. A lean team can be great for shipping features quickly, but even a lean team needs to sync regularly with its neighbors to stay aware of meaningful shifts in workload, traffic patterns, or interface behavior.
The same applies to incident response. When an incident or even a near miss occurs, both the team that owns the affected service and its upstream and downstream counterparts should be involved in the post-mortem. That is the only way the system, as a whole, can be improved rather than just one slice of it. In other words, a team can be agile, but it should never become a silo. A recurring operational review across adjacent teams is one of the simplest and most effective ways to break down those silos and keep everyone aligned on demand, design constraints, and risk.
Cloud technology has matured significantly. The core infrastructure is no longer the limiting factor it once was. What matters now is how people design systems, manage complexity, and operate under uncertainty.
About the Author: Ran Tao is a Cloud Support Engineer and Fellow of the British Computer Society (BCS) , specializing in cloud enablement and high-availability systems. He has led more than 1,200 enterprise customer engagements and serves as an accredited Subject Matter Expert for cloud messaging technologies and a lead technical interviewer. Tao also contributes to the broader technology community through leadership roles with IEEE initiatives, industry judging, and speaking engagements focused on cloud modernization, resilience engineering, and distributed systems.