The next observability gap may not be in the code.
It may be under the rack.
That sounds strange until you think about how AI incidents actually feel in the middle of an investigation. Latency moves around. A few accelerator nodes behave differently from their peers. A model-serving tier has a rough hour and then settles down. Logs are not empty, but they are not decisive. Traces show where the pain is showing up, not always why it started.
The application dashboard may be accurate. It may also be stopping at the wrong boundary.
AI systems depend on software, but they also depend on a dense physical stack: racks, power paths, thermal margin, maintenance activity and, in many environments, liquid cooling. Those physical dependencies can change slowly before they look like a software incident.
Application Signals Are Not the Whole Story
Application teams still need the usual evidence: service health, latency, traces, logs, node behavior and workload timing.
None of that becomes less important because the infrastructure stack is changing. It becomes more useful when the physical dependencies around the workload are visible too.
But AI workloads can make the physical context harder to ignore. A group of servers may share a rack, power path or cooling loop. A maintenance event may affect one slice of capacity. A thermal trend may be visible to facilities before the application team sees a clear pattern. A set of nodes may look logically unrelated in software while being physically related in the data hall.
During an incident, those facts are either close at hand or discovered slowly.
More Dashboards Are Not the Answer
The answer is not to push every facilities tag into a software dashboard. That is how teams get noisy dashboards that nobody trusts.
The better question is: which physical facts explain reliability risk?
Which racks support this workload?
Which shared dependencies serve those racks?
Did maintenance happen before the performance change?
Are the affected nodes physically related, or only logically related?
Is the thermal condition sudden, or is it the visible end of a trend that has been moving for weeks?
That is not facilities data for its own sake. It is reliability context.
Liquid Cooling Is a Useful Example
Liquid cooling shows the pattern clearly because some early signals can be quiet.
A loop can show acceptable temperatures while its condition is becoming less predictable. Filters can load. Particles can collect. Chemistry can drift. A loop can be opened during service and then return to normal operation with a new piece of history attached to it.
No operations team needs to become a coolant chemistry team. That is not the point.
The point is that if a cooling-loop dependency can explain workload behavior, the incident team should not have to find that dependency by accident. If a cluster problem follows service work on the same loop, that relationship should be visible. If a filter alarm affects the same racks as a latency complaint, someone should be able to connect those facts quickly.
Observability Needs a Better Map
This is partly a data-context issue, but mostly an operating model issue.
Software, infrastructure and facilities teams often use different systems, different language and different escalation paths. That separation worked better when workloads were less concentrated and physical margin was larger.
AI changes the cost of that separation.
The practical step is dependency mapping. Map workloads to racks. Map racks to shared physical dependencies. Bring maintenance events, alarms and slow-moving coolant or thermal trends into incident review when they can explain service behavior.
The goal is to make physical reliability context available before a small cooling issue becomes a larger operational mystery.
That starts by treating the rack, the loop and the maintenance history as part of the reliability picture.