With input from industry experts — both analysts and vendors — this 8-part blog series will explore what is driving the convergence of observability and security, the challenges and advantages, and how it may transform the IT landscape.

Start with: Exploring the Convergence of Observability and Security - Part 1

Start with: Exploring the Convergence of Observability and Security - Part 2: Logs, Metrics and Traces

Start with: Exploring the Convergence of Observability and Security - Part 3: Tools

Start with: Exploring the Convergence of Observability and Security - Part 4: Dashboards

In the previous blog, Part 4 in this series, we examined the convergence of tooling and dashboards. Now we ask the question: Will security teams converge with ITOps, NetOps and DevOps?

"Security needs to be part of operations," says Mike Loukides, VP of Emerging Tech Content at O'Reilly Media. "I don't see any other way to go forward that makes sense. A big problem with security has always been that it was an isolated team, and there was relatively little security expertise on the operations and development teams. That just doesn't make sense. That recipe institutionalized failure. Developers are incentivized to meet deadlines, not write secure code; ops is incentivized to keep the site up, not to keep it safe; and security comes to the end of the budget year saying, 'Nothing bad happened, but it could have, and that's why we need to spend 15% more next year.' These issues disappear when it all becomes a single team."

"For lean IT orgs that have a shared mindset to ITOps and SecOps, or even just smaller teams of dedicated professionals, the train has left the station, and they are already swimming in the huge scope of related responsibilities," Asaf Yigal, CTO of Logz.io adds. "For these orgs there is both the challenge and opportunity to utilize practices from employing shared observability and security in a single platform to building tighter integrations through automated workflows, when possible."

Taking It Slow

Most organizations are maintaining separate groups for IT Ops and security, but the groups are collaborating more often, according to Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at Enterprise Management Associates (EMA). "However, some are reporting convergence of these groups. Usually, it's only a partial convergence, where members of both teams are assigned to task forces that review tools and processes, approve and implement projects, and review and approve changes. A very small percentage of organizations have told EMA that they are fully converging their NOC and SOC into a muti-disciplinary operations center."

McGillicuddy describes multiple challenges to this convergence. First, teams can have issues with each other's data quality. Second, convergence can expose skills gaps that prevent people from collaborating effectively. Third, sometimes they fight over budgets.

"In larger, more mature orgs that might have an active SOC or the like, we see the convergence more around the data and process than the teams themselves," says Yigal from Logz.io. "Everyone has a role in security, and, to a certain extent, everyone has a role in ensuring the constant uptime and performance of the business-critical systems. What they need is stronger partnerships and the right data to share across responsibilities."

"So, for these organizations there's a convergence of the data, the platforms, and the workflows, and this is nothing new," Yigal says. "What is different is that they are being asked to work more closely together to support and secure fast-moving technologies like containers and Kubernetes, and this is driving the need for renewed or expanded partnerships and processes."

However, Yigal sees the convergence as a necessity mostly for smaller organizations. "The reality is that in large organizations with mature security shops, security pros are still going to own security, and they have a near endless array of tools to monitor their systems and defend against threats. At the same time, in smaller orgs or those with less security maturity, observability and security are clearly becoming more centralized. Many organizations do have teams that share oversight of ITOps and security, or DevOps and security, if not DevSecOps. So, it is and has been evolving slowly for years."

Keeping Teams Separate

Some experts argue that teams will remain separate, however.

"Security and performance teams will not necessarily converge," asserts Ajit Sancheti, GM, Falcon LogScale at CrowdStrike. "However, the convergence in visibility tools will reduce friction between DevOps, ITOps and SecOps teams."

Colin Fallwell, Field CTO of Sumo Logic agrees, "I don't see the convergence of teams happening as quickly as the convergence of data and processing. Perhaps some of this could be attributed to cultural differences, however, I attribute this more to the degrees of specialization needed to be an expert in their respective fields."

"Observability is a very wide field, whereas Security is narrower and deeper," Fallwell continues. "I find most operators and developers are contending with many facets at once and cannot be expected to take on the additional role of security and the entire MITRE framework."

"Another facet to consider is what these fields are after in driving outcomes. Operators and developers are focused on efficiency and reliability at high velocity. Security is focused on confidentiality, integrity, and availability. While there are some mutual aspects to this, separation of duties is a good thing. I think we probably want to keep our security specialists focused on their outcomes and our DevOps personas focused on theirs. This ensures that there remains a good system of checks and balances."

"Having said all this, I do think we'll see more security specialists becoming fluent in CI/CD, and Agile methodology, picking up skills in orchestration and automation more than they do today. I also see more operators and developers becoming more fluent in security," Fallwell adds.

"Even if the teams never fully converge, the lines between them will certainly blur over time," says Buddy Brewer, Chief Product Officer at Mezmo. "Before any convergence in teams can happen, however, the data must converge. It is not advisable for teams to let their data get locked away in specific tools, unavailable for other teams doing closely related work, and expect IT and security operations to run smoothly. It's important to eliminate the data divide between security and ITOps teams and establish mechanisms to access relevant data, allowing everyone access to the high-quality data they need."

Clash of the IT Titans

"There are definitely cultural issues between these diverse teams," Adam Hert, Director of Product at Riverbed points out. "Their jobs are vastly different in what they are trying to achieve and how they go about doing them."

"There will probably be some cultural issues here since traditionally security and operational performances have been handled by different teams," Roger Floren, Principal Product Manager at Red Hat agrees. "I don't think it's necessary for teams to converge since much can be done through cross-functional collaboration sharing knowledge and expertise."

Glenn Gray, Director of Product Marketing at Auvik believes that for larger organizations where IT departments are more siloed, the challenges will likely be cultural or heavily influenced by internal power structures. Can the CISO and CIO align their priorities and which take precedence? The CIO might be compelled to focus on providing IT solutions to create positive business outcomes. The CISO might be compelled to adhere to regulatory frameworks. They aren't mutually exclusive in all cases, but can create obstacles for the convergence of security and observability.

The DevSecOps and shift-left promise of efficiency and cost gains is attractive to business leaders because they see it as a way to reduce their security costs and optimize profitability, according to Esteban Gutierrez, CISO & VP, Information Security at New Relic. "While this can indeed be true, building and maintaining effective DevSecOps processes and managing vulnerabilities is still work that must be done. Simply shifting responsibilities to existing engineering teams not only puts more work on their plate, but it results in understaffed security teams to support them, answer their questions, and enable them. In other words, while security considerations have shifted left, so have potential security challenges and blockers. These factors create a perfect environment for an adversarial relationship between security and engineering teams, sabotaging a healthy partnership. Some responsibility falls on security teams — in many instances Security is the 'House of No,' overusing military metaphors to describe fighting righteous battles, rather than enabling business partners and helping them identify viable solutions."

"But some culture issues are systemic and environmental. Security teams set requirements and SLAs and interrupt engineers with unplanned, urgent priorities. Then when Engineering wants Security's help with high-impact work like helping design a secure architecture plan, they get frustrated that they have to wait because there is a backlog of teams seeking expert security services that the security team isn't staffed to keep up with."

Bridging the Cultural Divide

Loukides of O'Reilly Media also foresees some culture issues. "We can't do this because it's unsafe" will clash with "We have to do this because the site needs to stay up." But that's not a reason to avoid converging the teams.

"I don't know how those will be resolved, but we need to have these conversations about security and operations in the same group, not in isolated silos," he advises.

"There are definitely cultural issues to consider," Kirsten Newcomer, Director, Cloud and DevSecOps Strategy at Red Hat agrees. "When previously siloed teams are able to collaborate early and often, everyone learns from each other and new solutions may be found for existing security and performance requirements."

"Security is a rising concern for organizations across all industries, as many of them accumulated more attack surfaces over the last couple years," Gregg Ostrowski, CTO Adviser at Cisco AppDynamics contends. "We spoke with global IT professionals about this shift and the majority have determined a DevSecOps approach to be essential for effectively protecting against security attacks. Without a shared vision among teams, technologists are struggling to keep up with a rapidly changing security landscape and recognize that the convergence between security and observability could help."

"Technologists can no longer operate in silos," Ostrowski continues. "Converging security and observability tools requires teams to take a more unified DevSecOps approach to maintaining the IT stack. Culturally, technologists will need to operate as a multi-discipline team and embrace a more transparent mindset. Instead of focusing strictly on their individual specialty, technologists should look at how their role impacts the business, and they also need to expand their general expertise in other areas of IT as well."

Jam Leomi, Lead Security Engineer at Honeycomb feels it really depends on the business. "Location, industry, size, company culture, and a company's growth path, especially from the logging/monitoring space, really determine the answer to how teams converge or coexist. Leadership and company culture are also strong drivers as to whether security and growth into observability are priorities for the business."

Asaf Yigal of Logz.io agrees, "This is where organizations really are like snowflakes. Try to find two organizations with precisely the same approach to Ops or DevOps or SecOps. Real-world organizations are like the language we use to try to describe them — a set of Lego-like building blocks and responsibilities that they snap together to meet the requirements of their unique environments."

Go to: Exploring the Convergence of Observability and Security - Part 6: Challenges