Taming the Data Problem and Accelerating AIOps Implementations with Robotic Data Automation
May 06, 2021

Tejo Prayaga
CloudFabrix

Share this

What is Robotic Data Automation (RDA)?

Robotic Data Automation (RDA) is a new paradigm to help automate data integration and data preparation activities involved in dealing with machine data for Analytics and AI/Machine Learning applications. RDA is not just a framework, but also includes a set of technologies and product capabilities that help implement the data automation.


RDA enables enterprises to operationalize machine data at scale to drive AI & analytics driven decisions.

RDA has broad applicability within the enterprise realm, and to begin with CloudFabrix took the RDA framework and applied it to solve AIOps problems — to help simplify and accelerate AIOps implementations and make them more open and extensible.

RDA automates repetitive data integration, cleaning, verification, shaping, enrichment, and transformation activities using data bots that are invoked to work in succession in“no-code" data workflows or pipelines. RDA helps to move data in and out of AIOps systems easily, thereby simplifying, and accelerating AIOps implementations that otherwise would depend numerous manual data integrations and professional services activities.

Why RDA is Needed?

Artificial Intelligence for IT Operations (AIOps) requires processing vast amounts of data obtained from various hybrid IT data sources, that are spread across on-premises, cloud, and edge environments. This data comes in various formats and delivery modes.

Additionally, results and outcomes of such data processing need to be also exchanged with other tools in the IT ecosystem (Ex: ITSM/Closed loop automation/Collaboration Tools and BI/Reporting tools).

All of this requires integrating, ingesting, preparing, verifying, cleaning, transforming, shaping, analyzing and moving data in and out of AIOps systems in an efficient, reusable, and scalable manner. These essential tasks are most often overlooked in AIOps implementations and cause significant delays and increase costs of AIOps projects.

Challenges

Let us understand what some of the key challenges in data preparation & data integration activities are, when implementing AIOps projects.

■ Different data formats (text/binary/json/XML/CSV), data delivery modes (streaming, batch, bulk, notifications), programmatic interfaces (APIs/Webhooks/Queries/CLIs)

■ Complex data preparation activities involving integrity checks, cleaning, transforming, and shaping the data (aggregating/filtering/sorting)

■ Raw data often lacks application or service context, requiring real-time data enrichment bringing in context from external systems

■ Implementing data workflows require specialized programming/data science skill set

■ Changes in source or destination systems require rewriting/updating connectors

Traditional Approach of Data Handling in AIOps

In the traditional approach, AIOps vendors provide a set of out-of-the-box integrations and once you connect AIOps software to your data sources, you are now pretty much at the mercy of how your data gets utilized, processed for producing results & Outcomes.

■ Black box approach of data acquisition, processing, and integration

■ Use cases and scenarios limited to what the platform supports

■ Integrations mostly predefined/hard coded limiting reuse

■ Complex scripting modules or cookbooks requiring specialized/programming skills (Javscript, Python etc.)

■ Difficult to bring in external integrations for intermittent data processing (ex: enrichment)

■ Difficult to access data in a programmatic way for complementary functions (ex: data access for scripting, reporting, dashboarding, automation etc.)

These are all inhibitors to effective AIOps implementations by way of adding delays & costs (manual data prep/handling activities)

Robotic Data Automation (RDA), a key enabler for AIOps 2.0

RDA automates DataOps, similar to what RPA did to automate business processes. RDA is integral part of AIOps solution that provides augmented data preparation and integration capabilities. RDA is both a data automation framework and a toolkit to accelerate and simplify all data handling in AIOps implementations.

Highlights

■ Implement No-code Data Pipelines using Data bots

■ Native AI/ML bots

■ CFXQL — Uniform Query Language

■ Inline Data Mapping

■ Data Integrity Checks

■ Data masking, redaction, and encryption

■ Data Shaping: Aggregation/Filtering/Sorting

■ Data Extraction/Metrics Harvesting

■ Synthetic Data generation

Benefits

■ Simplify and Accelerate AIOps implementations

■ Reduces time/effort/costs tied to data prep and integrations

■ Suitable for DevOps/ProdOps personnel (no need of data scientist skills)

Tejo Prayaga is Sr. Director of Product Management & Marketing at CloudFabrix
Share this

The Latest

June 22, 2021

Your employees aren't coming back to the office, at least not in the traditional sense. The pandemic shifted almost all industries into remote work. And according to the results of Ivanti's Everywhere Workplace survey, they're not interested in going back to the way things once were ...

June 21, 2021

Respondents to an OpsRamp survey are moving forward with digital transformation, but many are re-evaluating the number and type of tools they're using. There are three main takeaways from the survey ...

June 17, 2021

More and more mainframe decision makers are becoming aware that the traditional way of handling mainframe operations will soon fall by the wayside. The ever-growing demand for newer, faster digital services has placed increased pressure on data centers to keep up as new applications come online, the volume of data handled continually increases, and workloads become increasingly unpredictable. In a recent Forrester Consulting AIOps survey, commissioned by BMC, the majority of respondents cited that they spend too much time reacting to incidents and not enough time finding ways to prevent them ...

June 16, 2021

In the age of digital transformation, enterprises are migrating to open source software (OSS) in droves to streamline operations and improve customer and employee experiences. However, to unlock the deluge of OSS benefits, it's not enough for organizations to simply implement the software. They must take the necessary steps to build an intentional OSS strategy rooted in ongoing third-party support and training ...

June 15, 2021

In Part 1 of this series, we explored the top pain points associated with managing Internet-based WANs today. This second installment will focus on today's most prevalent SD-WAN deployment challenges specifically and what you can do to better manage modern WANs overall ...

June 14, 2021

Enterprise wide-area networks (WANs) have undergone an incredible transformation over the past several years. More often than not, they're hybrid, offering multiple connection paths between WANs. This provides many benefits but also makes them more challenging to manage than ever before. In Part 1 of this series, we'll explore the top pain points associated with Internet-based WANs ...

June 10, 2021

As we have seen during this digital transformation boom during the pandemic, technologists are managing more applications and data than ever before, which has led three quarters of technologists to be concerned with increased IT complexity. Even more significant, 89% admitted to feeling under immense pressure to keep up with the churn, according to the recent AppDynamics Agents of Transformation report. It's clear that the pandemic has pushed many technologists to their breaking point. To help tackle IT burnout, tech professionals need a "canary" to help them streamline and catch the anomalies before they cause any major performance issues ...

June 09, 2021

An hour-long outage this Tuesday ground the Internet to a halt after popular Content Delivery Network (CDN) provider, Fastly, experienced a glitch that downed Reddit, Spotify, HBO Max, Shopify, Stripe and the BBC, to name just a few of properties affected ...

June 08, 2021

Digital experience has existed for a while now. We have now begun to scratch the surface to measure it. So that calls for Digital Experience Monitoring (DEM). DEM extends Application Performance Monitoring (APM) and Network Performance Management (NPM) to view and optimize application performance issues from the end-user perspective ...

June 07, 2021

The rising adoption of cloud-native architectures, DevOps, and agile methodologies has broken traditional approaches to application security, according to Precise, automatic risk and impact assessment is key for DevSecOps, a new report from Dynatrace, based on an independent global survey of 700 CISOs ...