Taming the Data Problem and Accelerating AIOps Implementations with Robotic Data Automation
May 06, 2021

Tejo Prayaga
CloudFabrix

Share this

What is Robotic Data Automation (RDA)?

Robotic Data Automation (RDA) is a new paradigm to help automate data integration and data preparation activities involved in dealing with machine data for Analytics and AI/Machine Learning applications. RDA is not just a framework, but also includes a set of technologies and product capabilities that help implement the data automation.


RDA enables enterprises to operationalize machine data at scale to drive AI & analytics driven decisions.

RDA has broad applicability within the enterprise realm, and to begin with CloudFabrix took the RDA framework and applied it to solve AIOps problems — to help simplify and accelerate AIOps implementations and make them more open and extensible.

RDA automates repetitive data integration, cleaning, verification, shaping, enrichment, and transformation activities using data bots that are invoked to work in succession in“no-code" data workflows or pipelines. RDA helps to move data in and out of AIOps systems easily, thereby simplifying, and accelerating AIOps implementations that otherwise would depend numerous manual data integrations and professional services activities.

Why RDA is Needed?

Artificial Intelligence for IT Operations (AIOps) requires processing vast amounts of data obtained from various hybrid IT data sources, that are spread across on-premises, cloud, and edge environments. This data comes in various formats and delivery modes.

Additionally, results and outcomes of such data processing need to be also exchanged with other tools in the IT ecosystem (Ex: ITSM/Closed loop automation/Collaboration Tools and BI/Reporting tools).

All of this requires integrating, ingesting, preparing, verifying, cleaning, transforming, shaping, analyzing and moving data in and out of AIOps systems in an efficient, reusable, and scalable manner. These essential tasks are most often overlooked in AIOps implementations and cause significant delays and increase costs of AIOps projects.

Challenges

Let us understand what some of the key challenges in data preparation & data integration activities are, when implementing AIOps projects.

■ Different data formats (text/binary/json/XML/CSV), data delivery modes (streaming, batch, bulk, notifications), programmatic interfaces (APIs/Webhooks/Queries/CLIs)

■ Complex data preparation activities involving integrity checks, cleaning, transforming, and shaping the data (aggregating/filtering/sorting)

■ Raw data often lacks application or service context, requiring real-time data enrichment bringing in context from external systems

■ Implementing data workflows require specialized programming/data science skill set

■ Changes in source or destination systems require rewriting/updating connectors

Traditional Approach of Data Handling in AIOps

In the traditional approach, AIOps vendors provide a set of out-of-the-box integrations and once you connect AIOps software to your data sources, you are now pretty much at the mercy of how your data gets utilized, processed for producing results & Outcomes.

■ Black box approach of data acquisition, processing, and integration

■ Use cases and scenarios limited to what the platform supports

■ Integrations mostly predefined/hard coded limiting reuse

■ Complex scripting modules or cookbooks requiring specialized/programming skills (Javscript, Python etc.)

■ Difficult to bring in external integrations for intermittent data processing (ex: enrichment)

■ Difficult to access data in a programmatic way for complementary functions (ex: data access for scripting, reporting, dashboarding, automation etc.)

These are all inhibitors to effective AIOps implementations by way of adding delays & costs (manual data prep/handling activities)

Robotic Data Automation (RDA), a key enabler for AIOps 2.0

RDA automates DataOps, similar to what RPA did to automate business processes. RDA is integral part of AIOps solution that provides augmented data preparation and integration capabilities. RDA is both a data automation framework and a toolkit to accelerate and simplify all data handling in AIOps implementations.

Highlights

■ Implement No-code Data Pipelines using Data bots

■ Native AI/ML bots

■ CFXQL — Uniform Query Language

■ Inline Data Mapping

■ Data Integrity Checks

■ Data masking, redaction, and encryption

■ Data Shaping: Aggregation/Filtering/Sorting

■ Data Extraction/Metrics Harvesting

■ Synthetic Data generation

Benefits

■ Simplify and Accelerate AIOps implementations

■ Reduces time/effort/costs tied to data prep and integrations

■ Suitable for DevOps/ProdOps personnel (no need of data scientist skills)

Tejo Prayaga is Sr. Director of Product Management & Marketing at CloudFabrix
Share this

The Latest

November 28, 2022

Many have assumed that the mainframe is a dying entity, but instead, a mainframe renaissance is underway. Despite this notion, we are ushering in a future of more strategic investments, increased capacity, and leading innovations ...

November 22, 2022

Most (85%) consumers shop online or via a mobile app, with 59% using these digital channels as their primary holiday shopping channel, according to the Black Friday Consumer Report from Perforce Software. As brands head into a highly profitable time of year, starting with Black Friday and Cyber Monday, it's imperative development teams prepare for peak traffic, optimal channel performance, and seamless user experiences to retain and attract shoppers ...

November 21, 2022

From staffing issues to ineffective cloud strategies, NetOps teams are looking at how to streamline processes, consolidate tools, and improve network monitoring. What are some best practices that can help achieve this? Let's dive into five ...

November 18, 2022

On November 1, Taylor Swift announced the Eras Tour ... the whole world is now standing in the same virtual queue, and even the most durable cloud architecture can't handle this level of deluge ...

November 17, 2022

OpenTelemetry, a collaborative open source observability project, has introduced a new network protocol that addresses the infrastructure management headache, coupled with collector configuration options to filter and reduce data volume ...

November 16, 2022

A unified view of digital infrastructure is essential for IT teams that must improve the digital user experience while boosting overall organizational productivity, according to a survey of IT managers in the United Arab Emirates (UAE), from Riverbed and market research firm IDC ...

November 15, 2022

Building the visibility infrastructure to make cloud networks observable is a complex technical challenge. But with careful planning and a few strategic decisions, it's possible to appropriately design, set up and manage visibility solutions for the cloud ...

November 14, 2022

According to a recent IT at Work: 2022 and Beyond study, there have been a few silver linings to the pandemic ... The study revealed some intriguing trends, which will be discussed in turn ...

November 09, 2022

The absence of topology can be a key inhibitor for AIOps tools, creating blind spots for AIOps as they only have access to event data. A topology, an IT service model, or a dependency map is a real-time picture of tools and services that are connected and dependent on each other to deliver an IT service ...

November 08, 2022

A modern data stack is a suite of technologies and apps built specifically to funnel data into an organization, transform it into actionable data, build a plan for acting on that data, and then implement that plan. The majority of modern data stacks are built on cloud-based services, composed of low- and no-code tools that enable a variety of groups within an organization to explore and use their data. Read on to learn how to optimize your data stack ...