Taming the Data Problem and Accelerating AIOps Implementations with Robotic Data Automation
May 06, 2021

Tejo Prayaga
CloudFabrix

Share this

What is Robotic Data Automation (RDA)?

Robotic Data Automation (RDA) is a new paradigm to help automate data integration and data preparation activities involved in dealing with machine data for Analytics and AI/Machine Learning applications. RDA is not just a framework, but also includes a set of technologies and product capabilities that help implement the data automation.


RDA enables enterprises to operationalize machine data at scale to drive AI & analytics driven decisions.

RDA has broad applicability within the enterprise realm, and to begin with CloudFabrix took the RDA framework and applied it to solve AIOps problems — to help simplify and accelerate AIOps implementations and make them more open and extensible.

RDA automates repetitive data integration, cleaning, verification, shaping, enrichment, and transformation activities using data bots that are invoked to work in succession in“no-code" data workflows or pipelines. RDA helps to move data in and out of AIOps systems easily, thereby simplifying, and accelerating AIOps implementations that otherwise would depend numerous manual data integrations and professional services activities.

Why RDA is Needed?

Artificial Intelligence for IT Operations (AIOps) requires processing vast amounts of data obtained from various hybrid IT data sources, that are spread across on-premises, cloud, and edge environments. This data comes in various formats and delivery modes.

Additionally, results and outcomes of such data processing need to be also exchanged with other tools in the IT ecosystem (Ex: ITSM/Closed loop automation/Collaboration Tools and BI/Reporting tools).

All of this requires integrating, ingesting, preparing, verifying, cleaning, transforming, shaping, analyzing and moving data in and out of AIOps systems in an efficient, reusable, and scalable manner. These essential tasks are most often overlooked in AIOps implementations and cause significant delays and increase costs of AIOps projects.

Challenges

Let us understand what some of the key challenges in data preparation & data integration activities are, when implementing AIOps projects.

■ Different data formats (text/binary/json/XML/CSV), data delivery modes (streaming, batch, bulk, notifications), programmatic interfaces (APIs/Webhooks/Queries/CLIs)

■ Complex data preparation activities involving integrity checks, cleaning, transforming, and shaping the data (aggregating/filtering/sorting)

■ Raw data often lacks application or service context, requiring real-time data enrichment bringing in context from external systems

■ Implementing data workflows require specialized programming/data science skill set

■ Changes in source or destination systems require rewriting/updating connectors

Traditional Approach of Data Handling in AIOps

In the traditional approach, AIOps vendors provide a set of out-of-the-box integrations and once you connect AIOps software to your data sources, you are now pretty much at the mercy of how your data gets utilized, processed for producing results & Outcomes.

■ Black box approach of data acquisition, processing, and integration

■ Use cases and scenarios limited to what the platform supports

■ Integrations mostly predefined/hard coded limiting reuse

■ Complex scripting modules or cookbooks requiring specialized/programming skills (Javscript, Python etc.)

■ Difficult to bring in external integrations for intermittent data processing (ex: enrichment)

■ Difficult to access data in a programmatic way for complementary functions (ex: data access for scripting, reporting, dashboarding, automation etc.)

These are all inhibitors to effective AIOps implementations by way of adding delays & costs (manual data prep/handling activities)

Robotic Data Automation (RDA), a key enabler for AIOps 2.0

RDA automates DataOps, similar to what RPA did to automate business processes. RDA is integral part of AIOps solution that provides augmented data preparation and integration capabilities. RDA is both a data automation framework and a toolkit to accelerate and simplify all data handling in AIOps implementations.

Highlights

■ Implement No-code Data Pipelines using Data bots

■ Native AI/ML bots

■ CFXQL — Uniform Query Language

■ Inline Data Mapping

■ Data Integrity Checks

■ Data masking, redaction, and encryption

■ Data Shaping: Aggregation/Filtering/Sorting

■ Data Extraction/Metrics Harvesting

■ Synthetic Data generation

Benefits

■ Simplify and Accelerate AIOps implementations

■ Reduces time/effort/costs tied to data prep and integrations

■ Suitable for DevOps/ProdOps personnel (no need of data scientist skills)

Tejo Prayaga is Sr. Director of Product Management & Marketing at CloudFabrix
Share this

The Latest

June 29, 2022

When it comes to AIOps predictions, there's no question of AI's value in predictive intelligence and faster problem resolution for IT teams. In fact, Gartner has reported that there is no future for IT Operations without AIOps. So, where is AIOps headed in five years? Here's what the vendors and thought leaders in the AIOps space had to share ...

June 27, 2022

A new study by OpsRamp on the state of the Managed Service Providers (MSP) market concludes that MSPs face a market of bountiful opportunities but must prepare for this growth by embracing complex technologies like hybrid cloud management, root cause analysis and automation ...

June 27, 2022

Hybrid work adoption and the accelerated pace of digital transformation are driving an increasing need for automation and site reliability engineering (SRE) practices, according to new research. In a new survey almost half of respondents (48.2%) said automation is a way to decrease Mean Time to Resolution/Repair (MTTR) and improve service management ...

June 23, 2022

Digital businesses don't invest in monitoring for monitoring's sake. They do it to make the business run better. Every dollar spent on observability — every hour your team spends using monitoring tools or responding to what they reveal — should tie back directly to business outcomes: conversions, revenues, brand equity. If they don't? You might be missing the forest for the trees ...

June 22, 2022

Every day, companies are missing customer experience (CX) "red flags" because they don't have the tools to observe CX processes or metrics. Even basic errors or defects in automated customer interactions are left undetected for days, weeks or months, leading to widespread customer dissatisfaction. In fact, poor CX and digital technology investments are costing enterprises billions of dollars in lost potential revenue ...

June 21, 2022

Organizations are moving to microservices and cloud native architectures at an increasing pace. The primary incentive for these transformation projects is typically to increase the agility and velocity of software release and product innovation. These dynamic systems, however, are far more complex to manage and monitor, and they generate far higher data volumes ...

June 16, 2022

Global IT teams adapted to remote work in 2021, resolving employee tickets 23% faster than the year before as overall resolution time for IT tickets went down by 7 hours, according to the Freshservice Service Management Benchmark Report from Freshworks ...

June 15, 2022

Once upon a time data lived in the data center. Now data lives everywhere. All this signals the need for a new approach to data management, a next-gen solution ...

June 14, 2022

Findings from the 2022 State of Edge Messaging Report from Ably and Coleman Parkes Research show that most organizations (65%) that have built edge messaging capabilities in house have experienced an outage or significant downtime in the last 12-18 months. Most of the current in-house real-time messaging services aren't cutting it ...

June 13, 2022
Today's users want a complete digital experience when dealing with a software product or system. They are not content with the page load speeds or features alone but want the software to perform optimally in an omnichannel environment comprising multiple platforms, browsers, devices, and networks. This calls into question the role of load testing services to check whether the given software under testing can perform optimally when subjected to peak load ...