ManageEngine Announces Support for Apache Spark
March 14, 2017
Share this

ManageEngine announced that Applications Manager, its application performance monitoring solution, now supports performance monitoring for Apache Spark.

The move enables development and operations teams in enterprises to gain visibility into the performance of the big data engine as well as the business-critical applications relying on Spark.

Apache Spark is an open source analytics engine built on top of the Hadoop Distributed File System (HDFS). Spark is steadily gaining prominence for its fast data processing capabilities and is being used for data streaming, fog computing, machine learning, and interactive analysis. It is one of the four widely used technologies in the Hadoop ecosystem. However, many components come together to make a Spark application work, so it presents unique complexities in monitoring and troubleshooting.

"Consumer-centric businesses are rapidly deploying data processing engines, such as Spark, to convert tons of data into quick business decisions. As these businesses scale their Spark deployments, it becomes more challenging for operations teams and data scientists to comprehend what is going on,” said Dev Anand, Director of Product Management at ManageEngine. “We want to uncomplicate performance monitoring for big data technologies so businesses can get the most out of their big data projects. That’s why we added performance monitoring and troubleshooting for Apache Spark in Applications Manager. Now, businesses can be more confident about deploying Spark and other big data applications in their production environments."
Proactive Monitoring and Troubleshooting for Apache Spark Clusters

Applications Manager enables comprehensive performance monitoring of the Apache Spark in-memory analytics engine to minimize downtime and performance degradation as well as take corrective actions before any problems arise. Applications Manager monitors key performance indicators of Apache Spark, including indicators related to drivers, executors, RDD blocks, tasks, job stages, CPU and memory usage, and JVM metrics.

The latest monitoring capabilities in Applications Manager help IT personnel:

- View a holistic picture of the health and performance of Apache Spark clusters, big data applications that rely on Spark, and associated infrastructure components using customer, interactive dashboards.

- Diagnose common causes of performance failures in the Spark infrastructure, drill down to their root cause and resolve them quickly before the failure propagates through the Spark infrastructure to the application.

- Gain insights into the overall cluster utilization and resource bottlenecks as well as plan capacity effectively to handle the increasing size and complexity of Spark workloads.

- Ensure Spark applications are consistently delivering a high-quality experience for end users.

Support for Apache Spark complements Applications Manager’s existing monitoring support for the Hadoop platform and key components of the Hadoop ecosystem, such as Apache HBase, Elasticsearch, ZooKeeper, Kafka and Solr search engine. Applications Manager also provides out-of-the-box support for more than 80 applications and infrastructure components, including other NoSQL and in-memory technologies such as Cassandra, MongoDB, Redis, Memcached, Couchbase, Oracle NoSQL, Oracle Coherence and SAP HANA.

Applications Manager 13.2 is available immediately.

Share this

The Latest

June 15, 2018

When it comes to their own companies, 50% of IT stakeholders think they are leaders and will disrupt, while 50% feel they are behind and will be disrupted by the competition in 2018, according to a new survey of IT stakeholders from Alfresco Software and Dimensional Research. The report, Digital Disruption: Disrupt or Be Disrupted, is a wake-up call for the C-suite ...

June 14, 2018

If you are like most IT professionals, which I am sure you are, you are dealing with a lot issues. Typical issues include ...

June 13, 2018

The importance of artificial intelligence and machine learning for customer insight, product support, operational efficiency, and capacity planning are well-established, however, the benefits of monitoring data in those use cases is still evolving. Three main factors obscuring the benefits of data monitoring are the infinite volume of data, its diversity, and inconsistency ...

June 11, 2018

Imagine this: after a fantastic night's sleep, you walk into the office ready to attack the day. You sit down at your desk ready to go, and your computer starts acting up. You call the help desk, but all IT can do is create a ticket for you and transfer it to another team to help you as soon as possible ...

June 08, 2018

As many IT workers develop greater technology skills and apply them to advance their careers, many digital workers in non-IT departments believe their CIO is out of touch with their technology needs. A Gartner, Inc. survey found that less than 50 percent of workers (both IT and non-IT) believe their CIOs are aware of digital technology problems that affect them ...

June 07, 2018

CIOs of 73% of organizations say the need for speed in digital innovation is putting customer experience at risk, according to an independent global survey of 800 CIOs commissioned by Dynatrace ...

June 05, 2018

Digital Transformation requires more than just the latest technology, it's a mindset that iterative change is on the way and should be embraced. This also requires us to factor in the people and process parts of the equation and find ways to measure the end-user-experience (EUE). One way to do this is to sponsor an Application Performance Monitoring (APM) initiative that can provide visibility to the business, help communicate the progress, and highlight the impacts to the organization. ...

June 01, 2018

While DevSecOps, much like DevOps itself, is more about changing IT culture than employing certain types of technology, some tools can be an important support. To find out what the right tools are, DEVOPSdigest asked experts from across the IT industry for their opinions on the top tools to support DevSecOps ...

May 31, 2018

Availability has cemented itself as one of the most essential elements of any successful data center. However today, what many organizations are beginning to realize — sometimes the hard way — is that traditional methodologies and technologies for high availability (HA) have limits. What's needed instead is a new approach that enables the dynamic transfer of workloads in IT environments based on optimizing the particular job at hand ...

May 30, 2018

Self-driving cars, integrated toys, smart home appliances, and even critical infrastructure have all become part of the ecosystem of Internet of Things (IoT) devices, which begs the concerning question, "How will network administrators process all the data generated?" ...