BlazeMeter Introduces Continuous-Testing-as-a-Service
February 03, 2015
Share this

BlazeMeter launched a continuous testing as a service platform, expanding the company’s portfolio options to meet the full spectrum of performance to mobile testing demands.

The new platform, using cloud and open source tools, reduces the barriers to easily create and run tests and integrate testing into the existing development environment. By facilitating continuous testing throughout the continuous delivery process, the platform enables enterprises to ensure a fast and error free user experience that provides shorter release cycles without compromising software quality.

As software increases in complexity, the number of required tests grows exponentially. While software development and deployment environments have advanced rapidly, testing is often still executed by people in different departments using different tools. To-date, testing has been a complex and resource intensive process with a workflow that is time consuming and extremely hard to maintain. The current process results in inadequate testing, poor software quality and an increased time to release, often compounded by human error.

“This ongoing shift in the industry is forcing developers to adopt agile continuous deployment and integration methodologies and move from perpetual to subscription-based solutions to keep up with the pace of the development cycles,” said Alon Girmonsky, BlazeMeter Founder and CEO. “Our open source and cloud based testing platform helps developers facilitate continuous testing as part of the product delivery cycle.”

BlazeMeter has broken through the barrier with its fully automated continuous testing that can now support dozens of engineering teams committing code, integrating and deploying continuously without compromising software quality. Now, testing can be done in a matter of hours, not days, eliminating friction and human error, and allowing teams to immediately identify and address any problems.

Continuous testing as a service using BlazeMeter’s testing platform, as well as industry standard tools and services such as Sauce Labs, Jenkins, GitHub and New Relic, employ the steps outlined below. Together with Sauce Labs, the two companies provide a complete continuous testing as a service solution for the entire testing lifecycle based on open source software and cloud based services.

“Automated testing is now a critical part of continuous delivery pipelines that are expected to run through complete test suites many times a day, with little tolerance for manual intervention, false failures, or reliability issues related to the test infrastructure," said Steven Hazel, Sauce Labs' Co-Founder and Chief Product Officer. “Combining functional testing from Sauce Labs and performance testing from BlazeMeter gives software teams a modern testing platform capable of meeting the demands of continuous delivery."

Testing starts by the developer using BlazeMeter and/or Sauce Labs to develop a comprehensive set of performance and functional tests that completely validate the quality of the developed module. This set of tests will be used in later stages to continue the validation of the module in the continuous delivery process.

BlazeMeter’s performance and Sauce Labs’ functional testing platforms support reading scripts and running tests with ‘DevOps’ friendly leading open source tools (e.g. JMeter, Selenium) and languages, such as Python, Ruby, or even Shell Scripts. The testing solutions provide libraries and APIs to create “homegrown” tests using common DSLs. With the new continuous testing service, the test creation and the running of the tests can be done with a robust API – everything can be automated regardless of the system you are using. Additionally, developers can replace scripts and use industry standard JSON files, that all developers are familiar with, to configure the load test. This reduces the developer’s need to learn a new language and allows them to perform tests much more quickly. Ideally, test scenarios and accompanying test codes are pulled directly from a source control tool such as GitHub.

A set of tests can be run with different provisioning. For example, a developer can run a module test from behind the firewall with local resources. On production, the same test can come from multi-geographies from the public cloud with or without load. When a test configuration is executed in an environment (e.g. Dev, CI, Pre-Prod, Post-Prod), it can easily adopt a different provisioning scheme. With BlazeMeter’s continuous testing as a service, users can now use the same tool and test definitions across the board, regardless of how they want to interact with the system. The company provides a single common platform regardless of how a user wants to interface with it, while accommodating both GUI and API access.

For an organization to run all required tests in parallel with zero time to test, sufficient resources need to be allocated for all teams according to their use case. BlazeMeter’s continuous testing on any test configuration, as well as on-demand, using simple API calls and running in parallel supports:

● Unlimited resources in public clouds (e.g. GCE, AWS, Azure, HP, Rackspace, Joyent)

● Private Cloud

● Running in the developer’s local environment or data center behind the firewall

BlazeMeter and Sauce Labs’ testing environments seamlessly integrate into existing reporting solutions such as Jenkins Performance Trend and existing standard open source and cloud-based tools in the continuous delivery lifecycle. The combined solutions provide “pass”/”fail” trend reports and “deep dive” reports for diagnostics and analysis. The testing solutions also overlay data from 3rd party systems such as New Relic to present a comprehensive picture.
Alerts In Case of Failures

The testing environments can indicate a “failure”, gather all test artifacts and immediately send them to the group of interested people (e.g. the developers). The solutions provide an alerting scheme per developer, module and project that allows running the exact failed tests all over again to identify the cause for failure.

Taking a snapshot of a build, release or production, provides different modules at different releases. To be able to comprehensively test every such snapshot, BlazeMeter can combine different test configuration fragments into one test configuration. The continuous testing solution now also granularly defines test result thresholds with pass/fail results saved in XML format and consumed by JUnit.
Version Control Friendly Incremental Testing

BlazeMeter’s continuous testing as a service framework is version control friendly and supports incremental testing to associate test configurations, set of tests and tests with versions (e.g code, build, RC, releases).

BlazeMeter’s Continuous Testing as a Service platform is immediately available.

Share this

The Latest

March 27, 2024

Nearly all (99%) globa IT decision makers, regardless of region or industry, recognize generative AI's (GenAI) transformative potential to influence change within their organizations, according to The Elastic Generative AI Report ...

March 27, 2024

Agent-based approaches to real user monitoring (RUM) simply do not work. If you are pitched to install an "agent" in your mobile or web environments, you should run for the hills ...

March 26, 2024

The world is now all about end-users. This paradigm of focusing on the end-user was simply not true a few years ago, as backend metrics generally revolved around uptime, SLAs, latency, and the like. DevOps teams always pitched and presented the metrics they thought were the most correlated to the end-user experience. But let's be blunt: Unless there was an egregious fire, the correlated metrics were super loose or entirely false ...

March 25, 2024

This year, New Relic published the State of Observability for Financial Services and Insurance Report to share insights derived from the 2023 Observability Forecast on the adoption and business value of observability across the financial services industry (FSI) and insurance sectors. Here are seven key takeaways from the report ...

March 22, 2024

In MEAN TIME TO INSIGHT Episode 4 - Part 2, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at Enterprise Management Associates (EMA) discusses artificial intelligence and AIOps ...

March 21, 2024

In the course of EMA research over the last twelve years, the message for IT organizations looking to pursue a forward path in AIOps adoption is overall a strongly positive one. The benefits achieved are growing in diversity and value ...

March 20, 2024

Today, as enterprises transcend into a new era of work, surpassing the revolution, they must shift their focus and strategies to thrive in this environment. Here are five key areas that organizations should prioritize to strengthen their foundation and steer themselves through the ever-changing digital world ...

March 19, 2024

If there's one thing we should tame in today's data-driven marketing landscape, this would be data debt, a silent menace threatening to undermine all the trust you've put in the data-driven decisions that guide your strategies. This blog aims to explore the true costs of data debt in marketing operations, offering four actionable strategies to mitigate them through enhanced marketing observability ...

March 18, 2024

Gartner has highlighted the top trends that will impact technology providers in 2024: Generative AI (GenAI) is dominating the technical and product agenda of nearly every tech provider ...

March 15, 2024

In MEAN TIME TO INSIGHT Episode 4 - Part 1, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at Enterprise Management Associates (EMA) discusses artificial intelligence and network management ...