BlazeMeter launched a continuous testing as a service platform, expanding the company’s portfolio options to meet the full spectrum of performance to mobile testing demands.

The new platform, using cloud and open source tools, reduces the barriers to easily create and run tests and integrate testing into the existing development environment. By facilitating continuous testing throughout the continuous delivery process, the platform enables enterprises to ensure a fast and error free user experience that provides shorter release cycles without compromising software quality.

As software increases in complexity, the number of required tests grows exponentially. While software development and deployment environments have advanced rapidly, testing is often still executed by people in different departments using different tools. To-date, testing has been a complex and resource intensive process with a workflow that is time consuming and extremely hard to maintain. The current process results in inadequate testing, poor software quality and an increased time to release, often compounded by human error.

“This ongoing shift in the industry is forcing developers to adopt agile continuous deployment and integration methodologies and move from perpetual to subscription-based solutions to keep up with the pace of the development cycles,” said Alon Girmonsky, BlazeMeter Founder and CEO. “Our open source and cloud based testing platform helps developers facilitate continuous testing as part of the product delivery cycle.”

BlazeMeter has broken through the barrier with its fully automated continuous testing that can now support dozens of engineering teams committing code, integrating and deploying continuously without compromising software quality. Now, testing can be done in a matter of hours, not days, eliminating friction and human error, and allowing teams to immediately identify and address any problems.

Continuous testing as a service using BlazeMeter’s testing platform, as well as industry standard tools and services such as Sauce Labs, Jenkins, GitHub and New Relic, employ the steps outlined below. Together with Sauce Labs, the two companies provide a complete continuous testing as a service solution for the entire testing lifecycle based on open source software and cloud based services.

“Automated testing is now a critical part of continuous delivery pipelines that are expected to run through complete test suites many times a day, with little tolerance for manual intervention, false failures, or reliability issues related to the test infrastructure," said Steven Hazel, Sauce Labs' Co-Founder and Chief Product Officer. “Combining functional testing from Sauce Labs and performance testing from BlazeMeter gives software teams a modern testing platform capable of meeting the demands of continuous delivery."

Testing starts by the developer using BlazeMeter and/or Sauce Labs to develop a comprehensive set of performance and functional tests that completely validate the quality of the developed module. This set of tests will be used in later stages to continue the validation of the module in the continuous delivery process.

BlazeMeter’s performance and Sauce Labs’ functional testing platforms support reading scripts and running tests with ‘DevOps’ friendly leading open source tools (e.g. JMeter, Selenium) and languages, such as Python, Ruby, or even Shell Scripts. The testing solutions provide libraries and APIs to create “homegrown” tests using common DSLs. With the new continuous testing service, the test creation and the running of the tests can be done with a robust API – everything can be automated regardless of the system you are using. Additionally, developers can replace scripts and use industry standard JSON files, that all developers are familiar with, to configure the load test. This reduces the developer’s need to learn a new language and allows them to perform tests much more quickly. Ideally, test scenarios and accompanying test codes are pulled directly from a source control tool such as GitHub.

A set of tests can be run with different provisioning. For example, a developer can run a module test from behind the firewall with local resources. On production, the same test can come from multi-geographies from the public cloud with or without load. When a test configuration is executed in an environment (e.g. Dev, CI, Pre-Prod, Post-Prod), it can easily adopt a different provisioning scheme. With BlazeMeter’s continuous testing as a service, users can now use the same tool and test definitions across the board, regardless of how they want to interact with the system. The company provides a single common platform regardless of how a user wants to interface with it, while accommodating both GUI and API access.

For an organization to run all required tests in parallel with zero time to test, sufficient resources need to be allocated for all teams according to their use case. BlazeMeter’s continuous testing on any test configuration, as well as on-demand, using simple API calls and running in parallel supports:

● Unlimited resources in public clouds (e.g. GCE, AWS, Azure, HP, Rackspace, Joyent)

● Private Cloud

● Running in the developer’s local environment or data center behind the firewall

BlazeMeter and Sauce Labs’ testing environments seamlessly integrate into existing reporting solutions such as Jenkins Performance Trend and existing standard open source and cloud-based tools in the continuous delivery lifecycle. The combined solutions provide “pass”/”fail” trend reports and “deep dive” reports for diagnostics and analysis. The testing solutions also overlay data from 3rd party systems such as New Relic to present a comprehensive picture.
Alerts In Case of Failures

The testing environments can indicate a “failure”, gather all test artifacts and immediately send them to the group of interested people (e.g. the developers). The solutions provide an alerting scheme per developer, module and project that allows running the exact failed tests all over again to identify the cause for failure.

Taking a snapshot of a build, release or production, provides different modules at different releases. To be able to comprehensively test every such snapshot, BlazeMeter can combine different test configuration fragments into one test configuration. The continuous testing solution now also granularly defines test result thresholds with pass/fail results saved in XML format and consumed by JUnit.
Version Control Friendly Incremental Testing

BlazeMeter’s continuous testing as a service framework is version control friendly and supports incremental testing to associate test configurations, set of tests and tests with versions (e.g code, build, RC, releases).

BlazeMeter’s Continuous Testing as a Service platform is immediately available.