BlazeMeter Introduces Continuous-Testing-as-a-Service
February 03, 2015
Share this

BlazeMeter launched a continuous testing as a service platform, expanding the company’s portfolio options to meet the full spectrum of performance to mobile testing demands.

The new platform, using cloud and open source tools, reduces the barriers to easily create and run tests and integrate testing into the existing development environment. By facilitating continuous testing throughout the continuous delivery process, the platform enables enterprises to ensure a fast and error free user experience that provides shorter release cycles without compromising software quality.

As software increases in complexity, the number of required tests grows exponentially. While software development and deployment environments have advanced rapidly, testing is often still executed by people in different departments using different tools. To-date, testing has been a complex and resource intensive process with a workflow that is time consuming and extremely hard to maintain. The current process results in inadequate testing, poor software quality and an increased time to release, often compounded by human error.

“This ongoing shift in the industry is forcing developers to adopt agile continuous deployment and integration methodologies and move from perpetual to subscription-based solutions to keep up with the pace of the development cycles,” said Alon Girmonsky, BlazeMeter Founder and CEO. “Our open source and cloud based testing platform helps developers facilitate continuous testing as part of the product delivery cycle.”

BlazeMeter has broken through the barrier with its fully automated continuous testing that can now support dozens of engineering teams committing code, integrating and deploying continuously without compromising software quality. Now, testing can be done in a matter of hours, not days, eliminating friction and human error, and allowing teams to immediately identify and address any problems.

Continuous testing as a service using BlazeMeter’s testing platform, as well as industry standard tools and services such as Sauce Labs, Jenkins, GitHub and New Relic, employ the steps outlined below. Together with Sauce Labs, the two companies provide a complete continuous testing as a service solution for the entire testing lifecycle based on open source software and cloud based services.

“Automated testing is now a critical part of continuous delivery pipelines that are expected to run through complete test suites many times a day, with little tolerance for manual intervention, false failures, or reliability issues related to the test infrastructure," said Steven Hazel, Sauce Labs' Co-Founder and Chief Product Officer. “Combining functional testing from Sauce Labs and performance testing from BlazeMeter gives software teams a modern testing platform capable of meeting the demands of continuous delivery."

Testing starts by the developer using BlazeMeter and/or Sauce Labs to develop a comprehensive set of performance and functional tests that completely validate the quality of the developed module. This set of tests will be used in later stages to continue the validation of the module in the continuous delivery process.

BlazeMeter’s performance and Sauce Labs’ functional testing platforms support reading scripts and running tests with ‘DevOps’ friendly leading open source tools (e.g. JMeter, Selenium) and languages, such as Python, Ruby, or even Shell Scripts. The testing solutions provide libraries and APIs to create “homegrown” tests using common DSLs. With the new continuous testing service, the test creation and the running of the tests can be done with a robust API – everything can be automated regardless of the system you are using. Additionally, developers can replace scripts and use industry standard JSON files, that all developers are familiar with, to configure the load test. This reduces the developer’s need to learn a new language and allows them to perform tests much more quickly. Ideally, test scenarios and accompanying test codes are pulled directly from a source control tool such as GitHub.

A set of tests can be run with different provisioning. For example, a developer can run a module test from behind the firewall with local resources. On production, the same test can come from multi-geographies from the public cloud with or without load. When a test configuration is executed in an environment (e.g. Dev, CI, Pre-Prod, Post-Prod), it can easily adopt a different provisioning scheme. With BlazeMeter’s continuous testing as a service, users can now use the same tool and test definitions across the board, regardless of how they want to interact with the system. The company provides a single common platform regardless of how a user wants to interface with it, while accommodating both GUI and API access.

For an organization to run all required tests in parallel with zero time to test, sufficient resources need to be allocated for all teams according to their use case. BlazeMeter’s continuous testing on any test configuration, as well as on-demand, using simple API calls and running in parallel supports:

● Unlimited resources in public clouds (e.g. GCE, AWS, Azure, HP, Rackspace, Joyent)

● Private Cloud

● Running in the developer’s local environment or data center behind the firewall

BlazeMeter and Sauce Labs’ testing environments seamlessly integrate into existing reporting solutions such as Jenkins Performance Trend and existing standard open source and cloud-based tools in the continuous delivery lifecycle. The combined solutions provide “pass”/”fail” trend reports and “deep dive” reports for diagnostics and analysis. The testing solutions also overlay data from 3rd party systems such as New Relic to present a comprehensive picture.
Alerts In Case of Failures

The testing environments can indicate a “failure”, gather all test artifacts and immediately send them to the group of interested people (e.g. the developers). The solutions provide an alerting scheme per developer, module and project that allows running the exact failed tests all over again to identify the cause for failure.

Taking a snapshot of a build, release or production, provides different modules at different releases. To be able to comprehensively test every such snapshot, BlazeMeter can combine different test configuration fragments into one test configuration. The continuous testing solution now also granularly defines test result thresholds with pass/fail results saved in XML format and consumed by JUnit.
Version Control Friendly Incremental Testing

BlazeMeter’s continuous testing as a service framework is version control friendly and supports incremental testing to associate test configurations, set of tests and tests with versions (e.g code, build, RC, releases).

BlazeMeter’s Continuous Testing as a Service platform is immediately available.

Share this

The Latest

January 16, 2020

Gartner highlighted the trends that infrastructure and operations (I&O) leaders must start preparing for to support digital infrastructure in 2020 ...

January 15, 2020

Edge computing usage is starting to increase. The obvious follow-up question is, "So, what can I do with edge computing?" I'm glad you asked. There are lots of things you can do ...

January 14, 2020

Industry experts offer predictions on how Network Performance Management (NPM) and related technologies will evolve and impact business in 2020. Part 2 offers predictions about 5G and more ...

January 13, 2020

Industry experts offer predictions on how Network Performance Management (NPM) and related technologies will evolve and impact business in 2020 ...

January 09, 2020

With AI on the edge, companies will more easily monitor desktops, tablets and other end-user devices. AIOps will enable IT to guide employees on improving productivity from the applications installed on their devices while delivering greater visibility and control around the entire IT environment ...

January 08, 2020

2020 will see AIOps adoption going mainstream as use cases crystallize for improving IT efficiencies and supporting faster decision-making. Expect AI-enhanced automation to become smarter and more contextual, move towards the edge, and used increasingly for customer and user experience analysis. Yet there are significant challenges and cautions, which will shape AI's development in not only IT but across business and society ...

January 07, 2020

Industry experts offer predictions on how Digital Transformation will evolve and impact business in 2020 ...

January 06, 2020

Industry experts offer predictions on how ITSM and related technologies will evolve and impact business in 2020 ...

December 19, 2019

Industry experts offer predictions on how APM and related technologies will evolve and impact business in 2020. Part 6 covers log analysis and the cloud ...

December 18, 2019

Industry experts offer predictions on how APM and related technologies will evolve and impact business in 2020. Part 5 covers monitoring ...