BrowserMob

In the past I've set up performance environments using the typical injector, system under test, and monitor configuration using tools such as JMeter or The Grinder. Since all the cool kids are in the "cloud" I had to try this stuff out. After some internal querying some people suggested BrowserMob.

The first thing I noticed is that web services aren't the focus of the solution. Oh well. It's not named "WebServiceMob" then I start reading the tutorial. I'm happy it's pretty darn easy to get things set up. You can upload a saved Selenium IDE test case, script one in their editor, or get really fancy and code up JavaScript to control the types of users.

I decided to upload a short Selenium IDE test case. The Selenium IDE source was converted into a BrowserMob script after it was validated.

That's nice but how can I start a load test? After you select "Schedule" a page is presented that lets you turn all sorts of knobs and dials related to virtual user configuration, scripts, script time allocation, and more.

Lots of knobs and dials to configure!




The test plan shows how many concurrent users were planned when the test was scheduled, and how many were actually active in the test run.

Data throughput displays how many bytes were received by transactions that completed during each minute of the test, both as a total number of bytes as well as broken down by each script associated in the test. Often when throughput levels off, you have reached the saturation point of the load test.

The average response time per minute, summarized for the total transaction and also broken down by each step. A site that scales well will have little variance in response time and very few associated errors. Response times that increase as load is applied indicates that the site may have reached its saturation point.

A per-minute summary of the total transactions, the total failures, and a break down of which step each failure took place. Often as response time begins to degrade, more errors will begin to take place. Note: a spike of errors at the end of the test is expected. It simply means that the final transactions were interrupted because the test was shutting down.

 For this particular run through I didn't create any failures but there was also a failure report which showed screenshots.

So I have a bunch of screenshots but how do I present reports to management or accumulate my load test data offsite? There's a MySQL data dump feature that gives you a SQL script that will recreate the data for you. Pretty nice. :)

This seems like a legitimately interesting tool. It didn't take ages to set up or get reports. It did bother me that if Selenium IDE can't do something or if the script won't validate I'd be stuck writing JavaScript. I'm really comfortable writing either Selenium or Selenium 2 code but I can't imagine maintaining JavaScript for multiple user scenarios on an actively developed product. What would really be awesome is something similar to the functionality that Sauce Labs has. If I could write a test harness and scenarios within a class and execute them with the same options as the scheduler that would be really powerful. The other thing I see would be monitoring on the server side to pair up the transaction times and responses with how the server was doing. There's still some collaboration between the cloud and the SUT.

Regardless, I hope to try this out in the recent future on a project and see how it goes!