I was asked to do some compliance analysis against a product. Some research had been performed for me and I was to take the page source and submit it to a online tool called The Ruby Accessibility Analysis Kit or Raakt for short. We chose the online submission because we're primarily a Java shop and writing Ruby code would take longer than just submitting to the site and parsing the results.

I spent a couple days coding up Java/Selenium and was able to output to CSV and nicely report issues but it worried me that this was a personal web site I was using for business purposes. I get up today and the URL comes up with "500 Internal Server Error." Ugh... Now I need to figure out how to call Raakt from Java.

I take a look at the Ruby script that is include on the Raakt site to try and see how I can pass things in and get them out of the script. In the original script it just uses the "puts" command to output to the console. I had already coded up a nice CSV output and wanted to get the result back into Java land. I tried various things but finally read something that "use of return is permissisble but unnecessary" in Ruby. So I tried it and lo and behold I get back a result from the script. I'm certain there's a better way but at the moment I'm just trying to get things rolling.

require 'rubygems'
require 'raakt'

# Set up the RAAKT test and pass in source html
raakttest = Raakt::Test.new(page)

#Run all checks and return them
result = raakttest.all
return result

After a little more Googling I find out that there's such a thing as JRuby and it supports Java integration in the direction that I want. I find that the ScriptingContainer needs to know where JRuby home is (after much head banging). Then I'm able to give the script my page source and get back the result. The only lame part is that it comes back as Object. Maybe I'll tokenize the issues that come back to pop them into my CSV as separate lines but I'm running out of billable hours. :)

public static Object validate(String pageSource) {
    ScriptingContainer container = new ScriptingContainer();
    container.setHomeDirectory("/Library/Frameworks/JRuby.framework/Versions/Current");
    container.put("page", pageSource);
    container.put("result", "");
    Object result = container.runScriptlet(PathType.RELATIVE, "./analyze.rb");
    return (result);

ScalaTest with FeatureSpec

The Scala fruits seem to be coming in the way of ScalaTest with FeatureSpec.

In a previous test harness I used cucumber-jvm and JUnit4 to be able to write Given/When/Then BDD style statements. I was excited when I found this but quickly tired of maintaining the code that supported the features. There's a resource directory that contains files with .feature as their extension. Those look like the following.

 Feature: As a user with a valid account I'd like to login to the system with a given role.  
 Scenario: Sign in as a Institutional Administrator role with a valid password.  
   Given a user with email address admin@example.com and password xxx
   When I click Sign In  
   Then I should be on the Institutional Administrator page  

Then a class annotated with @Feature and a custom JUnit Runner... See this post of why the test class and the class containing the steps are different.

 @RunWith(Cucumber.class)  
 @Feature("Login.feature")  
 public class LoginTest {  
 }  

Finally a class this contains the @Given/@When/@Then/@And annotated methods that have regex statements that match the feature file. Since I used JUnit4 I also have my @Before and @After methods in there.

 public class LoginSteps {  
   private WebDriver webDriver;  
   private LoginPage loginPage;  
   private InstitutionalAdminPage institutionalAdminPage;  
   private String username;  
   private String password;  
   @Before  
   public void prepare() {  
     webDriver = new FirefoxDriver();  
     loginPage = new LoginPage(webDriver).get();  
   }  
   @After  
   public void cleanUp() {  
       webDriver.quit();  
   }  
   @Given("^a user with email address (.*) and password (.*)$")  
   public void setLoginValues(String username, String password) {  
     this.username = username;  
     this.password = password;  
   }  
   @When("^I click Sign In")  
   public void signInAsUser() {  
       institutionalAdminPage = loginPage.loginAsInstitutionalAdmin(  
           this.username, this.password);
   }  
   @Then("^I should be on the (.*) page$")  
   public void assertLoginSuccess(String role) {  
     institutionalAdminPage.isLoaded();  
   }  
 }  

Here's essentially the same thing using ScalaTest extended by FeatureSpec with GivenWhenThen. Granted I'm don't have PageObjects coded up in Scala yet but it seems to have promise. One thing I see is that the Gherkin statements are now buried in code. :( This is unfortunate because it affects its maintainability and export to future tools. On the flip side of that upon execution the statements are output to the console so you could potentially "export" them later on.

class ExampleSpec extends FeatureSpec with GivenWhenThen with BeforeAndAfter {
    var driver: FirefoxDriver = _

    before {
        driver = new FirefoxDriver
        driver.get("http://localhost/")
    }

    after {
        driver.quit
    }

    feature("Login as Institutional Administrator") {
        info("As a user with a valid account")
        info("I want to be able to login as a Institutional Administrator")
        info("So that I can perform administrative functions")

        scenario("Attempt to sign in as a Institutional Administrator") {

            given("I am given a user with email admin@example.com and password xxx")
            val usernameField = driver.findElement(By.id("username"))
            usernameField.clear
            usernameField.sendKeys("admin@example.com")
            val passwordField = driver.findElement(By.id("password"))
            passwordField.clear
            passwordField.sendKeys("tissueLocator1")

            when("I click Sign In")
            val signInButton = driver.findElement(By.id("submitLogin"))
            signInButton.click

            then("I should be on the Institutional Administrator page")
            val pageTitle = driver.getTitle
            assert("Arizona Biospecimen Locator" === pageTitle)
        }
    }
}

The experiment shall continue.

OSS immaturity

I guess I'm not the only one...

"Unstable. Consider the blocks that build up Cuke4Duke: JVM, JRuby, RubyGems, Cucumber gem, Gherkin gem, Maven. For the sake of simplicity, let's assume that each of those exists in 4 versions. That means some 15.000 different combinations people might have on their machines. And then there is Windows/Linux/OS X differences on top of that."

Scala, IntelliJ, and Webdriver

I finally got it all working. Successfully writing and executing tests in Scala using Selenium2 (aka Webdriver). The IntelliJ IDE support for Scala seems way better than Eclipse though. Now to see if I can write page objects and use Cucumber. Yay BDD!

sbt

I tweeted a couple days ago that I was diving into Scala again. I did and the language is fine but the tools around it still seem pretty immature. Take for instance the build tool sbt not being backward compatible with each other. Googling for tutorials and grabbing things from github to try out isn't easy but now that I've stumbled a couple times I think I'm pushing through it. I did it with make, ant, and maven so I can do it again.

BrowserMob

In the past I've set up performance environments using the typical injector, system under test, and monitor configuration using tools such as JMeter or The Grinder. Since all the cool kids are in the "cloud" I had to try this stuff out. After some internal querying some people suggested BrowserMob.

The first thing I noticed is that web services aren't the focus of the solution. Oh well. It's not named "WebServiceMob" then I start reading the tutorial. I'm happy it's pretty darn easy to get things set up. You can upload a saved Selenium IDE test case, script one in their editor, or get really fancy and code up JavaScript to control the types of users.

I decided to upload a short Selenium IDE test case. The Selenium IDE source was converted into a BrowserMob script after it was validated.

That's nice but how can I start a load test? After you select "Schedule" a page is presented that lets you turn all sorts of knobs and dials related to virtual user configuration, scripts, script time allocation, and more.

Lots of knobs and dials to configure!




The test plan shows how many concurrent users were planned when the test was scheduled, and how many were actually active in the test run.

Data throughput displays how many bytes were received by transactions that completed during each minute of the test, both as a total number of bytes as well as broken down by each script associated in the test. Often when throughput levels off, you have reached the saturation point of the load test.

The average response time per minute, summarized for the total transaction and also broken down by each step. A site that scales well will have little variance in response time and very few associated errors. Response times that increase as load is applied indicates that the site may have reached its saturation point.

A per-minute summary of the total transactions, the total failures, and a break down of which step each failure took place. Often as response time begins to degrade, more errors will begin to take place. Note: a spike of errors at the end of the test is expected. It simply means that the final transactions were interrupted because the test was shutting down.

 For this particular run through I didn't create any failures but there was also a failure report which showed screenshots.

So I have a bunch of screenshots but how do I present reports to management or accumulate my load test data offsite? There's a MySQL data dump feature that gives you a SQL script that will recreate the data for you. Pretty nice. :)

This seems like a legitimately interesting tool. It didn't take ages to set up or get reports. It did bother me that if Selenium IDE can't do something or if the script won't validate I'd be stuck writing JavaScript. I'm really comfortable writing either Selenium or Selenium 2 code but I can't imagine maintaining JavaScript for multiple user scenarios on an actively developed product. What would really be awesome is something similar to the functionality that Sauce Labs has. If I could write a test harness and scenarios within a class and execute them with the same options as the scheduler that would be really powerful. The other thing I see would be monitoring on the server side to pair up the transaction times and responses with how the server was doing. There's still some collaboration between the cloud and the SUT.

Regardless, I hope to try this out in the recent future on a project and see how it goes!

Geek++

I updated my profile to "geek, husband, daddy, and triathlete" and I really haven't lived up to this in terms of my posting content. Here's to some geek++.

Left Big Toe...

I've been dealing with pain in my left big toe. In fact, it was so bad that I went to the doctor to find out wtf?! After a quick exam I was given a splint, some tape, and an X-ray appointment. The X-ray revealed nothing out of the ordinary and the exam didn't either. My mother suggested it could be gout but a quick email to the doctor confirmed that the likelihood of that is very low.

I have no idea what brought it on but I'm stuck riding the bicycle and using elliptical trainer in the garage. I foresee lots of practice going nowhere for the rest of the winter.

My coach has advised me to get back into the pool but I hate swimming alone. I might try and get involved with the Tacoma Center Master's Swim but the ones I can attend are 5:15-6:00am on Tuesday and Thursday.

Get Creative

"There once was an Indian medicine man whose responsibilities included creating hunting maps for his tribe. Whenever game got sparse, he'd lay a piece of fresh leather out in the sun to dry. Then he'd fold and twist it in his hands, say a few prayers over it, and smooth it out. The rawhide was now crisscrossed with lines and wrinkles. The medicine man marked some basic reference points on the rawhide, and--presto!--a new game map was created. The wrinkles represented new trails the hunters should follow. When the hunters followed the map's newly defined trails, they invariably discovered abundant  game.

Moral: By allowing the rawhide's random folds to represent hunting trails, he pointed the hunters to places they previously had not looked."

--Roger von Oech, A Whack on the Side of the Hea

Motivation

Great quote from the USAT Facebook page today.


“Those who say it cannot be done should not interrupt those who are doing it.”


Someone else on the feed pointed out that it's a snippet from a speech by Pres Theodore Roosevelt on April 23, 1910...


"It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat."