Wednesday, November 17, 2010

Cucumber + Selenium

Yah, Paul Mazak did a great job setting up a Hudson job that kicks off a Cucumber build which runs on Selenium. This was done with the purpose of having some sort of automated regression for our VERY rich ...RIA. Cucumber was meant for BDD, I think, but it seems perfect for automated regression. It is pretty neat to watch the Selenium browser click through the app. Usually, the most challenging part of writing a cucumber script is making selenium(or whatever else you choose your Cucumber to run on (HTMLUnit for example)) understand what element to click on, etc. If your mark-up is messed up, and if it does not have ids on the html elements, you are down for some trial and error until you get the xpath right(assuming you can't just add to the elements you need). However, truly ours Scott Basinger found a nice little FF plugin - the Selenium IDE, which records your interaction with the browser and can also convert that interaction to a unit test code...which has all you need to implement your Cucumber steps in your steps implementations files.

That's all great and groovy, but the issues I'm so far seeing are making me doubt Cucumber would be a good regression tool for our particular application. The reasons:

1. Tests brake intermittently with "No Data Received from server" popup - most likely a result of Selenium clicking too fast on things - We get around it through putting sleep here and there and waitForPageToLoad, whenever applicable. Update[09/26/2011] Credit to Scott Basinger - On a couple of occasions putting sleep in the Steps definition code did not resolve the "No Data Received from server" issue. What did solve it was to let selenium take things a little more slowly itself, as if you would adjust the Fast<->Slow slide bar in the selenium IDE for Firefox.  You accomplish that by calling selenium.setSpeed("1000") [or whatever value works for you] as the first line of code in the selenium test to be executed. Refer to Selenium API for more details.

2. We hardly have 5% of the regression suite converted into Cucumber Features, but it is already taking about 25 mins for the entire build to run - yah, I know - it takes longer when you have to slow things down with sleeps, but it's either that or js error, or just things won't be loaded when selenium is already looking for them

3. Some of the steps are not really reusable
Let's say I want to click on a check box and hit a delete button after that. In our app you can have a checkbox that comes with a delete button on multiple pages. The mark-up structure and the naming conventions are different from one page to another. So if you want your analyst or a QA person write a step
"Delete checkbox item"

you can't just have an implementation such as
@Then("^Delete checkbox item$")
public void deleteCheckboxItem() {"checkboxId");"deleteButtonId");

because the checkbox ids and/or the delete button ids are not the same at the different pages. SO, OK we could use xpath, but no...the structure of the markup is also different. What I end up doing is writing a very specific, not reusable steps/steps implementations....

Number 3 is pretty much unsolvable...I don't think.

1 and 2 though, I should get back and update this post if we figure something out...

No comments:

Post a Comment