Comments for Illiteration – Testing times in Software Testing

Comment on Does cucumber suck? by Thinking and Testing with Executable Specs – Part 1 | Stories from a Software Tester

Mon, 05 Dec 2011 13:28:27 +0000

[...] As a tester, when introducing yourself to this area of practice, you might read that You’re Cuking It Wrong or that You’re Cuking It Right. It still may not be clear which you are doing. You might read about being imperative versus declarative and wonder which is for you. (Can’t I do both?) You might even come to the conclusion that Cucumber sucks. [...]

Comment on JMeter annoyances with If Controller – missing ; before statement by Matt

Matt — Thu, 01 Dec 2011 01:13:03 +0000

I also had an issue with the IF Controller silently failing. My issue turned out to be that the IF Controller and it’s child were ignored if the only enabled child is a timer. I added a dummy sampler to the controller, and it started working.

Comment on Automating Flash, AJAX, Popups and more using Ruby, Watir and Sikuli by David

David — Sat, 19 Nov 2011 00:11:54 +0000

Just wanted to mention for those looking to have a test framework to automate something like

sikuli+web driver (or just watir)+mysqldb+jython (or jruby, or other platform)

one option to look at is http://www.robotframework.org. That framework is Jython/Python based, but supports remote library interface to work with test libraries (i.e. modules) built in other platforms. And there are already some prebuilt libraries (in varying stages of maturity) for it that are useful:

watir, Selenium, Selenium2, Sikuli, database access libraries (in Python or Java/Jython)

I also built wrapper library for Sikuli to be used via command line, or integrated for use with Robot Framework, or other tools (over XML-RPC) or to be called from Java as an API:

http://code.google.com/p/simplesikuli/

Comment on Automating Flash, AJAX, Popups and more using Ruby, Watir and Sikuli by Chuckvdl

Chuckvdl — Fri, 21 Oct 2011 16:12:17 +0000

FYI: when installing this on 10/21/11 I ran into an issue getting watir-webdriver installed due to a dependency of selenium-webdriver on v 1.0.9 of FFI.. (it pulled in 1.0.10 which was apparently too new). I had to use the -v switch on jgem install to manually pull down v1.0.9 of FFI first, and then was able to install the watir-webdriver stuff.

Comment on Stop Firefox asking where to save downloads with Watir-Webdriver by changing profile by Jared

Jared — Tue, 16 Aug 2011 12:20:03 +0000

Nice, thanks for the tip.

Comment on Stop Firefox asking where to save downloads with Watir-Webdriver by changing profile by Alister Scott

Alister Scott — Mon, 15 Aug 2011 13:21:19 +0000

What is even better is that you can do this programatically using Watir-WebDriver: http://watirwebdriver.com/browser-downloads/

Comment on Does cucumber suck? by Johnny Rocket

Johnny Rocket — Thu, 28 Apr 2011 04:14:30 +0000

We’ve been using Cucumber for about 9 months at my work. We used it to bang out lots and lots of GUI based tests in a very short period of time. So we wound up putting lots of detail in the tests. They basically turned out to be fancy “recorded” scripts, with things like object ids, link text, test data, etc, etc, right in the feature files. This is of course, entirely our fault, but to Jared’s point, it is the model promoted by the samples. Even our Cucumber expert said “That’s what makes it wonderful. Anybody can read and write it!” Apparently its maintenance be damned and full steam ahead.

In our efforts to refactor we’ve discovered several things which are bringing our test development to a grinding halt.

- Adoption: As Lisa states, we can’t get anybody outside the test team to write a feature file. They have other things to do not write our tests for us. Heck, we can’t even get 1/2 our test team to write them!

- Running Multiple Tests at Once: We’ve got lots of tests now. It can take up to 5-6 hours to run all our tests. So we’ve got the need for running multiple tests in parallel. And we don’t need the simple “run the same test in multiple browsers”. We need real concurrency. If I have 10 tests tagged with “@awesometests” we’d like to see those tests run at once. We can supply the hardware necessary for that, but Cucumber provides no manner for running multiple tests at the same time. So if we want to use Cucumber to do this we’ll have to tear it apart, alter it, and put it back together.

- Sharing Data: Very early we ran into the fact that we need to share data across step definitions. Our in-house Cucumber and Ruby expert suggested that we use global variables, which “should work fine, because all step defs are loaded in to memory and aren’t class instances, so they’ll all see the variables”. Which of course made us all cringe, as this a recipe for lost data, data changed by code that shouldn’t be changing it, one test changing another test’s data… the list goes on and on. Throw concurrency into the mix and the global var theory gets completely blown away.

- Tool support: The only IDE we’ve found that does a decent job of interpreting feature files is IntelliJ… at $600 a pop. A bit hefty for an open source tool.

So now we’ve got lots and lots of tests that have big time data problems, take a good deal of effort to refactor, and take waaaayy to long to run to be useful in a fast-paced agile environment. Some if it is our fault, some of it isn’t. but hey, we’ve got lots of tests! Yay!

Do we dare attempt to tear Cucumber apart and rebuild it to suit our needs? I don’t know….

Comment on Does cucumber suck? by Jared

Jared — Sun, 03 Apr 2011 11:39:42 +0000

That’s basically one of my points. Certainly, discussing test ideas and acceptance criteria prior to developing the story will in most cases be helpful. However, limiting that discussion to behavioural tests is a bit narrow. Specification by example has its own issues. Regardless, you don’t need cucumber to force the interaction you describe.

Another point relevant to your comment is that I feel the extra layers in the cucumber automation framework (as well as the idea that everything should be shoehorned into a behavioural testing framework) really aren’t helpful. Discuss the tests, take what’s important, implement it in the most efficient way you can. Based on my experience, using cucumber as a functional test framework seems a pretty inefficient use of my time.

By all means, put your behavioural tests in there. Do lots of other kinds of tests too though. While Dan North might think “It’s all behaviour”, but just because everything *can* be expressed as behaviour, doesn’t mean it’s efficient or effective to do so.

Comment on Does cucumber suck? by Mark Harris

Mark Harris — Fri, 18 Mar 2011 21:22:27 +0000

We are starting to use Cucumber for testing an agent portal implementation. With contractors and FTE’s mixed doing the specs and coding, and legacy code that we need to interact with, in many cases, the feature files are the best specification of how the product works for us.

When doing our initial proof-of-concept for this effort, we found that the process that the QA Analyst goes through with BA and dev to create the scenarios and steps at the beginning of the iteration forces the team to think through the broader user story. Without forcing that interaction, it’s all to easy to get back into the mold of test at the end of the iteration, and agile devolves into mini-waterfall.

In this case, then, *creating* the feature files is far more valuable than the feature files themselves. And you end up with a regression suite as a side-effect.

Comment on Automating Flash, AJAX, Popups and more using Ruby, Watir and Sikuli by Jared

Jared — Tue, 15 Feb 2011 01:55:07 +0000

I notice that this version in French beat me to the Sikuli/JRuby punch (but not the Watir combination) -

http://algorithmique.net/Snippets/2010/05/26/utiliser-sikuli-avec-jruby.html#

Provides some more examples anyway, particularly using the script part of the library to start a browser (and the fact that it’s on the mac).