Tim seems to be pretty busy now that he’s a free agent, so I’m looking forward to trying this new(ish) gem he announced recently for querying browser performance stats.

I can’t believe it’s been as long as it has since I last posted.  This has mostly been because I’ve been busy preparing to become a dad.  I’m a dad as of two weeks ago, and have a new job, so as things settle down I hope to get through my backlog of posts, mostly on technical topics and tools.

A frequently asked question for Watir seems to be whether there are any “record and playback” tools.  While most of the references point to a somewhat old tool, two viable alternatives seem much harder to find for some reason.

The first is Webmetrics‘ Global Watch Script recorder, available at http://www.webmetrics.com/products/script_recorder.html

The second, is CubicTest.  This uses SeleniumIDE to record the test, but can generate a script for Watir.

(Update) A third is the Testwise Watir Recorder, a Firefox plugin which I looked at a while back, but am unsure of it’s current state.

These will only get you so far, and have their own limitations, the most obvious of which is some corporate applications which only work in Internet Explorer. If a recorder is what you need though, there are a few options and these may get you further.  At any rate, hopefully they’re now easier for people to find.

Jonathan Kohl pointed me at Sikuli, a Python-based tool for automating applications using image recognition.  Unlike most tools, which attempt to identify objects via public APIs, Sikuli looks at the pixels on the screen and attempts to identify objects based on how they look.

This isn’t exactly a new approach, as commercial tools have had this feature for a long time as a means of creating custom objects.  So after playing with Sikuli, I wondered whether I could take advantage of it as a library to augment my Watir scripts in Ruby.

It turns out, the answer is ‘yes’, with a caveat.  You need to use JRuby (although you could probably do it in Ruby using the Ruby-Java bridge – It just looked a lot harder), and you also need to use Watir-Webdriver, a new implementation of Watir’s API which is used in a number of other automation frameworks.

Below is a simple example script and instructions to get you started.  It navigates to a website, then clicks on the flash control there. I’ve so far only tested this on Windows.  It should work on OSX and Linux, but perhaps not quite so easily (I’m waiting on some feedback).  Check the instructions for ‘Install OpenCV’ at the page with instructions for calling Sikuli from other tools.

Sikuli can be used to automate Flash components, any challenging AJAX elements of your web application, to dismiss pop-ups or probably even to inspect visual elements of the page (though I’d want to do this minimally).  It’s a little slow, but an interesting and immediately useful add-on to Watir or your favourite java-based testing tool.


#Install Java, or install the JRuby/JRE bundle at the next step
#Install JRuby 1.5.1 - http://jruby.org/download
#Install Sikuli - http://sikuli.org/download.shtml
#Install watir-webdriver (eg. jgem install watir-webdriver)
#Update ssh - jgem install jruby-openssl
#Copy sikuli-script.jar to \jruby-1.5.1\lib
#Get the test image
#Download http://www.software-testing.com.au/images/flashmap_middleeast.png and put it in the image folder as below
#See http://sikuli.org/trac/wiki/How to use Sikuli Script in your JAVA programs for examples

require ‘rubygems’
require ‘watir-webdriver’
require ‘java’

java_import “org.sikuli.script.SikuliScript”
java_import “org.sikuli.script.Region”
java_import “org.sikuli.script.Screen”

start_page=’http://www.lonelyplanet.com’
image_folder=”c:/sikuli_icons/”

$screen=Screen.new
$browser=Watir::Browser.new :ie
$browser.goto start_page

$screen.click(“#{image_folder}flashmap_middleeast.png”,0)

I’ve been finding popups in Watir a pain of late, with most of the solutions on watir.com not working for me.  Finally today, I found something that worked.  Steve Swanson’s solution did the trick.

However, before it could work, I had to figure out that the AutoIT dll wasn’t correctly registered.  If you try the solution above and find it to be behaving mysteriously, then try the steps on Watir.com’s FAQ page.

Today I found out that my post on Watir and TinyMCE let someone dump their Quick Test Pro licences.

A victory for common sense…

Nothing against QTP, but if you’re testing web applications, you should probably be looking at other tools.

Continuing the ‘what tool’ theme from last week, today’s topic is ‘Diff’.  

I frequently install windows versions of various Unix command line utilities via the Gnu Utilities for Win32 project.  Diff is particularly handy not just for the programming side of automation, but also for comparing output files from automation as well as database queries.  Occasionally though, I need the niceties of a graphical tool that handles side-by-side comparison of file differences a bit more nicely.  So I installed KDiff3 (http://kdiff3.sourceforge.net/), and it seems pretty good, supporting three-way comparisons.

I also looked at Winmerge which has a portable version.  It seems to have a nicer file diff view, but KDiff has a nicer view of folder differences.  Given that it’s portable, it will go onto my tester toolkit (although there’s some evidence KDiff may be portable enough for my needs).

Are there other diff tools I should know about?

When my testing gets technical, there are a lot of things that I only have to grapple with infrequently. Regular expressions are one of those in-again, out-again things for which my expertise varies depending on when you ask me. Today’s Ruby hacking saw me find RegExr which helps you build and test regular expressions, as well as having a lot of ready-to-go samples built into it.  You can also download it as an Air application if you need it locally.

I always seem to end up at regularexpressions.info as well to get help, but had issues with their regular expression tester today.  While the bugs were a hassle, it helped me in a way, because testing your regular expressions with a few different regular expression testers is always a good idea if you think you might need to use them in multiple environments or tools.

Coupled with JEdit or Ruby, regular expressions are a powerful part of your tester toolkit.

What other tools for regular expressions are you using?

A good testing habit when working with web apps is to monitor the log files of servers as you test.  In some cases this is easy, especially where there’s a single application server.  With the trend toward more service-oriented architectures, and server clusters for high-traffic applications, the environments I’m working in tend to have many log files spread over multiple servers and folders.  My old friend, ‘tail -f error.log’ becomes a bit more difficult.

I’d heard about Ruby’s SSH library quite a while back, but only recently have my spare time and my memory conspired to work together. The library lets you connect to a server via SSH, execute commands and see the result. So today I quickly hacked out a script to let me watch multiple (Linux) log files at the same time.  This might work for other environments that support SSH, but I only have it connecting to Linux servers at this time.

Code is below, very hacky, and tailored to my needs.  If you have any questions, leave a comment, but hopefully this helps someone get started!

To install, use the instructions for NET::SSH version one at http://net-ssh.rubyforge.org/ssh/v1/index.html. This will actually install version two though, so you’ll need the docs and examples from http://net-ssh.rubyforge.org/sftp/v2/api/index.html

require 'net/ssh'

class Logfile
  def initialize(name,server,filename)
    @name=name
    @server=server
    @filename=filename
    @current_file_size=0
    @ssh_server='server'
    @username='username'
    @password={:password=>'password'}
  end

  #You'll need to customise this method so that it generates the
  #full path of the log file you're interested in.  This one is for Splunk logs.

  def full_log_path
    today=Time.now
    day=today.day.to_s.rjust(2,'0')
    month=today.month.to_s.rjust(2,'0')
    year=today.year.to_s
    todays_log_folder="#{year}-#{month}-#{day}"
    full_path="/var/log/#{@server}/#{todays_log_folder}/#{@filename}"
    return full_path
  end

  def get_new_lines
    Net::SSH.start(@log_server,@username,@password) do |session|
      new_file_size=session.exec!("wc -l #{full_log_path}").split(" ")[0].to_i
      lines_to_get=new_file_size-@current_file_size

      # Don't generally need to get everything if the error logs are being flooded
      lines_to_get=100 if lines_to_get > 1000 

      new_logs=session.exec!("tail -n #{lines_to_get.to_s} #{full_log_path}")
      @current_file_size=new_file_size
      return new_logs
    end
  end

end

logfiles=[]
logfiles.push Logfile.new(:log_name1,"log_app_folder1","log_file_name1")
logfiles.push Logfile.new(:log_name2,"log_app_folder2","log_file_name2")

5.times do
  logfiles.each do | logfile |
    changes=logfile.get_new_lines
    puts changes if !changes.nil?
    $stdout.flush
  end
  sleep 10
end

After repeated issues trying to get what I thought would be a simple ‘If controller’ working in JMeter, it looks like my issue is to do with the poor examples on the JMeter site. Or else something is making my version and or/installation behave differently.

While the example provided on the JMeter site would have led me to use “${valid_data}”==”Y” as the Condition, with the ‘Interpret Condition as Variable Expression’ field unchecked, doing so was causing a silent failure.

After checking the log files, I could see this error:

“jmeter.control.IfController: missing ; before statement ”

After following some diagnostic advice that failed to work (the log function just seemed to repeatedly spit out my condition as a string, not evaluate it), and randomly experimenting with combinations, I still hadn’t managed to make it work as the log function was also seemingly not passing back the return value. So I removed it and then discovered that I’d somehow managed to make it work. The solution was to add a semicolon at the end:

“${valid_data}”==”Y”;

That was all it took. All good. Hope this helps someone.

I’ve been using the free Squirrel SQL SQL client under windows for a month or so now.  It’s a good tool, though somewhat annoying to get working.  Today it stopped working.  The loading splash screen would display, the progress bar would get about halfway through and then Squirrel would exit without any messages.  I had no desire to recreate all of my connections or reinstall the various drivers again, so I really wanted to fix my installation.

After trolling through forums, there was a suggestion that the problem may have been preferences related. No precise solution was offered, but I began to experiment to see if this was my problem.

First, I found the preferences folder, which lives in Windows’ documents and settings folder (eg. c:\Documents and Settings\). Inside this folder will be Squirrel’s preferences folder, named ‘.squirrel-sql’. I renamed this and restarted Squirrel. Things looked good with the application starting, so it seemed I was looking in the right place. In order to troubleshoot further, I wanted to restore the state of the application, so I renamed the new preferences folder that Squirrel had created and tried to rename the old preferences folder.

No luck! Windows didn’t like me trying to rename the folder back to its original name. I ran Squirrel again, which caused Squirrel to create another preferences folder. I now had three folders – squirrel-sql.old, squirrel-sql.new and the current preferences folder ‘.squirrel-sql’. I opened the old preferences folder, copied the contents and pasted them into the ‘.squirrel-sql’ folder.

Looking inside the preferences folder, I could two folders ‘plugins’, and ‘logs’. I could also see a number of xml files. Now that I had found the broad area I needed to investigate, I wanted to only change one element at a time. As my main objective in resurrecting Squirrel was to not lose my database connections and plugins, I ignored the xml files that were related to these, and looked at the most interestingly named file – ‘prefs.xml’. I renamed this to ‘prefs.xml.bak’ and restarted Squirrel. Still no joy, so I closed Squirrel and restored the original name of the file..

I repeated this step for ’sql_history.xml’, thinking that this file might be dynamic enough to cause problems. Again, Squirrel failed to start correctly.

Next, was a file named SQLAliases23_treeStructure.xml. Suspiciously, this was zero bytes, which seemed odd for something that looked like it was supposed to contain some kind of data structure. I added a ‘.bak’ extension to this and restarted Squirrel again.

Success! I closed Squrrel and I could see that it had recreated the SQLAliases treeStructure file again, this time with data. I restarted Squirrel one more time to make sure that there wasn’t some recurring problem with my database aliases, and it happily started again with my connections and query history intact.