Jonathan Kohl pointed me at Sikuli, a Python-based tool for automating applications using image recognition.  Unlike most tools, which attempt to identify objects via public APIs, Sikuli looks at the pixels on the screen and attempts to identify objects based on how they look.

This isn’t exactly a new approach, as commercial tools have had this feature for a long time as a means of creating custom objects.  So after playing with Sikuli, I wondered whether I could take advantage of it as a library to augment my Watir scripts in Ruby.

It turns out, the answer is ‘yes’, with a caveat.  You need to use JRuby (although you could probably do it in Ruby using the Ruby-Java bridge – It just looked a lot harder), and you also need to use Watir-Webdriver, a new implementation of Watir’s API which is used in a number of other automation frameworks.

Below is a simple example script and instructions to get you started.  It navigates to a website, then clicks on the flash control there. I’ve so far only tested this on Windows.  It should work on OSX and Linux, but perhaps not quite so easily (I’m waiting on some feedback).  Check the instructions for ‘Install OpenCV’ at the page with instructions for calling Sikuli from other tools.

Sikuli can be used to automate Flash components, any challenging AJAX elements of your web application, to dismiss pop-ups or probably even to inspect visual elements of the page (though I’d want to do this minimally).  It’s a little slow, but an interesting and immediately useful add-on to Watir or your favourite java-based testing tool.


#Install Java, or install the JRuby/JRE bundle at the next step
#Install JRuby 1.5.1 - http://jruby.org/download
#Install Sikuli - http://sikuli.org/download.shtml
#Install watir-webdriver (eg. jgem install watir-webdriver)
#Update ssh - jgem install jruby-openssl
#Copy sikuli-script.jar to \jruby-1.5.1\lib
#Get the test image
#Download http://www.software-testing.com.au/images/flashmap_middleeast.png and put it in the image folder as below
#See http://sikuli.org/trac/wiki/How to use Sikuli Script in your JAVA programs for examples

require ‘rubygems’
require ‘watir-webdriver’
require ‘java’

java_import “org.sikuli.script.SikuliScript”
java_import “org.sikuli.script.Region”
java_import “org.sikuli.script.Screen”

start_page=’http://www.lonelyplanet.com’
image_folder=”c:/sikuli_icons/”

$screen=Screen.new
$browser=Watir::Browser.new :ie
$browser.goto start_page

$screen.click(“#{image_folder}flashmap_middleeast.png”,0)

Continuing the ‘what tool’ theme from last week, today’s topic is ‘Diff’.  

I frequently install windows versions of various Unix command line utilities via the Gnu Utilities for Win32 project.  Diff is particularly handy not just for the programming side of automation, but also for comparing output files from automation as well as database queries.  Occasionally though, I need the niceties of a graphical tool that handles side-by-side comparison of file differences a bit more nicely.  So I installed KDiff3 (http://kdiff3.sourceforge.net/), and it seems pretty good, supporting three-way comparisons.

I also looked at Winmerge which has a portable version.  It seems to have a nicer file diff view, but KDiff has a nicer view of folder differences.  Given that it’s portable, it will go onto my tester toolkit (although there’s some evidence KDiff may be portable enough for my needs).

Are there other diff tools I should know about?

When my testing gets technical, there are a lot of things that I only have to grapple with infrequently. Regular expressions are one of those in-again, out-again things for which my expertise varies depending on when you ask me. Today’s Ruby hacking saw me find RegExr which helps you build and test regular expressions, as well as having a lot of ready-to-go samples built into it.  You can also download it as an Air application if you need it locally.

I always seem to end up at regularexpressions.info as well to get help, but had issues with their regular expression tester today.  While the bugs were a hassle, it helped me in a way, because testing your regular expressions with a few different regular expression testers is always a good idea if you think you might need to use them in multiple environments or tools.

Coupled with JEdit or Ruby, regular expressions are a powerful part of your tester toolkit.

What other tools for regular expressions are you using?

A good testing habit when working with web apps is to monitor the log files of servers as you test.  In some cases this is easy, especially where there’s a single application server.  With the trend toward more service-oriented architectures, and server clusters for high-traffic applications, the environments I’m working in tend to have many log files spread over multiple servers and folders.  My old friend, ‘tail -f error.log’ becomes a bit more difficult.

I’d heard about Ruby’s SSH library quite a while back, but only recently have my spare time and my memory conspired to work together. The library lets you connect to a server via SSH, execute commands and see the result. So today I quickly hacked out a script to let me watch multiple (Linux) log files at the same time.  This might work for other environments that support SSH, but I only have it connecting to Linux servers at this time.

Code is below, very hacky, and tailored to my needs.  If you have any questions, leave a comment, but hopefully this helps someone get started!

To install, use the instructions for NET::SSH version one at http://net-ssh.rubyforge.org/ssh/v1/index.html. This will actually install version two though, so you’ll need the docs and examples from http://net-ssh.rubyforge.org/sftp/v2/api/index.html

require 'net/ssh'

class Logfile
  def initialize(name,server,filename)
    @name=name
    @server=server
    @filename=filename
    @current_file_size=0
    @ssh_server='server'
    @username='username'
    @password={:password=>'password'}
  end

  #You'll need to customise this method so that it generates the
  #full path of the log file you're interested in.  This one is for Splunk logs.

  def full_log_path
    today=Time.now
    day=today.day.to_s.rjust(2,'0')
    month=today.month.to_s.rjust(2,'0')
    year=today.year.to_s
    todays_log_folder="#{year}-#{month}-#{day}"
    full_path="/var/log/#{@server}/#{todays_log_folder}/#{@filename}"
    return full_path
  end

  def get_new_lines
    Net::SSH.start(@log_server,@username,@password) do |session|
      new_file_size=session.exec!("wc -l #{full_log_path}").split(" ")[0].to_i
      lines_to_get=new_file_size-@current_file_size

      # Don't generally need to get everything if the error logs are being flooded
      lines_to_get=100 if lines_to_get > 1000 

      new_logs=session.exec!("tail -n #{lines_to_get.to_s} #{full_log_path}")
      @current_file_size=new_file_size
      return new_logs
    end
  end

end

logfiles=[]
logfiles.push Logfile.new(:log_name1,"log_app_folder1","log_file_name1")
logfiles.push Logfile.new(:log_name2,"log_app_folder2","log_file_name2")

5.times do
  logfiles.each do | logfile |
    changes=logfile.get_new_lines
    puts changes if !changes.nil?
    $stdout.flush
  end
  sleep 10
end