{"id":387,"date":"2010-05-19T13:43:57","date_gmt":"2010-05-19T03:43:57","guid":{"rendered":"http:\/\/software-testing.com.au\/blog\/?p=387"},"modified":"2010-05-20T15:51:18","modified_gmt":"2010-05-20T05:51:18","slug":"watchingtailing-multiple-log-files-at-the-same-time-with-rubys-netssh-library","status":"publish","type":"post","link":"http:\/\/www.software-testing.com.au\/blog\/2010\/05\/19\/watchingtailing-multiple-log-files-at-the-same-time-with-rubys-netssh-library\/","title":{"rendered":"Watching\/tailing multiple log files at the same time with Ruby&#8217;s NET::SSH library"},"content":{"rendered":"<p>A good testing habit when working with web apps is to monitor the log files of servers as you test.\u00a0 In some cases this is easy, especially where there&#8217;s a single application server.\u00a0 With the trend toward more service-oriented architectures, and server clusters for high-traffic applications, the environments I&#8217;m working in tend to have many log files spread over multiple servers and folders.\u00a0 My old friend, &#8216;tail -f error.log&#8217; becomes a bit more difficult.<\/p>\n<p>I&#8217;d heard about <a href=\"http:\/\/net-ssh.rubyforge.org\/\">Ruby&#8217;s SSH library<\/a> quite a while back, but only recently have my spare time and my memory conspired to work together. The library lets you connect to a server via SSH, execute commands and see the result. So today I quickly hacked out a script to let me watch multiple (Linux) log files at the same time.\u00a0 This might work for other environments that support SSH, but I only have it connecting to Linux servers at this time.<\/p>\n<p>Code is below, very hacky, and tailored to my needs.\u00a0 If you have any questions, leave a comment, but hopefully this helps someone get started!<\/p>\n<p>To install, use the instructions for NET::SSH version one at http:\/\/net-ssh.rubyforge.org\/ssh\/v1\/index.html.  This will actually install version two though, so you&#8217;ll need the docs and examples from http:\/\/net-ssh.rubyforge.org\/sftp\/v2\/api\/index.html<\/p>\n<pre>require 'net\/ssh'\r\n\r\nclass Logfile\r\n  def initialize(name,server,filename)\r\n    @name=name\r\n    @server=server\r\n    @filename=filename\r\n    @current_file_size=0\r\n    @ssh_server='server'\r\n    @username='username'\r\n    @password={:password=&gt;'password'}\r\n  end\r\n\r\n  #You'll need to customise this method so that it generates the\r\n  #full path of the log file you're interested in.  This one is for Splunk logs.\r\n\r\n  def full_log_path\r\n    today=Time.now\r\n    day=today.day.to_s.rjust(2,'0')\r\n    month=today.month.to_s.rjust(2,'0')\r\n    year=today.year.to_s\r\n    todays_log_folder=\"#{year}-#{month}-#{day}\"\r\n    full_path=\"\/var\/log\/#{@server}\/#{todays_log_folder}\/#{@filename}\"\r\n    return full_path\r\n  end\r\n\r\n  def get_new_lines\r\n    Net::SSH.start(@log_server,@username,@password) do |session|\r\n      new_file_size=session.exec!(\"wc -l #{full_log_path}\").split(\" \")[0].to_i\r\n      lines_to_get=new_file_size-@current_file_size\r\n\r\n      # Don't generally need to get everything if the error logs are being flooded\r\n      lines_to_get=100 if lines_to_get &gt; 1000 \r\n\r\n      new_logs=session.exec!(\"tail -n #{lines_to_get.to_s} #{full_log_path}\")\r\n      @current_file_size=new_file_size\r\n      return new_logs\r\n    end\r\n  end\r\n\r\nend\r\n\r\nlogfiles=[]\r\nlogfiles.push Logfile.new(:log_name1,\"log_app_folder1\",\"log_file_name1\")\r\nlogfiles.push Logfile.new(:log_name2,\"log_app_folder2\",\"log_file_name2\")\r\n\r\n5.times do\r\n  logfiles.each do | logfile |\r\n    changes=logfile.get_new_lines\r\n    puts changes if !changes.nil?\r\n    $stdout.flush\r\n  end\r\n  sleep 10\r\nend\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>A good testing habit when working with web apps is to monitor the log files of servers as you test.\u00a0 In some cases this is easy, especially where there&#8217;s a single application server.\u00a0 With the trend toward more service-oriented architectures, and server clusters for high-traffic applications, the environments I&#8217;m working in tend to have many [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[27,36,90],"tags":[91],"class_list":["post-387","post","type-post","status-publish","format-standard","hentry","category-ruby","category-test-tools","category-tool-assisted-testing","tag-netssh"],"_links":{"self":[{"href":"http:\/\/www.software-testing.com.au\/blog\/wp-json\/wp\/v2\/posts\/387","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.software-testing.com.au\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.software-testing.com.au\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.software-testing.com.au\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.software-testing.com.au\/blog\/wp-json\/wp\/v2\/comments?post=387"}],"version-history":[{"count":10,"href":"http:\/\/www.software-testing.com.au\/blog\/wp-json\/wp\/v2\/posts\/387\/revisions"}],"predecessor-version":[{"id":400,"href":"http:\/\/www.software-testing.com.au\/blog\/wp-json\/wp\/v2\/posts\/387\/revisions\/400"}],"wp:attachment":[{"href":"http:\/\/www.software-testing.com.au\/blog\/wp-json\/wp\/v2\/media?parent=387"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.software-testing.com.au\/blog\/wp-json\/wp\/v2\/categories?post=387"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.software-testing.com.au\/blog\/wp-json\/wp\/v2\/tags?post=387"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}