Swineherd

Swineherd is a useful tool for combining together multiple pig scripts, wukong scripts, and even R scripts into a workflow managed by rake. Here though I'd like to save the actual workflow part for a later post and just illustrate it's uniform interface to these scripts by showing how to launch a wukong script:

#!/usr/bin/env ruby

require 'rake'
require 'swineherd'

task :wukong_job do
  script = WukongScript.new('/path/to/wukong_script')
  script.options = {:some_option => "123", :another_option => "foobar"}
  script.input << '/path/to/input'
  script.output << '/path/to/output'
  script.run
end

You can save this into a file called "Rakefile" and run it by saying:

rake wukong_job