Al Hoang

Yet another reason to like Open Source

One thing I can say that I really enjoy about working with an Open Source project rather than commercial software is I can feel free to blog as many details as I want about getting something done in the hopes it helps me remember what I’ve done and perhaps be useful to others. No need to wrack my brains on whether I can or can’t release the information due to NDAs or other stupidity. The only thing holding me back is my laziness to actually say what I’ve done.

Not happy with the current Rakefile

Today, once again I couldn’t come up with a good idea how to start writing the documentation so I ended taking a detour and staring at the Rakefile (It’s pretty much a copy and paste of the Programming Ruby with parameters changed):

require 'rubygems'
Gem::manage_gems
require 'rake/gempackagetask'

spec = Gem::Specification.new do |s|
    s.name       = "openwferu"
    s.version    = "0.9.0"
    s.author     = "John Mettraux"
    s.email      = "john at openwfe dot org"
    s.homepage   = "http://rubyforge.org/projects/openwferu"
    s.platform   = Gem::Platform::RUBY
    s.summary    = "An OpenWFE-compatible workflow engine in Ruby"
    s.files      = FileList["{bin,docs,lib,test}/**/*"].exclude("rdoc").to_a
    s.require_path      = "lib"
    s.autorequire       = "openwferu"
    s.test_file         = "test/runtest.rb"
    s.has_rdoc          = true
    s.extra_rdoc_files  = ['README']
end

Rake::GemPackageTask.new(spec) do |pkg|
    pkg.need_tar = true
end

While the above Rakefile is quite short and is capable of making a gem via rake openwferu-0.9.0.gen I wasn’t happy I had to explicitly type out the name all the time. However, the Pickaxe book stopped at this point so guidance after this stage relied on my own motivation to go find out where I need to look next.

Getting Rake to build the gem easily

When I typed rake I got an error. I understood this to mean that rake had no idea what the default task so just bailed. So my current goal became, learning how to make the default task in a Rakefile create a gem.

I didn’t really know where to start so I flipped through the Rake Documentation and stared at the Rake API docs. After diddling around trying to use the User Guide and Tutorial to get me somewhere (they didn’t) I stared at the API again. This time I focused on the GemPackageTask object and what the heck it was doing. The above Rakefile creates a new instance of GemPackageTask and sends it a block. Okay fair enough. I tried adding a bit of stuff to get the Rakefile to default to building the package but was getting more confused. What was that block doing exactly anyways? I mean, glancing at it I sort of understood it but what was pkg really?

After digging through the source in the API docs (rdoc is really awesome for giving you quick access to the source code like this) I finally realized that the pkg variable is actually a reference to GemPackageTask itself however I couldn’t find need_tar in GemPackageTask. However, if you look at GemPackageTask’s parent PackageTask many of the details became much clearer. PackageTask holds the need_tar attribute which tells PackageTask to create a gzipped tarball.

One other thing that also became clear is that PackageTask also creates 3 extra tasks automatically on instantiation:

• :package
• :clobber_package
• :repackage

What this means is you get the following 3 commands available in rake automagically:

• rake package
• rake clobber_package
• rake repackage

At this point the dim light bulb in my head turns on. I only need to add one lousy line to the Rakefile:

task :default => [:package]

That’s it you say? Yep. That’s it. Now rake will default to creating a package if you type rake. Remember you also have rake package, rake repackage, rake clobber_package AND rake gem. When you create a task, it means you can also call that task from the rake utility by feeding it as a parameter to rake.

Now the Rakefile (gem spec elided) looks like this:

require 'rubygems'
Gem::manage_gems
require 'rake/gempackagetask'

spec = Gem::Specification.new do |s|
  ...
end

task :default => [:package]

Rake::GemPackageTask.new(spec) do |pkg|
    pkg.need_tar = true
end

Nifty, eh?

Cleaning up

Now that I was creating packages on the whim. I really wanted to be sure they got cleaned out so after a little more looking at the Rake documentation I noticed it was only an extra line I needed to add to indicate to the clean task which files I wanted deleted. The clean task is a built-in freebie with Rake however without some prodding on what to delete it won’t do much. I would suggest you be very careful not to setup an expression like CLEAN.include("**/*.rb") unless you’re itching for a rewrite. Here’s what I added above the Gem Specification object.

CLEAN.include(”pkg”)

Now I can type rake clean and it will clean out everything under the directory ‘pkg’ so I can rebuild a gem from scratch without worrying if there’s any cruft lying around. I also decided to modify the default task to rely on :clean as well as :repackage in order to really sure things were being cleaned up (this might be completely unncessary but it works for now). So my final Rakefile is now:

require 'rubygems'
Gem::manage_gems
require 'rake/gempackagetask'

CLEAN.include("pkg")

spec = Gem::Specification.new do |s|
  ...
end

task :default => [:clean, :repackage]

...

After adding only 2 lines, I gained a lot of functionality from this Rakefile. It was all there but heck if I knew it did that. I guess now I know. Later on, I also added unit tests and rdoc generation to the Rakefile but I’ll save that for another post.

John posted some of his thoughts on an article titled A brief history of workflow.

John Writes:

Maybe workflow history lies more into the “here is my recipe to sharpen sticks” or “here is my recipe to start a fire” (process/activity) and “hey, you, why don’t you sharpen sticks while we hunt, so that it’ll be easier for us to start a fire when we come back” (workflow).

I found John’s explanation brief and clear to understand about what workflow is about. However, I’m confused about what point history of a workflow is trying to make besides the author just blog brainstorming his understanding of workflow history.

While it might be an interesting academic exercise to trace the history. I think something more useful is to ponder on what it means for a software system to embody a set of steps a business organization makes to reach a goal.

Perhaps, this merely shows my utter ignorance about workflow engines (that I’m completely comfortable in admitting. Yes I’m clueless please make me ‘get it’) but from an outsider peering into the workflow engine community I’d feel the actual software artifacts of the workflow engine need to be documented in a more straightforward manner so people who don’t know ANYTHING about workflows can start to define, create, and excute workflows in a straightforward manner.

Perhaps some of the difficulties in going ‘aha’ on workflow engines and why they are necessary are:

1. At this point in time it’s just tough enough getting a working implementation
2. A software implementation of a rather abstract idea and trying to pin it down is not a trivial thing
3. To embody an abstract idea as a running piece of software, it is necessary to define terminology. WIth a name comes the power to be able to touch and manipulate it (for a programmer)
4. Enough example cases to sell that these engines are a decent bet

I believe for more widespread adoption of workflow engines it will be necessary to make it easier for people with very little understanding of the jargon, the theory, or even the nuts and bolts of these engines to be able to do something that provides a quick benefit to them. For now, many of these people will be programmers. So more concrete examples and explanations for many different scenarios (preferably a real scenario a programmer used to solve his/her problem) would far more helpful rather than histories or huge elaborate context stories that use Dilbert-management speak.

Hello (openwfe) World

John Mettraux who is the project lead for OpenWFE has given me commit rights to his Ruby port of
OpenWFE.

I have to say I’m flattered by the trust in receiving commit rights considering:

1. I have no experience with workflow systems
2. I’m not exactly a Ruby expert

First thing I did was check out the source and noticed that the rdoc for the source was reasonable but lacked examples for getting started. For a simpler library like Math this might be doable however a workflow engine has quite a few moving parts so it’s hard to know where to start.

I decided that my first task would be to add some examples to get someone else up to speed (such a myself) without having to crawl through the unit tests. However, after updating the README.txt I was noting things.

Restructuring the directory layout

When I started with openwferu I saw the following type of directory structure:

$ tree -d -L 2 --dirsfirst openwfe-ruby/
openwfe-ruby/
|-- lib
|   `-- ru
|-- tst
`-- website
    `-- src

Basically a decently structured project however one problem is that this directory structure does not adhere to subversion’s normal directory setup of branches/ tags/ trunk/ . This creates a few problems such as:

1. Force developers to learn a new directory layout convention
2. Makes it difficult to create tags and branches of the project
3. Forces customization of svn tools that rely on this directory structure

After discussing with John on this matter. He agreed we should restructure the directories around to adhere to a typical subversion layout.
After restructuring the codebase it now looks like this:

$ tree -d -L 3 --dirsfirst .
.
|-- branches
|-- tags
`-- trunk
    `-- openwfe-ruby
        |-- lib
        |-- pkg
        `-- test

Gemification of OpenWFEru

I was about to start up on documenting the codebase again when I realized that OpenWFEru’s release system was basically a shell script. Actually, most of the release tools were small shell scripts to handle tasks for running unit tests, creating a zipfile release, and generating rdoc from the sourcecode.

While this is okay, the Ruby world has built the Gems package management system to handle these problems. So I decided my next task was to bring openwferu into the gems world. This involved RTFM…

Unfortunately, I found the docs on the Gems website pretty weak at explaining the process besides some hand waving and a reference to some build tool called Rake. Luckily, I have a copy of Programming Ruby sitting on my desk here so flipped it open to Chapter 17 on package management with RubyGems and just followed through the example and substituted all relevant parameters with ones that were suitable for OpenWFE. A little bit later I had both a gemspec file and a Rakefile ready. Note that the Rakefile is a superset of the gemspec file (a Gemspec is basically embedded inside the Rakefile and some tasks are added to Rake). I took my shot at trying to generate a gem however failed…

The reason for the failure was based on OpenWFE following the version numbering policies of its java-based cousin (1.7.3pre1-something). No biggie, except the Ruby Gems website explains in detail why you should follow a version policy of major.minor.build (ex. 1.5.4). After discussing this with John, we decided to renumber OpenWFE to 0.9.0 as the API is mostly complete and the base functionality is there. However, there are some details that prevent it from being called a 1.0 release. I’ll let John decide on that.

After renumbering the versions, I ran the rake command listed in Programming Ruby that would generate a gem file (rake pkg/openwferu-0.9.0.gem). Lo and to my surprise, it built the gem all properly. No major fusses. A good portion of this can be attributed to John mostly following ruby conventions of keepign your code inside of lib/ rather than splattered all across the file system. Once nice thing about Rake and the Gemspec is that they are basically Ruby files so anything you can dream up in Ruby can be added into these files to help in customizing a build file. Luckily, for OpenWFE there wasn’t much needed. But it’s a double win as there is a much lower mental overhead in learning how to declare your build configuration without having to context switch between two different syntaxes (I mean you ant).

Now that gems can be easily created I can return to my original activity of trying to add examples and better documentation on using OpenWFE along with an extra activity of figuring out how to get the built gem files on RubyForge.