Al Hoang

Awhile back I installed Ruby 1.9.1 in such a way as to co-exist with my current Ruby installation [1], [2] (I should use rvm [3] these days…)

However, one issue that cropped up during an IRB session was I could not copy and paste Japanese characters into the IRB repl. This is very very painful for my day to day use with Ruby (Imagine not being able to use the ‘|’ character while writing UNIX pipelines).

Below is an example of me trying to enter the character あ into IRB and watching it fail.

$ irb
irb(main):001:0> ab = "?"    <---- Tried entering the character あ
SyntaxError: (irb):1: invalid multibyte char (UTF-8)
(irb):1: unterminated string meets end of file
from /usr/local/bin/irb19:12:in `'

After a lot of head scritching I was able to narrow it down to something with readline:

$ irb --noreadline
irb(main):002:0> ab = "あ"
=> "あ"
irb(main):003:0>

After some more digging into the issue. The root cause seems to be the lack of GNU readline. By default, Ruby will link in the system installed readline library on OS X which is called editline [4]. Unfortunately, editline does not support UTF8 or multi-byte character sets which makes this a no-go for daily usage.

Most of the other references suggest downloading readline from source and installing into /usr/local however I believe this defeats the purpose of using something like MacPorts. After a bit of finagling I found that this is the invocation to get things working.

wget ftp://ftp.ruby-lang.org/pub/ruby/1.9/rub...
tar xvzf ruby-1.9.1-p243.tar.gz
cd ruby-1.9.1-p243
# Don't trust MacPorts version of autoconf because it somehow nuked the
# --with-readline-dir option
/usr/bin/autoconf
./configure --with-readline-dir=/opt/local --enable-shared --program-suffix=19 --enable-pthread
make
sudo make install

I have it wrapped up in a script which you can see here.

References

[1] http://wonko.com/post/how-to-compile-rub…
[2] http://frozenplague.net/2009/01/ruby-191…
[3] http://rvm.beginrescueend.com/
[4] http://thrysoee.dk/editline/

In Mercurial I noticed one new feature in the commit command that lets you specify a commit date.

$  hg commit --help
 -d --date       record datecode as commit date

Too bad the help is too sparse to explain the commit date format. Luckily I found a good explanation at the URL below.

Thanks!
http://blog.littleimpact.de/index.php/2009/03/03/usage-of-hg-commit-date-mercurial/

I have a love-hate affair with Capistrano. It is a great tool if you are a Ruby person and need to do something NOW on a bunch of machines. But the docs are in a constant state of suck from my point of view.

The Capify.org website helps for remembering the ’simple’ details on what Capistrano can do. But where I waste a lot of my time is asking questions like, “How do I set the Capistano SSH gateway to a non-standard port?”. Luckily, Capistrano is written in Ruby so it is easy enough to glance through the code and finally find out where it is but this is why good tech docs exist. To give enough context to answer those questions.

To answer my own question, below is a snippet you can add to your capfile to use a non-standard port if you need to deploy through a SSH gateway that lives on a non-standard port

# Add this to your Capfile
# This sets the SSH gateway to a machine called mysshgateway.com on port 22222
set :gateway, ‘mysshgateway.com:22222′

After reading Game Developers and Porn Stars I started recollecting an earlier time in my life. At that point in time I was considering a life as a game developer. I had heard the rumors that life as a game developer was a meat grinder and had really long hours. I spent time reflecting on the choice I had. I really like video games and think they a great form of entertainment that has had a large influence on my life. But I still feel, at its core, video games are just entertainment. Sometimes, they can educate along with delight but that is all.

Ultimately, I took a different path than becoming a game developer. After reading Kill Ten Rat’s blog post on Game developers I am glad about my choices. I have pretty much erased almost any regrets on not taking that path in life. Although I AM sad to read such a story in 2009 because the decisions I made were over a decade ago and it is disappointing to hear the state of the game industry for a game programmer as a whole seems so soul crushing.

Seems someone else ran into issues while trying to deploy god.

While, I don’t think god sucks I definitely don’t endorse it. At this point I would only use it under the following conditions:

• Need for a process monitor tool with more dynamic configuration setups. This is where god really shines against monit’s simpler understanding of what process management is about.
• The host that needs monitoring can easily spare at least 16MB for a monitoring process. See below on why.
• I really want an all Ruby solution for all the tools in a system

In general, I am into the whole ‘It is Open Source. If you’re having issues, fix it’ deal so I am not nearly as angry sounding as Brad is about god. However, after having issues with god, I switched to monit for simple process monitoring and restarting. I had far less troubles and got on with other tasks that I considered more important than perfection in a process monitoring system.

For those that are curious here are the issues that I ran into with god:

• Daemonized Ruby took at least 8MB of RAM for the monitoring process. With RAM the way it is, this is not as big a deal. However, if you are trying to get by on a 128MB VPS host every kilobyte counts.
• God itself had issues just randomly dying after some time. Tom promptly fixed it after it was reported and that was great. However, it was a little disappointing that a monitoring process just died.
• Sparse documentation compared to monit’s. Then again this is typical from many Ruby projects and luckily Ruby code is readable enough
• Digging up known issues for god required noodling through groups, forums, and blog posts. Would have been nice to just have a friggin’ FAQ like other sys admin-targeted software I have seen.

I also DO agree as has been said in the comments on Brad’s post that it is the responsibility of the deployer of software to handle the issues with whatever they deploy and just deal with it. The reason I say this is because I fell for the hyped up description of god in the beginning and ultimately paid the price when it sucked up my time. I dealt with it but definitely am less impressed with overhyped marketing descriptions of software these days. Personally, I am not a fan of that type of marketing for software since it seems a little disingenuous to me. But that is just me.

I just read a blog post by Blaine Buxton describing the phenomenon of Forced Pairing. In a nutshell, pair programming has to take into consideration the human factor when programming. Some people need their own space to code well.

On reflection, this makes sense. When I have pair programmed, I have usually been supportive of the idea and want to share my thoughts and ideas with the person that I pair with. However, communication of thoughts and motives is at best an imprecise art. From what I have seen, pair programming can have issues at the ground level under the following circumstances:

• If one person in a pair is not willing to communicate with the other person
• If one person cannot express intentions well to the other person
• One person is moving too quickly and will not slow down enough for the other person to keep up (This is not fun at all)

I cannot imagine an environment where pair programming was taken so seriously that is has been codified as a law but if they do exist then Forced Pair Programming is definitely something to watch out for. However, I do believe that pair programming is an effective strategy for getting multiple developers up to speed on any codebase and avoids the Only One Developer Understands this Code syndrome.

I had a need for inserting the name of the current file into a bunch of files I was editing. I was pretty sure there was a function to do this in vim and after some searching I was right.

To insert the current filename. In Insert Mode, type CTRL-r % and it will insert the current filename.

Thanks to blog post for the tip!

Luke describes a nice methodology for using Mercurial as a way to track patches from a subversion checkout. This type of workflow stuff is very cool imo because it has the following:

1. Shows a concrete example of how to use a not so trivia tool
2. Works within constraints (in this example, playing with a subversion checkout)
3. Fills a need (Managing non-trivial changes to a centralized SCM model without sending tons of commits)

Read More

iBanjo talks about hgsubversion which seems to be a good mercurial-subversion bridge at last.

I’m surprised it took this long to show up but I guess the Mercurial community had not prioritized ‘Build a better git-svn’ as something someone should roll out… until now.

Installation still looks a little painful (and requires lots subversion 1.5 libraries) but I am optimistic this will start gaining traction for those that need to integrate with a subversion world.

Via Kvardek-du through Planet Lisp

So recently I saw this when doing a svn checkout of a project and ran into the following:

$ svn co http://svn.somewhere.com/svn/projects
svn: Unrecognized format for the relative external URL ”.

Wonderful. This indicated to me there was a problem with the svn externals somewhere. After noodling a little I decided to Google around and found this. Basically, duplicate listings in your svn:externals is a bad thing.

Read more

Doug Hellman’s svnbackup script tool is a really handy tool for setting up automated backups for a subversion repository.

However, the non-fun time comes when one wants to restore a subversion repository that has way too many dumpfiles parts. The instructions for restoration are basically ‘roll your own’ if you want to try to automate the restore procedure. What would be nicer is if there was the converse tool that made it easier to not have to figure out how to re-order the backup files in the proper manner in order to perform the restore.

I spent a few minutes thinking about it and wrote a small Ruby script to help with this that I call svnbackup-restore.rb. Here is the (hastily written) source code. (Download here)

#!/usr/bin/ruby
# Program Name: Restore the restore
# Purpose:      Take all the svn dumpfiles generated from svnbackup
#               sort them and try loading them via svnadmin
# Usage:        1. Create the new repo path with svnadmin create
#               2. Set repo_name to the repo backup file names
#               3. Set restore_path to the new path to restore to
#               4. Run it ./svnbackup-restore.rb
# Assumptions:  svnrestore-backup.rb is in the same dir as the svn dump files

# CHANGE PARAMETERS HERE
repo_name = 'myrepo'
restore_path = '/path/to/myrepo'

# Print out debugging?
DEBUG = true

# DON'T CHANGE BELOW

# Filename format is 'dumpfile---.bzip2'
svn_dumpfiles = Dir["dumpfile-#{repo_name}*.bzip2"]
sorted_files = svn_dumpfiles.sort do |a, b|
  left_rev = a.split('-')[2].to_i
  right_rev = b.split('-')[2].to_i
  left_rev  right_rev
end
sorted_files.each do |dump_part|
  results = `bzcat #{dump_part} | svnadmin load #{restore_path}; echo $?`
  puts results if DEBUG
  res = results.split.last.to_i
  if res != 0
    puts "Error on trying to load up #{dump_part}!"
    exit 1
  end
end

Lightning talks are one of the more interesting parts of a conference in my opinion since 5 minutes really forces the speaker to get to the point and it becomes painfully obvious if the presentation has no focus or if there has been real work to get across the main message in the shortest amount of time.

So here are some highlights

Kuwata on Java to Ruby

• No, not the book Java to Ruby
• Going from Java to Ruby requires a change in mindset. Not a change of code
• Some historical example: COBOL in Java (ed. That sounds frightening) is like Java in Ruby
• Don’t hold back experts and too many things spend too much time on beginners
• Do not keep beginners as beginners (teach them!)
• Bragging about the largeness of a project size is the wrong type of bragging (suggested large code + many devs == lack of ability

dRuby & Security by Nishiyama

• druby is not built by default to handle the wild Internet
• There is a feature called insecure method list that will help prevent certain methods from being invocated remotely
• Use $SAFE however it won’t save against a DoS. Also don’t forget about rlimit

Ruby + ODE by Sasaki

• Showed a very nice 3D demo walkthrough world controlled with a Wii-mote and looked like the Pitagora Switch world
• Can summon objects pressing one of the buttons
• Can stop time
• The project should be on Code Repos
• (ed. Seeing this in action was far more interesting than reading these notes)

• Conclusion first: Sorry they don’t
• Showed some very nice examples of using the Enumerator library for many bad cases (each_slice)
• Beginning Ruby programmers really love each to the point of going overboard

Folk programming with Ruby by mootoh

• Folk Programming was introduced at YAPC Asia 2008 (See my notes)
• Show examples outside of building web applications such as Rich UI exploration
• Showed examples of Plugins to other programs since it’s easier than writing a big app (although with Ruby the apps should be nice and small)
• Some examples: Safari + Hatena bookmark, Quicksilver + Twitter (looks dangerous), Quartz Composer + Gainer, Vim + Refe
• Ruby is a very good glue

Again as a Rubyist… toRuby by Ikezawa

• (ed. I liked this talk a lot since it wasn’t by some uber-Ruby hacker)
• Didn’t use Ruby until 2000
• Background was as a consultant since 1984. Helped found a Wapro Kissaten!
• Moving to Ruby hit the OOP barrier. Was not used to OOP methodologies at all
• Stopped however after a long time in June 2007, found a local Ruby guru to help him out and that got the ball rolling again
• Invites others to join in

Ruby 1.9 with Rails 2.1 by matsuda

• Went through the history of web programming (or his version of it)
• Ancient History – PHP
• Recent History – DB Framework with ORM (in Java… lots of these frameworks)
• Current – Rails
• However DB access is a Rails weakness
• Introduced named_scope feature (Is this a Rails 2.1 specific feature? Need to review this myself
• More info on this at blog.dio.jp

Read code with Testing by Endoh

• Rookie Ruby Committed
• His suggestion on learning is by reading real code
• Use TBCR (Test Based Code Reading)
• Run make test-all
• Find code paths that are never executed
• Add tests (requires more code reading)
• Using this method have increased Ruby’s test coverage to 85%
• In contrast Python is 80%, Perl 63%, PHP is 51% (plans to target PHP next)

Okay this is my notes from Matz’s Keynote. I was 5 minutes late since finding lunch took a really long time to find anything around the area unfortunately. Unfortunately, most of the visiting Rubyists decided to do KFC however Charles Nutter decided to stick it through and we finally found a nice Yakiniku restaurant to eat at but that ended up being the reason I was 5 minutes late… okay anyways here are my notes from in the middle of the talk…

Matz was talking about sanctuaries and how some technologies have built their sanctuaries over time. Here are some sanctuaries he mentioned and some of their defining characteristics:

• UNIX
• The filter (wahoo!)
• “Worse is better” aka the New Jersey School of Design (I always ask myself just how much worse though..)
• Convenience over perfection
• Smalltalk
• OOP, deep OOP
• Targeted at children (funny how only greybeards really use this language now discounting eToys)
• Bytecode VM but not the first to have one but one of the more prominent ones
• A dynamic language implementation with decades of hard-earned experience, wisdom and knowledge around it
• java
• Well Java seems to fulfill business and ‘enterprise needs’ (for now)
• Java was originally slow however time has changed that perception. Java’s speed complaints have mostly faded away (But I still complain about a 30 second startup time for the VM)
• Absorbed many other ideas from other languages (VM, Garbage Collection, Exception Handling)
• Java has one of the fastest Garbage Collected VMs now (with the amount of engineering effort thrown at it I’d hope so)

Matz then describes that Sanctuaries tend to go through the following phases

• Hackers gathered
• New technology is born
• The world changed (because of the technology? Missed this…)

Don’t forget about centripetal forces during these stages. The community matters quite a bit. According to Matz, 50% of the types of OSes out there in this world are some type of UNIX.

Finally, he goes on to describe the Ruby Sanctuary and its defining characteristics

• For Rubyists, feeling matters. Focus on the human and the joys programming
• It inherited many things from the past. Lisp metaprogramming, Smalltalk OOP, UNIX text processing
• Productivity matters. Machines are faster but people’s time is much more expensive
• Agility matters. Environments change and embracing the changes is the right thing

At this point a person from Rakuten comes up on stage and talks about some projects that Matz has been collaborating with.

One is called ROMA which is some sort of distributed memory data storage (think memcached with some more ability for data integrity in case of failures)

The other one is called fairy which is supposed to be a lightweight distributed programming framework.

Unfortunately it seems that both projects are rather delayed and it doesn’t seem apparent if these projects will be Open Sourced or not. That is rather disappointing to me but can’t have everything for free now can we?

Evan Phoenix on Rubinius

Evan started off with a quick intro on Rubinius and proceeded to talk about some of the big pieces of Rubinius from a 10,000 feet high view.

• The kernel of the system
• The C Compatibility Layer
• It’s a big project and Evan wants to have a conversation with anyone who wants to have one on it

For the VM nerds here are some feature highlights…

• Accurate generational Garbage Collector
• Bytecode based VM
• Capable of bootstrapping itself

Evan suggested we should think of Rubinius sort of like an OS.

Then he started diving right into a tour of some of the cool internals of Rubinius and explaining some of the internal objects that really are the heart of Rubinius (above the VM layer)

BTW what is the <<? Evan coins it the left chevron.

With MethodContext and CompiledMethod it’s possible to implement eval in Ruby and follow the principle of Code as Data. Taking advantage of Ryan Davis’s ParseTree (included in Rubinius) one can take something like

"1 + 1".to_sexp

Which will take 1 + 1 and change it into an s-expression, convert it into an AST, and a User Visitor Pattern implementation can walk through this and spit out bytecode. (I might have recollected this wrong.. if so, sorry.. Not a VM implementor)

Some other big features that Evan mentioned were

• Extensions
• Getting close to running Ruby C-extensions
• However they are still 2nd-class citizens (they will take a performance hit but working is better than fast and broken)
• MRI in this case is still faster
• Multi-VM implementation
• stdin and stdout are implemented as pipes
• Functional but still highly experimental (can try calculating Pi across a multi-core machine)
• args = ["-e", "puts", "hello"]
  vm = Rubinius::Vm.spawn args
  puts vm.stdout.gets
• Channels
• One of the main internal communication channels used in Rubinius
• I/O events use channels for example