Al Hoang

Had a case where I had some rails app that was using files for its session store and had been running like that for months. While it was a careless (and dumb) thing to run it that way, we had to do something about it since it was eating up close to 85% of the system partition. The dumb way to try to do this is below:

nice find /tmp/ -name 'ruby_sess.*' | xargs -n 100 rm -fv

However, the problem is that this causes insanity on IO which is NOT good for a running service. (Luckily this service was merely important instead of critical) So, I present to you my quick and dirty script that will clean up all those nefarious ruby sessions files that have run amok for months.

while true; do
    nice find /tmp/ -name 'ruby_sess.*' | head -n 20 | xargs rm -fv; sleep 10
done

Been thinking about the role of testing in a software organization. Steve Rowe of M$ has a great blog post on finding the right people for the job. He breaks it down into 3 types of roles that are useful in a test team:

• Runtime testers
• Scripters
• Tester Developer

Definitely a good read for understanding the type of personalities you would want when building a testing team.

It’s Common Sense, Stupid notes the difficulties in bootstrapping a GUI testing framework and what to expect out of it. These are good things to keep in mind when building out a UI test framework and putting realistic expectations on what you will get for the investment of effort.

I would also note that the blog posts on the difficulties of UI testing can be applied to larger scale system testing (ever test a router?) since one could view a GUI application is a little microcosm of a system that cannot be easily taken broken down into its components when trying to do integration testing.

The League of Professional Sys Admins (I always want to say Extraordinary Gentlemen but who would classify a sys admin as a gentleman?) has an interesting post on comparing systems monitoring to continuous integration

The concept of using monitoring as a continuous testing service to validate your changes against the environment in order to reduce alerts, breakage and general frustration seems odd to some people.

Doesn’t sound weird to me. Although one thing that would be nice is to establish some simple rule and patterns one could follow in bootstrapping the monitoring system to hook into a ’systems test framework suite’. As far as I know, that’s not something that I’ve seen something that can fit on a 3×5 card yet…

Interesting stuff. Here is a snippet…

Scale your application, while leaving your existing database untouched by front-ending the database with In-Memory-Data-Grid (IMDG) or caching technologies. The database acts as a persistence store in the background. I refer to this approach as Persistence as a Service (PaaS).

Although I do question how reliable the background persistence to a database is. Seems another layer of complexity that requires understanding in case it has issues of its own. After lookign at the Gigaspaces website, it looks like a typical Java-powered auto-generated website with tons of information all over the place but hard to find a good starting point (Yes I found the tutorials thank you) or any great documentation for a newbie.

• Read more on scaling out mysql
• Read more on Gigaspaces

From their website

OpenMac: The Smart Alternative to an Apple

The Psystar OpenMac works just like an Apple Macintosh

Yes, very desirable. However, we’ll see if Apple lets them continue with it…

Hope they don’t get sued out of existence

After a long time of not updating my trusty old iLamp iMac, I finally updated it to 10.4.11 over the weekend and let it lie. Later on I hear a report that Safari won’t start up.

That’s odd, I’ve never heard of Safari having launch problems before. I check the log and I see something like the following:

Date/Time:      2008-04-14 22:06:45.474 +0900
OS Version:     10.4.11 (Build 8S165)
Report Version: 4

Command: Safari
Path:    /Applications/Safari.app/Contents/MacOS/Safari
Parent:  WindowServer [86]

Version: 3.1 (4525.13)

PID:    20987
Thread: Unknown

Link (dyld) error:

Symbol not found: _WebDatabaseDirectoryDefaultsKey
  Referenced from: /Applications/Safari.app/Contents/MacOS/Safari
  Expected in: /System/Library/Frameworks/WebKit.framework/Versions/A/WebKit

Those type of messages go on and on in Console.app. My first round of Googling brought me to a Macrumors thread that recommended deleting the history.plist and other parts from ~/Library/Safari. I tried removing various files from that directory to no effect and finally just removing the directory altogether. Nada.

Troubleshooting a PC WITHOUT a web browser really sucks. I suggest everyone try it once in awhile to see just how much the nature of debugging IT problems has changed when you are deprived of a very powerful tool. Luckily, this Mac had a very crufty version of IE lurking so the first thing I tried was a download of Firefox. However, the disk image refused to be mounted! Seems this is also tied in with the security update. So basically, I’m left with using IE as my main tool for debugging on this Mac.

After a bit of googling, it seems that the fix is tied requiring the latest OS X security update (Available at the Support Download page). For OS X 10.4.x users you can find a link to the latest Universal here.

However, there is a catch. The security update is only available from that page as a Disk Image. Guess what you can’t open it up under the affected Mac. At this point you’re left with 2 options:

1. Burn the Disk Image to a CD-R/DVD-R and use it that way
2. Transfer the disk image to another Mac and unarchive it then push it back

I decided on the latter option since I hate wasting a CD-R if I don’t have to. After applying the Security Update and rebooting, Safari boots up and things seem chipper again but that was a serious side trip on something I’ve rarely seen Apple ever mess up. Anti-kudos to Apple for making this update on your ‘legacy’ OS really sucky.

Anytime I’ve been running a perl based script on my leopard box I got this really irritating output with whatever else I was expecting:

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
     LC_ALL = "En_US",
     LANG = (unset)
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

With a quick Google search I found an answer in this Rubify post. Basically the solution / fix is to make sure the following is set in either your ~/.profile or ~/.bashrc or ~/.cshrc

# This setting is for the new UTF-8 terminal support
export LC_CTYPE=en_US.UTF-8
export LC_ALL=en_US.UTF-8

Read more at Rubify!

Because the packaging certainly isn’t…

I had this task where I wanted to pull information from del.icio.us for some data processing so I decided to take a look at what was available for Ruby. Before, I say what path I took let’s go over the choices I dug up.

Ridiculous

First up was Ridiculous which labels it as a simple wrapper for the del.icio.us API. After looking at the API I definitely agree it is quite simple to use. So here are the highlights.

The good

• BSD Licensed
• API really is simple
• Has a rubygem (Any serious ruby tool should be available via rubygems)

The bad

• Seems to require logging into delicious
• Only seems to manage a delicious user’s bookmarks

Mirrored

Next was mirrored which looks like it is modeled on Active Record.

The Good

• Acts like an Active Record Model (nice for Rails guys)
• Supports both del.icio.us and ma.gnolia.com
• Has a rubygem
• MIT-ish licensed

The Bad

• A little more complex than ridiculous (counting # of classes)
• Only handles a logged in user scenario
• Requires logging in

ruby-Delicious

Next up was this blog post that pointed to ruby-Delicious. However, when I clicked on the link I got a Forbidden. I love abandon-ware. I decided not to look much farther since the author obviously wanted to talk about something he didn’t think about maintaining.

What I wanted

What I wanted was something a little more full-featured like Python’s pydelicious library. It handled both cases of being logged in and wanting to update bookmarks AND just the lazy joe who wanted to yank data from delicious without having to stuff it into a data structure by hand.

In the end I ended up using none of the above since none of them did what I really wanted and after a little of Googling saw many snippets [1], [2], [3] that showed how one could do it with just open-uri. In the end I decided to just use open-uri since it ended up being a big fat 3 lines of code to implement the base case of what I wanted with open-uri. With a few more lines I had exception handling and retrying on failure.

However, hopefully someone else will be able to make use of the above libraries since they do like quite useful for programatically accessing delicious.

[1] Use delicious API via HTTPS from Ruby
[2] Ruby delicious snippet
[3] Ruby, delicious and graphviz

I was thinking about the concept of naming machines and in general I try to first think of a theme to help decide on a naming convention.

I don’t know where I read about this… perhaps in the DNS & BIND book or some other website (before blogs really took off) however I just rediscovered RFC 1178 which offers plenty of suggestions and anti-suggestions for deciding on names for machines and like many of the earlier RFCs it is quite readable. Highly recommended read.

Read RFC 1178

While not much is known about Google and their Data Center operations this makes a decent attempt at aggregating all of it in one spot…

Read more

I love reading about these smart uses of waste byproducts I bet this might work well in Japan if it wasn’t for those pesky earthquakes.

A new data center in Switzerland is being used to heat a nearby swimming pool. In what appears to be a first, the town pool in Uitikon, Switzerland will be heated by waste heat from a data center

Read more at Datacenter Knowledge

Well worth the read..

Well, read it already!

Virishi tells a story of Meraki (an off-shoot of MIT Roof-Net) changing their tune.

Today I learn that my failure is due to the fact that Meraki has automatically updated the software on all of the units (including legacy, such as ours) so that you cannot install a different firmware on it, at all.

Not cool

Read more

Colin Percival had this to say on security:

Schneier suggests that this “particular way of looking at the world” is very difficult to train — far more difficult than the domain expertise relevant to security. I respectfully differ: In my opinion, this mindset is not particular to security professionals; and universities have been successfully training people to hold this mindset for centuries.

…

the entire mid-term examination consisted of writing proofs; and a proof isn’t correct unless it considers all possible cases. Forgot to prove that a limit exists before computing what it must be? Your proof is wrong. Assumed that your continuous function was uniformly continuous? Your proof is wrong. Jumped from having proven that a function is continuous to assuming that it is differentiable? Your proof is wrong. Made even the slightest unwarranted assumption, even if what you ended up thinking that you had proved was true? Sorry, your proof is wrong.

…

More importantly than this, however, is that the sort of edge cases which mathematicians are trained to think about in writing a proof are exactly the sort which cause most security issues

Colin’s view is that the security mindset is one where you should be thinking with a analytical (in a mathematical sense) mindset when truly considering security. This is not something that is that simple to do on a continuous basis.

My understanding that I took away from the article is that one should expend a lot of effort upfront building up a proof that the security of your system is secure and implement it following the proof. This is definitely an ideal that one should work towards when trying to apply security. However, I feel some things in the ‘real world’ are so fuzzy that this can’t apply to all facets of security evenly. What to do then?

When it comes to writing software this might be more tractable. However, at the systems level where it usually means integrating a set of systems together how does that work? I guess choose completely secure components and make sure they are configured to work in a secure manner when communicating with one another when you have the choice is a good one. How about when you don’t have that choice?