Al Hoang

With the announcement of OpenSolaris 2009.06 I thought it would be appropriate to blog a little about a tool I had been writing to help myself play with Zones a bit easier.

My overall goals were the following:

• Have each zone configured with its own virtual NIC (Crossbow)
• Allow easy creation of zones without having to type zonecfg crap over and over again
• Make it a stepping stone to automatically creating zones
• See how well ipkg branded Zones work
• Allow a Zone to get its IP and DNS configuration from DHCP

I had tried going through tutorials that I found on the web (See references below) for setting up Zones but sadly none of them worked to my frustration. After a lot of experimentation I finally pieced together a way to create zones quickly and (almost) automatically for simple configurations.

Howto

1. Create a template zone that will be used as the main clone Zone
2. Download setup-zone-exclusive.sh and modify lines 34-35 to match the name of your template zone and the real interface you want the zones to bind to
3. Download the DHCP event hook script from here and name it dhcp-client-event.sh if you want DHCP configuration
4. Run setup-zone-exclusive with the zonename and the virtual nic interface that you want

In more detail here are the steps below

First create a template zone (I call it barebones here)

# Create /zones as its own ZFS filesystem
$ pfexec zfs create rpool/zones
$ pfexec zfs set mountpoint=/zones rpool/zones
$ pfexec zfs create rpool/zones/barebones
$ pfexec chmod 0700 /zones/barebones
$ pfexec dladm create-vnic -l $REAL_IF vnic0
$ pfexec zonecfg -z barebones
barebones: No such zone configured
Use ‘create’ to begin configuring a new zone.
zonecfg:barebones> create
zonecfg:barebones> set zonepath=/zones/barebones
zonecfg:barebones> set ip-type=exclusive
zonecfg:barebones> add net
zonecfg:barebones:net> set physical=vnic0
zonecfg:barebones:net> end
zonecfg:barebones> exit

$ pfexec zoneadm -z barebones install

Get the script

I would suggest you create a project directory to hold things such as zonecreations.

Download from Github gists here. Name it setup-zone-exclusive.sh. Don’t forget to chmod +x the file so you can execute it

Download the DHCP event hook script

You can get that here. Make sure this script is in the same directory as wherever you saved setup-zone-exclusive.sh

Create a zone

You can now create zones like this:

cd zonecreations
pfexec ./setup-zone-exclusive.sh mycoolnewzone virtualnic1

Have fun!

Update: Fixed an error in the example for using dladm. It should be correct now. Thanks!

References

Downloads

http://gist.github.com/122220 (setup-zone-exclusive.sh)
A DHCP event script to make sure DNS is configured when DHCP acquires an IP

Older docs on setting up Zones on Solaris

How to use sysidcfg file in OpenSolaris 2008.11
Internal Zone Configuration docs
Performing the Initial Zone configuration
Preconfiguring with sysidcfg file
OpenSolaris FAQ on sysidcfg
Ben Rockwood’s blogpost on Zone creation
About /etc/.UNCONFIGURED

Helpful for understanding Zones and Crossbow

Crossbow on vnics

Finding out that there is a change in policy for setting root_password in sysidcfg files

PASSREQ is enforced
zlogin failure after zone setup

The following helped in understanding the role of IPS and ipkg inside a non-global Zone

Updating Zones in OpenSolaris 2008.x
A field guide to Zones in OpenSolaris 2008.05
OpenSolaris forum on sysidcfg and Zones

I had a need for inserting the name of the current file into a bunch of files I was editing. I was pretty sure there was a function to do this in vim and after some searching I was right.

To insert the current filename. In Insert Mode, type CTRL-r % and it will insert the current filename.

Thanks to blog post for the tip!

A nice back story on Writely (what is now known as Google Docs) via HN. I particularly like this snippet

[The creators] have been in the application software business for nearly 20 years… they understand the user problem so deeply that they can blend the advantages of each new platform with ‘document authoring problem’ to really build a platform-native solution, not a clone of someone else’s work.

Read more

Akihabara as many people in Japan know was originally the home for buying electronic goods in the Tokyo area. It still holds that reputation however the Anime Otaku crowd have changed the face of Akihabara to also accomodate their needs and desires.

One thing that is interesting is the origin of place name Akihabara. A friend of mine has an excellent post here

Read more!

I’ve been silently scanning some Solaris blogs and skimming some of the appropriate websites for documentation on some of the more interesting features of Solaris for awhile now. One thing that requires time to adjust to is the number of acronyms that the Solaris community has to describe their technologies. One of these is called L2ARC.

At first I thought it was some hardware device however after a bit more searching it turns out that it is part of the ZFS technology suite. L2ARC stands for second level ARC where ARC is a read cache system for ZFS that uses a system´s main memory for holding the cache. While ARC uses the hardware´s main memory, L2ARC is designed to take advantage of faster I/O media such as SSD devices to provide faster read throughput than what a typical hard drive can offer.

Brendan Gregg has an excellent overview explaining what the L2ARC is and some of the benefits it can give in accelerating random reads.

I love this quote

It’s obvious that GNU/Linux is a woman. She’s high maintenance, expects
everything to be given to her for free, and no matter what goes wrong…
it’s your fault.

Via Kvardek-du through Planet Lisp

There are many multiple file rename utilities that you can dig up.

One that I am used to on Debian/Ubuntu-based distros is called rename which is one that is derived from the Perl Cookbook. However, note that this rename script does not seem to exist on other UNIX variants in a packaged format. It definitely does not seem to exist on a Fedora Core 9 box I use nor does it exist on a FreeBSD machine.

While it would not be too hard to swipe the perl script and copy it in place onto a system. I hate having to remember moving that script around and around with me from place to place so I looked around for other equivalents and found one called mmv. One nice thing is that there are packages for at least FreeBSD, FC9, and Debian/Ubuntu variants so it should be quite lazy to install compared to rename.

Below I give an example of using it to do a multiple renaming where I want to append the value 1 to the end of a bunch of filenames.

$ ls
cat_100  cat_15  cat_200  cat_50  cat_500

$ mmv cat"*" cat#1_1
$ ls
cat_100_1  cat_15_1  cat_200_1  cat_500_1  cat_50_1

Lightning talks are one of the more interesting parts of a conference in my opinion since 5 minutes really forces the speaker to get to the point and it becomes painfully obvious if the presentation has no focus or if there has been real work to get across the main message in the shortest amount of time.

So here are some highlights

Kuwata on Java to Ruby

• No, not the book Java to Ruby
• Going from Java to Ruby requires a change in mindset. Not a change of code
• Some historical example: COBOL in Java (ed. That sounds frightening) is like Java in Ruby
• Don’t hold back experts and too many things spend too much time on beginners
• Do not keep beginners as beginners (teach them!)
• Bragging about the largeness of a project size is the wrong type of bragging (suggested large code + many devs == lack of ability

dRuby & Security by Nishiyama

• druby is not built by default to handle the wild Internet
• There is a feature called insecure method list that will help prevent certain methods from being invocated remotely
• Use $SAFE however it won’t save against a DoS. Also don’t forget about rlimit

Ruby + ODE by Sasaki

• Showed a very nice 3D demo walkthrough world controlled with a Wii-mote and looked like the Pitagora Switch world
• Can summon objects pressing one of the buttons
• Can stop time
• The project should be on Code Repos
• (ed. Seeing this in action was far more interesting than reading these notes)

• Conclusion first: Sorry they don’t
• Showed some very nice examples of using the Enumerator library for many bad cases (each_slice)
• Beginning Ruby programmers really love each to the point of going overboard

Folk programming with Ruby by mootoh

• Folk Programming was introduced at YAPC Asia 2008 (See my notes)
• Show examples outside of building web applications such as Rich UI exploration
• Showed examples of Plugins to other programs since it’s easier than writing a big app (although with Ruby the apps should be nice and small)
• Some examples: Safari + Hatena bookmark, Quicksilver + Twitter (looks dangerous), Quartz Composer + Gainer, Vim + Refe
• Ruby is a very good glue

Again as a Rubyist… toRuby by Ikezawa

• (ed. I liked this talk a lot since it wasn’t by some uber-Ruby hacker)
• Didn’t use Ruby until 2000
• Background was as a consultant since 1984. Helped found a Wapro Kissaten!
• Moving to Ruby hit the OOP barrier. Was not used to OOP methodologies at all
• Stopped however after a long time in June 2007, found a local Ruby guru to help him out and that got the ball rolling again
• Invites others to join in

Ruby 1.9 with Rails 2.1 by matsuda

• Went through the history of web programming (or his version of it)
• Ancient History – PHP
• Recent History – DB Framework with ORM (in Java… lots of these frameworks)
• Current – Rails
• However DB access is a Rails weakness
• Introduced named_scope feature (Is this a Rails 2.1 specific feature? Need to review this myself
• More info on this at blog.dio.jp

Read code with Testing by Endoh

• Rookie Ruby Committed
• His suggestion on learning is by reading real code
• Use TBCR (Test Based Code Reading)
• Run make test-all
• Find code paths that are never executed
• Add tests (requires more code reading)
• Using this method have increased Ruby’s test coverage to 85%
• In contrast Python is 80%, Perl 63%, PHP is 51% (plans to target PHP next)

Okay this is my notes from Matz’s Keynote. I was 5 minutes late since finding lunch took a really long time to find anything around the area unfortunately. Unfortunately, most of the visiting Rubyists decided to do KFC however Charles Nutter decided to stick it through and we finally found a nice Yakiniku restaurant to eat at but that ended up being the reason I was 5 minutes late… okay anyways here are my notes from in the middle of the talk…

Matz was talking about sanctuaries and how some technologies have built their sanctuaries over time. Here are some sanctuaries he mentioned and some of their defining characteristics:

• UNIX
• The filter (wahoo!)
• “Worse is better” aka the New Jersey School of Design (I always ask myself just how much worse though..)
• Convenience over perfection
• Smalltalk
• OOP, deep OOP
• Targeted at children (funny how only greybeards really use this language now discounting eToys)
• Bytecode VM but not the first to have one but one of the more prominent ones
• A dynamic language implementation with decades of hard-earned experience, wisdom and knowledge around it
• java
• Well Java seems to fulfill business and ‘enterprise needs’ (for now)
• Java was originally slow however time has changed that perception. Java’s speed complaints have mostly faded away (But I still complain about a 30 second startup time for the VM)
• Absorbed many other ideas from other languages (VM, Garbage Collection, Exception Handling)
• Java has one of the fastest Garbage Collected VMs now (with the amount of engineering effort thrown at it I’d hope so)

Matz then describes that Sanctuaries tend to go through the following phases

• Hackers gathered
• New technology is born
• The world changed (because of the technology? Missed this…)

Don’t forget about centripetal forces during these stages. The community matters quite a bit. According to Matz, 50% of the types of OSes out there in this world are some type of UNIX.

Finally, he goes on to describe the Ruby Sanctuary and its defining characteristics

• For Rubyists, feeling matters. Focus on the human and the joys programming
• It inherited many things from the past. Lisp metaprogramming, Smalltalk OOP, UNIX text processing
• Productivity matters. Machines are faster but people’s time is much more expensive
• Agility matters. Environments change and embracing the changes is the right thing

At this point a person from Rakuten comes up on stage and talks about some projects that Matz has been collaborating with.

One is called ROMA which is some sort of distributed memory data storage (think memcached with some more ability for data integrity in case of failures)

The other one is called fairy which is supposed to be a lightweight distributed programming framework.

Unfortunately it seems that both projects are rather delayed and it doesn’t seem apparent if these projects will be Open Sourced or not. That is rather disappointing to me but can’t have everything for free now can we?

Evan Phoenix on Rubinius

Evan started off with a quick intro on Rubinius and proceeded to talk about some of the big pieces of Rubinius from a 10,000 feet high view.

• The kernel of the system
• The C Compatibility Layer
• It’s a big project and Evan wants to have a conversation with anyone who wants to have one on it

For the VM nerds here are some feature highlights…

• Accurate generational Garbage Collector
• Bytecode based VM
• Capable of bootstrapping itself

Evan suggested we should think of Rubinius sort of like an OS.

Then he started diving right into a tour of some of the cool internals of Rubinius and explaining some of the internal objects that really are the heart of Rubinius (above the VM layer)

BTW what is the <<? Evan coins it the left chevron.

With MethodContext and CompiledMethod it’s possible to implement eval in Ruby and follow the principle of Code as Data. Taking advantage of Ryan Davis’s ParseTree (included in Rubinius) one can take something like

"1 + 1".to_sexp

Which will take 1 + 1 and change it into an s-expression, convert it into an AST, and a User Visitor Pattern implementation can walk through this and spit out bytecode. (I might have recollected this wrong.. if so, sorry.. Not a VM implementor)

Some other big features that Evan mentioned were

• Extensions
• Getting close to running Ruby C-extensions
• However they are still 2nd-class citizens (they will take a performance hit but working is better than fast and broken)
• MRI in this case is still faster
• Multi-VM implementation
• stdin and stdout are implemented as pipes
• Functional but still highly experimental (can try calculating Pi across a multi-core machine)
• args = ["-e", "puts", "hello"]
  vm = Rubinius::Vm.spawn args
  puts vm.stdout.gets
• Channels
• One of the main internal communication channels used in Rubinius
• I/O events use channels for example