Where's Waldo?

Programmers don’t juggle anymore.

There was a time, many moons ago, when you could pretty much assume that anyone who wrote code in a serious fashion knew how to juggle. Maybe not well, but at least well enough to keep three balls in the air for an extended period of time. Maybe even well enough to do passing patterns. You could go to a programmer’s office or desk and find a set of lacrosse balls (much better for juggling than tennis balls, until you miss).

There were other skills that were reasonably common, as well, such as lock picking. It was always nice to know that you couldn’t really be locked out of anyplace. Some programmers had real lock picking tools, while others made do with home-grown picks. It was part of what made the culture of programming somewhat unique.

I started thinking about this recently when I realized that none of the students that are around have any of these hobbies or skills. I wondered what had changed. And then I realized that it was because of the speed of the machines that we all use.

It used to be that part of programming was long periods of waiting. You put together a program of reasonable size, and it had to be compiled, which would take a while. And by a while I mean half an hour or more (I won’t bore you with the tales of waiting overnight for your deck to be run; go ask your parents). You needed something to do for the time it took the program to compile. You didn’t want to start on another program, because you wanted to be ready to fix any errors or start working on tests once the program compiled. So we did things that took enough attention to pass the time, but not so much brain power as to be actually useful. Juggling. Picking locks.

The exceptions to this were the programmers who worked in Lisp, who had instantaneous feedback from their integrated development environments. The Lisp community produced very good programmers, but they were (and this is going to get me in a lot of trouble) less interesting as people. Not only were they intolerably smug about their programming environments, but they didn’t juggle. How could they be real programmers?

Now, of course, our machines are so fast that we all use integrated programming environments that compile the whole system between our keystrokes. These interactive environments have brought all the speed and convenience of the Lisp environment to those of us who use compiled languages. The only time we wait for compiles now is when we download open-source software from someplace else and have to build the entire system from scratch (or while we wait for our IDE to initialize). But that wait isn’t long enough to do anything else other than, perhaps, read some email.

Mind you, I’m not really complaining. I wouldn’t want to go back to the days of 45 minute compiles (especially when those compiles found one typo, which got fixed in a few seconds and then required another 45 minute compile). But all this speed has made the community of programmers less interesting. No one juggles. No one picks locks. I sort of miss it.

At least the Lisp folks aren’t so smug anymore.

Well, something clearly went wrong with the last open office hour. Whether it was the day (Friday is probably better than Monday), or the weather (variously terrible), or the fact that there were preparations for a BBQ going on at the location (the perils of doing something unofficial), no one showed up. I hung around for 15 or so minutes and then headed off myself (sorry for anyone who came late that I missed). I’ll try again later this month.

Rather than talking about what went on there (a pretty short post), I thought I’d say a bit about some of my current thinking about open source. This can be something of a hot-button topic. But having lived in the open-source world for the last 10+ years of my time at Sun, I’ve thought about the topic quite a bit, and have some experience, as well.

Let’s start with a basic premise. Harvard is not a software company. There are those who say that Harvard is really a real-estate trust, or a hedge fund, but I’ll stick with the idea that Harvard is an institution that builds its value around education and research. The software we (at least those of us who are in the IT organization) build is meant to support that basic value, and to keep the overall institution running smoothly and efficiently. Other software is built to test research hypotheses. But none of it is built to sell. Since software is expense to develop and (especially) to maintain, we should only build our own software when we absolutely have to.

This means that, if we can fit our needs to a commercially available software package, we should probably do so rather than build the software ourself. But higher education is a small niche for a software solution, so there aren’t a lot of major players building software for this market. Where Harvard is like other businesses (for things like human resource management or ERP systems) we can find commercial offerings. But there are other areas (like student information systems, or classroom scheduling) where the market is just too small for the major players.

This puts us in an interesting bind. We don’t want to be writing and maintaining software, because doing that is expensive and not part of our core mission. But we are not part of a large enough market to attract the major (or even many minor) software companies. And, more and more, we need software to support our core mission. This need will only expand in the future, as we figure out new ways of using technology to support both teaching and research.

Open source is one of the ways to get around this dilemma. There is a lot of open source software that supports various academic endeavors. This existing code may not be exactly what we want, but we have the source so we can change it as needed. Even if an open-source package only does 50% of what we want, that is more than what we have if we start from scratch. It isn’t actually 50% more, since there is time and effort in learning an open-source code base. But it is a start.

Even more, it means that there are already some developers who are working on the problem, who can be used to leverage our own development efforts (although they may view us as leveraging theirs). And, if the package is successful, there is a group of maintenance engineers larger than the staff we can use to maintain the package (which is also true for everyone else that is working on the package).

Open source gives a nice mix of low price, high safety, and the ability to customize. The low price is actually quite a bit higher than free; you need to learn the software, be ready to use the open-source community for maintenance, and be prepared to join and support the community. And you could be stuck with maintaining the software by yourself (which is one of the things that argues against writing our own) if the community around the open source evaporates. But you still have the source, and can make changes, and the initial price is low.

If there is no open-source package, then we may have to write some software from the ground up. But if we do so, we should build that package with the intention of making it open-source, or just start the endeavor as an open-source project. With any luck, others will join in and help in the development or the maintenance. There is some cost to this (building and working with an open-source community takes effort), but the end result can be much better than if we keep the code to ourselves and will never be much worse.

Harvard has perhaps the best brand in the world of higher education. If we start writing open-source software in support of our teaching and research missions, I’m pretty sure we would have others joining the effort. If we are really lucky, someone will start a company around the software that we help build (perhaps current members of the IT staff, or perhaps students). If that happens, we can trade our intellectual property rights for maintenance, freeing up our development resources for something new.

So, to a first approximation, here is my crude view on how we should approach software at Harvard. If there is an existing commercial application offered by a major vendor, we should buy it. If there is an open-source offering that even approximates what we need, we should adopt it, join in the community, and alter it (with the alterations going back to the community) for our needs. And if there is nothing, we should write the software as an open-source project, inviting others in to help in the development and maintenance.

One of the suggestions that came out of the discussion when I spoke to the ABCD group was that I hold some sort of open office hours. The idea is that I could just be available for anyone who wanted to come by and talk. No agenda. No plan. Just discussion.

I thought it was a great idea, but wondered if it would work. But I tried it out a couple of weeks ago, and to my surprise (and delight) about 8 people showed up. We had a terrific conversation about what they did, what got in their way, and what made their part of Harvard unique (the person from Classics spends lots of time support odd and non-standard fonts, which was interesting to hear and not surprising once she said it, but hadn’t occurred to me before). The second attempt (last Friday) had about a dozen people, and we ended up talking a lot about open source and the role it could play at Harvard. Both great discussions.

So I’m going to have another. While Friday afternoon is a good time for this sort of thing, I’m going to be out of town (vacation) the end of next week, so I’m going to try varying the day to see what happens. So the next CTO open office hour will be next Monday, August 8, at 4 p.m. in the ground floor Maxwell Dworkin lounge. I’ll put up a sign of some sort (probably on my computer screen) to identify myself.

So come one, come all. The discussions are good, I’m learning a lot, and it is your chance to be heard…

And in an attempt to get some comments, where are some other places on campus that would be good to hold this? I’m looking for someplace informal, that needs no particular scheduling, and where I won’t be embarrassed if I end up spending the hour by myself but where a group of 10 to 20 can sit and chat. I know the Engineering part of campus pretty well (which is why I’ve been doing this in Maxwell Dworkin and Northeast Labs), but it would be good to move to other parts of campus as well. So if you have a suggestion, tack a comment on this post and I’ll entertain other options.

The great thing about starting up a blog is that you get asked interesting questions. Even something as vanilla as my introductory post brought some good comments (and some help in setting up the Word Press theme; thanks!). Then along comes Ruby Bright and asks one of the big questions. As Ruby so beautify put it,

I’m not all that sure that I have the answer to this question; if I did I’d be rich and famous. People have written books about the problem (Crossing the Chasm is probably the most famous). I’ve thought about the problem a lot. And so have a lot of others, much smarter than I. When I worked at Sun Microsystems, this was a constant topic of conversation. It caused the company to move from workstations to servers (an innovation that worked), and to support Java (much more innovative, but maybe not so successful financially), but in the end we didn’t find another innovation that the company could support. I’m sure that this is a topic of conversation (all the time) at places like Google, and Facebook, and Microsoft. Doing something new is really hard, especially when what you are currently doing is working well. It is even harder when you are well known, because then everything you do is watched by everyone, and there is the worry that failure will dull (or even ruin) your reputation.

I remember seeing this risk-aversion develop in the Java organization. When Java first came out, the people working on the project thought of it as a new way of writing software, with the virtual machine, object-oriented language, and portability from machine to machine no matter what the operating system. There were all sorts of new things that we wanted to try. Innovation piled on innovation. It was really fun.

But then Java took off, and we had other companies as partners and the price of the company stock was determined by the latest news story on something that had gone wrong (or, less often, something that had gone right) with Java. Within a couple of months the question stopped being “what new can we do with this shiny thing?” and became “how do we keep from making any mistakes?” And a lot of innovation was stopped or never started.

Fortunately for those of us who want to innovate in the IT world of Harvard, we aren’t under quite the same set of constraints. Harvard is not, at core, an IT organization (or at least it doesn’t believe that it is, at core, an IT organization– this may be the subject of a future post). Since IT is a support for the core activities of teaching and research, we are not under quite the same microscope as we would be if Harvard were a hi-tech company.

And innovation, as I continually tell my colleagues in HUIT, is not an activity that can be planned, or managed, or controlled by some process. It is messy, and unpredictable, and requires leaps of faith. It is hard. But there are a couple of things that seem to help.

I’m a big fan of skunkworks projects– small groups of engineers that go off and build something out of sight of most everyone. This is a great way to build a prototype. You need to do it fast, and you need to understand that you might fail, and be willing to take the risk. But with the right people this can work amazingly well. The first version of what became the Common Object Request Broker Architecture (CORBA) was written by a group of 5 engineers in six months in a skunkworks (I had the great pleasure of leading that team). When we did the Jini work at Sun, the team was hidden from the rest of the organization by being put (organizationally) in Human Resources. Making such an investment is something I hope we soon have the freedom to do, at least at some scale. It may be that we involve students to keep the costs down, but we should try something.

I’m also a fan of importing good ideas from elsewhere, and then making them our own. This may be in the form of adopting an open-source solution that we then customize for Harvard. It may be in the form of taking work someone else has done here, and adapting it for a new purpose. To quote the sage Tom Lehrer, “brilliance borrows, genius steals.” We should be unashamed of using the work of others (always respecting IP laws, of course, and giving credit where it is due).

But most of all, we need people to be willing to take risks. Managing projects in any organization is like managing a financial portfolio– if you want major rewards, you need to allow some risk. You don’t want every investment to involve major risk, but you need to have some to balance the low-risk parts of what you do. It is best to manage risk with some vision of where you want to go; this is a vision of the future. I’m trying to encourage some risk taking in the organization, and there is plenty of support for trying this out. We will see if it will allow us to innovate. I think it will, if we let it.

It is an odd thing being the first Chief Technology Officer for Harvard University. There is no history of what the job entails. There is no precedent. I am, quite frankly, making things up as I go along (with the help and advice of others, of course).

But that is part of the attraction of the job. I’ve been a software developer most of my life, and I have always enjoyed the activity of building things. Now I have a chance to build something that (among other things) builds software. Raising the activity up a meta-level means thinking about things differently, and trying things that I’ve not done before.

Now is an exciting time to be in information technology at Harvard. There are a lot of folks who understand that technology is key to the way the university will work moving forward, and are willing to try new things. While there is always some resistance to change (especially at an institution that has been so successful in the past), I believe that there is a real chance of doing some transformative things as CTO.

The purpose of this blog is to communicate what I am doing, and to hear from readers if they think that this is the right set of things to do. I’m hoping that this (along with some other things that I’m trying out) will become a mechanism for two-way communication on the changes that we will be undertaking. I remember once being told that people like change, but resist being changed, so it is important to bring them in to the discussion of what is going to change and how the change is going to happen. I’ll try to be as transparent as I can in writing this, and hope that any readers are honest with me about their own thoughts.