CQ2 | Ed Murphy

I spend too much time these days talking about what seems to me to be basic, obvious facts about professional services.  So, to save time, three quick points about project management, PSO economics, and metrics.

(more…)

I’ve recently moved into a new job at Novell, working on our strategy for worldwide services and planning for our next fiscal year is keeping me busy.  But I still, fortunately, deal with real clients and real problems too.  This one is classic: the client has several hundred old Unix and RHEL servers that they want to move to SLES.  Great!  We want to help.  So they negotiate the server deal and then want to know the cost to migrate.  How much is it going to cost, in total, to go from what they have today to what they want tomorrow?  They ask for estimates on a per-server basis; how many hours would it take to migrate a Solaris server to SLES?  Ten hours?  A thousand hours?  So they bring in the consultants, the dreaded consultants.  They’ve tried to avoid slowing down the deal but there’s no avoiding it now.

Well, you’ve done this before, they say, you’re grizzled veterans of the data center; is it two or ten hours for a server?  And the consultant — and I’ve been in this situation, believe me, it sucks — has to say, “Well, it depends.  It could be a thousand hours.”  Which is what everyone is expecting him to say because you can’t get a straight answer out of a consultant.  They’re always going to tell you “it depends.”  Right.

And even if there is all the time in the world, this particular answer needs to be in writing on the buyer’s desk by EOD today or the sales guy isn’t going to make his number for the quarter which means that he’s not going to make ‘club’ (his incentive travel event), which his wife is really looking forward to, so this damned consultant is not only not answering a simple, reasonable question from the beloved customer but they are also very directly making his wife mad at him, with the attendant consequences.

Perhaps you think I joke?  Or exaggerate?

Making matters worse, some nerd named Chad has downloaded OpenSUSE onto a machine in their testing lab and moved a couple of apps without incident (some directory changes, a few lines of code) and based on that experience has estimated that moving the three hundred servers will take approximately an hour each.  Seriously: we have clients who want us to tell them that moving unknown production workloads from one operating system to another will take less than two hours per server.

So the consultant sighs and starts to ask questions: What do the workloads on these servers actually do?  Online banking is different from warehouse management.  What platforms are they running?  (What version of J2EE?  What version of RHEL?  What version of Manugistics?)  Are they going to change anything else besides the operating system when they do this move?  Is the software custom or off-the-shelf?  What’s it written in?  If they say something like current Java apps running on a 2.6 kernel going to the same JVM on another distribution, that would be one thing.  If you are looking at non-ANSI C custom code on RHEL 3 on a complex multi-tiered app, that’s something else.  (Moving from the 2.4 kernel to the 2.6 kernel on any distribution is much harder than moving from one current distribution to another.)  What about storage, and backup, and disaster recovery?  Systems management?  There are a thousand more architectural details that you need to understand (one data center or many?  resource utilization?) but everyone is getting impatient with you and your endless questions.

Then you start getting into the enterprise-y aspects, which is where the real time and cost come in.  There’s a difference between Chad moving an app from one platform to another as a technical exercise and the actual time that it takes production applications to go from one to another.  What’s the testing regime?  I would expect that production code moving from one distribution to another would require real testing (stress/performance, UAT, etc.).  Would you include that in the estimate?  What about security?  Does the new OS have to go through a security audit at the company?  (Answer: yes, and it’s going to take a long time for the online banking app, believe me.)  Documentation?

This is all super-boring and bureaucratic and definitely not technical so the nerds aren’t interested and think it’s worthless and the sales guy is hearing his wife screaming at him and the buyer is saying, “Why is this so complicated?”

So, should we skip the backup part?

Really, the way to do this kind of thing is to do a quick assessment and figure out some kind of prioritization and rough sequencing, but that would require the client to spend time and money helping you to figure out how much to charge them and they are naturally leery of such a thing.  You desperately want to avoid getting locked into a fixed figure because you still have no real idea how complex the problem your being asked to solve is, but that is what the client and the others are asking for.

So you end up with a fudge; you commit to moving some edge servers and a cluster of supposedly simple apps and you sign up to do a security-approved core build and an assessment for the rest so that the project can get started and the customer can show progress to their boss and the sales guy can make his number.

Now you’re faced with months in the lab at the client site with Chad explaining to you how completely screwed up their environment is and how there’s no way that he’s going to give up his Solaris servers and anyway they’ve tried to do this themselves a bunch of times already and it never works because it’s not really a current release of Manugistics and they did some customization that they probably shouldn’t have…

Michael Nygard has his head in the computing clouds, suggesting that not only is cloud computing in our future, but that there’ll be many of them. He’s right.

Everyone who runs a large data center is today faced with the same set of interconnected environmental problems; space, power, and heating/cooling. And these are environmental not just in the sense of tree-hugging but also in a straightforward practical sense: there is no more space, there is no more power, there is too much heat and not enough cooling. These problems were the domain of junior people a few years ago, worrying about where, physically, to locate all the new Windows boxes. Then it was middle managers trying to sort out power and HVAC issues: “If we deploy a new phone system in our building we won’t have enough power to do any upgrades in the data center,” that sort of thing. Now environmental issues are front-and-center for senior IT management and if you’re a “red-shift” kind of company, for senior corporate leadership too.

You can cloak it if you want to in green terms but businesses are faced with real operational issues that they need to address regardless of their perspective on global warming or riverine dolphins.

Alongside these environmental issues, data centers are also facing a crisis of manageability. A large enterprise data center is a staggeringly complex thing, too complicated. Also, if the truth be told, most of them are not that well run; would you expect, for example, that an auto parts distributor would have great technology management skills? No, of course not, and the fact is that they probably wouldn’t want to spend the money to acquire that talent and technology even in they could; their differentiation, the competitive advantage of their business, lies elsewhere. So they have a complicated, and sub-optimized, technology infrastructure.

The answer to all of these problems — Monday edition — supposedly lies in virtualization. Novell gets brought into these conversations because inevitably data center managers have a roadmap that looks something like this:

(more…)

The question that I’ve always had about agile methods is: where does the project come from? Based on my limited knowledge (and I’m like a like a pagan at a theology convention here), the agile movement assumes a defined project or problem at the outset and then figures out where people should sit: by themselves, with a friend, or with a group. This is all fine to me; you take your Mountain Dew and sit wherever you want. But where is the problem coming from? Are you working on the right problem? How do you know?

I understand user stories and all that, but at that point you’ve already dedicated a team to working on the problem and so they go and — gasp! — actually talk to users. But how did they get tasked with that problem, the redesign of the inventory reorder system say, rather than some other problem, updating the contractor billing system, say? Do agile methods go upstream?

12 April 1865, Appomattomax courthouse.

Joshua Chamberlain was selected to receive the Confederate surrender. He describes the scene thus:

The momentous meaning of this occasion impressed me deeply. I resolved to mark it by some token of recognition, which could be no other than a salute of arms. Well aware of the responsibility assumed, and of the criticisms that would follow, as the sequel proved, nothing of that kind could move me in the least. The act could be defended, if needful, by the suggestion that such a salute was not to the cause for which the flag of the Confederacy stood, but to its going down before the flag of the Union. My main reason, however, was one for which I sought no authority nor asked forgiveness. Before us in proud humiliation stood the embodiment of manhood: men whom neither toils and sufferings, nor the fact of death, nor disaster, nor hopelessness could bend from their resolve; standing before us now, thin, worn, and famished, but erect, and with eyes looking level into ours, waking memories that bound us together as no other bond;–was not such manhood to be welcomed back into a Union so tested and assured?

Instructions had been given; and when the head of each division column comes opposite our group, our bugle sounds the signal and instantly our whole line from right to left, regiment by regiment in succession, gives the soldiers salutation, from the “order arms” to the old “carry”–the marching salute. Gordon at the head of the column, riding with heavy spirit and. downcast face, catches the sound of shifting arms, looks up, and, taking the meaning, wheels superbly, making with himself and his horse one uplifted figure, with profound salutation as he drops the point of his sword to the boot toe; then facing to his own command, gives word for his successive brigades to pass us with the same position of the manual,–honor answering honor. On our part not a sound of trumpet more, nor roll of drum; not a cheer, nor word nor whisper of vain-glorying, nor motion of man standing again at the order, but an awed stillness rather, and breath-holding, as if it were the passing of the dead!

I think you’d have to be more hard-hearted than me to not be moved by that scene. Chamberlain, hero of the second day of Gettysburg at Little Round Top with the 20th Maine and later president of Bowdoin College, continues:

What visions thronged as we looked into each others eyes! Here pass the men of Antietam, the Bloody Lane, the Sunken Road, the Cornfield, the Burnside-Bridge; the men whom Stonewall Jackson on the second night at Fredericksburg begged Lee to let him take and crush the two corps of the Army of the Potomac huddled in the streets in darkness and confusion; the men who swept away the Eleventh Corps at Chancellorsville; who left six thousand of their companions around the bases of Culps and Cemetery Hills at Gettysburg; these survivors of the terrible Wilderness, the Bloody-Angle at Spottsylvania, the slaughter pen of Cold Harbor, the whirlpool of Bethesda Church!

The whole passage is well worth reading, especially on the anniversary of that “chill gray morning.”