One of the things that I did down in Silicon Valley was attend a lecture by Alan Cooper, notable for his authorship of the book About Face in 1995 and his criticism of exposing the hierarchical file system and the RAM/disk distinction to end-users. Cooper’s latest project is “ending the death march”, figuring out how to fix the chronically broken process of software development. Cooper starts from the proposition that software development projects are opaque to management, generally failures, and therefore management underfunds and understaffs them, hoping that, if failure is inevitable, at least it will be a cheaper failure.
Cooper wants to see interaction design, the software’s behavior, made more explicit in the design. He notes that he has met two kinds of programmers in this world: (1) those who like to get stuff right, and (2) those who like to ship product and get it into users’ hands. His proposal is “the Triad” in which three kinds of people are given equal status on the project: Interaction Designers, Design Engineers, and Production Programmers. The design engineers figure out how to get state of the art technology and algorithms into the product and they may write a lot of test code that will eventually be chucked. The production programmers handle all of the edge cases and compatibility/installation/deinstallation concerns. He presented a lot of graphs for CEO-types about measuring return on investment at various stages. I didn’t understand these.
I personally enjoyed seeing Cooper’s elevation of interaction design. In the course that we teach at MIT, we try to hammer into students the importance of planning the data model (what can be represented in between clicks and sessions) and page flow (the interaction design itself) before writing the pages. For all too many systems, the interaction design is only specified implicitly in a huge pile of CGI scripts or whatever.
The question for commenters is “Do we believe that changing the management process can fix software development?”
Let’s look at some potential arguments against Cooper’s proposal.
First, it is unclear that the average software project is underfunded or understaffed. The U.S. govenrment regularly throws $billions at seemingly fairly simple software projects that a handful of expert programmers ought to be able to do by themselves. A typical project has lots of mediocre programmers on it whose contributions are negative. In this sense, the project is clearly overstaffed because it would go faster if you fired 50 percent of the people.
Second, the fundamental tools and debuggers for software have changed but little in 50 years. During that time, many tens of thousands of organizations have executed software development projects. Among these tens of thousands of organizations, every conceivable management structure has already been tried. If there were a silver bullet, it would already have been discovered by one of these companies, perhaps randomly, and then propagated to the world at large.
Third, when genuine progress has been made in software development productivity and software quality, it has come from improved tools such as SQL (a declarative language that replaced a lot of unreliable and unreadable imperative code). Maybe the reason software development projects fail is that we’re using the same tools, the programmers have the same IQ as in decades past (or perhaps lower as the smart ones trot off to professional schools), and the requirements have become more complex.
Fourth, when you see a company whose products are better than competitors, it is usually because they have programmers with higher IQs, not a magic management technique. Google has the smart people; the federal government (and its contractors) has the dullards. Google gets innovative stuff out on time; the federal government ends up years behind schedule on a simple rewrite.
Fifth, over the life cycle of a software product, we already almost have the kind of structure and division of labor proposed by Cooper. The programmers who like blazing new trails, albeit somewhat sloppily, show up on Day 1. They solve the hardest algorithm problems and get a working v1.0 out the door. By the time v3.0 is shipped, natural turnover within the development group has resulted in the code being worked on only by production programmers who like methodically sweating the details.
Some arguments in favor of Cooper’s proposal:
Managers of software development projects tend to be so incompetent and lacking in information that they can’t recognize and reward the strong contributors (this results in the best programmers getting paid only 20-30 percent more than the mediocre ones (compared to a 500 percent ratio in most areas of law, for example)).
Making the interaction design explicit makes it easy for decision-makers to evaluate whether programmers are accomplishing anything.