As a lawyer-cum-techie at Massachusetts Law Reform Institute in the mid-2000s, I became aware of computer system called Beacon used by the MA Department of Transitional Assistance (a/k/a “welfare”) to distribute various benefits such as food stamps to Massachusetts residents. Occasionally, our clients would have their benefits reduced or cut off because of errors in Beacon programming, and our advocates would fight not only to restore their aid, but to fix the system.

What was happening in Massachusetts was happening around the nation, and indeed our errors were relatively benign by comparison. In Colorado, faulty software generated hundreds of thousands of incorrect benefits calculations, and in New York the state’s benefits distribution system was so egregiously broken that our colleagues there brought suit in federal court and won sweeping changes.

These are some of the mundane but vitally important ways in which software is becoming the mechanism whereby government executes laws. It’s not hard to find other examples, from “deadbeat dad” lists to terrorist “no fly lists” to the inner workings of voting machines, tax calculators, and even Predator drones. (Professor Danielle Citron, to whom I owe much of the following analysis, has documented many more examples.) So perhaps Lawrence Lessig’s profound observation, code is law, has a corollary: law is code. That is to say, if software is increasingly the guise under which laws manifest in our daily lives, it behooves a democratic society to begin treating that software as law.

Software that executes law (“codelaw”) presents a number of challenges to a democracy. The simplest are bugs, coding errors that lead to wrong results. Bugs present relatively easy cases: like potholes, if you find them, you fix them. As with potholes, the reality may be harder – a common excuse we heard was that the state just didn’t have the money to hire someone to patch the software – but in principle everyone agreed that these problems should be fixed.

The larger democratic challenge arises when codelaw isn’t so much wrong as it is not necessarily right: while it may not contradict the law, neither is this particular implementation the only way to construe the law. In short, the software assumes a particular interpretation of an ambiguous law, and in so doing, it essentially makes law.

By using codelaw to carry out policy, government shoves analog pegs into round holes, resulting in the same loss of fidelity that occurs when music is ripped into digital formats. The 20th-century administrative state in America relies on a particular cascade of power, carefully tweaked to ensure democratic accountability: the elected legislature passes law; an administrative agency, with public input, promulgates rules to implement that legislation; and agency workers carry out the rules. The gradual replacement of agency workers with codelaw reveals the cracks in this system. Because legislatures lack time and expertise to tight-fitting law, they delegate specifics to agencies for further rulemaking. But rulemaking isn’t comprehensive either: nuanced decision-making still resides in agency workers who interpret and apply the rules. Codelaw takes discretion out of the hands of human beings.

Eliminating discretion can be good governance: people are notoriously susceptible to bias, corruption, and just plain meanness. The real problem for democracy is the gap between the round curves of human laws and the sharp edges of computer code. Agencies have traditionally promulgated rules expecting people to fill the gaps later. With codelaw, the people who fill the gaps are not trained government employees, but software developers, often with no substantive knowledge of the law nor accountability to the general public.

So what can be done to ensure that in an era of increasing automation, codelaw remains accountable to the people?

First, software should be fully open for inspection. Democracy depends on laws and rules being accessible to the people; codelaw should be no exception. But because only the best-resourced lobbyists can bug-check machine code, mere transparency is not enough. There must be meaningful participation.

Existing principles that cover traditional (legal) code offer guidance on handling codelaw. For example, most state and federal rulemaking require a period of public “notice and comment,” during which concerned citizens can offer input. A publicly accessible quality assurance cycle would create a parallel process for codelaw. So when Massachusetts prepares to release Beacon 2.0, it should enable people like my colleagues at MLRI to submit tricky food stamp scenarios to test that the software gets the right results.

In the long run, new forms of “semantic code” that’s both human-readable and machine-executable may narrow the gap between fuzzy legislation and binary software. “Legalese” as a software language may not deepen public confidence, but it would at least enable more precision for lawmakers.

Finally, we need a more nuanced understanding of the appropriate role of codelaw. Deployed properly, software can ameliorate systemic human failings such as sexism and susceptibility to scams. But conversely, we should also recognize the limits of software, and identify the aspects of governance best entrusted to thinking and feeling human beings. Codelaw may herald a terrifying dystopia where machines arbitrarily decide our fate. But it also invites us to imagine a world where software augments our greatest capacity for just, compassionate, and human governance.

Professor Citron, of the University of Maryland School of Law, has written the opening chapter to an important new field of legal study, one hinted at in Lawrence Lessig’s “Code is Law” formulation and which I had dubbed “Law as Code,” but which until now has not received direct scholarly investigation. Her article, forthcoming in the Washington University Law Review, identifies the need for “technological due process” in software that executes government policies – from public benefits to no-fly lists.

In brief, Professor Citron describes how software code increasingly executes our public laws. Decision support systems, she convincingly argues, quickly become decision making systems. And invariably, the vagaries of the legislative and administrative processes leave large gaps in the specifics of how a given law should be executed. Without firmer guidance from proper governmental bodies, the programmers charged with translating legal code into software code essentially wind up creating law to fill the gaps. (I describe this as “shoving analog pegs into digital slots”). From a procedural – even a Constitutional – perspective, this is a grievously inappropriate delegation of governmental functions to the private sector, not unlike the hiring of Blackwater mercenaries to achieve military objectives. Professor Citron finds, therefore, the need for “technological due process”: safeguards to ensure that software is literally up to code.

In exploring what I’d described as “Law as Code,” and now significantly better-informed by the invaluable analysis of “Technological Due Process,” this fall the Berkman Center started seeking out areas of public law at risk of being improperly executed by unaccountable software code. In my work in legal aid I’d already identified the distribution of federal food stamps as one such area. Here in Massachusetts, the BEACON software system is riddled with errors and misinterpretations of the law. In New York State, Federal District Court Judge Rakoff had similar invalidated similar software allocating that state’s food stamps (445 F.Supp.2d 400). Rather than going after broken code, we’re much more keen on identifying code not yet in place, where identifying and implementing some best practices can go much farther than litigating for change in a system already starved for resources.

“Technological Due Process” points the way to new vistas of research. Perhaps most important among them are questions about how to bring democratic accountability back into the system. From my perspective, at least two paths present themselves. The first lies in the realm of classic administrative law. There is no question in my mind that many instances of government software meet the definition of agency rules: they are rules of execution, prospective in nature, applied uniformly. Meeting that standard is significant: it would subject such code to rigid requirements such as public participation in their creation (notice-and-comment) and judicial review in their actual application.

But as James Grimmelmann, now a professor at New York Law, pointed out when I’d first suggested this approach to the CyberScholars this past spring, many lawyers and scholars perceive ad law as broken. Furthermore, perhaps software is sui generis and deserves its own method of review tailored to its unique nature.

Berkman clinical student and JOLT editor Ryan Trinkle has proven invaluable in guiding our search for review methods native to software. As both a student and practitioner of the coding arts, Ryan pointed out that software quality assurance (QA) might provide some excellent models for how to achieve the due process goals of ad law. We ultimately struck on one potentially elegant merger of QA and notice-and-comment: obligate vendors to include testing suites with their software, and allow the public to submit specific cases that the test suites would calculate. In the case of food stamp software, for example, advocates for domestic violence survivors might submit a battery of both mainstream and “outside case” scenarios and evaluate the results for accuracy. (The New York food stamp laws had specifically tripped up on a category of undocumented immigrants who were also domestic violence victims). The proof of the software, then, would be in its results — and tested publicly before deployment.

Prof. David Super, a colleague of Prof. Citron and visiting this year at Harvard, has also suggested avenues for federal change, namely, the Office for Management and Budget under its “Management” mandate. OMB issues circulars governing how the quality of products and services purchased by the federal government which include software (§277), but most if it pertains to accounting except for §277.18. We might consider contacting the Office of Information and Regulatory Affairs (OIRA), whose mission is ensuring good government, and making the dual argument that the software the government currently procures is (a) bad, and (b) burdensome to maintain.

Ultimately we predict that legal and software code will merge as semantic computing becomes more powerful. Indeed, perhaps “legalese” will evolve into an even more technical, even self-executing, language. One way or another this evolution will also call for more specially-trained lawyers who can bridge the two faces of code.

But until then, I think our best hope for democratic accountability over the software that increasingly shapes public life will be robust testing suites as Ryan suggests and further research down the paths that Prof. Citron has broken open for us. And, perhaps, never giving one of our artificially intelligent agents a nuke.

…[T]he form that Virginia courts use to notify state police about a mental health disqualification addresses only the state criteria, which list two potential categories that would warrant notification to the state police: someone who was “involuntarily committed” or ruled mentally “incapacitated.”

However, Mr. Cho belonged to a third category: “determination by a court, board, commission or other lawful authority” that as a result of mental illness, the person is a “danger to himself or others.” Thus, a special justice’s order that he seek outpatient care and that also declared him an imminent danger to himself was never transmitted to the federal system of handling background checks for handgun purchases.

The article mentions Representative Carolyn McCarthy’s efforts to “automate their criminal history records so computer databases used to conduct background checks on gun buyers are more complete.” McCarthy (who happens to represent my hometown) introduced in January H.R. 297. The bill would require state officials to report disqualifications to the National Instant Criminal Background Check System (NICS) as well as provide funds to fund “establish or upgrade information and identification technologies for firearms eligibility determinations” and “improve the automation and transmittal to federal and state record repositories” disqualifying factors.

Monday’s tragedy offers an extreme example of what happens when jurisdictions fail to reconcile discrepancies in their laws. The answer, however, doesn’t really lie in information technology. Virginia laws didn’t match federal laws, no matter what the technological implementation; no amount of software coding would have changed that. IT can speed up the transfer of information, but an information pipe with no connection on the other side would still be a road to nowhere. Fixing state-federal disconnects will require more than just software code: it will take monkeying around with old-fashioned legal code.

(See also my personal response to the Virginia Tech shootings).

In the Matrix, Code is Law

I began seeing the very mundane, but vitally important, ways in which software is increasingly becoming the mechanism whereby government executes laws. Lessig’s profound observation, Code is Law, has a corollary: law is code. That is to say, software code is one way in which laws enter our daily lives.

In every state I’ve surveyed, it is software that calculates the amount of money that a family receives in food stamps. But code is everywhere: the controversy over voting machines’ source code is another, highly-publicized, example of this idea.

If software code really is an embodiment of our laws, then I argue that the same principles, procedures, and safeguards that apply to the promulgation of (administrative) code should also apply to the implementation of (software) code. That is to say, the concepts of administrative law — and specifically, rulemaking — should guide how governmental software becomes, effectively, law.

The Matrix, infiltrating the Real World.

I was privileged to give a short talk on this idea last Tuesday at the monthly Cyberscholars meeting here at Harvard; here is the PowerPoint. I’ll be adding a few more posts in the future about some of the specific ideas and cases in that presentation, as well as some of the very helpful feedback I’ve received.