- Rethinking Online Culpability: The Amazon “Keep Calm” Shirts Controversy (Part 4: Concluding Thoughts)
In early March, the online retailer Solid Gold Bomb provoked outrage when customers discovered that its Amazon store, which featured apparel bearing dozens of variants on the famed “Keep Calm [and Carry On]” slogan, included a t-shirt that read “Keep Calm and Rape A Lot.” Solid Gold Bomb generated the shirts, and Amazon offered them for sale in its marketplace. To complicate matters, it appears that Amazon doesn’t review the stores in its marketplace like a mall owner might review physical storefronts, and, particularly unusual, Solid Gold Bomb didn’t review the shirts they offered for sale: the designs were computer generated. How far, then, should blame extend? When unsupervised automation produces results that everyone regrets, how do we decide whom to hold responsible, and when do we decide to hold anyone responsible in the first place? This post is the fourth installment in a four-part discussion of the issue. Part 1 can be found here.
It is difficult to discern how much blame companies like Amazon and Google deserve for content that appears on their platforms, and where exactly their purview begins and ends. Decisively untangling the mess of overlapping responsibilities in the “Keep Calm” shirts controversy is a complicated process. As humans and computers further overlap in the generation and dissemination of online content, questions like these will only become more convoluted. It seems, however, that in our general reflexes to blame platform curators, we rarely factor in the degree of supervision those curators have. Since human and automated curation so frequently overlap, at what point is it most judicious to censor the results of algorithmic processes that individuals find objectionable?
We are left with a choice: we can demand that internet shopping portals show the same sort of curating we’d expect from a brick-and-mortar mall, or we can change our expectations to reflect changes in business practices. Given our options, what sort of Amazon is the best Amazon? Do we want a marketplace that curates its content based on popular reaction; or a marketplace that is regulated in accordance with the law, but not individual taste?
In truth, hands-off marketplaces may be the only real option, thanks to the ubiquity of massive A/B tests and other automated algorithms. The Amazon Marketplace is already much too large for each item to be supervised by humans in any meaningful way. Fortunately, uncurated marketplaces may not be all that bad. After all, we don’t blame the creators or retailers of Scrabble simply because you can spell out “keep calm and rape a lot” with the included tiles. And while uncurated marketplaces may lead to a flourishing of distasteful content, commercial sensitivity to popular opinion can give rise to events like Wal-Mart pulling a shirt that featured the text “Someday, a Woman Will Be President.” With the expansion and automation of online retail may come a diminishing of our perceived entitlement not to be offended by the items for sale at the stores we patronize.
There is remarkable potential for the enriching proliferation of efficient—and offensive—computer-aided commerce (and algorithmic serendipity like horse_ebooks). But the receding presence of direct human supervision means we need to make a thoughtful assessment of the way we express, and regulate, outrage in digital commerce.
- Ben Sobel, Kendra Albert, and JZ
- Rethinking Online Culpability: The Amazon “Keep Calm” Shirts Controversy (Part 3: (Un)supervised Algorithms)
In early March, the online retailer Solid Gold Bomb provoked outrage when customers discovered that its Amazon store, which featured apparel bearing dozens of variants on the famed “Keep Calm [and Carry On]” slogan, included a t-shirt that read “Keep Calm and Rape A Lot.” Solid Gold Bomb generated the shirts, and Amazon offered them for sale in its marketplace. To complicate matters, it appears that Amazon doesn’t review the stores in its marketplace like a mall owner might review physical storefronts, and, particularly unusual, Solid Gold Bomb didn’t review the shirts they offered for sale: the designs were computer generated. How far, then, should blame extend? When unsupervised automation produces results that everyone regrets, how do we decide whom to hold responsible, and when do we decide to hold anyone responsible in the first place? This post is the third installment in a four-part discussion of the issue. Part 1 can be found here.
There is at least one analogue to Solid Gold Bomb and Amazon’s implicit endorsement of the T-shirts. Google’s search algorithm has come to be viewed as an arbiter of online content, much like the way Amazon’s Marketplace dominates internet commerce. For the most part, Google searches will lead users to unfiltered results, without insinuating that Google approves or disapproves of these results. For example, although a search for “stormfront” yields a popular white supremacist website as the first hit, nobody can seriously accuse Google of sympathizing with neo-Nazis.
Sometimes, though, the content Google returns does represent a conscious position. While most of the time, Google’s suggested search algorithm is driven exclusively by “the search activity of all web users and the content of web pages indexed by Google,” the company acknowledges that it occasionally curates its suggested searches to avoid steering users towards pornography and copyright violations. This partial curation makes it impossible to discern whether content on Google is simply an algorithm’s presentation of the “state of the internet,” or if it is in fact a result of a deliberate corporate stance. Just as it’s difficult to gauge Amazon’s support of each product in its Marketplace, it’s very difficult to tell exactly what search results Google does and does not endorse.
One particular case highlights the ambiguity of Google’s role as an internet gatekeeper: in September 2012, Bettina Wulff, the wife of the former president of Germany, sued Google because slanderous terms like “prostitute” would appear as suggested searches after her name was typed. Should these suggested searches be understood as Google’s opinion, or an unfiltered snapshot of an offensive internet community? When Google search results occasionally do represent the company’s views, how can users decide when particular results indicate a deliberate position and when they are simply the output of an algorithm?
Continued here.
- Ben Sobel, Kendra Albert, and JZ
- Rethinking Online Culpability: The Amazon “Keep Calm” Shirts Controversy (Part 2: the Extension of Branding)
In early March, the online retailer Solid Gold Bomb provoked outrage when customers discovered that its Amazon store, which featured apparel bearing dozens of variants on the famed “Keep Calm [and Carry On]” slogan, included a t-shirt that read “Keep Calm and Rape A Lot.” Solid Gold Bomb generated the shirts, and Amazon offered them for sale in its marketplace. To complicate matters, it appears that Amazon doesn’t review the stores in its marketplace like a mall owner might review physical storefronts, and, particularly unusual, Solid Gold Bomb didn’t review the shirts they offered for sale: the designs were computer generated. How far, then, should blame extend? When unsupervised automation produces results that everyone regrets, how do we decide whom to hold responsible, and when do we decide to hold anyone responsible in the first place? This post is the second installment in a four-part discussion of the issue. Part 1 can be found here.
Ultimately, the way we evaluate Amazon’s responsibility in the Solid Gold Bomb embarrassment hinges on the way we perceive the Amazon brand’s interactions with Marketplace vendors and customers. The Amazon Marketplace has rules in place that restrict items that can be sold, including a prohibition on “products that promote or glorify hatred, violence, racial, sexual or religious intolerance or promote organizations with such views.” eBay and Etsy have similar provisions. The fact that numerous permutations of a reprehensible—and, accordingly, forbidden—item like the “Rape a Lot” shirt could appear in Amazon’s Marketplace is a testament to the limitations of platform proprietors’ abilities to police their services’ offerings.
But doesn’t Amazon’s relationship to products in the Amazon Marketplace feel somehow different from the way eBay and Etsy associate themselves with products sold through their portals? There are plenty of utterly reprehensible things for sale on the latter two platforms, such as this rape-satirizing T-shirt for sale on Etsy, which, according to its description, was made from an iron-on transfer purchased on eBay. For whatever reason, however, it seems more to be expected that eBay and Etsy might offer the offensive output of rogue users. Amazon feels different, and this difference lies in Amazon’s vigorous standardization and extension of their brand. Thanks to things like uniform, white-background product photos (absent on Etsy and eBay) and product pages that mention the seller only once, in ten-point font, non-Amazon sellers look like Amazon. Amazon has characterized itself as a place where anyone can buy anything, but not just anyone can sell anything—though, as the Solid Gold Bomb case demonstrates, this may be far from true. However, it certainly is a clever marketing strategy. Although much of what you can find for sale on the Amazon.com domain isn’t in Amazon warehouses, people still think of Amazon—and not Amazon marketplace vendors—when setting out to buy nearly anything on the internet. This sort of vast, recognizable umbrella branding is so effective at creating trust and simulating corporate endorsement that it even worked for a guy who modeled his resume on the Amazon site design. Amazon, more than most other major online retailers, appears to style itself to tacitly endorse everything available in its Marketplace. This is what made the outrage at Solid Gold Bomb’s products lead back to Amazon. Had the “Keep Calm” shirts been posted on, say, Craigslist, it’s doubtful that anyone would have lashed out at the platform used to sell them.[1]
- Ben Sobel, Kendra Albert, and JZ
[Continued here.]
- Rethinking Online Culpability: The Amazon “Keep Calm” Shirts Controversy (Part 1: A/B Testing)
In early March, the online retailer Solid Gold Bomb provoked outrage when customers discovered that its Amazon store, which featured apparel bearing dozens of variants on the “Keep Calm [and Carry On]” slogan, included a t-shirt that read “Keep Calm and Rape A Lot.” Solid Gold Bomb generated the shirts, and Amazon offered them for sale in its marketplace. To complicate matters, it appears that Amazon doesn’t review the stores in its marketplace like a mall owner might review physical storefronts, and, particularly unusual, Solid Gold Bomb didn’t review the shirts it offered for sale: the designs were computer generated. How far, then, should blame extend? When unsupervised automation produces results that everyone regrets, how do we decide whom to hold responsible, and when do we decide to hold anyone responsible in the first place?
Solid Gold Bomb’s official apology explained that its Amazon store featured millions of hypothetical shirts to be produced on-demand, should anyone order one. The “Keep Calm” debacle resulted from an automated script that generated words to approximately fit the design’s syntax and layout. The resulting list, says SGB owner Michael Fowler, “was culled from 202k words to around 1100 and ultimately slightly more than 700 were used due to character length and the fact that I wanted to closely reflect the appearance of the original slogan graphically.” Clearly, the vendor is at fault for failing to eliminate possible ending phrases to the Keep Calm slogan like “rape a lot” and “choke her” from a 700-word list. However, similarly automated practices regularly take place on a much larger scale across the internet. Determining accountability for these widespread and fundamental operations can be much less straightforward.
In some ways, Solid Gold Bomb’s generation of the offensive shirts can be seen merely as A/B testing gone awry. Offering thousands of options and printing shirts to order is a way of using user behavior to cull successful products. Presumably, if one of the quasi-randomly-generated shirts began to outstrip the others in sales, Solid Gold Bomb would have adjusted its inventory and marketing accordingly.
With A/B testing, the line between savvy capitalism and unethical business practice can get fairly nebulous. Zynga, for example, relies on a practice that CEO Mark Pincus calls “ghetto testing.” One of Zynga’s approaches to game development is to advertise games that do not yet exist, in order to test consumer response to a basic premise. Says Pincus,
“We’ll put up a link for five minutes saying, ‘Hey! Do you ever fantasize about running your own hospital?’…We’ll put that up for five minutes, and the link will maybe take you to a survey, where you give us your email and we say when this comes out we’ll contact you. If you’re really doing ghetto, it says ‘404 not found’. That’s bad. So first you try to get the heat around it, you see how much do people like it, then…”
This isn’t all that dissimilar to Solid Gold Bomb’s approach. Like Zynga’s “ghetto-tested” games, the “Rape a Lot” shirts didn’t actually exist, and would only have been produced in accordance with user demand. In fact, Solid Gold Bomb didn’t misdirect potential buyers as deliberately as Zynga’s “ghetto testing” approach does.
In large, computer-conducted A/B testing campaigns, it becomes impossible to demand human supervision of every output. Solid Gold Bomb’s 700-word list for generating T-shirts should have been thoroughly scrutinized, of course, but operations with more permutations of A’s and B’s seem less accountable for each potential outcome. For example, it’s hard to believe it would be within a webmaster’s responsibility—or even her ability—to make sure that every possible banner ad on every single page of a site doesn’t combine unfortunately with the page’s content.
A/B testing is practically ubiquitous online, and most of its applications are unequivocally benign. Wikipedia, for one, famously self-published the test results of its 2010 fundraising push. Moreover, unsupervised, computer-conducted A/B testing can produce serendipitous results that no human could ever have engineered or anticipated. The popular twitter handle @horse_ebooks, for example, began as a poorly functioning spam account intended to drive traffic to an e-book site. But its garbled messages are so striking—and occasionally poignant (cf. a recent example)—that the bot currently has over 170,000 followers.
The problem, then, is that our expectations for internet commerce haven’t quite caught up with the techniques that drive internet commerce. If a store offers things for sale that we find offensive, our typical reaction is to get mad at the store—after all, being willing to profit off an item seems to imply some kind of endorsement of that item. Today, however, these assumptions about endorsement are challenged by the ubiquity of A/B testing and other automated content generators. A “ghetto test” by Zynga might not mean that the company fully endorses a game that simulates running a hospital. Similarly, the presence of an item in the Amazon Marketplace might not be enough to presume Amazon’s approval of that item.
[Continued here.]
- Ben Sobel, Kendra Albert, and JZ
- The Future of the Internet: Five Years Later
In 2008, The Future of the Internet called attention to a “sea change” in the way consumer devices interact with the Internet. “The future is not one of generative PCs attached to a generative network,” the book warns; “it is instead one of sterile appliances tethered to a network of control.” In response to the security threats posed by malicious third-party code, increasing numbers of users will likely gravitate towards gadgets “tethered” by continuous communication between product and vendor. And this proliferation of tethered computing—the “appliancization” of PCs—will deal a serious blow to the principles of generativity and free expression that drove the early Internet.
Since the publication of The Future of the Internet, the ethos of strict appliancization has taken a new turn. In 2011, Professor Zittrain wrote an update on the book’s message: “at the time of the book’s drafting, the alternatives seemed stark: the “sterile” iPhone that ran only Apple’s software on the one hand, and the chaotic PC that ran anything ending in .exe on the other. The iPhone’s openness to outside code beginning in ’08 changed all that. It became what I call “contingently generative” — it runs outside code after approval (and then until it doesn’t).” This trend towards contingently generative models continues into the present day, and represents a shift similar in many respects to the one The Future of the Internet predicted.
Jon Brodkin and Peter Bright’s Ars Technica op-ed on the Microsoft Metro app store offers some valuable commentary on a big development in this “sea change.” The article recognizes that “Microsoft is imitating Apple in one very bad way, by limiting the distribution of Metro applications to a Microsoft-controlled app store… by bringing Windows to tablets, Microsoft could strike a blow for openness in a market dominated by a closed system. Instead, Microsoft is bringing the same restrictions found on iPads to both Windows tablets and PCs.” As forecasted by The Future of the Internet, devices that only run approved code are gaining popularity. Metro, the curated user interface that has found its way onto Microsoft’s tablets and PCs (in the case of the PCs, alongside a fully-functional desktop mode capable of side-loading non-Windows Store applications), won’t run applications from outside the Windows Store. Moreover, the apps available through the Store are subject to a bevy of restrictions on content. With these restrictions on installable applications come the restrictions on generativity that The Future of the Internet anticipated: “lock down the device, and network censorship and control can be extraordinarily reinforced.” And, as the Ars Technica piece observes, the Windows Store’s rules would exclude critically-acclaimed content like the video game Elder Scrolls: Skyrim, simply for its PEGI 18/ESRB M rating. It isn’t hard to extrapolate, as Brodkin and Bright do, that these rules could give rise to debacles similar to Apple’s (repealed) ban of a satire app developed by a Pulitzer Prize winner.
Though the Windows Store’s restrictions resemble Apple’s policies in many ways, there is a crucial difference: Metro-running Windows 8 products are designed as PC replacements, rather than sui generis devices like the iPad. And since Windows desktops have long been preferred gaming platforms, the theoretical exclusion of content like Skyrim from the Windows Store makes Windows 8’s emphasis on the Metro interface particularly jarring.
With Metro, Microsoft has made a decisive move towards contingent generativity. Brodkin and Bright note that “there are security benefits to a closed app store model, particularly for less tech-savvy users who may not understand all the dangers on the Web. There are also, arguably, convenience benefits; end-users can be reasonably confident that the apps they download will work correctly and be at least marginally useful…But while these security and convenience benefits might be enough to justify the existence of a curated app store, they don’t justify the decision to make that store the only option for all users. Informed users should be allowed to install applications from wherever they want.” Brodkin and Bright prefer a system like Gatekeeper, a fixture in newer versions of Apple’s OS X, from Mountain Lion forward. Gatekeeper gives users the choice to restrict their operating system to App Store apps and outside apps that have been signed with Apple-issued Developer IDs, or open up the device to all programs, whether or not they’ve been vetted by Apple. The “Future of the Internet” Blog is fairly enthusiastic about Gatekeeper: about a year ago, a post here suggested that “the middle ground of allowing non-App Store signed code may represent the best of both worlds.” But we were quick to warn that Gatekeeper strikes a tenuous balance: “one small tweak — lose that Control-click for sideloading — and OS X could fully merge with iOS, both in functionality and in security methods.” Metro’s riff on content control could be just that sort of tweak—especially given recent speculation that Microsoft may dump desktop mode in Windows 9, leaving only Metro.
Moreover, a contingently generative business model like the Windows Store’s carries some ethical implications that, while not damning, are certainly worth examining. Distribution systems like the Windows Store, Apple’s App Store, and the Android Market receive 30% of the sales revenue from applications sold in their stores (in the Windows Store, this cut drops to 20% after an app reaches $25,000 USD in revenue). Further restrictions on side-loading in new operating systems would drive a great deal of business towards big companies’ proprietary marketplaces—and with that traffic would come big payouts. With the uptick in store traffic that tighter gatekeeping would engender, it’s easy to imagine the equilibrium of Mac’s OS X Gatekeeper being forsaken for more restrictive, and more lucrative, operating systems. To analogize, a la The Future of the Internet: when the company that makes your computer requires you to install programs through their official store, it isn’t so different from the company that makes your toaster forcing you to buy from their bakery—and taking a cut out of every bread purchase you make.
Even though Windows 8 PC users can still make use of a fully-functioning desktop operating system, Microsoft’s failure to include a side-loading option for the heavily-emphasized Metro interface—particularly in devices marketed as PC replacements—is a step in the wrong direction. It’s also an indication that the seas are changing in the way The Future of the Internet predicted. Given that Android’s more open approach to outside applications[1] still leaves the Android Market increasingly economically viable, Ars Technica is right to voice its disappointment in xenophobic operating systems like iOS and Metro.
- Ben Sobel, Kendra Albert, and JZ
January 17th, 2009 at 3:01 am (#)
[...] Moving Towards Generativity :: The Future of the Internet — And How to Stop It A very good article discussing the generative capabilities of the Android Mobile OS (tags: Google Internet mobile) [...]