Internet & Democracy Blog

Question: What role, then, is the Internet playing in Russian media?

—Interview by Josh Tapper, Nieman Journalism Lab

In the above quote, Elena Vartanova echos two key research questions we have for Russian Media Cloud:

1. Do blogs and other online media provide an alternative public sphere, and;
2. What role do they play in agenda setting of the news.

To begin to test these hypotheses we have built off the hard work by Ethan Zuckerman, Hal Roberts, David Larochelle, Yochi Benkler and Zoe Fraade-Blanar on English Media Cloud, which collects data on different sets of English language blogs and popular traditional media available online (mostly newspapers). For the Russia effort we have an even larger and more varied set of feeds, including:

1. 1000 popular Russian blogs: The Yandex Top 1000 list

2. Over 11,000 Russian language blogs divided into link-based attentive clusters, based on the results of our previous Russian blog research

3. 1000 random, or long tail, blogs based on our own spider of the Russian blogosphere

4. Top 25 ‘mainstream media’: This is currently the Google Ad Planner list of the top 25 most popular news Web sites in Russia, which we filtered to remove sites any sites that are not news related or not primarily about Russia (*See list at bottom of this post)

5. Russian TV news feeds from: Channel 1, Vesti, REN TV, TV Tsentra, NTV, Channel 5, Mir, Zvezda, and TV Stolitsa

6. Russian government Web sites: President Medvedev’s official site, Putin’s official site, the Russian government portal government.ru, and sites of the Ministry of Emergency Situations, Ministry of Justice, Ministry of Defense, and the Ministry of Foreign Affairs

Using the same method as Ethan describes in his blog post on calculating cosine similarity among sources and sets of sources, we are able to draw a visual map that shows how similar these different sets of feeds are to one another, based on content (as opposed to links). What this method allows us to do, and what we have done with all of the below examples, is compare the similarity of bags of words in different media sets. Media Cloud outputs alone do not say anything about the meaning behind those differences between different sources. However, with additional context about what we know of the political situation and media ownership in Russia, as well qualitative analysis of sentences within queries, we can begin to hypothesize about the possible meaning behind similarity scores, word clouds, polar maps and other automated outputs.

As Ethan writes about cosine similarity:

This is a technique computer scientists use to detect a type of similarity between documents. Basically, a computer program counts the appearances of words in a document (in this case, a week’s worth of media coverage by 25 outlets) and compares that frequency list to that of another document. If those documents are identical in word frequency – both mention Obama 23 times, Libya 5 times and basketball twice – they score a 1. If they’ve got no words in common, they score a zero.

(The actual math behind this is wonderfully cool, if slightly mind-bending. Imagine a set of documents with only two words in them – “Obama” and “NCAA”. In source A, Obama is mentioned 8 times, NCAA 2 times. Put a point on a graph at (8,2) – Obama’s our X axis, NCAA our Y axis, and draw a line that passes through 0,0 and 8,2 – that’s the vector that represents set A. In source B, Obama gets mentioned twice, NCAA 8 times – put the point at 2,8 and draw the vector for source B. The angle between vectors A and B is a measure of how similar the sets are, and taking the cosine of that angle is a simple way to scale the value to be between 0 and 1 for angles between 0 and 90 degrees. The trick, of course, is that documents contain words other than Obama and NCAA, and cosine similarity adds a new dimension to our graph for each new term. So the vectors we’re measuring when we compare all the words in 25 media sources over a week to another comparable week exist in 3000-dimensional space. Don’t bother imagining 3000-dimensional space – it will make your head hurt. Just imagine three dimensional space and think about two vectors that each emerge from 0,0,0 and each pass through an arbitrary point in positive x,y,z space – it’s easy enough to imagine measuring the angle between those two vectors. Then take it on faith that, mathematically, you can do the same thing in many-dimensional space.)

Popular Blogs Compared to the Government and Traditional Media

As a first test of whether blogs are different than Russian traditional media and government information channels, in the first polar map we compare the similarity of the Yandex Top 1000 popular blogs compared to the Russian government, TV news transcripts, and top 25 MSM over the period of December 15, 2010 to February 21, 2011. The center node, or pole around which the map is drawn, is the collective content of Russian government feeds over that same time period. The further a source is from the black dot in the center, the more different it is from Russian government feeds. What we see at first glance from this map is that, although fairly overwhelming because of their large number, most blogs are located near the outer ring of this map, while the government, MSM and TV sources are located more closely to the center of the map, showing that the media are more similar to the government than most blogs. This is probably at least in part due to the fact that Russian popular blogs are not focused exclusively on politics, which we see from the content clustering (color) process.

Polar Map

Center Node: Russian Government

Read the rest of this entry »

Let your Russian friends and colleagues know that we’ve just released the full Russian translation of our paper (pdf), “Public Discourse in the Russian Blogosphere: Mapping RuNet Politics and Mobilization.” Accompanying the release is a very nice summary and additional commentary on the role of blogs in Russia by project team members Gregory Asmolov and Karina Alexanyan that is available on Ivan Zassoursky’s excellent Web site Chastnyi Korrespondent (which our research shows is a top outlink for Russian bloggers). Gregory and Karina have some new analysis and additional blog maps not found in the original paper, so if you read Russian it is well worth checking out. We are turning our focus next to Twitter in Russia and have nearly completed an update crawl of Russian tweets, John Kelly has created an alpha Twitter map, and we hope to release a working paper in the spring. The Russian version of Media Cloud is also up and running and we hope to begin blogging some of our early finding from that tool in the next couple weeks, so check back here often for additional research on the role of the Internet in Russia.

Attention holiday shoppers! Still searching for the perfect (read: free) gift for the Internet thinker in your family? Look no further than two recent publication from the Internet & Democracy team, released just in time for solstice.

First, John Kelly, Robert Faris, John Palfrey and I adapted of our earlier Arabic blogosphere case study into a journal article, “Mapping the Arabic Blogosphere: Politics and Dissent Online,” which appears in the 2010 issue of New Media & Society. You can download the pdf (here) (available online as of December 21, 2010). In that article we focus on the political aspects of the Arabic blogosphere and evaluate whether it meets the conditions for a networked public sphere, as defined by Yochai Benkler.

We also wrote a short think piece on online organizing that was published this month in the SAIS Review of International Affairs. Here is the abstract from the paper, “Political Change in the Digital Age: The Fragility and Promise of Online Organizing:”

In this paper, we discuss the possible impact of digital technologies in authoritarian and semi-authoritarian regimes. We conclude that policymakers and scholars that have been most optimistic about the impact of digital tools have over-emphasized the role of information, specifically access to alternative and independent sources of information and unfiltered access to the Internet. We argue, in contrast, that more attention should be paid to the means of overcoming the difficulties of online organization in the face of authoritarian governments in an increasingly digital geopolitical environment.

If that’s not enough to satisfy your loved ones’ reading habits during the darkest days of the year, my colleagues here at Berkman (Ethan Zuckerman, Hal Roberts, Ryan McGrady, Jillian York and John Palfrey) released this week an excellent study on DDOS attacks against human rights groups and independent media, and I also just discovered that Berkman Fellow Clay Shirky has a smart, balanced essay out in Foreign Affairs, “The Political Power of Social Media.”

Check ’em out. I hear several appear somewhere in print form as well.

On Tuesday, Google released a Transparency Report that shows the number of government inquiries it receives for information about users and requests for Google to take down or censor content.

The report, which Google presents in the form of an interactive map and traffic graphs, reveals the number of content removal and data requests that it received from government agencies around the world during the first 6 months of 2010. (See this Herdict post on the traffic graphs.)

The map tool allows one to see:

The number of government requests we received to remove content, per country;

The number of individual items asked to be removed, per country (as there may be many URLs per request) — this is new for 2010 data;

The products that contained the content; and

The percentage of those requests that we fully or partially complied with on a country-by-country basis.

Internet censorship around the world is on the rise, according to the report. A comparison of data from the internet service database that it released 5 months ago and the current version shows that there has been an increase in the number of government requests, suggesting that internet freedom may be taking a turn for the worst. Note: the map tool excludes data request information for a number of countries, most notably China and Iran. Using the traffic graphs, however, one can see how much traffic Google services get in those countries. The U.S. currently occupies the #1 spot with  4,287 government requests for data information during the first half of 2010. Directly behind it are Brazil with 2,435 and the U.K. with 1,343. Just five months ago, however, Brazil was in first place, the U.S. in second, and the U.K. in third.

While the report does contain important and telling information about censorship levels worldwide, the data that Google provides is not totally comprehensive and accurate. It fails to address a couple of major criticisms that arose when it first released an internet service database. Most notably, the  tool does not reveal query specifics, such as how many requests government agencies made for the same user data and what the total amount of requests across all categories of content is. Google also admits that among its limitations is the fact that that it has not included statistics for countries in which fewer than 30 requests for user data in criminal cases during the observation period.

Moreover, the report does not account for government-mandated service blockages—only content removal requests. A major addition that Google has made to the tool, however, is the ability to see how many requests it has honored: 138 in the US, 398 in Brazil, and 48 in the UK.

Google says that it would  like to be able to share more information with the public, but lack of standardization among user data requests, as well as the difficulty in categorizing and quantifying those requests, prevent it from doing so. But the internet search giant has expressed its intention to offer more information in the future, as well as a desire to fuel debate on government censorship.

“We hope this step toward greater transparency will help in ongoing discussions about the appropriate scope and authority of government requests,”  Corporate Development and Chief Legal Officer David Drummond said in a post on Google’s official blog.

By Marianna Tishchenko

This front page New York Times article, which describes how Russian officials have used trumped up charges of software piracy to pressure Russian advocacy groups, led quickly to a blanket license for Russian NGOs from Microsoft. The article appeared on Sunday; the blanket license was announced on Monday. Microsoft lawyers are now also are effectively prohibited from assisting in these cases.

The raids, however, have taken place since 2007. Russian security services have conducted dozens of raids on Russian NGOs and media organizations that voice their opposition to the country’s leadership, according to the article. Although Russian officials have claimed that the raids have no political basis, the Times notes the government “rarely if ever” conducts such inspections for advocacy groups and news organizations that support the government, suggesting that officials have used piracy concerns as an excuse to silence dissenters.

Cliff Levy of the Times focuses on one case involving the Russian environmental NGO Baikal Environmental Wave. Russian police stormed the NGO in January—supposedly in order to investigate its possession of illegal software after receiving a ‘civilian complaint’—and confiscated staff computers that may have contained illegally downloaded software. The NGO insisted that it had paperwork that could prove that the Microsoft products had been legally purchased. Russian officials were not interested in seeing it, however.

The fact that Baikal Environmental Wave had been planning protests against a government-sponsored reopening of a paper factory near Lake Baikal when Russian officials stormed their facility makes their intentions suspect.

Since the release of the NYT article, a great deal of speculation has surrounded Microsoft’s role in raids. And in an attempt to dispel rumors and ward off criticism, Microsoft issued a statement on Sept. 13, in which it expressed strong disapproval of the use of software piracy concerns as a scapegoat for “nefarious purposes.” The company also announced its intention to create a unilateral software license for NGOs that will give them free and legal software.

“Whatever the circumstances of the particular cases the New York Times described, we want to be clear that we unequivocally abhor any attempt to leverage intellectual property rights to stifle political advocacy or pursue improper personal gain,” Senior Vice President and Microsoft General Counsel Brad Smith said in the statement. “We are moving swiftly to seek to remove any incentive or ability to engage in such behavior.”

One hopes that Microsoft will get out ahead of any efforts by other governments to carry out similar raids in other countries.

This is a visual representation of the Moscow metro bombing discussion on Twitter. It is a word cloud of 1000 random tweets from the #moscow hashtag (which saw a five fold increase after the bombings) based on research we’re doing with the Sam Gilbert and the Web Ecology Project. We are still working through #metro29 and other hashtags that were more popular among Russian language Twitter users. Turns out it’s pretty hard to say anything too original in 140 characters.

And for comparison, here’s a word cloud representing the discussion on the bombings in the US and Russian press, drawn from the full text of 68 articles related to the bombing from Johnson’s Russia List. Both clouds were created with Wordle.

This is intended mostly as a fun experiment as we build more accurate tools to make these type of inter platform comparisons, but it is still pretty striking to see how limiting Twitter can be when trying to tell a story. In any case, it is still far better than Russian TV news.

Update: I replaced an earlier Twitter cloud to strip out the date, which I think was a cut and paste error on my part, since that doesn’t appear in the text of the Tweets.

As I wrote last week, Twitter is starting to expand beyond its English-language roots in the US. Yesterday the Russian search engine Yandex released a short fact sheet on Russian language Twitter users. Here are the highlights (via Nick Wilson):

* Yandex estimates 183 thousand Russian accounts on Twitter
* More than 60% of Russian-speaking users update their Twitter stream every day
* 67% of all Tweets contain links, 8% of Tweets contain links to the media.
* During the winter 2009-2010 the number of Russian-speaking users on Twitter has
increased by 42%. In the year period from March 2009 to March 2010, by 26%.
* About 150 thousand Tweets (messages) are posted each day in Russian. 5% of them are ReTweets.
* There are more than 125 thousands links published on Twitter each day.
* Yandex studies more than 20 microblogging services in RuNet. Every day, more than 2 millions entries are made.

Perhaps most interesting is how strong the growth is during the winter compared to spring – gotta do something during those long Russian winters I guess. Yandex has also created a list of the most popular Russian Twitter users. Not much on the substance of the discussions taking place, but we are starting to dig into that now.

This morning the New York Times quotes a ‘senior administration official’ who says that the US is set to relax sanctions against Iran, Syria and Cuba to allow US companies such as Google, Microsoft and Yahoo to allow downloads of personal Web-based services in those countries. Around the water cooler this morning, my colleague Jill York correctly pointed out that the article appears to conflate too many things together when it describes ‘Internet services’ that are currently banned, and that might be allowed as part of the planned waiver. My understanding is that any service that is based ‘in the cloud’ (gmail, twitter, etc.) is currently allowed to be used in Iran, Syria and Cuba because they do not require users to download software to use those services. It seems that Treasury’s Office of Foreign Assets Control (OFAC) is considering a blanket waiver that will permit US companies to allow users in those sanctioned countries to also use services (such as MSN chat) that require a download. It’s not clear if the type of software downloads will be limited to ‘communications’ services, or if broader downloads of services such as Google Earth or Adobe Photoshop will be allowed. It strikes me that the broader the type of downloads allowed the better, since the more services available not only allow for greater creativity in how users mash-up blogs, video, photos, email, etc., but that also makes it harder for states to block one type of service if many are available and being used together. However, given that circumvention tools will not be included in this waiver, it appears that the language may be fairly restrictive. Regardless of how the technicalities shake out, this seems like a positive step forward and I’m hopeful even smaller companies like Blue Host, that have been denying use of their Web hosting service in any country with even fairly limited US sanctions, will again make their services more widely available.

UPDATE: Here’s the official Treasury announcement and the updated rule–looks like Syria loses out on this one. From Deputy Treasury Secretary Wolin:

The new general licenses authorize exports from the United States or by U.S. persons to persons in Iran and Sudan of services and software related to the exchange of personal communications over the Internet, including web browsing, blogging, email, instant messaging, and chat; social networking; and photo and movie sharing. Today’s amendments also provide that specific licenses may be issued on a case-by-case basis for the exportation of services and software used to share information over the Internet that not covered by the general licenses.

According to the Twitter blog, last year Twitter use grew by 1,400%, and now there are over 50 million tweets a day, or 600 a second on average. Our friends at the Web Ecology Project have done some of the best early research on Twitter. While #iranelection was a major story in 2009, it pales in comparison to the number of Tweets about Michael Jackson’s death (78 per second at its peak) over a similar two week period. In fact, it appears that Jackson’s death actually sucked all the air out of the Iran election discussion on Twitter, according to what Ethan Zuckerman tells me based on Media Cloud data.

While this remains primarily an English language and US-centered technology, it has been interesting to see the growth of other languages, including Russian, which we are digging into more deeply, (to say nothing of the strength of Malay(!) on Twitter). The Russian search giant Yandex already has a number of Tweeters among their top 1000 bloggers, and they reported last year that microblogging platforms have seen impressive growth, led by Twitter but also including Juick, a Jabber application. Whether or not all of this is a good thing we’ll leave up to others to debate, but it seems that the people are voting with their feet on this one.

Hat Tip: The Daily Dish

On February 20, Harvard Business School hosted its eleventh annual African Business Conference in Cambridge, Massachusetts.  While the conference broadly addressed the continent, a number of breakout events focused on the centrality of mobile technology, as addressed in titles such as “Reaching Africa’s Un-banked Population,” and “African Telecom: Growth and New Opportunities.” The over 4 billion mobile phones now buzzing around the planet, however, hide the fact that in Africa mobile penetration is only 40 percent.

Mobile mixed with e-commerce may yet be the most empowering platform, in Africa and beyond. This “e-Mobility” is taking shape, as mobile payment platforms improve access to services, and reduce transaction costs associated with payments.  Lowering transaction costs saves time and travel, effects that improves household disposable income by between 5 and 30 percent. A CGAP report indicates that mobile transactions cost one-fifth of traditional ones.

Mobile banking and transaction enabled phones bring the un-banked population into the formal folds of financial services, a delicate process necessitating new trust, but one through which inflows of capital expand the ability of local banks to provide entrepreneurial loans, expanding business and jobs, and fueling growth.

One HBS panelist and McKinsey principal, Zakir Gaibi, focused on subscriber growth as a vital first step to improving mobile access.  Foremost, was the need to reduce handset costs to facilitate access within the ultra poor demographics of the regions in Africa, such as the Sahel.  In Niger, Chad, and even in the Democratic Republic of Congo, he noted that mobile penetration is cited as low as 10 percent.  High tele-density is not yet universal.

Subscriber growth in unsaturated markets will be contingent on reduced costs of access, notably in headset costs, and in voice service costs.  Due to poor power infrastructure, African telecommunications companies rely on expensive generators that drive up costs.  It’s often quipped that second to power companies telecommunication groups produce the most energy in Africa.  However, these costs are passed onto consumers in the form of expensive voice and data plans, some approaching per minute costs in developed markets, despite drastic differences in purchasing power.  Moreover, concurrent deficiencies in local human capital necessitate talent sourcing abroad, and ex-patriot retention packages impact consumer prices, further impeding subscriber growth in the poorest markets.

Countries that recognize mobile trends could capitalize in the form of economic growth, and increased employment opportunities.  Countries like Morocco have built upon expanding tele-density, and created an infrastructure that has enabled them to provide services to the European market.  Given their geographic and temporal proximity to the EU, and their linguistic overlap in French, Spanish, and English, mobile penetration, tele-density, and infrastructure investment have created a conduit for service export, economic growth, and human capacity building.

Perhaps investment in reducing headset materials cost could increase mobile penetration, improve access to mobile banking, help formalize financial services the poorest regions of Africa, facilitate greater access to capital, and improve growth.  Or then again maybe cheaper headsets coupled with mobile promotions and spotty service will mean that those Africans who currently carry two phones to compensate for poor coverage might just pick up a third.