(Trying To) Picture Cooperation On The Web
Sorry about the week delay in posting, caught up in a wave of work and this new home for updating on the project totally slipped my mind. We’ll get the hang of this, folks — stay with us.
As you might remember from our opening post, the current aim of the Cooperation project is simple to state:
What creates cooperation online? What community rules and structural design mechanisms can be put into place to encourage collaboration?
Obviously, easier said that done: the aim of exploring cooperation (broadly speaking) online is huge — and requires the development of two things. First, some kind of representative or defensible set of examples of cooperation that can be sorted into a typology. And second, some kind of standardized data set and rigorous way of gathering that data set.
Generating these two things has been our work for the past few months, and it’s been a crazy time. Since both of these parts have enough moving parts that they deserve their own post — we’ll start with the latter.
Since most of the kinds of levers we’re interested in are hard to mechanically scrape from websites in an automated way (what norms are used on the site? who has decision-making power in the site?), the current vision is to conduct a massive human effort using what is effectively a 100+ question survey (a “codebook”) that captures the information we’re interested in learning en masse about different sites. Broadly, this includes aspects of Organization and Activity Structure (how cooperative activity is organized in terms of people), Inputs/Resources/Contributions (the types of things that are leveraged in creating the cooperative activity), Motivations and Discipline (the types of norms and personal incentives appealed to), Scope and Participation (size of community, etc), and Technological Affordances (the mechanisms used). Though any other suggestions on things that might be interesting to focus on at this point are definitely welcome.
Obviously, the perpetual balance with this sort of thing is to capture enough complexity to make the resulting data set interesting, while avoiding the vagueness or subjectiveness that tends to make the dataset meaningless. After conducting an initial pilot of the codebook on a pre-selected group of sites and using Krippendorff statistics to measure reliability, we are currently in the process of re-writing and repiloting to increase accuracy and precision when we test a question out over many different respondents. (We’ll be making this coding tool public once we’ve got the final kinks worked out).
Eventually, the idea is that this data will eventually allow us to conduct some kind of deeper observational dive into particular clusters or types of cases that have been coded with our survey tool — what we’re calling at this point as “Phase II” studies. It’ll also allow us to build some hypothesis about the kinds of structural community features that are associated with one another (“Turns out commercial sites always implement similar kinds of reputation systems for their users — wonder why that is?”)
To gain some kind of preliminary picture of how this will go, we’ve been testing the codebook out on small sets of 10-15 cases and taking a closer look at that set. This’ll allow us a) to assess how well our codebook is able to pick up on, b) how best to carve out domains that yield interesting questions. Team members Roxie (online video), Dharmishta (collaborative art online), Yael (community news reporting), and Ayelet (negative/illegal collaboration online) have been tackling some of this during the December/January time split, and you can expect the first of these to be coming out in early/mid-February.
Explore posts in the same categories: project update