Jim Moore’s blog: Innovation, Strategy, Public Policy

The value of directories created by people: a brief history

Dave Winer has for some time been running an OPML directoryroll instead
of a blogroll on his site. What this directoryroll establishes on Dave’s
site is something akin to the original Yahoo directory–a set of
intelligently categorized links that goes deep into the web.  See the original Yahoo directory, October 17, 1996, courtesy of the Wayback machine Yahoo archive. 

Indeed, as David Mercer has stated so well this morning, it recapitulates Gopher and the earliest pre-directory days of the
web. Thanks Dave for this link also, which started my rumination today.  As Mercer says,

Back in the days when Gopher ruled supreme, and the Web was just a
little Cern linemode client icky-ness, where you ‘clicked a link’ by
typing the number by it at the bottom of the screen, Gopher had
structure.

You could search gopher-space, and someone had
categorized that information into a hierarchy just by dint of putting
it up on a gopher site: gopher sites were merely hierarchical menus
that had content as the leaf nodes.

Yahoo and other earlier directories more or less institutionalized the structure of Gopher.

The Yahoo directory required a team of editors, and the editorial team
could not scale with the web.  Thus came, to forshorten history a bit, Google with the page
rank systems.  Instead of Yahoo’s team of trained, paid editors
sorting the web by explicit categories, Google organized the web by
examining the implicit categories of vast hordes of unpaid and
untrained editors.  The links into a site from site rolls and blog
rolls determined important semantic nodes and their interlinked
networks of sites, and then keyword analysis was used to determine the
nature of the resulting clusters.  Google had the advantage of
scaleability, but the disadvantage that human judgment, while still
central to its system, was embedded and implicit rather than direct and
explicit, as in the Yahoo system.

Directoryrolls combine advantages of Google scaleability and the early Yahoo editorial oversight

What is cool is how the directoryroll reintroduces the original
Yahoo vision, of hierarchical lists of intelligently-categorized
selections made by editors, and do so in a highly distributed
participative way.  In the Scripting News example, Dave is able to
maintain his own
top
level Yahoo-style directory, with his own selection of editors, and the
entire structure is kept fresh by the magic of syndication.  Any
one of us can do his or her own
version, too!

OPML-enabled directories combine the advantages of  Google–of scaleability, harnessing
millions of unpaid volunteers, and maintaining an ever expanding
population of “reviewers” as well as “reviewed”–with those of the original Yahoo editorial team.

People are now consciously creating and sharing reading lists and OPML
directories of sites of interest. But more
important, as OPML management tools proliferate people are assembling their own
directories of directories.  That is, they are assembling for
themselves carefully selected groups of web editors, who are themselves
creating and maintaining directories, in the Yahoo fashion.  And
all this is being done in an open and unpaid fashion–sometimes as a
completely voluntary initiative, and sometimes supported by ads.

And every person who has a site can in turn subscribe to OPML outlines
which in turn reference layers upon layers of directories, maintained
by layers and layers of
editors, pointing to vast, folk-structured rivers and oceans of
content.  And all made manageable by easily expandible, scannable,
and searchable OPML directories.

OPML-enabled directory trees are folksonomies on steriods,
personalizable by each of us as individuals, explorable and enjoyable
in aggregate by all of us.

Directoryrolls target dynamic content

But there is another fundamental difference between the web of then and the web of
now.  The most important new web content is dynamic.  It is less a universe of
places, and more a universe of rivers and flows. 

In the case of the Yahoo directory, it’s editors identified
the most
innovative sites of the time, which were in HTML and were static. 
The Google has added blog searches, it’s forte is referencing static
sites.

In
the case of Dave’s directory, Web 2.0, and the world of OPML
directories, new tools help us make sense of a web that flows. 
They help us make sense of meme flows, meme propogation, meme transfer,
memes swimming in a world of feeds.  The new web is based on
continously updated, vastly open sources made
available by the syndication paradigm. These sources are OPML, RSS, and
blogger-oriented, conversational, HTML pages.  They are podcast
and videoblogs.  They are emerging patterns in tag clouds and
across tag-based communities.

Directoryrolls enable participatory, distributed improvements to categories and hierachies

Moreover, the categorization schemes we use to point to patterns in
the flows are themselves of necessity fluid if they are to be
relevant.  The syndication of directories
(e.g. by OPML), in addition to content (RSS), enables people who do not
know each other to
participate in the shared creation of a world of directories, and to do
so in
real-time without the intervention or oversight of any top level
directory owner.  Each incorporated directory in a tree has its
own authors/editors, and the tree as a whole may have dozens or even
hundreds of editors.  These editors are free to modify their
directories as they see fit, whenever they find it necesssary or
helpful to their anticipated users.

Today I experienced this flexibility in a minor but telling way.  Dave kindly put
TopTenSources and its OPML on his directory roll.  Unfortunately, the OPML was messed up and created lots of
uglies when viewed in the renderer.  Yuck!  Fortunately–and this is my
point–we put in a fix.  This required changing in subtle but vital ways the directory
structure and user experience that led to and framed the content on
TopTenSouces.  And when we changed our OPML, it was in turn updated on Dave’s
site.   All this was accomplished without any direct communication this morning with Dave.