Just want to share what a trip it is to have the TopTenSources daily OPML summary syndicated as a directory on Dave Winer’s Scripting News site. Yesterday we did lots of scrambling to
improve the user experience of the directory, to respond to
needs/problems that were not apparent to us until Dave put us up on his directory roll.
For those in the OPML community who are interested in syndicated
directories, here is a summary of our experiences. Some of our
“lessons” are relevant only to our situation, but some may be of
broader interest.
1. Our OPML did not validate. It showed up on Dave’s OPML directory sub-site,
more or less, but blew up and returned ugly error messages if a user tried
to click through to a post. This same OPML had validated in the
past, but with the format going through revisions, we now realize that
it is important to validate regularly and proactively.
2. Our first version of our OPML daily summary was based on a
listing of the items a user could click to from our home page. As
a result, we inadvertantly left out a link to the central content
of our home page, which is our daily blog. After seeing our OPML
rendered as a directory, the omission was apparent. We added to
our OPML output a link that goes to our daily blog.
3.
Our original OPML was designed to be used in newsreaders like Bloglines, so we
pointed our OPML to the RSS/XML for our sources. The RSS in turn
came in many varieties, which rendered poorly in a browser when clicked
on in Dave’s directory. So we concluded that we needed to rewrite
our OPML to point to HTML links, rather than or in addition to RSS/XML
feeds of our sources.
Currently we are outputing two OPML files
now from TopTenSouces, one for directory applications and one for news
aggregators. We believe that we will ultimately be able to output
one OPML file with pointers to both HTML and RSS/XML, and depend on the
application that renders the OPML file to decide what is appropriate. In many cases the directory
renderer would “prefer” the HTML or incorporate RSS/XML
renderers, and aggregators would prefer RSS/XML.
4. RSS/XML are usually made available by publishers at the
source/site/or feed level, and our original OPML
pointed to one feed per source. A problem arises when one
switches from RSS/XML to HTML. When pointing to HTML, there is
no predictable single navigation entry point to a site. Pointing
to the
home page of a site is not necessarily similar to picking up the main
RSS/XML feed of a site, and in many cases HTML home pages have special
display programming that makes matters especially
difficult. Thus we concluded that if we are linking to HTML
rather than RSS/XML feeds, the ten sources in our OPML-based directory
need to be joined by post-level OPML directories that in turn link to
HTML of specific posts. Here is an example from one of today’s
sources:
Top > Top 10 Sources > Astroblog
| 1. | This weeks fuzzy planet photos are … |
| 2. | PT, the place Science refers its readers to… |
| 3. | Searching for Venus’s Shadow. |
| 4. | Martian Anniversary |
| 5. | Twinkle, Twinkle little star |
5. Once we started using directories like that above, we were
delighted to discover that they were very helpful to users.
We liked the experience of being able to navigate two
levels of directory–the overall list of sources, and each source’s
list of posts–before going to the sites themselves.
A two-level
OPML-based directory made consistent and scanable all of the navigation
schemes across all ten source sites. This is a tremendous joy, and is
exactly parallel to the increase in scan speed and comprehension that
results from HTML sites being reduced to RSS, and eliminating most
site-specific formating. OPML directories eliminate most site-specific
navigation formating.
6. The overall mindblower is that we were able to do our experiments and improvements post
hoc, after Dave linked to us, and in real time. By the magic of
syndication our changes were immediately reflected on Dave’s site, and
experienced by Scripting News users.
We started scambling around 8
AM EST, and were conscious that we needed to get things worked out
before the west coast woke up. Credit Bela, Andrew Dallas and Justin
Russo. During the couple of hours that we worked, we generated some
minor disasters, and of course were freaked out when they showed up
live on Dave’s site. However, the only way we could test our fixes was by
experiencing them on the site. Strong motivation! When we finally got things more or
less right, we felt a great sense of satisfaction.
Moreover, through the power of syndication we automatically
provide a new
directory to Scripting readers each and every day, with a fresh
collection of TopTenSources. Yesterday our focus was online deals and coupons sites. Today we have ten on astronomy. Tomorrow, only our
editors know.

Thanks Dave and other members of the community for bringing life to the OPML fishpond!
Addendum:
Danny Ayers had this to say about my previous post on Dave’s directory roll. Here are Danny’s comments:
The directoryroll is a an interesting model, in fact I’m looking forward to Gopher NG.
But what you see on Dave’s web site is HTML. The actual functionality
needed for what you are describing is already with us in HTML and the
web browser. Even this page contains a directory of links, which could
easily link to other people’s lists and directories of links. OPML may
be an aid to imagination, it’s easier to think out of the box with new
toys (or rather reappraise old tricks like Gopher and Yahoo!
directories), but there’s absolutely no need to create a whole new
stack of formats and tools. I could just as easily subscribe to the
links on this page as the links in an OPML file you provide. The only
real difference being that I’d need a special tool to view the OPML,
whereas the HTML links are usable in the browser. We already have this
technology, let’s move existing tools forward, rather than being led
down a cul-de-sac.
Sometimes painters who get stuck in their work go back to pen and ink,
or charcoal, for inspiration. A reduced palette can lead to expanded
possibilities…
I agree that directory rolls can be done with HTML. On the other
hand, what I found powerful was that OPML gave me a “reduced palette”
compared with HTML, and thus my and others’ directories were stripped
to their essentials and could be easily traversed and scanned, combined
and modified.
Moreover, I suspect that the standardization and structure of directories expressed in OPML may give rise to new forms
of search and sharing and combinations and deconstructions and
interlinking, as are forshadowed/anticipated in the sharing features of
the OPML editor, and in the category (aka “tag”) searches and publishing features of OPMLsearch.
Like HTML reduced and structured into RSS.