Internet Librarian: Chris Sherman: Search Engine Report
Chris Sherman is of Search Engine Watch. With Gary Price, he coauthored a book about the invisible Web a few years ago.
I am torn between sessions in the Web Design and Development track and the Information Discovery and Search track. The Web Design and Development track has a lot of Web 2.0 stuff in it. Since I like learning about those technologies and many of you are hoping I’ll write about that kind of stuff here, I feel like I should go. Since I often don’t have the chance to learn about search engines through professional development events and I haven’t yet heard Chris Sherman speak, I’m starting out in his talk. Unfortunately, there are odd technical problems beyond his control preventing his slides from displaying on the screen. The tech folks can’t seem to solve the problem, either.
“Ask is on fire,” he exclaims. Ask is part of a larger group including a number of other non-search sites, but the CEO says they plan to stick with search. The CEO believes they can be “the Avis of search.” He thinks they may not be the number one search engine, but they plan to become a strong number two player with a lot of great and unique tools.
People are wondering what Google is doing. Are they still a search engine? Are they a portal? With new acquisitions, like YouTube, does Google have a strategy or are they just throwing things at the wall to see what sticks? They have a list of the top 100 goals. Many projects still relate to search.
What’s up with Microsoft? Sherman’s slide (yes, the tech folks finally got them working) says “Brilliant features marred by confusing & inconsistent interface.” He believes their image search is one of the best on the Web.
Yahoo! is has become a major propoent of social search. They’re revamping their major sites and becoming more of a medial company. Maybe Yahoo! is trying to figure out what to do to compete with some of the other top contenders. Sherman doesn’t think they’ve lost their touch. He thinks they’re just doing a sloppy job of communicating what they’re up to with the outside world. Yahoo! has hired economists to study how people are using the Internet because it mimics how people behave in the marketplace.
All of the major search engines have blogs. One of the best ways to learn what’s happening in search is to keep an eye on the blogs.
Sherman is like me–he doesn’t really get Web 2.0 as a term. “Web 1.0 was all about connecting people. It was an interactive space, and I think Web 2.0 is of course a piece of jargon, nobody even knows what it means. If Web 2.0 for you is blogs and wikis, then that is
people to people. But that was what the Web was supposed to be all along.” –Tim Berners-Lee [more to the quote]
Web 2.0 search tools: Kosmix, liveplasma visual query map of musical artists. Links are based on whether they’ve played together or have similar music styles. [This map is the kind of map I want to see with bloggers in the blogosphere or with friends via Orkut.] Last fm depends on people tagging their music collections. Zillow is aggregating real estate data and breaking the hold of the multiple listing service (MLS). Yahoo! Search Builder allows people to build their own specilized search, which can be customized for blogs. Farecast aggregates airline pricing data and guesses at what day and time will be the best value for a particular airline ticket. SeatGuru has maps of airline seats and tells you specifics about airline seats, like which ones are near the bathroom or have limited leg room. The 2004 Underground Map shows some of the fine details about the London subway map coupled with a real London map. [I can’t find this link now. It is a neat image-based site with some interactive elements.]
Sherman predicts we’ll hear more and more about social search. He defines it as “Internet wayfinding tools informed by human judgment.” Then, he explains how “informed” can also be uninformed because someone who doesn’t know something or who is really working to misinform people can influence results. Social search isn’t new: the first Web guide came out in 1990. Yahoo!’s directory, which was originally made by people, is another example. Algorithmic search is social because they reflect the bias of the programmers. Search engines often observe human behavior and use it to modify searches. And, of course, there are personalized search engines.
Algorithmic search isn’t making the strides it once was. Innovation is more difficult. “Humans are still better at some things than computers.” Many social search services, like del.ici.ous, are based on the work of volunteers. Social search tools are often incredibly cost-effective because the company doesn’t have to pay for it.
Social search: Del.icio.us, Shadows, MyWeb, Furl, Diigo
Engines: [Feedster A few years ago, lots of people would have mentioned Feedster. Now, no one is. I wonder why.], Technorati, and Bloglines
Coolaborative directories: Open Directory Project, Prefound, Zimbio, Wikipedia Sherman demonstrates how Wikipedia has gained Web directory qualities because of the citations in the encyclopedia articles.
Personalized verticals: Eurekster, Rollyo; Collaborative harvesters: Digg, Netscape, Reddit(!), popurls.com agreegates these aggregators; social Q&A sites: Google Answers, Yahoo! Answers, Answerbag, About.com
Sherman mentioned some problems with the Web. The scale and scope of the Web is enormous. How do we search all of it, especially since the Web is still growing by thousands of pages per day. What do we do about multilingual content? Is it enough to have search engines focusing on sites written in English? With the growth of tagging comes another set of problems: language is ambiguous, there’s no controlled vocabulary, people are often lazy and, well, some are even idiots. Spam is a major problem, especially from malicious spammers.
Ultimately, what will work with social search? An optimal search will be a mix of algorithmic and people-mediated search. Trust networks are key. Trust networks include things like reviews on Amazon.com or ratings of answers in FAQ lists. Personalization and user control over results filtering. Social search could work the best with non-text content, like photos, music, and videos.
There’s a blurring of content and process. In the future, there will be more personalization, dynamic interaction, and more vertical specialized sites and mashups. The speed at which content is created is problematic for search services: how can they index blogs as quickly as bloggers can blog?
Questions:
Q: What role does the government play in future trends?
A: It depends on the country. The US government is fairly hands-off, but other countries want more control. China is often held up as the prime example of a country that censors, but at least about 20 other countries do that, too. It won’t be too long before Chinese Internet users outnumber other groups of Internet users.
Q: Question about purchasing things online related to online transactions
A: Google Checkout is really powerful and popular.
Q: Do you think the Web is becoming more intuitive?
A: The semantic Web is moving us beyond keyword based searching to intelligent searching. Google Book Search is now parterning with OCLC to help people find the books in libraries.
Q: about Homeland Security
A: Sherman believes the search engines take our privacy very seriously.
[I am going to post this live and clean it up later. I am missing a lot of Mary Ellen Bates’ talk by playing with this post right now.]
Tag: il2006
