Dark Seo

Fellow “former VC who returned to the bright side” Tony Conrad kindly gave me a preview of Sphere, ex-Yodel Search, at the Web 2.0 conference, and re-kindly gave me access to the beta version of this new blog search engine, “that uses an advanced algorithm to discover high-quality, relevant, and timely blog posts”.

You might think: why do we need yet another blog search engine – especially after Google launched BlogSearch ? The team behind Sphere provides a pre-scripted answer, since one can bet they’ll hear that question many times over:

The first part of that question is easy: We thought we could build a much better search engine to serve the rapidly growing blogosphere.

When we started building Sphere, there were around five million blogs. Nine months later, there were more than 19 million blogs. With so many people reading, writing, and commenting on blogs, finding high-quality, relevant content has become difficult. For a variety of complex technical reasons (such as an exclusive emphasis on freshness, or an overly simplistic computation of a blogger’s authority) other blog search services deliver less-than-satisfying results. Our new, advanced algorithm rapidly sorts through all blogs to find high-quality, relevant content that matches a blog search query.

The second part of that question (you know, the Google part) is a little bit harder to explain. Our corporate therapist hasn’t led us to the answer yet, but we think it’s because we saw firsthand through Oddpost that size doesn’t always matter. We like our product and hope you will, too. And who doesn’t love an underdog anyway?

In no particular order, here are a few initial remarks I have regarding Sphere (initial because I am “waking up” in three two hours to catch a plane to NYC):

*
It crawls the blog website, extracting the full text from all posts, and not the feed. Its current archive dates back January 2003 but the index can be back-filled by crawling blog archives.
*
The engine seems to do a good job at extracting the actual content of pages, and not include “surroundings” like blogrolls. A search on “Jeff Clavier” only returned posts containing my name (sorry, for now only the beta users will be able to access these results). The engine actually behaves more like a feed search engine since basic search unit is the individual post.
*
There is however an issue with the fact that some of these aggregator blogs that suck my feed (and others) to build vertical content sites get predominantly displayed. This proves the fact that Sphere does a good job at clustering information, but these should be removed from the index – or they should be given lower priority.
*
Sphere SearchSphere seems to do a great job queries containing a few keywords. A search on “open source crm” retrieves very relevant posts, much more so than other blog search engines. Switching the sorting to ‘date’ returns results which are far less relevant, and include a bunch of spam blogs.
*
I like the idea of displaying relevant blogs and news article for a particular search term. What is not clear to me is which news source are used currently (CNet ?), and how “relevant” blogs are picked up (search term included in the blog description?). For example, none of the blogs related to Venture Capital include the established VC blogs.
*
Also a good idea, building a profile for each blog included in the index. The information currently displayed is not that interesting though (avg posts per week, avg number of words per post, and recent outbound links). I’d rather see recent posts – including the number of references on each post, recent inbound links, etc.
*
The overall UI is actually quite clean. I sense the touch of Adaptive Path in the design (?).

We’ll have to see how well the engine scales at capacity – once the index has been backfilled and users are banging on it. But it certainly looks interesting and seems to deliver on the promise of higher relevance which is a big deal. I should point out that Om and Mike recently wrote about Sphere as well.

Congrats to Tony and team!

PS: You can leave your email address on the Sphere home page if you are interested in joining the beta program, but I understand that there are thousands of people on the waiting list. If you are at BlogOn ‘05, ask Tony for a demo.
PPS: Almost forgot - I don’t have any ties with Sphere (especially since I am an angel investor in Feedster but Tony and I share a number of good friends here and in France.