I wonder if Digg could ever build up enough posts and acquire new posts at a fast enough rate that it could compete with a specialty search engine, like Technorati?
As long as users are posting articles by hand, Digg will never have enough articles to rival any search engine, but if a crawler could be created to automatically post articles to Digg with a decent summary and title (which would require some NLP, and AI), then perhaps Digg could be a search engine where instead of pagerank as a ranking, the ranking would be popularity based on votes. That’s all that pagerank is now anyway, a proxy for popularity. So, why not cut out the proxy?
Of course, the crawler would make or break the implementation. If the fresh diggs posted by the crawler were total garbage, nothing would ever get traction because any half-decent post would drown in the noise, and digg would crumble. But if the crawler could discern some interestingness algorithmically (see last post for more on that topic) and post only interesting articles, then the signal to noise ratio could be high enough that really great posts could get traction in voting. The secret sauce in the crawler could make digg comprehensive without drowning users in a sea of adsense leeches and spam.
10 Responses to “Could Digg Ever Rival a Search Engine?”
- 1 Pingback on Sep 3rd, 2006 at 8:46 am
- 2 Pingback on Oct 3rd, 2006 at 5:42 pm
- 3 Pingback on Oct 4th, 2006 at 5:25 am
- 4 Pingback on Oct 26th, 2006 at 10:50 pm
that was where we were going with delicious before it was sold to Yahoo!
i think that’s an obvious thing to do with an index of URLs that is user generated
try searching the web here
http://del.icio.us/search/?
The Democracizing of Interestingness is an interesting idea in and of itself. But it is kind of like baseball statistics. The further back in history the more radical the variance, and the statistics are more notable because of the variance– if and only if you factor in the steroid cheating. I think the Sabermaticians refer to it as “reversion to the mean.” I think our democracy of interestingness has gone through a similar process. Britney Spears is interesting to the whole nation at large. So is Jessica Simpson and Linsay Lohan and everyone else you see online at the grocery store. They’re intesting according to this democratic means of determining interest. It’s worse in our country’s politics (see: present day Oval Office). Mediocrity reigns via tyranny of the majority. Because these websites are by similar standards still in their inception, the variance in the “interest quotient” is then still quite wide, but time takes its toll on mountains and statistics I guess.
Terrific comment Miles. Would you then expect that as Flickr grows larger that its interestingness will also revert to the mean? I suspect not, and whatever they do to prevent that problem is what would need to be leveraged here too.
delicious was sold to Yahoo? I guess I missed that headline!
:(
What in the world makes you compair a search engine with a website?
Its apples and oranges
Looks like this post got dugg, and I have been officially defamed in the comments of the digg post. Brutal. To set the record straight for anyone else coming from digg. I’m not a VC, though I do work in VC. Furthermore, I don’t necessarily think this is going to happen; it’s a thought experiment. Ponderings… Anyway, glad to know my hosting can survive being dugg.