You could, quite reasonably, argue that more than any other technological advance including microchips, nanotechnology, fiber optics, you name it; computer algorithms are what created the 21st century as we know it.
Trivia digression: The word “algorithm” is derived from the name of a 9th century Persian mathematician and astronomer, Abu Abdullah Muhammad ibn Musa al-Khwarizmi, who was the first person known to write about what we now call algebra.
Ao, here’s an interesting question: Which computer algorithms have had the greatest impact on computing? There’s a paper by Barry Cipra archived at the Department of Applied Mathematics, C. U. Boulder, that discusses the article “Top 10 Algorithms of the Century” published in the January/February 2000 issue of “Computing in Science & Engineering.” This collection is interesting but misses what I would argue has become the most broadly valuable algorithm of modern times: The Google Page Rank algorithm.
Page Rank, the algorithm that drives the Google search engine, is important not just for what is does, which is to rank the relevancy of Web page content for a given search term, but also for the bigger picture impact it has had on business and the economy. It is the prime driver behind the entire Internet search business eclipsing the impact and importance of all previous search algorithms by orders of magnitude.
The power of Page Rank is what gave Google a 64.2% search market share for April 2009 which was more than three times the market share of its nearest competitor, Yahoo (20.4%), and almost eight times more than the third place player, Microsoft (8.2%).
So, what are a thousand companies looking for? Yep, a better algorithm than Page Rank. But what does “better” mean? At first blush you might think better relevancy of results, a faster response (although how much faster than Google’s response would be useful is arguable), or some other critical technical factor would be needed.
Here’s the problem; to be judged to be significantly better by the market (in other words not just a technical improvement) would require an almost godlike result – something that was so undeniably better, so much more relevant, so incredibly useful that it is actually hard to imagine such a thing existing.
Vertical search engines that is, search services that focus on providing depth in a limited field, for example, tropical skin diseases or the tort law, can produce quantifiably better results but that’s almost akin to a trick, the algorithms aren’t generalized and the data that is being searched is usually marshaled, pre-formatted, and otherwise manipulated to make the algorithms as successful as possible. Just limiting the scope of what can be searched thereby eliminating “noise” will improve vertical search results enormously.
No, when it comes to improving general purpose search, finding a new algorithm that can better Page Rank is an epic challenge and not just in improving the results but also getting people to use whatever service it powers because what is it that the public tends towards? That which they know.
To get the public to use something new you need an edge, something that makes them understand and want the value proposition that the new thing claims to provide.
Next week, I’ll look at a new search service that does just that, sort of …
