Google, Mahalo and SEO

A bunch of people have come out against Robert Scoble's 30+ minute video about how Mahalo will crush Google within four years. Here is one and here is another. Now it's time to weigh in with my opinion...

I believe I have posted on this topic before, but to reiterate, I do think that SEO has negatively effected Google's results. Not necessarily SEO but the awareness of PageRank has enabled people to artificially get results in. SEO is a part of that, but not the whole thing.

I was just thinking a few weeks ago about this and I was trying to come up with a new method to generate search results that would bypass the spam that now clogs Google. Of course, any algorithm, once it is made public knowledge, will be able to be manipulated and hacked but a new one would at least give people clean results for a few years, like Google was for it's first half decade or so.

I have been very clear on my feelings about Mahalo and why I don't think it will ever work. Basically it is because, as everyone points out, Mahalo is not a search engine. It is a directory of links. If I want to search for a combination of words no one has ever searched for before (which I quite often do) Mahalo will not be able to give me any useful results. What we need is an automatic search algorithm that can filter out spam. Or better yet exclude spam.

I believe that Google is adjusting their algorithms to try to filter out the spam results, but the problem is that filtering won't work. The spammers will always be at most one step behind your filters, as evidenced by the plethora of email spam I spend hours deleting every day. At best filtering spam will give you a couple of days or weeks of peace before they work out a way to bypass the filters.

What we really need is a completely new way to do search. As I said, it will only yield clean results for at most a couple of years before people find ways to manipulate it and get their results in. The real holy grail in this situation would be an algorithm that completely defeats spam, but I don't think that is ever going to be possible. The nature of an algorithm is that it yields predictable results and once you know how the algorithm works and what the expected results are it is always going to be possible to manipulate the data to manipulate the results.

So, to summarize - Mahalo, Facebook and Techmeme are not Google killers. I don't think Google is going to be the Google killer, unless they buy the solution from someone else who comes up with it, which is exactly what I would do if I were them. PageRank is too widely known and understood to remain an effective search algorithm and spam is a huge problem with Google. I remember in the year 2000 you could type something into Google and you would get a list of very relevant results. Now you have to wade through pages of spam results to find the legitimate ones.

If I had the solution to this problem I would sell it to Google and be a wealthy man. Unfortunately for me, Google probably has people a lot smarter than me working on this right now. If Google could provide me with a sample of the data they collect on each page they index I would be more than happy to stare at it until I got some ideas but I don't think they will do that.

So for now we will have to continue to wade through spam to get our sparse relevant results, or use directories like Mahalo, which have historically failed miserably, until someone comes up with an idea to solve this problem. Many people have tried to get rid of spam, mostly in email, and none have succeeded so getting rid of it in search results will be a gargantuan task.