Search Engines’ Choices: Google v.s. hakia.com

We join others and applaud Google as one of the pioneers of search – a company with a solid stance about its choices and guiding principles. Since search engines control the way we access the Internet, we receive many questions about our algorithm and the choices we make. I was intrigued when I recently read a paper by James Grimmelmann, an Associate Professor of Law at the New York Law School, with the title “The Google Dilemma”. He questions some of Google’s choices and poses an interesting question: Where is the line between a search engine’s First Amendment right to build its system as it likes, and its responsibility as a public corporate citizen guiding users in the Web jungle?

James takes the reader through five seemingly harmless cases and questions how search engines:

• Organize information on the Internet and rank search results – for example, searches for Mongolian gerbils and talentless hack (for more you should read the paper)
• Deal with the dilemma of controversial searches such as a search for jew
• Intervene with human oversight when their algorithms are intentionally tricked, as was the case for Search King
• Customize search results by accommodating local laws and regulations which explains the different images that one sees for the search “Tiananmen”: in Google.com vs Google.cn?

We, architects of search engines, make choices when building our algorithms. These choices are extremely important, as searchers usually look only at the first page of search results and our development decisions limit them to see what the Internet has to offer through our lenses. Naturally, we are guided and grounded by our beliefs and how we perceive the world. Hence, our corporate values directly impact the Internet’s information seekers, which is a huge responsibility. For instance, Google has built a very elegant search solution that assumes that people will not “be evil”. But some of us are acting for self-promotion and as a result, the search eco-system is suffering from evil-doers who have:

• Made statements with search bombs. Google’s trust that people who link to other sites will use an actual word or phrase or meaning in the linked Web text was not returned in a small percentage of cases.
• Built link farms that unfairly boost the rank of commercial sites
• Used the Internet for political, racist and other extremist propaganda

We at hakia believe that searchers suffer from information pollution, and the time for a guided search experience has arrived.

We believe in letting computers do the work without human interference and have algorithms extract meaning from Web text directly, and refuse to consider link statistics. If there is an economic benefit, simple rules will always be violated and it is not humanly possible to monitor mischief.

We believe the Internet is in a mature stage and information scientists- the librarians- can point users to credible Web sources- an effort already under way with great projects like the LII, IPL and more.

I think it is best to demonstrate what we mean with an example. Before we do that, I have three disclaimers: 1) We are still in development and are rendering credible content recommended by librarians to our QDEX system. Our credible content universe is limited to health and medical sources recommended by the Medical Library Association for now; 2) We will not touch upon the superior relevancy performance of semantic search technology. 3) We will not discuss hakia’s immunity to search bombs to keep this blog entry to a manageable size.

We ran the query: “what prevents migraine” both at Google and hakia. When you view the hakia SERP, you will note that we have identified PubMed, MayoClinic and Clinicaltrials.gov as credible source picks and Wikipedia as user generated content. We would like to assist the searcher to identify relevant and fresh information from credible sources as the searcher is not expected to be an expert in every field to make this determination. In this case, the medical librarians were the experts. You can compare the same SERP of Google. Google has made a different choice.

As hakia matures, credible sources will be surfaced – when they are the most meaningful match to the query- in every search. We will always visually mark the credible search results. We believe that our approach will offer searchers a different way to access the vast Internet. This is hakia’s way of using its First Amendment right and filling its public corporate citizen shoes.

As said, we all make choices when we set out on a given journey. Google has made theirs, and we have made ours. But the most important traveler on the Internet’s road is the searcher – you. What will your choice be? Or, can you afford seeing only one perspective and ignore the other?

        

This entry was posted on Friday, September 5th, 2008 at 1:56 am and is filed under Technology. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.