The Good, the Bad, and the Ugly side of Web Search

April 16th, 2008 by Dr. Riza C Berkan, CEO

Our recent BETA update stirred interesting conversation on the blogsphere along with questions and scepticism. We welcome all comments and take the feedback from the users to improve our search engine. I want to take this conversation one step further.

When you are searching for any topic on the Web, no matter what, you will find yourself in between the worlds of different views. There will be good and bad things about it written by opposing views, a part of which will have a commercial interest.

When the good and bad are clear, there is no problem. For example, burning coal for energy is good because it is cheap, but it is also bad because it pollutes the environment. The distinction is as clear as it gets.

When the good and bad are not clear, there is uncertainty which is the ugly. What makes uncertainty ugly is the commercial motivation to interpret it one way or another. In medicine, law, finance, business, politics, and many other topics, uncertainty “the ugly” has been the fertilizer to grow cash and personal gain out of misleading. Some success come from using the ugly in a very unsettled way.

If a corporation, say X, is manufacturing vaccinations in the market, and financially sponsoring some independent organizations, say Y, and if you read the benefits of a certain vaccination on the Website of Y, where does this leave you as the consumer? Does it put you in the ugly? There are tons of Xs and Ys in the world involved in medications, treatments, therapies, and what not.

Unlike in the movie, the ugly is not just “bad looks” in Web search. It may cause pain, both mentally and physically. Forming opinions and making decisions by using misleading information can have serious consequences. To avoid it, you have to spend a lot of time trying to assess the legitimacy of the information found.

As a search engine user, you shouldn’t worry about the ugly. You shouldn’t be tracing relationships to assess the legitimacy of information. This is one of the aspects where hakia differentiates itself from the others. Starting a week ago, we are displaying search results at top positions from credible sources recommended by expert librarians. Those are the ugly-free results.

The user can point the mouse on the corner of a search result to see the following details:

We display the full name of the organization, the librarians who vetted the sources, and the date of capturing this search result. We call it the “quality stamp”. If we display a source like Wikipedia, which is not as credible as the others, we show its nature: “User Generated Content”.

Our objective is to be the first general purpose search engine where the results are ranked by quality rather than popularity. As a one-stop destination site, we want to offer the users credible and fresh results in every vertical. This is possible, thanks to our scientists, building semantic algorithms and resources in the back-room.

Semantic search technology enables accurate retrieval of information via concept/meaning match. It is very effective, and perhaps the only method, in application to credible and dynamic content. Popularity algorithms cannot work effectively beyond common queries because most of the credible and dynamic content are statistically flat (infertile). That is how semantics is related to quality as the enabling force.

There are more innovations coming your way from hakia. That’s the good thing. It is taking quite an effort and time. That’s the bad thing. And the ugly, we’ve decided to stand up to it.

And Eli Wallach says: Those who are making comments on hakia’s QDEXing ought to read this page before writing a blog post. He means it!
        

This entry was posted on Wednesday, April 16th, 2008 at 9:09 am and is filed under News, Technology. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.