Accountability in Web Search
One very important issue that has been systematically evaded by the current (conventional) search engines is “accountability.” For an introduction on the topic, you can read James Grimmelmann’s paper “The Google Dilemma.” If a search engine is bringing 10 results to a query, ranking these results based on some algorithm, should the search engine be somewhat accountable for what it presents? Obviously, “do no evil” type of tag-lines do not answer the question.
Today, there is certain level of commercialism in disguise in search engines. The marking of “Sponsored Links” is only a disclaimer to distinguish how the advertising money is changing hands. It does not mean that the rest of the search results are free of commercial bias. This is an important detail when searching in gray-shaded areas, like the treatment options for certain diseases. It obviously gets more important when the queries are life-and-death questions.
At hakia, we have deployed one solution for the accountability problem. The search results are separated by their credibility based on source Website’s level of peer-review and editorial control. We took librarians’ recommendations to identify sources that are free of commercial bias. This is an on-going process at hakia.
If a search engine cannot be held accountable for what it presents, it should at least be kind enough to separate visually what is credible and what is not. Expecting the end user to recognize credible Websites (and perform site searches) is as unrealistic as expecting everyone to be a physician.
Check the following query at hakia to visualize this concept:
shoulder injury
The user can jump to specific verticals by clicking on the “more” buttons. We have also deployed destinations such as:
health.hakia.com – for consumer health
pubmed.hakia.com – for medical science
where searches are confined to these collections of Web sources.
If a search engine’s algorithm is popularity ranking, then bringing relevant results from credible sources represents a problem, especially when vast portions of such domains are statistically infertile (like the holes in Swiss cheese). That is where semantic search technology makes a tremendous difference because of its systematic coverage without relying on statistical clues.
I am an advocate of information quality, as some of you may have read my earlier posts. I claim that exposure to more information can cause miseducation (if there is such a word). If we are the pioneers of Web search today, accountability is something we should all pay attention to. A world full of miseducated people would be a miserable place, and bad business in the long run.
October 27th, 2008 at 12:09 pm
How do you decide what groups in society should be consulted when determining the knowledge claims, on the part of websites, that you consider to be credible? You seem to assume that librarians are not in possession of vested interests that might influence their identification of the websites that are deemed credible.
For instance, since librarians have a professional interest in maintaining the impression that editing and archiving are best performed by centralized authorities – such as peer review – they would have an interest in promoting websites that generate materials according to processes that embody the same epistemic prejudices upon which the librarian profession is legitimated.
Therefore, the librarian has an interest in promoting Britannica as opposed to Wikipedia. However, as studies, such as the one performed by Nature, are beginning to establish, the Wikipedia can produce materials that contain only marginally greater rates of errata, and the errors expressed in the Wikipedia’s contents are corrected much faster.
On a final note, you seem to suggest that Google has not implemented any instruments designed to weed out websites that have little credibility but, nevertheless, engage in SEO or PPC, allowing them to achieve high SERPs. This is patently false. Google has errected functions in its algorithms that penalized for practices, such as paid-for-links, and so forth.
What is different between the practices of Hakia and Google is that Google is democratically inclined while Hakia lends itself to a form of authoritarianism.
Russell Cole
October 30th, 2008 at 12:27 pm
Dear Russel Cole
Good points.
First, you have missed few important details. Britannica is a commercial entity. hakia uses librarian’s recommendations for non-commercial entities. If you mouse-over CREDIBILITY column, it tells you along with a link for the criteria. So, most of your arguments above are not applicable.
Second, the credibility (centuries old criteria) comes from peerview process and editorial control by experts. Wikipedia is not a peerreview or editorial environment. Anyone with any authority can change Wikipedia. For example, things written about hakia on Wikipedia are not accurate, and the moment we correct it, someone else takes it off. Who are these people to know hakia more than us? It is a joke, and happens frequently in every subject matter.
Third, Google results are not democratic. If you think it is, you are in illusion. A few people in Google can change weights of the (black-box) algorithm for any reason. In your line of thinking, this is dictatorship who says “do no evil”. No dictator in the history had this much power to control the flow of information with so little effort. But this is an analogy, and only used in response to your line of thinking.
Last, you call hakia’s practice a form of authoritarianism. No, we just separate results visually as to their editorial integrity recommended by OTHERS. If I had a bigger font for OTHERS, I would use it here. And these OTHERS have profession and expertise in this area. Consumers have a right to see this differentiation.
Thanks for your comments.
November 2nd, 2008 at 6:16 pm
By dismissing my counterfactual, in which I compared the Wikipedia to Britannica, you have completely missed the point. I compared the two respective reference sources because one, Britannica, subscribes to the peer-review editing methodology, while the other, Wikipedia, does not. Whether you would exclude Britannica from your indexes on the basis that it is not free is entirely beside the point.
The point of this counterfactual is to demonstrate that peer review, which is practiced by Britannica, does not necessarily result in the production of contents that possess substantially decreased rates of errata. Furthermore, because editing by the Wikipedia is decentralized, the Wikipedia corrects its errata at a substantially faster pace. This has been established by empirical studies, of which you appear to be ignorant. The most salient of these studies was conducted by Nature about a year and a half ago.
You are certainly correct to say that anyone can edit and author the materials expressed in the contents of the Wikipedia. However, you must account for the fact that, once again, according to comparisons between the Wikipedia and Britannica, the Wikipedia, nevertheless, publishes contents with only marginally greater rates of errata.
It is this point, that peer review does not necessarily result in the production of superior materials, that you need to address. By simply disqualifying the argument on a basis that is altogether irrelevant to the argument’s soundness and its implications is merely a slight of hand that amounts to a thoroughly unimpressive rhetorical tactic.
Even more disconcerting, you attempt to defend peer-review by stating that it has been used for centuries. You are quite correct. However, your polemic possess solely as its pillars of support, tradition and authority. Such defenses of an intellectual position might fair well in the deliberations of those who practice scholasticism. However, it fails to offer any convincing reason why those of us who are alternatively swayed by objectivism would adopt your thesis as credible.