Google and the Evolution of Search I: Human Evaluators, by John Paczkowski, Digital Daily (june3)
Are people involved in adjusting the ranking og Google's search results?
"Google, for example, employs a vast team of human search “Quality Raters” (You’ll find a copy of an old training manual here). Spread out around the world, these evaluators, mostly college students, review search returns against established criteria–testing different algorithms and see which works “best” in predicting the quality of a site (though not directly judging the quality of any individual site itself).
They’re aided by Google’s own registered users, who can now, when logged into their Google accounts, promote and delete sites from their own search returns according to their preferences."
Would be helpful to have an estimate of the number of registered users who bother to adjust the rankings.
This is a three-part series of interviews with Engineering director Scott Huffman of the search evaluation team. Senior Google software engineer Matt Cutts, and Google Fellow Amit Singhal.
Amit Singhal closed wtih "AS: I believe that the role of the human evaluator in search will be there until we can understand language by computers, which is a far distance from where we are today. You know, we have made great advances but by no means is our language understanding technology close to saying this person really meant to get this document or not."