background preloader

Human-evaluation

Facebook Twitter

Science

Iga. Avoiding. A Speculative Post on the Idea of Algorithmic Authority « Clay S. Jack Balkin invited me to be on a panel yesterday at Yale’s Information Society Project conference, Journalism & The New Media Ecology, and I used my remarks to observe that one of the things up for grabs in the current news environment is the nature of authority. In particular, I noted that people trust new classes of aggregators and filters, whether Google or Twitter or Wikipedia (in its ‘breaking news’ mode.)

I called this tendency algorithmic authority. I hadn’t used that phrase before yesterday, so it’s not well worked out (and I didn’t coin it — as Jeff Jarvis noted at the time, Google lists a hundred or so previous occurrences.) There’s a lot to be said on the subject, but as a placeholder for a well-worked-out post, I wanted to offer a rough and ready definition here. As this is the first time I’ve written about this idea, this a bit of a ramble. Khotyn is a small town in Moldova. Do you trust me?

And this is where authority begins to work its magic. Community Blog: Todd Speaks: 1/2 Stars. Dear blogfans: Todd is apparently more busy than I am and consequently this was the only way I could drag him on stage here. Be nice or he won’t come back. Todd? I’ve got a gajillion people here who would really like to know why we don’t have ½ stars. You’ve read their comments; you saw the poll we ran. Is this going to happen or not, and if not, why not? Todd: Hello Everyone in blogland. Now from the Netflix vantage point.As I said, your hunches are almost identical to mine. I like the improved accuracy, but I’d rather have more people rate than fewer.

I have not given up on this issue, because you and I generally agree it seems like it would be a good thing. Google's Scott Huffman: Many More Search Features Coming - Posted by: Rob Hof on October 03, 2009 Scott Huffman runs one of the least-known units at Google: the evaluation team that measures the impact of every little proposed change to the leading search engine.

And with some 6,000 experiments run annually, he’s pretty busy. Not to mention, he runs mobile search at Google, too. In a recent interview for my story on how Google’s trying to stay ahead in search, Huffman explained in detail how Google runs all those experiments—which include the use of hundreds of human evaluators in addition to Google’s massive computer infrastructure. This is the third of a four-part series that began with search chief Udi Manber and Google Fellow Amit Singhal, who heads the core search result ranking unit. Next up on Sunday is Matt Cutts, head of the anti-Web spam unit. Q: What does the evaluation unit do? A: We try to measure every possible which way we can think of how good is Google, how good are our search results, how well are they serving our users. Should YouTube Scrap its Ratings System and Rely on Implicit Use. Last week YouTube blogged that it is considering moving away from the familiar 5-star system of reviews. According to YouTube product manager Shiva Rajaraman, the stars system is being used bluntly by the majority of YouTube users - most give videos a perfect 5 star rating.

Rajaraman noted that "when it comes to ratings it's pretty much all or nothing. " When you also consider that the wisdom of the crowds is often dominated by small, powerful groups, then the validity of user ratings is further called into question. So why not just get rid of explicit user ratings and use implicit recommendations instead? YouTube graph showing the dominance of full 5-star ratings YouTube wants to know if a thumbs up/thumbs down system would be be more effective (two options), or even just favoriting (one explicit action to say you like an item).

However possibly a better option is to remove explicit ratings altogether. Are explicit user ratings still valid in consumer apps such as YouTube and Amazon? CommandShift3 - It's like Hot or Not for web design. Blog Archive » User Ratings. Google's Human Quality Evaluation: How To Spot Offensive Si.