News Trust dot Net

A non-profit initiative called Newstrust is developing a system for rating news from a very broad array of sources. And unlike previous efforts that employed teams of paid reviewers (a model that proved economically unviable), this one is a social network model which uses the intellect of the masses to rate all manner of news content and news sources. In beta now and due out in early 2008, Newstrust will not only be a stand-alone site where consumers can come to find the best journalism as ranked by an army of volunteer media reviewers, but more importantly it will (we can hope) be deployed over all manner of online news sources so that readers will on any news-related website see an objective rating of that site's quality and of specific news content.

....

Anyone can sign up to be a Newstrust reviewer, with nothing more than an e-mail address. That gets you the right to start reviewing news sources (websites, blogs, etc.) and individual articles. Reviewers are asked to rate news sources using a simple 5-star selection. Each source gets an aggregate score. Ratings for individual stories are more interesting and deeper; story reviewers are asked a series of questions about the article (rate with 1-5 stars):

Is it a good story?

Do you trust this publication?

Is the story informative?

Is the story fair?

Is the story well sourced?

Does the story show the "big picture"

The reviewer also can leave a text review, which will show up on Newstrust along with the story.

....

More importantly, Newstrust's developers are building bias filters into the algorithm that drives the site. Every member reviewer of Newstrust gets a "member rating," which is an assessment of their performance as a reviewer. When administrators notice a reviewer that consistently reviews sources and articles with a particular political bias, for example, that reviewer's member rating gets knocked down. While the biased member can continue rating things, his impact on the overall scores is neglible, and his written reviews are either placed where few will see them or not shown at all.

Administrators actively look for reviewers who are trying to game the system and influence ratings toward a particular point of view (e.g., a conservative reviewer who regularly gives 5 stars to opinion pieces on Fox News while ranking New York Times articles with 1 star). Such people are then sent an e-mail with a request to comply with the non-partisan spirit of Newstrust and start reviewing content and sources objectively.

The site also has tiers of reviewers. Everyone starts out as a Reviewer, but if you really get into this and perform in an objective manner, you can be promoted to a Host or even an Editor.

A future enhancement to the site will be bias-adjusted reviews – that is, optional ratings based exclusively on reviews by Hosts and Editors. (That may be a paid premium service.)