Flawed assumption in Tier determination

emtilt · Post by **emtilt** » Tue Dec 25, 2007 3:21 am

There is a flawed assumption in the way Criticker determines the \"tiers.\" For the tier ranking to work, the assumption must be made that all people see the same ratio of \"good\" films to \"bad\" films and that this corresponds to the divisions set in Criticker\'s ranking system. This isn\'t always the case, though. This could be remedied by allowing users to adjust where their tiers cut off. That way, if they only see one great movie and nine good movies and nothing else, they could have their tiers set so that they have one Tier 10 movie and nine Tier 7 movies and nothing else.

Of course, this introduces it\'s own problems, such as everyone\'s tiers not corresponding to similar ideas of ranking. This could be remedied by leaving the system as it currently is by default, but allowing users to change it only if they feel the way it works now isn\'t working for their rankings. Further, you could help standardize the meanings of the individual tiers by providing an explanation before it is modified of what type of opinion it should correspond to (eg, \"Your Tier 10 should correspond to those films that you think are the best of the best,\" or whatever).

Personally, I think this would improve Criticker\'s usefulness a great deal. The majority of users would likely leave the Tiers as they are by default, while users who are not satisfied with how their films are being divided up can customize them. There could, for instance, be a simple option on a preferences page: either leave on the default percentage based system (you could even make this the recommended method) or turn on a hard numbers based tier system.

Even if my specific ideas are not something that you guys would want to implement, you should think about some way of addressing this issue in the future. I think it seriously skews the accuracy of some of Criticker\'s rankings.

KGB · Post by **KGB** » Tue Dec 25, 2007 4:09 pm

A public survey reveals shocking results: 4 out of 5 Criticker users just love complaining about this, and always recieve the answer that changing this is messing with Criticker\'s heart (which is true). Me included.

AFlickering · Post by **AFlickering** » Wed Dec 26, 2007 9:34 am

I\'m yet to see an idea for solving this issue that\'s really appealed to me - personally I\'m happy with the admittedly imperfect system, largely because I happen to have seen a pretty even spread of films from awesome to awful. It\'s worth thinking about though, I agree.

A bigger annoyance for me is that tier 8 is dark green, should be light green methinks. Anybody who would call the films in their 8th tier \"great\" is either very lucky or has seriously low standards.

mpowell · Post by **mpowell** » Thu Dec 27, 2007 9:19 am

Hi Guys,

This is a good point, and remains the most frequent complaint of Criticker's system -- the fact that you can't manually set what tiers mean.

The solution we\'re opting for (and one we've been promising for a long time, I know) is to allow users to manually define tiers themselves. This means, you can say "For me, good films range between scores 61 - 75, great = 76 - 82, etc". The underlying computations will be largely unchanged, but you will have far greater control in how your film lists are displayed. You will also be able to define your own "quips" ... by which I mean the short describing words like "Great", "Awesome", "Not So Hot", etc.

We really plan on getting this out the door soon, but the coming month is already shaping up to be a very busy one. Rest assured, though, this is one of the main improvements on our mind.