From Tiers to Percentiles: Introducing a Big New Change to Criticker

Ideas to improve Criticker and new feature requests, as well as announcements about new enhancements.
kyvetti
Posts: 57
182 Ratings
Your TCI: na
Joined: Tue Apr 21, 2009 11:42 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by kyvetti »

livelove wrote:
kyvetti wrote:ranking inflation portrayed by people giving rankings like 70 or 80 to films they think suck tells me many people also wouldn't be any good at assigning any arbitrary midpoints or other milestones.

Ok. But what about people who rank movies they deem "very good" at 87 and have them in their 1% percentile ?


Actually, for people like this the more important feature is the users who have ranked a lot of the same films and the list of other films those users have ranked, the actual rankings being less interesting. And that feature already exists for subscribers.

cke
Posts: 8
107 Ratings
Your TCI: na
Joined: Fri Dec 12, 2014 2:32 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by cke »

BadCosmonaut wrote:
livelove wrote:Same here.
I think the takeaway of the thread: What happens to tiers if a user only rates movies he likes? and the thread: Disagree with the Ranking Scale is that we have to rate bad movies.

I am also wondering, whether there is another (better) way.
But I admit, that I haven't yet found a solution either.


Under the new system, I don't think you have to rate a bunch of bad movies. You just have to rate 1 bad movie for it to register that tier. As long as a rating (and therefore a tier) exists for each different rating you can possibly give on your personal rating scale, then I don't see how the old problem being discussed could still be happening.

For example, let's say you've seen 1,000 movies and use a 10 point scale. Let's say it looks like this: 200 movies at 10, 200 at 9, 200 at 8, 200 at 7, 195 at 6, and 1 movie rated for each of 5, 4, 3, 2, 1. With this scale, the new system won't try to force any less than 10 tiers. It used to force tiers into 10% of your total amount rated (to the best of its ability). Now it doesn't need to do that, since you can have up to 100 tiers. And if that's true, then I can't see how the old problem is still a problem. There is no grouping of different ratings. There is no forced merging of what should be different tiers.

The example above would work just as well up to a 100 point scale. No more than that is needed, since the maximum scale you can have is a 100 point scale.

Do I misunderstand how the new system works? If not, then I don't see the issue.


Sorry mate, don't have the time to go into detail but yes you misunderstand the issue we're talking about.

It's about a user's distribution of rankings and how (un)even it might be. The % system does not change this issue, it only makes the rankings have finer fidelity.

Edit: Livelove and AFlickering have explained.

cke
Posts: 8
107 Ratings
Your TCI: na
Joined: Fri Dec 12, 2014 2:32 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by cke »

paulofilmo wrote:i hope people unwilling to experiment with a range of scores will get confused and frustrated

The issue affects all users because TCI works both ways.

livelove
Posts: 759
0 Ratings
Your TCI: na
Joined: Thu Sep 17, 2015 9:36 pm

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by livelove »

cke wrote:The % system does not change this issue.
cke wrote:The issue affects all users because TCI works both ways.

Yes. Thanks for helping to point this out.
No wonder new users might feel “confused and frustrated” if even long-time residents here grapple with some of criticker's basic concepts.

djross
Posts: 1212
10 Ratings
Your TCI: na
Joined: Sun Apr 16, 2006 12:56 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by djross »

I just want to say thank you to the guys who run Criticker for working so hard and for so long to continuously improve this site. I'm sure this switch to percentiles has been more than a little complicated and time-consuming, but I think it succeeds in taking what was a clear and simple predictive system to a higher level while retaining that simplicity and clarity. When a user (trippingly) pointed out that the formula for calculating percentiles wasn't quite right, you listened, acknowledged the error, thanked him and made the change, showing once again the perhaps unique way that Criticker deals with its members.

As everyone knows, aesthetic taste is not a science, even if marketing departments want to believe otherwise, but Criticker provides a fun and interesting algorithmic experiment that also proves to be fairly accurate a surprising amount of the time and definitely succeeds in helping users to discover worthwhile movies. After 12 years as a member, I can honestly say that virtually all the decisions taken by the guys who run the site over that time have been for the benefit of the site and its users. I can't think of another website about which I would consider making such a claim. Thanks again.

krizoo
Posts: 4
0 Ratings
Your TCI: na
Joined: Mon Dec 14, 2015 1:59 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by krizoo »

Wow, what an awful lot of bootlicking. If you had posted that somewhere else, I would have taken it at face value for what it is. But by placing it here abruptly amidst this discussion you insinuate in a veiled fashion that some or all of us here are not thankful to Criticker or its owners, just because we discuss statistical problems and look for ways to improve Criticker. That is not only nonsensical (because it's evidence to the contrary) but also very disloyal to your fellow Criticker users.

AFlickering
Posts: 641
287 Ratings
Your TCI: na
Joined: Mon Feb 26, 2007 6:15 pm

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by AFlickering »

djross wrote:I just want to say thank you to the guys who run Criticker for working so hard and for so long to continuously improve this site. I'm sure this switch to percentiles has been more than a little complicated and time-consuming, but I think it succeeds in taking what was a clear and simple predictive system to a higher level while retaining that simplicity and clarity. When a user (trippingly) pointed out that the formula for calculating percentiles wasn't quite right, you listened, acknowledged the error, thanked him and made the change, showing once again the perhaps unique way that Criticker deals with its members.

As everyone knows, aesthetic taste is not a science, even if marketing departments want to believe otherwise, but Criticker provides a fun and interesting algorithmic experiment that also proves to be fairly accurate a surprising amount of the time and definitely succeeds in helping users to discover worthwhile movies. After 12 years as a member, I can honestly say that virtually all the decisions taken by the guys who run the site over that time have been for the benefit of the site and its users. I can't think of another website about which I would consider making such a claim. Thanks again.


seconded. and i recommend resisting the urge to rise to the bizarre post above lol

BadCosmonaut
Posts: 355
0 Ratings
Your TCI: na
Joined: Mon Sep 16, 2013 7:08 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by BadCosmonaut »

mpowell wrote:
BadCosmonaut wrote:One thing I didn't like about the implementation of the tier system was that the system would not force at least 10 different tiers for users who use at least 10 different ranks. For example, let's say I rate movies one of the following ratings: 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10. That's 10 total. It was possible (depending on how many movies were in each rating) to just have 7 tiers. I wrote about it here and in a few other threads before. I could never understand the reasoning for not forcing 10 tiers (for users with at least 10 different ratings).

Does this new percentile system suffer from that same problem? Like I just made the switch to percentiles. Behind the scenes, are my scores still divided up into any kind of tier system that forces them into just 7 tiers? Or are they split up basically into 101 tiers now (one for each whole number percentile including zero)?


They're now basically split into 100 "tiers" (0-99), with no grouping. A 62 percentile is considered slightly different to a 63 percentile. For this reason, the resulting PSIs are slightly more accurate, as there is a finer granularity.

Also, I forgot to mention, we've revamped our "Help & Information" section, to help explain Percentiles, TCIs and PSIs.


This chart makes it look like there are still a maximum of 10 tiers (each trying to force itself to be 10% of your total rankings). Is this chart accurate? Or is it outdated/not getting updated under the new system/unable to handle the new system?

mpowell
Posts: 3851
1 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by mpowell »

BadCosmonaut wrote:This chart makes it look like there are still a maximum of 10 tiers (each trying to force itself to be 10% of your total rankings). Is this chart accurate? Or is it outdated/not getting updated under the new system/unable to handle the new system?


I think the chart is accurate, though it's probably more interesting for someone who employs a larger range of ratings. For myself, I can see that my top 10% of rankings span the scores 86 to 100, while the bottom 10% go from 0 to 38.

It's true that the format of the chart is the result of shoehorning the new Percentile information into the existing Tier table. We didn't want 100 lines here, though there's probably something we can do to improve it.

BadCosmonaut
Posts: 355
0 Ratings
Your TCI: na
Joined: Mon Sep 16, 2013 7:08 am

Re: From Tiers to Percentiles: Introducing a Big New Change to Criticker

Post by BadCosmonaut »

deleted post

Post Reply