PSI Bug

Encounter an error, or something which isn't working correctly? Please, let us know
lineuphere
Posts: 10
0 Ratings
Your TCI: na
Joined: Mon Jun 27, 2011 4:54 am

PSI Bug

Post by lineuphere »

Before changing over to the percentile system, a large number of my films' scores matched their PSI. What happened? Why is the PSI's accuracy failing? I score my movies on a 0-100 scale and I now have zero movies in the database with a PSI over 94, yet I have over 60 films with ratings above 94. Something's is off. I miss the fun of the PSI being more accurate.

mpowell
Posts: 3851
1 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: PSI Bug

Post by mpowell »

That does sound like a bug -- the switch to Percentiles should have been more cosmetic, as the same basic backend calculations are used. We'll take a look into your account, and let you know what we find!

mpowell
Posts: 3851
1 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: PSI Bug

Post by mpowell »

After looking into this a little, it seems as though the correct calculations are being used. Your score of "94" corresponds to the percentiles 96.45 - 97.25%

If you look at the PSI Explain for As I Was Moving Ahead Occasionally I Saw Brief Glimpses of Beauty, you'll see that it's in your 97th percentile -- nearly perfect, but Lucas M's percentile of 85 brings it down just enough.

I hope this makes sense! It would have (hopefully) been the same under the old system. It's definitely possible that we could enhance this further, and make it even more accurate, and we'll take a look at doing so!

lineuphere
Posts: 10
0 Ratings
Your TCI: na
Joined: Mon Jun 27, 2011 4:54 am

Re: PSI Bug

Post by lineuphere »

Thanks for investigating. I blocked Lucas M temporarily just to test the rating and I see the PSI jumped up 1 point. However, in addition to that, I ran a test where I blocked all users whose score for As I Was Moving Ahead Occasionally I Saw Brief Glimpses of Beauty was not at 99 percentile. With my top 10 TCIs at 99%, my PSI for the film is still only 96. Why not 99?

So, I understand how this statement works with the percentile system:
Your score of "94" corresponds to the percentiles 96.45 - 97.25%
However, I suppose (because I grade on a 0-100 scale), my brain wants to make 96.85 the average score for that percentile. So essentially, a PSI of 94 feels like it should be labeled a PSI of 97 instead.

Under the tier system, I recall a considerable number of PSIs in the 95-100 range -- and further, as low as 0 (currently, I have no PSI under 10 - not that that is as important; and I understand why, based on the mechanics of the percentile system). The middle-range movies do seem more accurate, but the very high and very low end PSIs seemed to have been shaved off just a hair with the implementation of the percentile system.

Nonetheless, I'm enjoying Criticker as always! Thanks for all the awesome work on the site.

livelove
Posts: 759
0 Ratings
Your TCI: na
Joined: Thu Sep 17, 2015 9:36 pm

Re: PSI Bug

Post by livelove »

mpowell wrote: It's definitely possible that we could enhance this further, and make it even more accurate, and we'll take a look at doing so!
do you mean something like a high-precision normalized score ?

mpowell
Posts: 3851
1 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: PSI Bug

Post by mpowell »

We've recently introduced an enhancement to the calculation of PSIs, which makes the "percentiles" a lot more accurate, by storing them as floating point numbers, rather than integers. This allows a maximum "percentile" of 99.9999 as opposed to 99.0, and from our testing, has had the desired affect of greater accuracy.

Thanks again for pointing this out!

Post Reply