New Sponsor Enhancement: Prediction Accuracy

Ideas to improve Criticker and new feature requests, as well as announcements about new enhancements.
mpowell
Posts: 3851
1 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

New Sponsor Enhancement: Prediction Accuracy

Post by mpowell »

We've just released a new feature for Criticker Sponsors that we're calling Prediction Accuracy. This is a sortable list of all your scores, alongside the PSI which Criticker predicted for you... and the differences between them.

Image

If you've ever wondered how often Criticker hits the nail on the head ... or how often we miss the nail completely and hit your big toe ... this is the place to check that out. You can sort the list by Name, Score, PSI or Difference.

We also show you the average and median difference between score and PSI of all the titles in the list. If we had investors, they'd never allow this, since it's basically a measure of how wrong we are. But we don't have investors, so screw it! We've always been proponents of sharing as much data as possible, and are confident enough in Criticker's accuracy that we don't feel (too) insecure.

If you've been on the fence about supporting Criticker, now is a great time to do so. We've totally revamped our sponsorship mechanism, enabling you to pay with credit card, in addition to Paypal. This opens up sponsorship to a whole range of people who didn't want to use Paypal, or were prohibited from doing so by their countries. More details here.

saudade
Posts: 167
24 Ratings
Your TCI: na
Joined: Fri Dec 09, 2011 7:27 am

Re: New Sponsor Enhancement: Prediction Accuracy

Post by saudade »

You are the sunshine of my life.

stsakiri4
Posts: 9
0 Ratings
Your TCI: na
Joined: Fri Oct 06, 2017 6:45 am

Re: New Sponsor Enhancement: Prediction Accuracy

Post by stsakiri4 »

Great Addition! Another useful feature would be to show us movies, whose PSIs are a lot bigger than the average rating.

Ramy
Posts: 86
169 Ratings
Your TCI: na
Joined: Sat Aug 08, 2015 5:46 pm

Re: New Sponsor Enhancement: Prediction Accuracy

Post by Ramy »

stsakiri4 wrote:Great Addition! Another useful feature would be to show us movies, whose PSIs are a lot bigger than the average rating.


I'd love to have this. This is what Criticker is all about!

livelove
Posts: 759
0 Ratings
Your TCI: na
Joined: Thu Sep 17, 2015 9:36 pm

Re: New Sponsor Enhancement: Prediction Accuracy

Post by livelove »


cross-referencing related thread: Prediction Accuracy Index




Last edited by livelove on Sat May 12, 2018 3:16 pm, edited 6 times in total.

BadCosmonaut
Posts: 355
0 Ratings
Your TCI: na
Joined: Mon Sep 16, 2013 7:08 am

Re: New Sponsor Enhancement: Prediction Accuracy

Post by BadCosmonaut »

I just accessed this system for the first time. It is pretty neat and helpful, but can you check to make sure it's working properly?

Specifically, "# Way-Off (> 40 Diff):" shows 0 for me, which if I understand correctly means that none of my ratings are more than 40 points off. The reason I think this isn't really accurate though is because I don't use a 100 point scale. I use a 10 point scale (1-10), so it's impossible for anything to be 40 points off. If so, then the "way off" meter isn't scaling like it should. The way off meter should scale to a user's own scale. For me, way off should be more than 4 points difference. For a user who uses a 5 point scale (1-5), way off for that user would be more than 2 points difference. For a user who uses a 5 point scale (20, 40, 60, 80, 100), way off would be more than 40 points difference.

Do I understand right? Does that make sense?

[spoiler]Image[/spoiler]

mpowell
Posts: 3851
1 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: New Sponsor Enhancement: Prediction Accuracy

Post by mpowell »

Yes, that does make sense, and it should be working the way you'd expect. Okay, thanks for pointing this out, we'll take a look as soon as we get a chance.

Thanks again for supporting Criticker, by the way!

BadCosmonaut
Posts: 355
0 Ratings
Your TCI: na
Joined: Mon Sep 16, 2013 7:08 am

Re: New Sponsor Enhancement: Prediction Accuracy

Post by BadCosmonaut »

This is more of a quality of life suggestion, but the name column should be sorted alphanumerically when sorting by score, PSI, or diff. See image. All my titles rated 10 are in a mixed up order. They should be in alphabetical order.

The way it works is that the chart should be sorted by the selected method first, then apply alphanumeric sorting second. So in the image, I've picked Score (descending) as my primary sorting. After it does that, it should also sort all the 10s alphabetically. Then all the 9s alphabetically. Then 8s, etc. down to 1. Basically, no matter which primary sorting method I choose, the secondary sorting method should always be alphabetical. If I choose alphabetical as my primary sorting method, then there would be no secondary sorting method since there can't be two titles with the exact same title.

As a side note, IMDB does not have secondary sorting like this. I suggested it to them years ago, and the people who replied couldn't understand how there could be more than one sorting method applied at a time.

mpowell
Posts: 3851
1 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: New Sponsor Enhancement: Prediction Accuracy

Post by mpowell »

Heh, well we at least understand the concept of secondary sorting! I'll make a note of that, it shouldn't be too difficult.

mpowell
Posts: 3851
1 Ratings
Your TCI: na
Joined: Fri Sep 09, 2005 10:22 am

Re: New Sponsor Enhancement: Prediction Accuracy

Post by mpowell »

We've fixed both the issues you've brought up -- the secondary sorting on title name, as well as the fact that a ratings span of 10 shouldn't have a difference of 40 to be considered "way off". Check it out, and thanks for bringing both problems to our attention!

Post Reply