How Criticker fits into my personal movie recommendations...

Introduce yourself to the community or chat with other users about whatever is on your mind
AFlickering
Posts: 641
287 Ratings
Your TCI: na
Joined: Mon Feb 26, 2007 6:15 pm

Re: How Criticker fits into my personal movie recommendations...

Post by AFlickering »

Magb wrote:While we're discussing these things I'd like to talk a bit about something that I've unsuccessfully tried to put into words before, but which I now think I understand better. I think most mathematical representations of subjective taste in film (or anything, for that matter) are going to be somewhat hampered by the fact that there's no practical way to determine how much the score of any given film should count in the total analysis. As I see it there are (at least) two different scores that you can give for a movie you've seen. One is the obvious one: how much do you like the movie, or how good do you think the movie is? The other is more subtle: how much do you care about how much you like the movie? For instance, I've given the Ken Burns documentary "The Civil War" a score of 100 because I think it's virtually flawless, but I'm nowhere near as enthusiastic about that film as I am about a film like "2001: A Space Odyssey" or "Casablanca". On the other end of the scale I've given "Freddy Got Fingered" a 0 because I think it's completely terrible in every way, but I wouldn't really hold it against someone if they loved it the way I would with a film like "The Day After Tomorrow" or "Die Another Day". Finally, in the middle of my ratings list I have both films like "Vacation" that I'm just completely indifferent about and films like "Titanic", which I strongly believe are neither good nor bad but aggressively average and mediocre.


i dunno, i've never really had this attitude, i can't imagine ever giving a movie full marks unless i absolutely unconditionally adored it, or giving something no marks if it's just harmless laughably bad shit. my marks are based solely on my personal response to a film, any notion of objective quality which doesn't correspond to that is irrelevant to me.

Magb
Posts: 47
0 Ratings
Your TCI: na
Joined: Tue Mar 21, 2006 11:40 pm

Re: How Criticker fits into my personal movie recommendations...

Post by Magb »

AFlickering wrote:i dunno, i've never really had this attitude, i can't imagine ever giving a movie full marks unless i absolutely unconditionally adored it, or giving something no marks if it's just harmless laughably bad shit. my marks are based solely on my personal response to a film, any notion of objective quality which doesn't correspond to that is irrelevant to me.

That's not really what I'm talking about, though. I also rate movies mostly based on how much I like them intuitively and I rarely make a sharp distinction between how much I like a film and how good I think it is. There's no such thing as objective quality as far as I'm concerned. What I'm saying here is that there are movies I like a lot but that still don't really interest me that much compared to other films, and there are movies I strongly dislike that don't actively offend me the way some other movies do. And most importantly there's the distinction I mentioned between movies I just think are completely uninteresting and therefore give noncommital scores, and movies that I strongly believe deserve "average" scores because they're either a mix of good and bad or oppressively mediocre -- typically movies that could and should've been better, such as Titanic.

Still, you're certainly right that this is not something that will be very important to everyone. And your point about never giving 100 to movies that you don't love unconditionally is also a good one. On re-reflection I think what I'm talking about here -- at least for me -- applies more to average movies than to very good or very bad movies. I'm imagining a Bell curve type thing where the closer a movie is to being perfectly average the more likely it is that the "judgment confidence" score is low. You're very likely to be very secure in your opinions about the movies that you love or hate, but less so with the movies that fall somewhere in the middle.

Quicky
Posts: 451
0 Ratings
Your TCI: na
Joined: Mon Feb 25, 2008 2:53 pm

Re: How Criticker fits into my personal movie recommendations...

Post by Quicky »

td888 wrote:Can you give mine a swirl again? I send you my pw via PM... :)

Done ;) - http://thequicky.net/files/td888-qmdb-r ... tions3.pdf
Your PSI correlations jumped from 0.41 to 0.43, have you been adjusting some ratings or is this just a coincidence that your PSI is now performing ever so slightly better?

Magb wrote:I'm imagining a Bell curve type thing where the closer a movie is to being perfectly average the more likely it is that the "judgment confidence" score is low. You're very likely to be very secure in your opinions about the movies that you love or hate, but less so with the movies that fall somewhere in the middle.

This is the reason why I've always felt it was a bit unlogical that Criticker uses 10% tiers, instead of bell-curve tier system, like me and td888 mentioned here.

td888
Posts: 836
0 Ratings
Your TCI: na
Joined: Thu Jul 12, 2007 10:44 am

Re: How Criticker fits into my personal movie recommendations...

Post by td888 »

Quicky wrote:Your PSI correlations jumped from 0.41 to 0.43, have you been adjusting some ratings or is this just a coincidence that your PSI is now performing ever so slightly better?


Not really, but I rated some new movies in the 70-75 range. I noticed before there's some delicate balance which tips my Tiers in either direction when I rank movies in this range.... For instance, at the moment I have no movies in tier 3. Because of this the PSI differ I guess...

Quicky wrote:This is the reason why I've always felt it was a bit unlogical that Criticker uses 10% tiers, instead of bell-curve tier system, like me and td888 mentioned here.


Yes, this would solve my missing Tiers problem.

Quicky
Posts: 451
0 Ratings
Your TCI: na
Joined: Mon Feb 25, 2008 2:53 pm

Re: How Criticker fits into my personal movie recommendations...

Post by Quicky »

td888 wrote:Yes, this would solve my missing Tiers problem.

Not necessarily. Let's assume that user X rates every one of his movies as '70', then in a bell-curve tier system, Criticker would still not be able to make more than 1 tier, because it simply cannot decide which movies would belong to the upper tier and which to the lower tier.

I don't really understand the reason though why Criticker tries to keep movies with the same rating together in the same tier. Myeah, maybe it's more logical, but the math just doesn't get better because of it, at all. Why not split up movies with the same rating based on their PSI's or based on their average tiers over all Criticker users? Even though you've already rated a movie, it still has a PSI, even though it's not visible to the normal user. Then those movies with the higher PSI's would be put in the higher tier, and those movies with the lower PSI's would be put in the lower tier. Maybe average tier would be better than PSI because then you'd have a circular calculation. Your tiers influence your PSI's and your PSI's would influence your tiers.

td888
Posts: 836
0 Ratings
Your TCI: na
Joined: Thu Jul 12, 2007 10:44 am

Re: How Criticker fits into my personal movie recommendations...

Post by td888 »

Quicky wrote:
td888 wrote:Yes, this would solve my missing Tiers problem.

Not necessarily. Let's assume that user X rates every one of his movies as '70', then in a bell-curve tier system, Criticker would still not be able to make more than 1 tier, because it simply cannot decide which movies would belong to the upper tier and which to the lower tier.


Yes, off-course, but the assumption is that a normal person ranks movies to distinguish between good and bad movies. Not rank them all the same. If you do this, criticker is useless, it gives only an indication if you've seen a movie or not. Even if that user ONLY has seen movies worth a score of 70, he should rank them between 70 and 79. If he''ll do this, you'll see that some have a score of 71,72,78,79 and most fall in the middle (75). This also creates a bell-curve (a small one, but still a bell-curve...). I have movies in my tier 10 all with a score of 90, but I still have a preference between them, but it's not necessary for me to distinguish between them (for now). They're all in tier 10.

I'm pretty sure 80 to 90% of criticker's users has relative few movies in tier 1 and a few in tier 10 and the rest falls in the middle (the bell-curve). Only for some the curve'll be to the left of the 'center' and for some the curve'll be to the right. The trick is to use a ranking-system which matches the user's bell-curves....

Quicky
Posts: 451
0 Ratings
Your TCI: na
Joined: Mon Feb 25, 2008 2:53 pm

Re: How Criticker fits into my personal movie recommendations...

Post by Quicky »

td888 wrote:Yes, off-course, but the assumption is that a normal person ranks movies to distinguish between good and bad movies. Not rank them all the same.

Sure, but I was just giving an extreme example of your case where you have 25% of your movies ranked as '70'. Using a bell-curve system will not split these movies into separate tiers, at least not when it's set up the way it is now.

For clarity, here's what a bell curve 5-tier system might look like for a typical user. (Only 5 tiers for simplicity)
On the left, you have the bell curve system, on the right you have the evenly separated system that Criticker now uses.

Image

I find a bell curve system to be more logical. It will put more emphasis on the top and bottom ranges of the spectrum, which I think will generate more useful TCI's. What do you guys think?

Quicky
Posts: 451
0 Ratings
Your TCI: na
Joined: Mon Feb 25, 2008 2:53 pm

Re: How Criticker fits into my personal movie recommendations...

Post by Quicky »

Here's a summary of how my system has performed for all of the 8 users I've so far tried it on. The ones that haven't been mentioned here, Bosmarmot and Isobelmoon are a friend and my girl friend respectively.

Image

From this you can see that PSI is the best performer of all 7 parameters for everyone, so for all of us PSI is more useful than IMDB for example. Kudos to Criticker :).
Besides that you can see that we all seem to dislike blockbuster movies. I think Criticker users might be a bit biased in that regard, since I would guess that people who go to the movies to see the latest chick flicks and stuff like that are underrepresented on Criticker. It seems I'm also pretty much the only one who favours the more recent movies over the older ones :|. I guess I don't appreciate classics as much as I should :-S.

My system seems to work best for KGB and worst for td888.

td888
Posts: 836
0 Ratings
Your TCI: na
Joined: Thu Jul 12, 2007 10:44 am

Re: How Criticker fits into my personal movie recommendations...

Post by td888 »

Quicky wrote:Sure, but I was just giving an extreme example of your case where you have 25% of your movies ranked as '70'. Using a bell-curve system will not split these movies into separate tiers, at least not when it's set up the way it is now.


I totally agree, but that's 'bad' ranking from my site ;) I still need to 'correct' my rankings in my 75-65 range... Anyway, I think we agree on this subject.

For me the ideal situation would be ranking based on a bell-curve-system. If there's an option to display the rankings in a diagram, you get a much better idea about the rankings in general. In my case there'll be a misformed bell-curve, maybe other users will have two overlapping curves, etc... But if gives an better insight in the spread of your rankings and how to 'correct' or 'optimize' them.

td888
Posts: 836
0 Ratings
Your TCI: na
Joined: Thu Jul 12, 2007 10:44 am

Re: How Criticker fits into my personal movie recommendations...

Post by td888 »

Quicky wrote:My system seems to work best for KGB and worst for td888.


I'll see if i can 'fix' my rankings this weekend, see if it improves after ;)

Post Reply