How granular are your ratings ?

Introduce yourself to the community or chat with other users about whatever is on your mind
livelove
Posts: 759
0 Ratings
Your TCI: na
Joined: Thu Sep 17, 2015 9:36 pm

How granular are your ratings ?

Post by livelove »

I often struggle with how to accurately score movies. This topic is part of a series dealing with voting-related problems, challenges, phenomenons and paradoxes — all as part of a quest whose end-goal is to correctly reflect my appreciation of movies when rating them:


Granularity
One of the things I learned is that many times I am pretty sure about the minimum/maximum score I would give a movie (for example I would score movie XY in the 20-40 range) but I find it hard putting an exact number on a film. I hear you say: "Well, the obvious choice then is to reduce the ranking granularity". Not really. Because then I'm lacking intermediate steps when I need them.

By way of experimentation I found out that my ideal granularity is 20.

I could use 0-20 in steps of 1.
Or 0-10 in steps of 0,5.
I chose 0-100 in steps of 5. It's all the same, but it somehow helps to put a percentage on the movie. Like "I liked it to about 75%", which means I would rank the movie at 75.

  • • A higher granularity (0-100 in steps of 1) wouldn't work for me, as I am unable to distinguish between 42 and 43 for example. Or between 17 and 18.
  • • A lower granularity (say 0-10) would make the choice easier, but then I often get the feeling "I would like something right in-between" (like 7,5).

So 20 shades is the most I can distinguish with a certain sense of certainty.
Beyond that, I just struggle and my votes would constantly change (back and forth between 42 on one day and 43 the next).

What is your experience on that subject ?
Last edited by livelove on Sat Sep 28, 2019 4:08 am, edited 6 times in total.

chmul_cr0n
Posts: 27
28 Ratings
Your TCI: na
Joined: Mon Nov 14, 2011 9:46 pm

Re: How granular are your ratings ?

Post by chmul_cr0n »

Being used to imdb I was always a little frustrated there weren't more rating options. So criticker was a revelation, but also a little overwhelming at first. Initially I based all my rankings on my imdb scores and then rockpaperscissored a little within the tiers. So weaker 7s would get a 67, solid 7s would get a 70, 7-8-inbetweeners would get a 75, and so on...

Now I'm so used to criticker that I sometimes almost miss some sort of way to distinguish between let's say better or worse 73s for example. But that only goes for the upper half (50-100...which covers about 85% of my ratings) and specifically for the 62-77 area (which basically covers anything from "enjoyable" to "pretty awesome" :>), where half of my movies lie rated. When it comes to these kind of ratings, my instinct has become very numerical in a way, and I can pretty intuitively come up with a number, which I always base on the impression the movie left on me. Not necessarily taking into account craftsmanship or substance. At least not consciously. Numbers ranging from 0-100 obviously. Which then get translated into imdb scores. Sometimes I'm unsure, and I'll change it between a 74 or a 75 a couple of times, but most numbers have so many movies assigned to it (about 2000 movies spread out over these 16 rating options) that I can compare (instinctively) and make a confident decision.
It should be obvious by now that I'm naturally drawn o numbers and ranking things, and criticker has definitly made it worse. :D
I have thought about maybe expanding said area, but I'm not really a fan of that idea. The orderly numerophile in me says its the movies' fault if they all evoke similar reactions in me. :>

I'm still getting used to the 0-49 stuff. Which is mainly due to my instinct as to what movies I'm probably not gonna enjoy being fairly well-developed. :)

livelove
Posts: 759
0 Ratings
Your TCI: na
Joined: Thu Sep 17, 2015 9:36 pm

Re: How granular are your ratings ?

Post by livelove »

@ chmul_cr0n:
How do you distinguish between 42 and 43 ?
Or between 17 and 18 ?

SpikyCactus
Posts: 6
5 Ratings
Your TCI: na
Joined: Sun Jul 31, 2011 2:04 pm

Re: How granular are your ratings ?

Post by SpikyCactus »

I score in multiples of ten. Anything more granular hurts my head and the variations between smaller numbers would get swamped by whatever mood I was in at the time I watched the movie, my changing opinions more generally over time and even if I've watched it (and can remember what it's about) before. If you've ever tried rating a film you've rated before without knowing what you gave it previously, you may see an example of what I mean.

However, if a film has something intrinsically special about it I add an extra five. This could be for anything, from a specific actor I especially like being in it, great individual bits of music or the concept of the story, through to it having a specifically good scene, or evoking an especially strong emotional reaction from me.

70 is my base score for a decent film that I enjoyed but probably wouldn't miss much if it vanished from existence tomorrow. This is quite high, but I mostly watch things that, for better or for worse, I've bought a copy of; and I try not to buy things I don't think I'd like much. (However, I wish with hindsight that I'd picked 50 instead.)

Also, it's easy initially to try and be more specific with your scores, but after you've rated hundreds of films I'm personally not that interested in how the individual ones in the middle of the pack relate to one another. I'm interested in the ones that get the best and worst scores, but I'm not really bothered if the one that's number 435 actually is better than the one that's at 436.

livelove
Posts: 759
0 Ratings
Your TCI: na
Joined: Thu Sep 17, 2015 9:36 pm

Re: How granular are your ratings ?

Post by livelove »

SpikyCactus wrote:If you've ever tried rating a film you've rated before without knowing what you gave it previously, you may see an example of what I mean.
that's actually a pretty good test everyone should try who rates in steps of 1

SpikyCactus wrote:Anything more granular hurts my head and the variations between smaller numbers would get swamped by whatever mood I was in at the time I watched the movie, my changing opinions more generally over time.
same here! (except that I score in multiples of 5)
Personally, I can't fathom how it should be possible to precisely narrow down an emotional reaction to an artwork to 1 shade out of 100.

SpikyCactus wrote:even if I've watched it (and can remember what it's about) before.
sorry, I didn't get that part. what do you mean?

PrestoBix
Posts: 83
0 Ratings
Your TCI: na
Joined: Thu Aug 20, 2015 6:48 am

Re: How granular are your ratings ?

Post by PrestoBix »

The better a movie is the easier it is for me to be precise, and vice versa. When I know a film is going to be in the 90 to 100 range, it's easy for me to pick a number. But when a movie is in the sub-50 range, it's harder for me to pick it out, but I also don't care, so often I just say 2.9, sounds good, and am done with it.

If I'm having trouble, I'll just look at similarly ranked movies and ask which one is better, and use that as a check. But, I've been using the same rating scale since 2008, so I'm quite used to it by now. Criticker is a godsend for people like me.

livelove
Posts: 759
0 Ratings
Your TCI: na
Joined: Thu Sep 17, 2015 9:36 pm

Re: How granular are your ratings ?

Post by livelove »

PrestoBix wrote:The better a movie is the easier it is for me to be precise, and vice versa. When I know a film is going to be in the 90 to 100 range, it's easy for me to pick a number. But when a movie is in the sub-50 range, it's harder for me to pick it out
couldn't this be a sign that your ratings are skewed towards the top? 70 is only in your 25%. Which means that you have to cram 75% of your films in the 70-100 range. The consequence is that you need a very fine granularity there (1 step), whereas this 1-step-granularity is way too fine in the lower range, as you point out yourself. This is a consequence of the skewed distribution. If you "stretched" your ratings more to the bottom (more towards a Bell curve) — or in other words: be stricter with your ratings — you could satisfactorily use a lower granularity for the entire 0-100 range and encounter less difficulties in picking the right number.
Image

PrestoBix wrote:often I just say 2.9, sounds good
2.9 out of 100? What does that mean?

SpikyCactus
Posts: 6
5 Ratings
Your TCI: na
Joined: Sun Jul 31, 2011 2:04 pm

Re: How granular are your ratings ?

Post by SpikyCactus »

livelove wrote:
SpikyCactus wrote:If you've ever tried rating a film you've rated before without knowing what you gave it previously, you may see an example of what I mean.
that's actually a pretty good test everyone should try who rates in steps of 1
Thank you. :)

SpikyCactus wrote:even if I've watched it (and can remember what it's about) before.
sorry, I didn't get that part. what do you mean?[/quote]

Whoops. What I meant was that if I've watched a film before (and can remember anything about it), then the fact I've seen it before will colour how good I think it is on the second or subsequent viewings. Some movies can be seen again and again without really seeming to be better or worse, whilst others lose a lot of their impact after the first time, or actually get better on repeated viewings.

livelove
Posts: 759
0 Ratings
Your TCI: na
Joined: Thu Sep 17, 2015 9:36 pm

Re: How granular are your ratings ?

Post by livelove »

thanks for the explanation, SpikyCactus.
I opened a new discussion for the interesting issue you raised, if you like to comment.

SpikyCactus wrote:70 is my base score for a decent film that I enjoyed but probably wouldn't miss much if it vanished from existence tomorrow. This is quite high, […] (However, I wish with hindsight that I'd picked 50 instead.)
I'm aware that you'd choose 50 now, but I am just curious:
Why did you choose such a high number in the first place for a film described in these rather negative terms?

SpikyCactus
Posts: 6
5 Ratings
Your TCI: na
Joined: Sun Jul 31, 2011 2:04 pm

Re: How granular are your ratings ?

Post by SpikyCactus »

livelove wrote:thanks for the explanation, SpikyCactus.
I opened a new discussion for the interesting issue you raised, if you like to comment.

SpikyCactus wrote:70 is my base score for a decent film that I enjoyed but probably wouldn't miss much if it vanished from existence tomorrow. This is quite high, […] (However, I wish with hindsight that I'd picked 50 instead.)
I'm aware that you'd choose 50 now, but I am just curious:
Why did you choose such a high number in the first place for a film described in these rather negative terms?


I wouldn't call my description negative, as I do say a 70 is a film I enjoyed. I think watching a really good film (like any really good experience in life) will be something you look back on afterwards with a degree of pleasure and remember that you enjoyed, even if you can't always recall the details. However, there're many things in life that I enjoy at the time but not long after have faded from memory. I go to a lot of gigs and I enjoy nearly all of them, but few remain in my memory for long afterwards. However, a few I recall for years afterwards as being amazing, even when I can't really recall the details of why. It's the same with films, I've lots of 70 films that I can't recall anything about, but from the score I know I enjoyed watching at the time. The ones with even higher scores I think I can generally remember more details from, although this is enhanced in many cases by multiple viewings.

Post Reply