The Elimination of Dead Weight

[mr rigard]

I feel there might be a lot of temporary users who come onto criticker, get all excited during their intial visit or two, mindlessly rate a whole bunch of movies, then move on to nowheresville. The problem that I have with these users is that they appear on my TSI list and irrelevantly effect my PSI's.

Let's face it, they aren't contributing to the community by seeing movies at the movie theater or rating new releases that come out on DVD at the video store. The net result is a a PSI list that is top-heavy with older movies.

Always wanting the best for the Criticker Community, I think it would be nice to implement some type of inactive policy or feature. Maybe if somebody hasn't rated a movie in the past month, somehow make them inactive and take them "off the table" until they become active.

Users that come onboard for a week or so, rate a whole bunch of movies that they have seen YEARS AGO and don't actually remember isn't a useful factor in comparative minds.

[Quicky]

The net result is a a PSI list that is top-heavy with older movies.

Actually this is not true. Users who only rated older movies do not cause PSI's to be higher for those movies. On average they will even out. The only thing you can say is that they're not contributing to new movies and thus are not solving the issue where PSI's can't be calculated due to a lack of ratings for the newer movies. However, getting rid of these users is not going to solve this issue either. If you find that your PSI list is top-heavy with older movies, it's for an entirely different reason and is not related to the amount of inactive users.

It's been mentioned before in a similar topic that you cannot cheat Criticker's recommendation system or influence it in a bad way. Every contribution is a useful one for those people who have similar tastes. It is however true that nobody wants to waste time checking up on people who aren't active in the community anymore. So probably it would indeed be useful to 'hide' inactive users from your TCI list.

[mpowell]

Yes, this is something that has been brought up before. In the past, we never really had to think about it much, since Criticker itself was a "newbie" but now we're starting to mature, and a lot of users aren't really active anymore.

We've got an enhancement in our database for a way to filter out users who haven't ranked films in a certain amount of time. We don't want to narrow down our "active" user base too much and dilute the wealth of data available, but I agree that eventually newer films will be underrepresented in PSIs. So, it will be about finding the right balance.

[paulofilmo]

You've got me afeared that I'll go on some two year hippie expedition and come back to find I've been deleted.

[mpowell]

Oh no, we'd never delete data. This would just take the form of allowing people to exclude "stale" users from their PSI calculations.

[Quicky]

Oh no, we'd never delete data. This would just take the form of allowing people to exclude "stale" users from their PSI calculations.

At the risk of being an annoying and inquisitive whiner, may I ask why you'd exclude users from PSI calculations (as opposed to just excluding them from visible TCI lists)? To me it seems that excluding users from PSI calculations does not give any mathematical advantage, quite to the contrary... or do you feel this loss of data is worth the gain from having more active people in your TCI's?

[mpowell]

At the risk of being an annoying and inquisitive whiner, may I ask why you'd exclude users from PSI calculations (as opposed to just excluding them from visible TCI lists)? To me it seems that excluding users from PSI calculations does not give any mathematical advantage, quite to the contrary... or do you feel this loss of data is worth the gain from having more active people in your TCI's?

I think I wasn't very clear. What we'd probably do is give people to filter their TCI lists, so that "stale" users are removed. Once you no longer have TCIs with stale users, they wouldn't be used to generate PSIs (because we just use your top TCIs for this). As to the advantage... consider that if 90% of your TCIs haven't logged in for a year, you would very rarely get recommendations for films added to our system in the past year. That percentage is unlikely for now, but eventually it's true that more and more of your top users would become "stale".

Again, the filtering would be something users could opt in or out of. And, I'd like to stress that we haven't started to work on this at all, or really even think about it very hard ... this is just the direction we probably will go once we get to that point.

[paulofilmo]

Oh no, we'd never delete data. This would just take the form of allowing people to exclude "stale" users from their PSI calculations.

*whew*. Ta. That's reassuring.

[Quicky]

I'm not sure I agree on the math here. But... please allow me to split the issue into two different ones for clarity:

1. Are TCI's from users who are not active anymore less reliable than those from active users? And should they thus be filtered out for PSI calculations?
2. Assuming that the TCI's calculated in the first issue are completely accurate and reliable, will it be a) mathematically advantageous or b) disadvantageous to remove inactive TCI's from the PSI calculation or will it be c) irrelevant.

The point I've been trying to make before was that for question 2) I believe the answer is b): it will be disadvantageous to remove those inactive users because if there's a user with a lower TCI available, taking it into account for PSI calculating will always improve the accuracy (or at least the probability for being accurate). However, I'm starting to realize that what you are all probably instinctively feeling, is that the answer to question 1) might be yes, TCI's are less reliable for inactive users.

Why would they be less reliable? Maybe because these users are by definition biased towards less recent movies? Maybe because they're biased towards less committed users who rate more randomly? Maybe you can think of a few other reasons... I don't know. If it turns out that TCI's from inactive users are indeed less reliable, then the point I was trying to make before, i.e. issue 2), becomes irrelevant...

[mpowell]

Well, like I said earlier, this is a concept which we haven't even really started to think about very much yet, let alone work on. It seems that there's interest for such a feature, so we've written it down as something to look into in the future...

All the thoughts here are definitely valid, and everything will be taken into consideration once we do begin to look at this.