What's going on! - Music - Forums : Best Ever Albums

albummaster

Janitor

Gender: Male

Site Admin

soundguardian wrote:

I wasn't too active on this site last year, tbh. at least, not this time of the year. I've come to realize that ratings on this site are HIGHLY unreliable. Especially for tracks; I recently saw the ratings on the Spiritualized album Ladies and Gentlemen.... Three months ago, "Ladies and Gentlemen..." and "Broken Heart" had amazing ratings (~88 or 89). Then 5 or so people skew the results by giving the songs 5/100. It clearly marks a simple biasing behavior, and angers me that people are not rating things fairly. There's no reason anyone should give those tracks a 5/100 (especially not 5 people in the expanse of 3 months) unless they do it purposely to skew ratings. And of course people could be like "well, that's just like... their opinions, man...", but statistically they're all considered outliers... by several standard deviations... so statistically their ratings aren't very sound. Which indicates that bias has seeped into the tracks' ratings. Not just preference, but bias. And the averaging algorithm that BEA has implemented apparently can't support nullifying the effects of outliers as well as I first thought. Even if it could, more people would probably come and rate things lowly until the scores DO go down... .

Anything with a rating outside three standard deviations of an item's mean average is not included as part of the overall item average. I've just spot-checked the tracks that you mentioned to see if there is a problem & the standard deviation for those tracks is high. For example, the first track you mentioned, the STD is over 26 and so anything within three standard deviations is included which gives you a range from about 5 to 100 (for that track) where all ratings are counted.

EDIT: I'll also check into the ratings as well to see if there might a gaming issue as it is unusual to see so many low ratings scattered together for the same track.

EDIT 2: I've just banned nine users who had the same IP address (one user with multiple accounts) who had rated over 1,000 different items with each account (including the above tracks) in a clear pattern of gamesmanship (rating the same items at similar times etc). Thank you for mentioning the original issue.

soundguardian

Gender: Male

Age: 34

Location: alone in the superunknown

albummaster wrote:

Anything with a rating outside three standard deviations of an item's mean average is not included as part of the overall item average. I've just spot-checked the tracks that you mentioned to see if there is a problem & the standard deviation for those tracks is high. For example, the first track you mentioned, the STD is over 26 and so anything within three standard deviations is included which gives you a range from about 5 to 100 (for that track) where all ratings are counted.

thanks for the clarification on how the algorithm works!

albummaster wrote:

EDIT: I'll also check into the ratings as well to see if there might a gaming issue as it is unusual to see so many low ratings scattered together for the same track.

EDIT 2: I've just banned nine users who had the same IP address (one user with multiple accounts) who had rated over 1,000 different items with each account (including the above tracks) in a clear pattern of gamesmanship (rating the same items at similar times etc). Thank you for mentioning the original issue.

Ok, that's good to hear. I was also a little suspicious about that. I'm glad I could be helpful! 😄

also, in the future, if I see any more signs of potential gamesmanship, is there somewhere specific that I should go to notify about it?
_________________

<--suddenly...Bora!
#BodyRollSwag

albummaster

Janitor

Gender: Male

Site Admin

soundguardian wrote:

Ok, that's good to hear. I was also a little suspicious about that. I'm glad I could be helpful! 😄

also, in the future, if I see any more signs of potential gamesmanship, is there somewhere specific that I should go to notify about it?

Please just leave a post in the suggestions forum. I think its best to be as open as possible about these types of things. I may also add more info where the ratings are displayed to show the STD and trimmed mean etc. Items with a high STD probably need flagging on the site somewhere as they may indicate funny business is going on.