Rating Bad: Selective Memory in Film and TV Reviews
Posted by JP Kelly
Last week The Guardian reported that, in terms of page views, Breaking Bad was ranked as the most popular television series on Wikipedia in 2013 (and third most viewed Wikipedia page overall – trailing only to Facebook at number one and the rather morbid “Deaths in 2013” at number two). With this kind of evidence there seems to be no disputing the series’ popularity but I do still struggle to understand why it has received such high praise from critics and audiences alike (for the record, I think it’s good, just not the greatest television show ever made).
Indeed, we need only look at IMDb [The Internet Movie Database] where Breaking Bad has an average rating of 9.6 (out of 10)* making it the highest rated TV show on the site. For regular visitors to IMDb, you’ll know that this score is very high. To give you a sense of perspective, even the highest rated movie on IMDb is only 9.2 (The Shawshank Redemption in case you were wondering). Once you get to the fourth most highly rated film (Pulp Fiction) you’re already down to 8.9. If we compare this to the top rated TV series of all time on IMDb, it isn’t until we get to the 24th most highly rated series (Friends) that we find a rating as low as 8.9. Although it’s like comparing apples and oranges (as Walter White might say) is Friends (the 24th best television series of all time) really on a par with Pulp Fiction (the fourth best film of all time)? Does it make sense that only two of the top rated films of all time (The Shawshank Redemption and The Godfather) would even make it into a top ten list of film and TV combined?
This disparity between film and TV ratings seems to suggest one of two things: either people hold television in higher regard than film (which I personally do), or — and this is what I’d like to explore in the remainder of this post — their feedback on TV series is not entirely accurate.
The first reason why television ratings may be skewed is to do with fundamental differences in format. Film is a relatively self-contained media text usually lasting around 90-120 mins (or needlessly longer as has become the recent trend). When people rate a film via a site such as IMDb they do so based on those 90-120 mins. However, a television series such as Breaking Bad has a runtime in the thousands of minutes. This raises a number of questions in terms of how accurately viewers can evaluate a series. For instance, will viewers have seen ALL of a series they are rating (assuming it has already ended)? Even if they have, how can they make a fair and accurate assessment of thousands of minutes of television – their final verdict somehow distilled to a figure between 1 and 10. I’m not suggesting that this is easy to do with film, but it is arguably a much simpler task given the significantly smaller volume of content that needs to be considered.
A second potential problem with IMDb’s ratings is the possibility that viewers will rate a series during a particularly good/bad spell (i.e. whilst it is still on-air) thus providing a specific (rather than general) evaluation. In other words, ratings will often represent someone’s view of a TV series at a particular point in time, rather than their overall impression of the programme as a whole. The fact that series often get worse (and sometimes better) suggests that ratings that are submitted while a programme is still on air don’t necessarily reflect a viewer’s overall impression. The chart below showing the average episode and season ratings for Breaking Bad is testament to this fact. If you followed the series right through to the end (which, in my opinion, was far better than the beginning) there’s a good chance that your rating will be inflated given that you will recently have watched the “good part” whilst the “average part” was several years ago.
- Breaking Bad IMDb Ratings | Credit to http://graphtv.kevinformatics.com/
The third (and most interesting) possibility is that people appear to have “selective memory” when it comes to rating television series. In Television’s Second Golden Age (1997), Robert J. Thompson questions the widely-held belief that US television was at its peak (in terms of quality) during the 1950s and early 1960s – a period often referred to as “The Golden Age”. In doing so, he points out that many inferior quality programmes simply disappeared from television history (and more importantly, from people’s memories) due to their lack of syndication and repetition. Meanwhile, the more successful / superior quality series will inevitably be those chosen for syndication and repetition – television is a business after all. For those who prefer programming of yesteryear, this recollection of television’s past as superior to the present is therefore partly due to the fact that the most of the “bad stuff” never made it through.
In addition to highlighting the ephemerality of inferior programming (in schedules and in our memories), Thompson argues that people will tend to feel nostalgic when recalling television series – selectively remembering the good episodes, and forgetting the bad or average ones. Although it might be a stretch to suggest that people selectively (i.e. nostalgically) remember Breaking Bad considering that it has only recently concluded, there is some evidence to suggest that this is the case. If we compare the ratings that viewers have submitted to the main page of Breaking Bad on IMDb (i.e. the overall rating – not based on any individual episode but given to the series as a whole) with the average episode rating, there is a notable difference. The overall rating based on the score for the series as whole is 9.6 while the average rating based on individual episodes is only 9 (or 9.00483871 to be more precise!) In real terms, this represents a 6% difference in how people rate/remember the series as a whole versus how they rate it via the more thorough and precise method of calculating an average based on an episode by episode basis. Ultimately this suggests that there may be some truth to Thompson’s claims about television viewing and selective memory – in other words, it indicates that people tend to forget or ignore lower rated episodes when making an evaluation of a series as a whole. Of course, people are perfectly entitled to do this; to ignore the bad episodes and to pick and choose their favourites as they see fit. But it’s still interesting to note the discrepancy between individual episode averages and overall scores.
It is also worth pointing out that IMDb TV ratings often decline over time. In January of 2010, The Wire topped the TV list with a rating of 9.7 (yes, higher than Breaking Bad) but in the space of four years has fallen to fourth place with an average rating of 9.4. Although this is a fairly marginal drop, it does highlight the way that buzz around current TV series may contribute to inflated ratings. In short, it’ll be interesting to look back at Breaking Bad in four years time to see how it has stood the test of time.
Breaking Bad isn’t unique in terms of a discrepancy between overall rating vs. average episode rating. Game of Thrones, which is joint second on the list of highest rated TV series with an overall rating of 9.5, only has 8.86 when calculating an average based on individual episode reviews. While time doesn’t permit me to calculate the average for other examples, I suspect that this may be the case for many if not most television series.
So what does all of this tell us? And what is the point of deconstructing IMDb’s ratings in this way? The first thing it tells us is that we should take IMDb’s television ratings with a pinch of salt. If, like me, or the countless other people I know who use IMDb to make a decision about what to watch, you probably need to recalibrate your expectations.
Finally, it’s reassuring for those of us who either don’t like Breaking Bad or don’t think it’s the “greatest television series ever” and spend so much time and energy having to defend our opinions. If anything, it provides fairly substantial evidence that people think it’s better than it actually is (specifically in terms of how they remember and rate TV as an overall experience versus the much more thorough method of averaging out the score of every individual episode). So there you have it, hard evidence to prove that Breaking Bad isn’t as good you think it is. As Jesse Pinkman would say, “Yeah! Science bitch!” Well, rudimentary mathematics anyway.
* The ratings cited in this post were accurate as of January 16th, 2013.
About JP KellyLecturer in Film and Television at Royal Holloway, University of London. Twitter: @jippykelly Web: www.johnpaulkelly.co.uk
Posted on January 16, 2014, in AMC, Breaking Bad, Game of Thrones, IMDb, JP Kelly, Ratings, Research, Royal Holloway, Television, Television History, The Wire and tagged Breaking Bad, IMDb, JP Kelly, Ratings, Royal Holloway. Bookmark the permalink. 7 Comments.