The 2021 Hugo Awards: "No Award" Statistics

Last year, I did a post analyzing various statistics in the Hugo Awards voting data, mostly relating to the use of "No Award."* I want to do that again this year, especially as this year's Hugo team provided us with more data than is usual about the use of No Award.

But first, here's all the categories ranked in terms of how many ballots were received: (Note that this does not include ballots where "No Award" was the first preference.)

As is, I believe, almost always the case (it would be interesting to go back and verify), the prose fiction categories typically do the best, and the fan endeavor categories the worst. Interestingly, Best Novel (won by Network Effect) and Best Series (won by The Muderbot Diaries) were the top two, even though those are the two categories that require the most work for a conscientious voter. This year's special category, Best Video Game (won by Hades), come in a little bit below average. I have no sense if the Business Meeting will make this one permanent going forward. (I have no interest in it myself, but I am not philosophically opposed to as I am some other categories.)

Last year, the bottom category was a third of the top one, but this year it is only a quarter. One should note that as per the World Science Fiction Society Constitution, "'No Award' shall be given whenever the total number of valid ballots cast for a specific category (excluding those cast for 'No Award' in first place) is less than twenty-five percent (25%) of the total number of final Award ballots received" (3.2.12). Best Fancast came very close this year, with just 26.8%.

Next, here are all the categories ranked by how many ballots placed No Award as first preference:

You'll note that here, it's mostly the inverse of the other graph. Best Editor (Long Form) (won by Diana M. Pho) is the highest. As I said last year, this could mean two things: "First, it could mean that you find the concept of the category invalid. Every year, I vote No Award for Best Series, Best Editor, and a couple other categories, for example, and leave the rest of my ballot blank. I have some fundamental disagreements with the premises of those categories, and do not think they should be awarded. (Very few Hugo voters agree with me, though, clearly.) It could also mean that you just found everything in that category subpar [in which case you might rank finalists below that point anyway]." The other thing to note, I think, is that Best Video Game is a little higher than we might expect by its placement on the first graph.

One interesting piece of information given on the results is how many ballots ranked No Award above the eventual winner. This could mean 1) the ballot just No Awarded the entire category, or it could mean that 2) on that voter's ballot, No Award was placed higher than the winner. (This is reported because if more ballots place No Award above the winner than placed the winner above No Award, No Award wins, even if it doesn't on the normal counting method. I don't think this has ever actually come to pass, and I think it would require an incredibly polarizing winner for it to do so. This is called the "run-off.")

So here are all the categories ranked in terms of how many ballots had No Award ranked above the eventual winner:

This lets us identify particularly controversial winners.This year, though, it's not too different from the other above graphs.

We can highlight controversial winners even more by subtracting the number of people who ranked No Award first from the number of people who ranked it above the winner. So this shows how many ballots had No Award in 2nd place or lower, but above the winner:

What this reveals, I would argue, is how many voters believed a category had some basic level of validity (so they didn't put No Award first for whatever reason), but didn't like what actually ended up winning. There's nothing as standout as last year's Best Related Work win, but Best Fanzine (won by Nerds of a Feather, Flock Together) comes out on top, and this year's Best Related Work (won by Beowulf: A New Translation) comes in next. On the other hand, there were just two people who placed No Award above Sara Felix in Best Fan Artist!

Last year I posited the existence of what I've come to think of as the "No Award Curve," an inverse relationship from the total number of ballots cast in a category and the number of first-preference No Award votes. I asked, "Does it indicate that the bigger categories are bigger for a reason, i.e., people view them as more legitimate?" It seems to hold up this year:

As I did last year, what I find most interesting about it are the categories that are off the curve, where more people voted No Award than you would expect given the size of the category. In the past, Best Series has been an outlier, but this year it's right where it should be. Everyone sure does love Murderbot, I guess! The most noticeable outlier is Best Video Game: there is a sizeable contingent of Hugo voters who do not believe this category should exist. Not as big of outliers are Best Related Work and Best Dramatic Presentation (Long Form). It's nice to have validated my belief that The Old Guard did not deserve a Hugo.

Last year, this is where I wrapped up, but this year's Hugo team provided us with extra data: they ran the run-off test when determining every place, not just first. So we know not only how many people placed No Award above the first place winner of each category, but the second, and the third, and so on. So this year, we can identify not just particularly controversial winners, but controversial finalists.

I experimented with a number of ways of showing this, but ultimately settled on plotting the number of ballots ranking a finalist above No Award against the number of ballots ranking No Award above a finalist:

You'll see that in 95.0% of cases, no more than 150 ballots ranked No Award above a finalist. Indeed, 92.4% were within one standard deviation of the mean. The outliers were:

Category	Finalist	Ballots with No Award above Finalist
Video Game	Last of Us pt. 2	136
Semiprozine	Strange Horizons	139
Novelette	"Helicopter Story"	148
Dramatic (Long)	Tenet	166
Dramatic (Long)	Eurovision Song Contest	200
Related Work	Last Bronycon	211
Related Work	FIYAHCON	257
Related Work	CoNZealand Fringe	265
Related Work	"GRRM Can Fuck Off into the Sun"	358

So these are this year's controversial finalists. I can't comment to all, but in most, the reasoning seems pretty obvious. "Helicopter Story," as I have said, was pretty controversial; one notes that it got the most first-place preferences in Best Novelette and the most placing No Award above it. I guess people really did not care for Tenet or Eurovision Song Contest; the latter does not surprise me but the former kind of does.

What is particularly notable is how high all the Best Related Work finalist land... and indeed, all of these except The Last Bronycon I placed below No Award myself. A significant contingent of Hugo voters were clearly displeased with what Hugo nominators got onto the ballot in this category, even though "George R.R. Martin Can Fuck Off into the Sun" came in fourth, a full 24.2% of voters in this category ranked No Award above it.

I think, though, there are more variables here than I can account for as an amateur with a spreadsheet! I'd be curious to see things like, "what finalist got more No Award votes than we would expect based on its rank and number of ballots cast?" But that is pretty far beyond me!

* In the process of making this post, I realized I made an error in last year's: what is listed on the official Hugo voting statistics as "Cast Ballots" for each category excludes ballots cast that listed No Award in first place. This year I added those back in when calculating percentages.