Safeguarding Your IF Voting From Animal Attack

By Ted Casaubon

Last year, sci-fi fans were appalled to discover that the nominations for the 2015 Hugo Awards were dominated by a small but disgruntled and well-organized minority of the voters called the Puppies, who publicized a slate of candidates and voted mostly in lockstep. Almost all the Hugo nominees are unread by me, but the consensus was that the Puppies’ picks were motivated by racial and gender bias (at least in the case of the larger of the two Puppy factions, the Rabid Puppies) as well as a preference for the old-fashioned where plots are concerned. Some important categories didn’t have a single nominee that wasn’t drawn from the Puppy slates, leaving many voters with nothing to mark on their ballots but “No Award.” The sci-fi community quickly began debating voting reforms to stop the Puppies from repeating their victory in 2016.

Does the IF community have anything to learn from the sci-fi fans’ electoral strategizing? Are IF awards like the IF Comp and the XYZZY at risk of a similar raid by canines, alligators, or their evolutionary descendants?

In one sense the IF awards look even more vulnerable than the Hugos. The WorldCon organization requires voters to pay at least a $40 “supporting membership” fee to submit a Hugo ballot, but voting for the Comp and XYZZY awards has been free to anyone in the world whos willing to register on a website. That means it would be easier to coordinate a bloc of voters who otherwise wouldn’t vote at all. For the most part, open-voting IF awards’ obscurity has kept them secure. But history has shown that on the Internet, an angry mob can materialize overnight, and an IF game isn’t necessarily too small a target to attract trolls’ fury. And IF awards have few enough voters that even a small effort may prove statistically significant.

The XYZZYs have a somewhat vaguely-defined rule against voting blocs. Traditionally, entrants are given this guidance: “canvasing for votes is strongly discouraged, either for your own game or on behalf of others. It’s fine to talk about the XYZZYs – but if doing so results in a flood of voters all voting for the same game, those votes will be discounted.” For the 2011 XYZZY awards, Choice of Games reported that they effectively overwhelmed the voting with a single blog post encouraging their fans to vote for an eligible ChoiceScript game that year. (In a further affront to certain segments of the traditional IF community, the game that received the flood of votes was about zombies.) That year the XYZZY organizers threw out every ballot that didn’t vote for at least one game not written in ChoiceScript, and another game was crowned the winner. (To their credit, 2012’s post was scrupulous to avoid even the slightest inkling of a repeat.)

The IF Comp’s corresponding rule on bad-faith voting, adopted before the 2014 Comp, says that every judge who rates a game should make “a good-faith effort to actually play that game as intended” and the organizers “reserve the right to disqualify any ratings that appear to have been submitted under any other lcircumstances.” However, there’s no guarantee that there’ll always be a way to distinguish bloc voters’ ballots from everybody else’s, or any way to reconstruct what the result of an election would have been if an undesired “flood” of voters hadn’t happened. (This makes everything murky; last year’s comp saw rumblings of the fact, or perhaps coincidence, that every Twine game in the 2015 comp, without exception, received two 1/10 votes.)

I wish to apply a very, very light touch to this sort of negative enforcement,” IF Comp organizer Jason McIntosh wrote. “One of the best aspects of the IFComp is the annual array of very thoughtful reviews that all sorts of people write and publish. Some of these have, in recent years, begun to appear in high-profile commercial media. I would certainly not wish to conflate audience enthusiasm — resulting, I hope, in audience growth — with Sad/Rabid Puppies-style, bad-faith bloc voting!”

There is a limited release of post-Comp voting stats that allow fans to speculate about whether there was any unfair pattern in the ballots. McIntosh also pointed to the longstanding requirement for judges to rate at least five games as an important safeguard against “thoughtless ‘drive-by’ votes, or even well-intentioned votes where a person read a high-profile glowing review of a game, played it, and then did nothing else — which might prove imbalancing to the comp overall.”

One unique feature of the XYZZYs seems to give them an advantage over the Hugos in diluting voting slates: the XYZZY website lets each voter nominate only a single work per category in the nominating round. That means it wouldn’t be possible for a bunch of prospective ballot-stuffers to copy-paste a slate of games from a blog and turn that same slate into the entire second round list of nominees. It would at least demand a certain minimum amount of logistical effort for the bloc voters to coordinate their votes to make sure each entry on their slate scored over the threshold.

A proposal that’s been thrown around for Hugo voting is to allow a limited number of anti-votes in the nominating round. The IF Comp essentially has anti-votes already, because you can assign games you don’t like a “1.” For the XYZZYs (like the Hugos), there’s such a long tail of eligible but unworthy works each year that it wouldn’t be feasible to anti-vote for all of them. So, the only value in allowing anti-votes would be as failsafe against voting slates: when you hear that some person you don’t like has created a slate, you anti-vote for everything on it. On the downside, anti-votes would create a new weapon for trolls to wage personal vendettas and target quality works for political reasons, and the XYZZYs would have to join the Comp in awarding an annual Golden Banana of Discord.

A more interesting alternative was proposed in a discussion about the Hugos on Quora. The idea was to use “reweighted approval voting” in the nominating round, which would mean that after the candidate with the most nominating votes was placed on the nominee list, an elector who voted for that candidate would get only half a vote for deciding the second entry on the nominee list. If the elector voted for both of the first two candidates who made in on the nominee list, then the elector would get only a quarter of a vote for deciding the third nominee, and so on. Reweighted approval voting has the benefit of diluting coordinated voting blocs, but it also encourages tactical voting. If you’re confident your preferred candidate is the front-runner, you have a disincentive to vote for that candidate, because by doing so you dilute the votes that you could have used to decide who became the second nominee, the third, and so on.

Then there was the long discussion on Making Light that ended with another proposal inspired by the idea of diluting ballots that have votes for more than one front-runner. It uses a “single divisible vote with least popular elimination” system that asksgood-faith voters to divide up their vote among (ideally long) lists of potential nominees, and then whittles away the list of finalists by removing the least popular ones first and reassigning their share of the vote to the other candidates that shared the same ballots with them. In theory, the good-faith electors’ voting power will end up concentrated behind only the most popular entries on their longlists, while the bloc’s voting power will remain diluted if all the entries on their voting slate are about equally popular. Still, the good-faith voters may face a dilemma because if they nominate too many works, they’ll have no way to express a preference between them, and works with broad but lukewarm support might become the finalists.

The voting systems described above are all intended to ensure that a minority bloc doesn’t thwart the will of the majority. But the reality is that a majority voting bloc could be just as harmful to the integrity of an IF award, if it was the result of a raid on the polls from outside the community. The only real way to prevent that would be to limit who gets to vote. A time-tested way to keep out undesirables is to use Academy-style voting like the Oscars, Emmys, and Grammys, or a critics’ award like the Golden Globes. But it’s not easy to keep track of the membership of an academy, especially when the award organization is a few volunteers working for free. An IF academy would present particular problems because there’s no clear definition ofwhich works count as “IF,” and it’s almost certain that technological changes are going to expand the world of IF in unexpected directions. The enthusiastic ChoiceScript supporters were seen as invaders by the IF community in the 2011 XYZZYs, but with Creatures Such as We taking second place in the 2014 Comp and Scarlet Sails taking 7th in 2015, they probably wouldn’t be considered such outsiders today. Without knowing which works are IF, it’s hard to say whos a part of the IF community, and still harder to form an electorate that includes all of those people while keeping others out. No electoral system can be perfect, and the more disagreement there is about whose interests the system should serve, the more challenging it’ll be to craft rules that make the outcome feel legitimate to everybody.

4 thoughts on “Safeguarding Your IF Voting From Animal Attack

  1. Dan Fabulich

    For the historical record, after the 2011 XYZZYs, we did another *very* careful blog post for the 2012 XYZZYs. https://www.choiceofgames.com/2013/04/vote-for-interactive-fiction-in-the-xyzzy-awards-for-2012/

    The result was not very encouraging. There was less bloc voting than in the prior year, but that’s all the good news.

    As a result, we have refrained from posting about the IF community’s open-voting competitions on our blog ever since, which only reinforces the problem that the ChoiceScript community as a whole isn’t very interested in (or knowledgeable about) non-ChoiceScript IF, even Twine games.

  2. Katherine Morayati Post author

    Thanks for the note — I’ve updated the author’s copy to reflect this.

  3. A

    XYZZY and IFComp are two different situations, I should think. Nominating candidates from a vast field of works post-facto vs. rating a limited pool of works that were released simultaneously and for the same purpose. Therefore their considerations for avoiding bad-faith votes differ.

Leave a Reply

Your email address will not be published. Required fields are marked *