The OpenWater Blog

Learning Series: Types of Judging

Once you have your judges selected and your entries are in and accounted for, the next logical step is to judge each entry. There are three main ways you can judge an entry and, if you’re like most organizations, you’ll use a combination of all three methods for your competition.

Method #1: Judge by Criteria

Judging by criteria eliminates something called “The Halo Effect” described in Daniel Kahneman’s New York Times Bestseller “Thinking, Fast and Slow.”

The Halo Effect means that we judge based on a summary conclusion of pass/fail rather than a set of criteria. In this sense, the Halo Effect is the aura surrounding what’s being judged. The summary conclusions we generate as we evaluate an entry gives us our general sense of each entry and affects the judging outcome (even though you don’t want it to).

To eliminate this effect, you should set forth a number of criteria that will measure the quality of each entry. Typically, programs choose between three and six criteria to do so to ensure precision without overwhelming judges. Then, by asking judges to assign a score between 1 and 5 to each criteria and totaling the score from all categories, you can rest assured that you have a good representation of each entry.

Popularity Vote

Another way to judge entrants is through a popularity vote or the People’s Choice vote. To do this, open all the entries to the public or select a group and allow them to vote for which entry should win.

There are multiple ways to conduct a popularity vote depending on your program. You may want each person voting to only have one vote across all entries, one vote per category, or allow people to vote for as many entries as they want. Many competitions impose time restraints on allowing multiple votes by limiting each person to submitting only one vote in a 24-hour period.

Popularity votes shouldn’t be used to determine the winner of your program. Instead, they should combine with judging criteria to weight scores accordingly. For example, you may want the popular vote to account for 20 percent of the total score whereas the judging criteria account for the remaining 80 percent of the score.

Ranking

The final judging method is ranking. Ranking allows judges to compare entries and then determine which they prefer from best to worst.

Like the popularity vote, ranking should only be used in combination with other judging methods, particularly judging criteria. Many programs use criteria to determine the top 5 or 10 entrants and then allow judges to rank to determine which entrant wins.

Strike the Right Balance in Your Judging Process

It’s important that you determine which types of judging are most appropriate for your program and weigh your options to figure out what is most appropriate. After all, you may have the best panel of judges but if your judging process is unfair, entrants won’t care who the judges are.

Reinforce industry experts with a bias-free, well thought out judging process for your best program yet.

Ashley Surinak

Ashley Surinak is the resident OpenWater expert on all things content. From the blog to guides and beyond, you'll find her at every turn in your OpenWater journey.

How to Use Content to Attract Entries During the Call for Papers

Stretch Your Budget With Mobile Engagement