Judging best practices for awards programs

by | May 4, 2021 | Articles, Blog

There’s more than one way to pick a winner.

In fact, in the Award Force judging suite the options for setting up and managing an effective and successful assessment process are practically endless. 

Award Force judging review

While the options are aplenty, judging configuration can sometimes feel like a daunting task. Where should you start? How can you decide which judging mode (or combination of modes) is right for your program?

We’ll break it down for you in this article with an overview of the Award Force judging suite, some helpful tips, use cases and best practices on how to set up your awards program judging process for ultimate success.

A judging mode for every program type

There is no one way to judge an awards entry. That’s why Award Force provides an entire suite of sophisticated judging tools to help you assess each submission. All our judging modes are designed to be quick, friendly and easy to use for judges. 

We offer the following four judging modes:

Qualifying judging mode

Qualifying judging mode in Award Force

Qualifying is used to make simple pass/fail decisions. This is an ideal method for cleaning up a large volume of awards program entries before moving them on to a more in-depth judging round. 

Top Pick

Award Force Top Pick judging

 

The Top Pick judging mode employs the STV method (Single Transferable Vote – widely used in Western elections)) of calculating results. It’s designed to achieve proportional representation through ranked preferences. Perfect for multi-winner outcomes or to draw out consensus amongst a large judging panel.

With this method, you can decide how many preferential picks each judge gets, and they have a live progress bar to help them keep track of their choices. 

Voting

People's choice voting in Award Force


The
Voting judging mode is a “people’s choice” option and is great for soliciting votes from the public. It’s also useful for garnering votes from a closed audience, like a specific organisation,  industry body or other members-only groups. 

VIP Judging (score-based assessment) 

Score-based judging in Award Force

VIP Judging is a highly configurable score-based assessment method and is best for in-depth reviews performed by industry, organisational or expert judges. You have complete control over the judging criteria, which your VIP judges can see and refer to throughout the judging process. 

Choosing the right judging tools for your awards program

In Award Force, you can tailor your judging configuration to suit your program by using one or more judging modes to support your judging process. You can use multiple judging modes in tandem or set them up to run consecutively, moving entries through several different rounds of qualitative and quantitative assessment for the best judging outcomes. 

To help you visualise what’s possible, we’ve compiled a few common use cases to illustrate just some of the ideal judging configurations.

A high volume photography contest 

Photo contest in Award Force


What:
A highly visual arts contest that solicits a high volume of photographic entries.

Suggested judging mode(s): Qualifying followed by VIP judging with an online gallery for sharing (shown above in the Award Force platform)

How it works: Using the Qualifying judging mode, program managers (or the judges themselves) can quickly pass/fail entries based on a benchmarked quality standard. Once the entry has passed the qualification round, it can be automatically sent to expert judges using the VIP judging mode. These experts then assess and score based on  specified criteria with the highest scored entry being declared the winner.

Other tips: Create categories to help organise the entries as they are submitted. Consider setting up an online gallery (also possible in Award Force) to share shortlisted entries with the public. 

A “best in industry” association awards program with many judges

What: An association-sponsored awards program to recognise the best in industry, using a large community of board members to serve as judges.

Suggested judging mode(s): Top Pick followed by VIP judging

How it works: The association wants all the board members involved in the awards selection process but finding consensus amongst a large judging panel can be challenging, so we suggest using Top Pick to help find the top 10 preferred entries and then follow that up with a round of expert review using VIP judging. Initially, using Top Pick, each judge would simply pick 10 of their preferred entries. These individual picks will then be calculated using the Single Transferable Vote method and the top 10 preferred entries will move forward in the judging process. Then, the association’s expert judges will assess these 10 entries against a set of criteria and apply a score for each criteria set. The winner is declared based on the highest scoring entry. 

Other tips: You can optionally allow judges to discuss and collaborate with each other during the judging process by enabling judges to comment and/or view comments from other judges on a per entry basis.

A portfolio assessment program with expert academic reviewers

What: A university portfolio assessment program for graduate students requiring extensive research and written portfolio assessment.

Suggested judging mode(s): Qualifying round with plagiarism detection followed by VIP Judging 

Plagiarism detection in Award Force

 

How it works: During the qualified round, a plagiarism detection scan can be run on all submitted entries to search for any plagiarised portfolios. If plagiarism is found, these portfolios can be quickly discarded. Once applications have passed the plagiarism scan, they can be automatically sent for expert review using the score-based VIP judging. 

The score-based judging can involve multiple rounds, with different criteria and judges for each round to help assess each portfolio in-depth.Scores can be given to the portfolio as a whole or you can create individual scoring criteria for each section of the portfolio. Different scoring criteria can be given different weightings. 

It’s also possible to provide access to others to participate in a peer review process. Automations can be configured to set up a workflow where the student is notified when the portfolio has been submitted, qualified, scored or whether they will be required to perform further actions.

 

Other tips: Use entry categories to organise portfolios up by topic, subject, grade, year or level. 

A people’s choice podcast audio awards program

What: A nationwide podcast awards program which accepts mp3 files of podcast samples, plus images of podcast art and advertisements There are several categories in this awards program, including “Best presenter” and “Best topical podcast”.

Suggested judging mode(s): Qualifying round followed by a Public Voting round 

How it works: Every entry goes through a quick pass/fail qualifying round to ensure entry quality. Once the qualifying round is complete, all qualifying entries will be made available for public voting, where members of the public can easily vote for their favourites. 

Each member of the public is allowed to vote once per category. And the winner in each category is determined by the highest number of public votes. 

Other tips: Require voters to register before they can participate. This cuts down on any abusive voting practices such as syndicate voting schemes or bot voting. Get more tips on creating a buzzworthy public voting program.

An employee recognition program with a nomination process

What: A staff excellence program where employees are nominated and then approved by their  manager before assessment by a review panel. 

Suggested judging mode(s): Review flow and VIP judging

Awards management review flow

How it works: A nominator submits an entry, the nominee is notified and decides whether they would like to enter by accepting or declining their nomination. If they accept, the entry is sent to their manager who either approves the nomination for judging, or requests a new submission from the original nominator. Manager approved submissions then move forward into the next round of VIP judging and a winner is selected. 

Other tips: You can also optionally allow the nominee to edit their submission before it is sent to their manager for approval. This makes it easy for nominees to edit or correct the submission and ensure it is both factual and compelling. 

Judging best practices for your awards program

Now that you have an idea of what’s possible when it comes to actual judging configuration, what else can you do to make sure the process runs as smoothly as possible? We’re glad you asked. Providing a delightful experience is often in the details. Here are some tips from our judging software experts at Award Force.

Make it easy for your judges with contextual information + instructions

Contextual information for judges


Your judges are likely very busy people. It’s important to make sure the judging experience is both easy and fast! The Award Force platform is intuitive from the start. But there are extra steps you can take to make sure your judges have a seamless experience, and that starts with providing all the information they need, right where they need it. 

In Award Force, you can add contextual information, which we call content blocks, throughout the judging view. In these locations you can provide judging instructions, images, or even videos to convey your message or content. In the scoring section, it’s easy to add help text right where they’ll add scores, making it easy for them to score without any confusion or questions.

Communicate often with your judges

In addition to providing helpful content for your judges, it’s also important to communicate regularly with them. This could include general updates about your program, important dates for judging, reminders on when their judging will start, whether there are any new entries for them to assess and other important program announcements. 

In Award Force, you can send personalised emails to your judges directly from the platform, complete with your branding. No jumping between systems, software or other websites required. 

Consider your judging criteria carefully

It’s important to create an entry form that your judges will love—from the very beginning. This means only asking for the information that judges need to make informed decisions about each entry. Consider applying word or character limits to entry fields to keep the entries in check and require consistency in any required attachment files. This will save your judges time and help them easily view and assess each entry efficiently. 

Reduce biases with recusal and abstention options

Manage potential conflicts of interest or bias with options for you to recuse and judges to abstain from judging.

Avoiding bias in judging


In Award Force’s judging software, you can easily recuse a judge from assessing specific entries and re-assign any relevant entries in the Assignments view and Panels. And, if a judge identifies a potential conflict of interest, they can also abstain from assessing it with a simple click on the relevant entry. 

It’s also possible to anonymise entries to remove potential judging bias. Use the field visibility tool to hide personal entrant information on entries (includes downloads and attachments), to ensure your judges won’t know who submitted what. 

Easy judging, happy judges 

The judging round of an awards program is an important part of every program. So, it’s vital to provide a simple and efficient experience for your judges. 

The Award Force judging suite gives you ultimate control to configure your judging quickly and easily and with flexible options you can create a judging workflow as unique as your awards. 

 

Learn more about Award Force’s powerful judging suite. Take a tour or book a live demo with one of our friendly team members.

Search our blog

Categories

Follow our blog!