Candidate Testing: 10 Questions to Ask Before Buying

Candidate tests are a great solution for companies looking to improve hiring outcomes, such as new hire performance, time to hire, and retention and your choice of assessment – it’s features and pricing structure – will shape your hiring process for years to come. It will ultimately influence hiring decisions and can, positively or negatively, impact recruiters’ relationships with hiring managers. As your company grows, choosing the right candidate testing provider, one that can adapt as you grow becomes hugely important.

To help you find the ideal testing tool, we’ve put together a list of 10 simple questions to ask before you buy.

1. What are your measures of success?
2. How are the tests priced?
3. Who will be using the test?
4. What are the strongest features of the test?
5. What reporting is available?
6. What’s the candidate experience like?
7. How is the test validated?
8. What are the training and set-up requirements?
9. What are the litigation risks?
10. What support will I get?

What is candidate testing?

A candidate test, sometimes known as a pre-employment assessment or pre-hire assessment, is a tool or method used to evaluate job candidates with consistency. Tests can range from hard skills tests such as coding, typing or cognitive ability to “softer” skills tests looking at behaviour, personality or situational judgement, or realistic tasks and scenarios.

A job interview is actually a type of candidate test, however, what most companies refer to when they talk about candidate testing is online assessments to judge candidate capability and role relevance.

1. What are your measures of success?

The most important thing to have in place before you start reviewing candidate test providers is some key goals and measures of success. These will preferably be tied to a clear business outcome and provide tangible ROI. Common examples include:

• Improve new hire retention by X%
• Reduce time to hire by X days
• Improve new hire performance by X%
• Reduce the number of phone screens conducted by X%

Each of these measures should have a financial benefit attached to them. e.g. reducing the number of phone screens saves our recruitment team X hours per week and a cost saving of £X. If your primary goal is to reduce the number of new hires who leave in the first 90 days, you can model your cost saving by using our cost of failed hires calculator.

2. How are the tests priced?

Is pricing based on a charge per candidate – or is it a subscription with unlimited use? Will you be billed monthly, yearly, or at a different frequency? It’s best to avoid tests that increase their prices based on usage as you’ll quickly hit those caps and incur unnecessary fees. Look for any add-on fees for extras, such as support, integrations, reporting, or training.

You’ll want to look for a test provider that offers:

• Simple pricing, making it easy to understand your costs and budget appropriately.
• Unlimited tests, so you can use it on every new vacancy without worrying about restrictions.
• Low (preferably no) integration fees; these can quickly rack up with the inevitable changes to your ATS configuration in future.

Chances are, you’ll want a test that you’ll use for years to come, so look at both the current cost and the future cost. Does the pricing model make sense as you grow?

3. Who will be using the test?

Are the results and interview reports going to be used by recruiters, hiring managers or both? If hiring managers will be using the information, then consider how easily the results will be understood by them. If it’s just recruiters, then think about how they cascade information to hiring managers and the potential pitfalls there.

It’s also worth thinking about how people will access the data. Will the test scores and data be accessed through the ATS or will people have to log into a new system? Reducing the friction to test use and usability is a major factor in ensuring your candidate testing gains adoption.

4. What are the strongest features of the candidate test?

Most tests are built around a vision of “improving hiring decisions”. While the right answer to this question will be unique to your business, there are a few features that every recruiter will benefit from such as:

• Great candidate experience
• Seamless ATS integration
• Easy to understand interview reports

The strongest features should closely align with the hiring goals and success measures you’ve already set, for example:

• [Goal] To reduce time to hire
• [Metric] To reduce phone screens by 40%
• [Desired Feature 1] A test that communicates the realities of life in the role so candidates can de-select themselves from the role
• [Desired Feature 2] A test that scores and ranks candidates within our ATS so that recruiters can prioritise the best talent

3rd party websites can be a god source of information to know what features each test provider can deliver. Lists like the top 11 recruitment assessment tools from Recruiters Lineup can be useful.

5. What reporting is available?

Reporting features are crucial for understanding and improving your hiring process over time.

Testing platforms should offer two types of reports; candidate reports that help your team make better hiring decisions, and performance reports that provide you insights on how your test is performing against your pre-defined goals.

• Candidate reports should be detailed, yet easy to digest. Rather than just showing an overall score, they should include data on the individual selection criteria you are measuring. Some testing tools, such as ThriveMap’s work simulation assessments can also auto-generate relevant interview questions for managers to ask.
• Performance reports should include data on how candidates are performing on the test, completion rates and how long it takes candidates to complete. You’ll want to partner with a company that can optimise your test over time and show you which questions are predicting success in your organisation.

Consider asking senior leadership what kind of hiring data matters most to them and ask test providers how they will measure their performance against these criteria.

6. What’s the candidate experience like?

The candidate testing market is in the midst of a wave of rapid innovation with new technologies, such as Artificial Intelligence, Virtual Reality, and Gamification all being lauded as the next big thing. However, it’s worth considering the entire candidate journey before deciding on a solution.

A candidate test needs to feel appropriate for the role being recruited and congruent with the rest of the recruitment experience. If you choose a gamified testing provider then it’s important to note that not everyone likes playing games and not everyone has access to the latest smartphones or fast internet connections. If some of your candidates apply offline then you may want to ask how the provider ensures they can complete the test too.

7. How is the test validated?

Validity and reliability are two critical measures when considering the quality of a candidate test.

• Validity is a measure of how well a test predicts success.
• Reliability is a measure of whether if people were to take the test again they would get the same result.

The most valid and reliable tests are ones that are customised to your specific job roles and cultural context. Questions that model what someone will actually be doing in a job and that reflect the behaviours and skills you require of candidates have more predictive validity than non-contextual tests that are modelled on psychometric theories. If this is important to you then you’ll need to decide if you want to buy an off the shelf solution or build a bespoke assessment to suit your needs.

You’ll want to find a test provider that clearly explains the validation process they go through for each test they create. How they test the control group and how they improve the predictive validity of the test over time.

8. What are the training and set up requirements?

It’s important to understand the practicalities of getting started with your new testing tool. Some products are complicated and require hours of consulting services and training just to get started. Often, new users have to sift through training materials or gain a qualification before they can start using them.

To use the test, will managers need to be trained? If so, how much time will this take? What effort is involved in setting this up? How long will it take to roll out the system and integrate with your ATS? All these are important questions that will shape how soon your test can be deployed.

To make implementation easy, look for a testing partner that includes:

• Comprehensive set up and consultancy, with an expert team to help create and validate the test
• Little (or no) product training, with an intuitive interface and straightforward roll-out process
• Native integration, with the tools you use every day – especially your ATS

There’s no need to settle for clunky, complicated business software. Today’s recruiting tools should look and feel much like a consumer app and that includes getting up-and-running in weeks, not months.

9. What are the litigation risks?

A robust hiring process should consider the possible risks of hiring discrimination and institutional bias. Although candidate tests use generally reduces these risks two important questions to ask when choosing a test are: “Are the questions relevant to the work to be done?”, and “Can I explain how the result was reached?”. If the answers are no, then you are at serious risk of litigation from disgruntled candidates.

There are many different candidate tests to choose from but broadly speaking they fall into two buckets; contextual assessments and non-contextual assessments.

• Contextual tests, which ask questions directly related to the job
• Non-contextual tests, which ask abstract questions – behavioural or situational – that are not directly related to the job

Non-contextual tests such as games or personality tests put companies at risk of unfair selection practices. Vendors that sell the benefits of using Artificial Intelligence to calculate scores or predict success are particularly problematic as you will need to be able to clearly explain to any candidate how a result is calculated or risk legal action being taken against you.

10. What support will I get?

Responsive customer support is important for all businesses, but it becomes even more important as you grow. Before you make a purchase, try to understand what customer support is provided.

You’ll want to find a candidate testing provider that offers:

• Candidate support. Your recruiters shouldn’t feel the brunt if the test goes down. Find a provider that offers candidate support too.
• Multi-channel support. Candidates will access your test across all devices and web browsers. A great partner will make it easy to access support wherever and whenever it’s required—via chat, email and phone if required.
• “Value-add” account management. Your candidate testing partner should be proactive, offering ongoing check-ins and performance reviews to help you get the most from your software.

Great customer support offers more than just a safety net in case problems crop up: it helps you get the most from your candidate tests.

This article is from our free Ebook: A buyers guide to candidate testing