Construct Validity: How to Measure It and Why It Matters

5 minute read

Posted by Chris Platts on 22 April 2021

Imagine you’re about to shoot an arrow at a target. You’ve been boasting to your friends about how accurate a shot you are, and this is your opportunity to prove it to them. You shoot the arrow and it hits the centre of the target. You load up the next arrow, it hits the centre again. And the next, and the next, same result. You’ve just validated your claim to be an accurate archer. 

Now think of this analogy in terms of your job as a recruiter or hiring manager. The arrow is your assessment, and the target represents what you want to hire for. You claim that your assessment is accurate, but before you can continue, you have to prove it.

In order for an assessment, a questionnaire or any selection method to be effective, it needs to accurately measure the criteria that it claims to measure. This is broadly known as test validity. There are many strings to validity, so if you’re using assessments or tests of any kind, it’s important you understand the various types and how to know your assessment is valid. One key area which we’ll cover in this post is construct validity. We’ll explore how to measure construct validity to find out whether your assessment is accurate or not.

What is Construct Validity?

Construct validity is established by measuring a test’s ability to measure the attribute that it says it measures.  Sounds confusing? Ok, let’s break it down.

For example, let’s say you want to measure a candidate’s “interpersonal skills”. 

First, you have to ask whether or not the candidate really needs to have good interpersonal skills to be successful at this job. Conducting a thorough job analysis should have helped here – but if you’re yet to do a Job Analysis, our new job analysis tool can help.

Next, you need to measure the assessment’s construct validity by asking if this test is actually an accurate measure of a person’s interpersonal skills. 

That requires a shared definition of what you mean by “interpersonal skills,” as well as some sort of data or evidence that the assessment is hitting the desired target.

How to Measure Construct Validity

Determining whether your test has construct validity entails a series of steps:

Step 1: Define the term you are attempting to measure

Different people will have different understandings of the attribute you’re trying to assess. In the words of Professor William M.K. Trochim, an author and assistant professor at Cornell University, the construct (term) should be set within a “semantic net.” Simply put, the test provider and the employer should share a similar understanding of the term.  This is a massive grey area – and cause for much concern with generic tests – that’s why at ThriveMap we enable each company to define their own attributes.

Step 2: Provide evidence that your test controls what you are trying to measure

You need to be able to explain why you asked the questions you did to establish whether someone has evidenced the attribute. First, prove why your questions relate to the term you defined, then explain why you believe their answers demonstrate their abilities in that area. 

Let’s take the example we used earlier. If you are trying to measure the candidate’s interpersonal skills, you need to explain your definition of interpersonal skills and how the questions and possible responses control the outcome. One way to do this would be to create a double-blind study to compare the human assessment of interpersonal skills against a test’s assessment of the same attribute to validate its accuracy.

Step 3: Provide evidence that your test correlates with other similar tests (if you intend to use it outside of it’s original context)

If you intend to use your assessment outside of the context in which it was created, you’ll need to further validate its broader use. In this example, your definition of “interpersonal skills” is “how well the person can carry a conversation.” In order to prove that your test is valid in different contexts, you need to find other tests that also measure how well a person can “carry a conversation” and compare the results of the two tests. Analyze the trends in your data and see if they compare with those of the other test. If the data in two, or preferably multiple, tests correlate, your test is likely valid. 

It’s worth reiterating that step 3 is only required should you choose to develop a non-contextual assessment, which is not advised for recruitment.

Closing

As a recruitment professional, it is your responsibility to make sure that your pre-employment tests are accurate and effective. Ensuring construct validity in your assessment process is a key step in hiring the right candidates for your jobs. Don’t waste your time assessing your candidates with tests that don’t really matter; use tests that will give your organisation the best chance to succeed.

If you liked reading this post you may also like reading the following:

Want help building a realistic job assessment for your business? Request a demo and learn more about how ThriveMap can reduce hiring mistakes!

Share

The ThriveMap Newsroom

Subscribe for insights, debunks and what amounts to a free, up-to-date recruitment toolkit.

About ThriveMap

ThriveMap creates customised assessments for high volume roles, which take candidates through an online “day in the life” experience of work in your company. Our assessments have been proven to reduce staff turnover, reduce time to hire, and improve quality of hire.

Not sure what type of assessment is right for your business? Read our guide.

Other articles you might be interested in

Banner image for this post

How to tackle attrition in hiring

Attrition is often the silent drain on recruitment efforts, impacting businesses in ways that are too costly to ignore. From inflated hiring budgets to declining team morale, productivity, and customer satisfaction, the effects of attrition ripple across the organisation. Yet, it’s rarely tackled head-on. The reality? Attrition isn’t just a recruitment challenge; it’s a business-wide […]

Continue reading
Banner image for this post

ThriveMap Wins Best Candidate Assessment Tool For Berkeley’s Diversity Results

ThriveMap’s innovative Realistic Job Assessments (RJAs) have earned top honors at the In-house Recruitment Awards, winning the Best Candidate Assessment category. The award recognises ThriveMap’s pivotal role in transforming Berkeley Group’s apprenticeship recruitment strategy, resulting in groundbreaking diversity outcomes for the construction sector. Revolutionising recruitment in construction The construction industry faces a critical skills shortage […]

Continue reading
Banner image for this post

Mitie’s new approach to prison recruitment: Solving prison service attrition challenges with Realistic Job Assessments

The UK prison service is grappling with persistent recruitment and retention issues that undermine workforce stability and operational efficiency. Attrition rates reached 13.4% as of June 2023, highlighting the urgent need for innovative hiring practices. The expectation gap challenge The Efficiency Spotlight Report from the Criminal Justice Joint Inspection highlights critical challenges in the prison […]

Continue reading

View all articles