Construct Validity: How to Measure It and Why It Matters

5 minute read

Posted by Chris Platts on 22 April 2021

Imagine you’re about to shoot an arrow at a target. You’ve been boasting to your friends about how accurate a shot you are, and this is your opportunity to prove it to them. You shoot the arrow and it hits the centre of the target. You load up the next arrow, it hits the centre again. And the next, and the next, same result. You’ve just validated your claim to be an accurate archer. 

Now think of this analogy in terms of your job as a recruiter or hiring manager. The arrow is your assessment, and the target represents what you want to hire for. You claim that your assessment is accurate, but before you can continue, you have to prove it.

In order for an assessment, a questionnaire or any selection method to be effective, it needs to accurately measure the criteria that it claims to measure. This is broadly known as test validity. There are many strings to validity, so if you’re using assessments or tests of any kind, it’s important you understand the various types and how to know your assessment is valid. One key area which we’ll cover in this post is construct validity. We’ll explore how to measure construct validity to find out whether your assessment is accurate or not.

What is Construct Validity?

Construct validity is established by measuring a test’s ability to measure the attribute that it says it measures.  Sounds confusing? Ok, let’s break it down.

For example, let’s say you want to measure a candidate’s “interpersonal skills”. 

First, you have to ask whether or not the candidate really needs to have good interpersonal skills to be successful at this job. Conducting a thorough job analysis should have helped here – but if you’re yet to do a Job Analysis, our new job analysis tool can help.

Next, you need to measure the assessment’s construct validity by asking if this test is actually an accurate measure of a person’s interpersonal skills. 

That requires a shared definition of what you mean by “interpersonal skills,” as well as some sort of data or evidence that the assessment is hitting the desired target.

How to Measure Construct Validity

Determining whether your test has construct validity entails a series of steps:

Step 1: Define the term you are attempting to measure

Different people will have different understandings of the attribute you’re trying to assess. In the words of Professor William M.K. Trochim, an author and assistant professor at Cornell University, the construct (term) should be set within a “semantic net.” Simply put, the test provider and the employer should share a similar understanding of the term.  This is a massive grey area – and cause for much concern with generic tests – that’s why at ThriveMap we enable each company to define their own attributes.

Step 2: Provide evidence that your test controls what you are trying to measure

You need to be able to explain why you asked the questions you did to establish whether someone has evidenced the attribute. First, prove why your questions relate to the term you defined, then explain why you believe their answers demonstrate their abilities in that area. 

Let’s take the example we used earlier. If you are trying to measure the candidate’s interpersonal skills, you need to explain your definition of interpersonal skills and how the questions and possible responses control the outcome. One way to do this would be to create a double-blind study to compare the human assessment of interpersonal skills against a test’s assessment of the same attribute to validate its accuracy.

Step 3: Provide evidence that your test correlates with other similar tests (if you intend to use it outside of it’s original context)

If you intend to use your assessment outside of the context in which it was created, you’ll need to further validate its broader use. In this example, your definition of “interpersonal skills” is “how well the person can carry a conversation.” In order to prove that your test is valid in different contexts, you need to find other tests that also measure how well a person can “carry a conversation” and compare the results of the two tests. Analyze the trends in your data and see if they compare with those of the other test. If the data in two, or preferably multiple, tests correlate, your test is likely valid. 

It’s worth reiterating that step 3 is only required should you choose to develop a non-contextual assessment, which is not advised for recruitment.

Closing

As a recruitment professional, it is your responsibility to make sure that your pre-employment tests are accurate and effective. Ensuring construct validity in your assessment process is a key step in hiring the right candidates for your jobs. Don’t waste your time assessing your candidates with tests that don’t really matter; use tests that will give your organisation the best chance to succeed.

If you liked reading this post you may also like reading the following:

Want help building a realistic job assessment for your business? Request a demo and learn more about how ThriveMap can reduce hiring mistakes!

Share

The ThriveMap Newsroom

Subscribe for insights, debunks and what amounts to a free, up-to-date recruitment toolkit.

About ThriveMap

ThriveMap creates customised assessments for high volume roles, which take candidates through an online “day in the life” experience of work in your company. Our assessments have been proven to reduce staff turnover, reduce time to hire, and improve quality of hire.

Not sure what type of assessment is right for your business? Read our guide.

Other articles you might be interested in

Banner image for this post

Best recruitment methods for call centres

Call centers are the backbone of customer service for many businesses, handling a myriad of enquiries and concerns daily. However, the traditional recruitment methods used by most call centers often fall short, leading to challenges such as high attrition rates and mismatched skill sets among employees. Pitfall of current call centre recruitment methods At the […]

Continue reading
Banner image for this post

Navigating recruitment technology compliance: Key considerations and how to future-proof your investments

Recruitment technology is rapidly evolving and regulators across the world are playing catch up to ensure candidates are getting a fair experience. This article discusses the key considerations that you, as a recruitment leader, must consider before investing in new tools, to both future-proof from impending regulations and to uphold equality, inclusion, and diversity in […]

Continue reading
Banner image for this post

Skills-Based Hiring: What You Need to Know

In the rapidly evolving job market, skills-based hiring is becoming increasingly significant, heralding what some call the “death of the degree.” With technology advancing at breakneck speed, employees find themselves in a constant cycle of acquiring new skills to stay relevant. This shift has led many job seekers and employers alike to prioritize skills and […]

Continue reading

View all articles