Construct Validity: How to Measure It and Why It Matters

Imagine you’re about to shoot an arrow at a target. You’ve been boasting to your friends about how accurate a shot you are, and this is your opportunity to prove it to them. You shoot the arrow and it hits the centre of the target. You load up the next arrow, it hits the centre again. And the next, and the next, same result. You’ve just validated your claim to be an accurate archer. 

Now think of this analogy in terms of your job as a recruiter or hiring manager. The arrow is your assessment, and the target represents what you want to hire for. You claim that your assessment is accurate, but before you can continue, you have to prove it.

In order for an assessment, a questionnaire or any selection method to be effective, it needs to accurately measure the criteria that it claims to measure. This is broadly known as test validity. There are many strings to validity, so if you’re using assessments or tests of any kind, it’s important you understand the various types and how to know your assessment is valid. One key area which we’ll cover in this post is construct validity. We’ll explore how to measure construct validity to find out whether your assessment is accurate or not.

What is Construct Validity?

Construct validity is established by measuring a test’s ability to measure the attribute that it says it measures.  Sounds confusing? Ok, let’s break it down.

For example, let’s say you want to measure a candidate’s “interpersonal skills”. 

First, you have to ask whether or not the candidate really needs to have good interpersonal skills to be successful at this job. Conducting a thorough job analysis should have helped here – but if you’re yet to do a Job Analysis, our new job analysis tool can help.

Next, you need to measure the assessment’s construct validity by asking if this test is actually an accurate measure of a person’s interpersonal skills. 

That requires a shared definition of what you mean by “interpersonal skills,” as well as some sort of data or evidence that the assessment is hitting the desired target.

How to Measure Construct Validity

Determining whether your test has construct validity entails a series of steps:

Step 1: Define the term you are attempting to measure

Different people will have different understandings of the attribute you’re trying to assess. In the words of Professor William M.K. Trochim, an author and assistant professor at Cornell University, the construct (term) should be set within a “semantic net.” Simply put, the test provider and the employer should share a similar understanding of the term.  This is a massive grey area – and cause for much concern with generic tests – that’s why at ThriveMap we enable each company to define their own attributes.

Step 2: Provide evidence that your test controls what you are trying to measure

You need to be able to explain why you asked the questions you did to establish whether someone has evidenced the attribute. First, prove why your questions relate to the term you defined, then explain why you believe their answers demonstrate their abilities in that area. 

Let’s take the example we used earlier. If you are trying to measure the candidate’s interpersonal skills, you need to explain your definition of interpersonal skills and how the questions and possible responses control the outcome. One way to do this would be to create a double-blind study to compare the human assessment of interpersonal skills against a test’s assessment of the same attribute to validate its accuracy.

Step 3: Provide evidence that your test correlates with other similar tests (if you intend to use it outside of it’s original context)

If you intend to use your assessment outside of the context in which it was created, you’ll need to further validate its broader use. In this example, your definition of “interpersonal skills” is “how well the person can carry a conversation.” In order to prove that your test is valid in different contexts, you need to find other tests that also measure how well a person can “carry a conversation” and compare the results of the two tests. Analyze the trends in your data and see if they compare with those of the other test. If the data in two, or preferably multiple, tests correlate, your test is likely valid. 

It’s worth reiterating that step 3 is only required should you choose to develop a non-contextual assessment, which is not advised for recruitment.

Closing

As a recruitment professional, it is your responsibility to make sure that your pre-employment tests are accurate and effective. Ensuring construct validity in your assessment process is a key step in hiring the right candidates for your jobs. Don’t waste your time assessing your candidates with tests that don’t really matter; use tests that will give your organisation the best chance to succeed.

If you liked reading this post you may also like reading the following:

• Criterion Validity: How and Why To Measure It
• What is a Realistic Job Assessment and how does it work?
• Pre-Employment Test Validity vs Test Reliability
• Situational Judgement Test: How to Create Your Own
• Job analysis: The ultimate guide to job analysis

Want help building a realistic job assessment for your business? Request a demo and learn more about how ThriveMap can reduce hiring mistakes!