How Do You Prove Validity?
How Do You Prove Validity?
(Can be sung to the tune “How do you solve a problem like Maria?” from The Sound of Music)
Validity is a pretty dry topic but it is consistently the subject of the most frequently asked questions about the use of Prevue Assessments. It has therefore earned feature article status. If you can persevere through to the final part of this article you will find the offer of a reward for your dedication.
Validity questions generally break down into the following three topics:
- What is validity?
- Why is it important?
- How is it measured?
What is Validity? (this is the easy part)
In the simplest terms validity confirms an assessment tool or process “measures what it purports to measure”, “is job relevant” and “does assist in making more effective decisions”.
Keep in mind that the reference to assessment tools or processes doesn’t just refer to Prevue Assessments. The need to confirm validity applies to any and all tools and procedures that are used in assessing individuals including the collection of information from job applicants, content of screening and interview questions, any kind of testing used in the hiring process and even background and reference checks.
Why is it Important?
There are two basic reasons why employers should be concerned about validity:
- First you don’t want to be wasting your time and money using assessment tools or processes that are not effective. Not only do you want to know they are effective, you should also want to determine their degree of effectiveness – in other words, your return on investment (ROI) on the time and money invested in those tool and processes. Proving validity and ROI are virtually the same process.
- Second (first for some) the law requires that you be able to demonstrate the assessment tools or processes used in the hiring process are valid. In the USA this is contained in the Civil Rights Act, EEOC Guidelines and host of other federal, state and local laws. In Canada, Australia, the UK and other common law jurisdictions the requirement is contained in a wide range of human rights and employment legislation at all levels of government.
How is it Measured? (this is the heavy going part)
Evidence of Validity should be available from the following two sources:
- First, proof of validity should be provided by the supplier of any assessment tool or process you utilize. For the Prevue Assessments this is comprised of the Prevue Technical Manual together with the Prevue Technical Bulletin, “Measures of Reliability and Validity”.
- Second is confirmation of validity through validation studies conducted by the user of any assessment tool or process, sometimes referred to as internal validity. This is what the law expects the user to demonstrate in the course of responding to a complaint filed under any of the legislation noted above or in civil proceedings that involve employment issues.
The specific degree or level of validity is measured by validity coefficients reported as a number between 0 and 1.00. The US Dept. of Labor’s guide on Testing and Assessment confirms that validity coefficients of .21 to .35 are typical of most assessment tools with scores over .35 being very beneficial. You can see in the above-noted Technical Bulletin that the validity coefficients for the Prevue Assessments generally exceed .35.
Validity and ROI are confirmed by conducting one or more of the following types of validation studies:
- Criterion-related validation to demonstrate the relationship between assessment performance and job performance that confirms either concurrent validity or predictive validity;
- Content-related validation to confirm the assessment is relevant to and measures requirements or qualifications for the job; and
- Construct-related validation to exhibit the assessment does measure the constructs it purports to measure and those constructs are important to job performance.
You Can Do It – We Can Help (finally, the reward part – more work for all of us)
The Prevue Technical Manual confirms the Prevue Assessments have been validated with all three types of studies outlined above. The obligation on employers to confirm Internal Validity we know can be both time consuming and expensive. Prevue has therefore developed the “Prevue Internal Validation/ROI Program” to assist Prevue users to confirm the internal validity and/or the ROI of their use of Prevue Assessments. You can obtain an outline of the Prevue Internal Validation/ROI Program from your authorized Prevue Distributor or by email request to [email protected] .
For those of you masochists who may be interested in finding out more about this subject we would refer you to chapter 3 of the “Employers Guide to Testing and Assessment” published by the US department of Labor, Employment and Training Administration, atwww.onetcenter.org/dl_files/empTestAsse.pdf .
Written By: Ken Danderer