Don’t Be Fooled
A consumer’s guide to purchasing employment tests
By Carl Greenberg
In order TO ensure they are providing high-quality workers, many staffing firms augment their résumé review and employment interview process with various pre-employment tests. But with so many test vendors out there, how do you decide which tests are really going to help identify high-quality workers?
Often, companies choose testing vendors based on convenience, such as those that offer a one-stop-shop for all your testing needs.
Some make buying decisions based on the product’s sales sheet. But all test product sales sheets give the appearance that their test is a good one to use. They are designed to persuade you to buy the test and usually do not provide sufficient information to make an informed decision.
With this in mind, here are a few tips on how best to evaluate tests and other assessment products:
- Ask for a technical report that demonstrates the test’s validity. This document provides evidence that the test was developed in accordance with professional standards. Quality tests need to follow the standards set forth in professional practice documents by such organizations as the Society of Industrial and Organizational Psychology. If technical documentation does not exist, walk away from that vendor.
- Review the technical report for proof that the test predicts job performance. Even if a technical report exists, it may not provide evidence that people who score better on the test do better on the job. I have often come across technical reports that show how the test was constructed and some information on test result norms. But it is the information about the validity of the test that will tell you if using the test will result in selecting higher quality employees.
- Review the test itself. This serves multiple purposes. First, it provides you with a sense of the test’s face validity. That is, on the surface, the test appears to be relevant for the job. This is not evidence of the test’s validity and usefulness in selecting better workers, but it does affect the candidate’s experience with your pre-employment process. Moreover, carefully look at the test items themselves. Poorly written items are easy to detect, especially on hard skill tests. For example, if items have multiple and obvious incorrect choices, they may measure a candidate’s test-taking ability more than actual job skills.
- Understand what the test is really measuring. There are tests in the marketplace whose names suggest they measure particular competencies, but in fact do not. I’ve come across one such test labeled by the vendor as measuring customer service skill, but when looking at the test questions it became quite apparent that the test measures knowledge of customer service procedures. These are distinctly different concepts. Organizations can readily teach their new hires how to handle various customer situations, but have a more difficult time teaching people the interpersonal skills necessary to carry out the procedure effectively. Tests that measure difficult-to-train competencies are more useful than those that assess easily trainable competencies.
- Focus on the test’s performance outcomes. Carefully evaluate the appropriateness of the performance outcomes used to support the test’s validity. If the test purports to predict job performance, how does the test publisher define that? Is it based on supervisory ratings, efficiency measures or quantity?
- Don’t be wowed by technology. Today, many vendors are creating computerbased job simulations. Just because a test creates the appearance of what someone does on a job, it does not necessarily mean the test is accurately measuring critical competencies. Look beyond the superficial aspects of the test and make sure it is measuring what it says it is. Selection tests are a critical part of your service quality proposition. Not all tests are alike, and purchasing a lemon could cause considerable damage to your firm’s profitability and reputation. Taking care in choosing the right tests, though, can create a competitive advantage.
Carl Greenberg, Ph.D., is founder of Pragmatic HR Consulting. He has more than 30 years of individual and organizational measurement experience and has held executive positions in the consulting and staffing industries. He can be reached at firstname.lastname@example.org.