Category Archives: Assessments

How to Use Personality Tests for Employee Selection and Rejection

rejectedIn my experience, most users of personality tests aren’t aware of the practical distinction between using these tests to select candidates and using tests to reject them. In fact, selection and rejection are very different testing stratagems and, for both decision-making and legal reasons, users need to be quite clear on what those differences are.

Candidate selection is what most people associate with testing, but the reality is that tests do a better job of signaling who is unlikely to perform well in a job than they do of predicting who will likely succeed. Even one behavioral factor, by its presence or absence, might greatly increase the odds of job failure, but it rarely works the other way around. Complex roles are dependent upon so many diverse variables that no one test can accurately predict high-level performance on a sustained basis.

Using a test as a method of rejection means employing it in a frontline screening role where a cut-off score of some type determines who gets weeded out and who continues through the evaluation process. This is ideal for high-volume hiring situations where speed is essential. To some people, this seems terribly unfair or, at the very least, impersonal. Quite to the contrary, so long as the organization undertakes an objective, statistically based study of the job, preferably one in compliance with EEOC guidelines for such studies, it is very fair. Personality tests have not been found to produce an adverse impact on any protected group, so tests do not cull out candidates in any disproportionate way, and they provide an objective, as opposed to a subjective, means of differentiating job-performance potential.

The critical consideration for a rejection process to work is the quality of the job analysis. Sometimes less formal, but statistically sound methods work.  For example, accomplishing an indiscriminate group analysis of top and bottom performers in a job, but only where clear behavioral or trait differences exist between the two groups, can be an effective means of measuring candidate potential. Better still is an in-depth statistical study that generates at least one or more statistically significant correlations between the test measures and actual job performance outcomes. Such a study might show that sales revenues increase in unison with increases in assertiveness or initiative, in which case, candidates with more of those qualities will be more likely to generate higher sales.

Without rigorous job analysis, even ignoring possible legal concerns, the screening process could produce false positives, filtering out potentially higher performers in lieu of those with less likelihood to succeed. As implied above, when used in isolation, tests can only in the broadest way predict performance. In reality, no tool or process unilaterally has that type of predictive capability.

Alternatively, employing valid personality tests at the selection stage in conjunction with job benchmarks should enable users to objectively extract talents, motivational factors, as well as job-related behaviors that can only be inferred from interviews, yet with more precise calibration. One of the nuances of understanding personality is that the degree of various traits a candidate possesses is as crucial a consideration as the traits themselves! In some jobs, this gradation can be the one differentiating variable between top and middling performance.

At the selection stage, tests should also help validate and amplify what is learned from other tools and procedures, such as simulations, reference checks, résumés, and interviews. Effective assessment should leave no loose ends, so tests should either confirm insights from these sources or raise red flags where inconsistencies are evident.

Hiring is much like gambling: It’s always a game of odds. The issue is how to shift those odds more consistently in your favor. Tests can do that, but you have to be realistic in your expectations and know what you want testing to contribute to the overall assessment process. Even shifting those odds a couple points can have a tremendous accumulative effect on improving hiring success and job performance, and minimizing hiring and onboarding costs.

__________________________________________________________________________13259f4For more than forty years, Frank Gump has been helping corporations become more productive and profitable by helping management teams identify and hire top performers and manage them most effectively. Developed and refined through extensive experience in more than 1200 organizations in the United States, Canada, England, and Australia, ADGI’s Organizational Management System (OMS) is a finely calibrated, technologically advanced decision-making process offering the potential for enormous payback. Contact ADGI for more insight and connect with Frank on LinkedIn. Follow ADGI on Twitter @ADGIGroup. Like ADGI on Facebook and follow us on Google+.

Picking a Test that Works and Suits Your Needs, Part 2

Furthering the discussion regarding test validity and reliability, here is a final tip to assist you with your analysis.

Avoid getting hoodwinked

testing mrgAs a reminder, a test should be able to predict in a statistically significant way performance differences among people or some performance outcome. Validity is always a statistical determination and never a subjective one. What is called face validity is not validity in the true sense of the word, but is really more akin to Facebook Likes and Dislikes. You should be justifiably cautious of any test that makes a claim, such as “89% of those who received feedback said the results described them accurately,” particularly if no specific statistical data is also provided. A test is not valid simply because people like what it says about them.

Validity and reliability are expressed as correlation coefficients, which essentially mean the extent to which two things move in unison and which evidence a cause–and-effect relationship. For example, in the first two years of life, we would expect to see a high correlation between the weight and height of babies. Correlations express likelihood – the extent to which one variable likely influences something else. So, if a vendor tries to explain validity in some other way, for example as an accuracy percentage, there is simply no scientific basis for that. It’s baloney.

As noted above, in this era of big data, spin is becoming more prevalent, and you need to watch out for it. As an example, in measuring test reliability, the generally-accepted cutoff for a trait scale would be a .70 correlation. The higher the correlation, the greater the reliability, so .85 is a lot better. Tests have multiple scales, so if one falls slightly below .70, that does not nullify the value of the test or mean that it shouldn’t be used. It simply means that specific scale should be treated more cautiously. The spin angle is apparent today with several instruments that have numerous scales that fall well below the traditional cutoff. The reality is that the scales are weak and their value is questionable. One vendor in particular is using a white paper to rationalize many weak scales by claiming new and more subjective measures of reliability make the .70 threshold less meaningful. That’s obfuscation by complexity just to defend something that may be indefensible.  If you drill into their literature and see scales where r=.55 or something similar, understand that the scale is weak and a poor measure of whatever it’s attempting to identify.

Follow up?

There’s much more to understanding all the considerations of test construction and validation than what can be covered in the space of two blogs, but as they say, this is a start. Please email me at fgump@2oms.com with questions, or comment below. You can also reach us on Twitter at @ADGIGroup or on Facebook.

__________________________________________________________________________13259f4For more than forty years, Frank Gump has been helping corporations become more productive and profitable by helping management teams identify and hire top performers and manage them most effectively. Developed and refined through extensive experience in more than 1200 organizations in the United States, Canada, England, and Australia, ADGI’s Organizational Management System (OMS) is a finely calibrated, technologically advanced decision-making process offering the potential for enormous payback. Contact ADGI for more insight and connect with Frank on LinkedIn. Follow ADGI on Twitter @ADGIGroup. Like ADGI on Facebook and follow us on Google+.

Picking a Test that Works and Suits Your Needs, Part 1

test mrg Trying to navigate through test validity and reliability is a jungle! After reviewing a myriad of validation claims over the years, you begin to realize that truth is sometimes hard to find.

Here are a couple of tips to help you with any investigation you might want to do. Remember, the goal of any test is to add situationally-relevant insight, so if it doesn’t do that, you need to move on to something that will.

“Frankly, I’m shocked!”

Whereas claims from some test vendors are straightforward, others are disingenuous. Some vendors simply make claims with no supporting documentation, others publish weighty tomes with irrelevant content in the belief that people will associate truth with weight and technical complexity, and still others try to support their claims with nonsensical information. And now there’s a new twist: Some vendors are trying to reframe accepted measures of validity or reliability to make their instruments look better than they really are. Lipstick on a pig? Sure sounds like it…

Pick the right tool for what you want to do.

If you are going to use a behavioral assessment, you first need to make sure that you are selecting the right type of instrument for your needs. There are two types of tests to choose from: a normative design and an ipsative design. A normative test is intended for decision making, because it compares individuals to a work group or a defined population and allows individuals to be compared to one another. In other words, when you’re trying to determine whether or not a new hire fits your company culture, this is the best option In contrast, ipsative instruments are most appropriate for personal discovery or group-understanding applications where people are not compared with one another and decisions are unnecessary. Such tests are based upon self-referent measures of relative behaviors and strengths and don’t offer a meaningful basis for comparing people. Ipsative tests are primarily intended for coaches and trainers who are trying to identify the talents of their clients and teams.

Although some vendors of ipsative instruments point out the purposes and limitations of their test design, others don’t. Here’s where the spin comes in: At least one vendor goes so far as to claim that, because they have more than 10 scales, their results approximate those of a normative test, which then begs the question: Why not just use a normative test rather than a wannabe?

The bottom line is: Don’t get blinded by brand or fooled by spin. Find out which tools are appropriate for your applications and information needs.

Understand what to look for.

Validity and reliability in a business decision-making context are really very simple:

A test or instrument should measure what it claims to measure, which is called construct validity. For example, if a test measures social initiative and friendliness, does it accurately distinguish between those who more sociable and those who are not?

A test should show evidence that the scales have internal consistency and that repeated test results are consistent. This is reliability. If that test supposedly measuring social initiative shows different results over several administrations, then it’s really not measuring anything.

Finally, a test should be able to predict in a statistically significant way some performance outcome. This is criterion or predictive validity. If you are using a test to make more placement decisions, then more accurately predicting performance or some dimension of performance is the goal.

In the next post, you can learn how to avoid statistical data spins.

__________________________________________________________________________13259f4For more than forty years, Frank Gump has been helping corporations become more productive and profitable by helping management teams identify and hire top performers and manage them most effectively. Developed and refined through extensive experience in more than 1200 organizations in the United States, Canada, England, and Australia, ADGI’s Organizational Management System (OMS) is a finely calibrated, technologically advanced decision-making process offering the potential for enormous payback. Contact ADGI for more insight and connect with Frank on LinkedIn. Follow ADGI on Twitter @ADGIGroup. Like ADGI on Facebook and follow us on Google+.

How to Get Real Payback from Behavioral Assessment

measurementAre you using some sort of testing to assess candidate or employee behaviors? Recent studies indicate that between 20 and 33 percent of employers now use behavioral tests and diagnostics, so it’s worthwhile to ask, as Dr. Phil would say, “How’s that working for you?”

For more than 40 years we have been observing how companies use tests and assessments, and it’s our belief that most organizations really don’t know how well their assessment tools are working. Of course, this is true for a lot of what goes on in HR, because a lot of what goes on is, in fact, difficult to isolate and measure in it’s own right.

But measuring the operational and financial impact of testing is essentially just keeping score. What really matters more is how organizations implement and manage the assessments they use. In a series of subsequent articles, I will uncover for you what we believe are the six most critical reasons why behavioral testing fails to deliver and fails to meet what should be reasonable expectations in any organization. Address these, and testing can work for you!

Where are we going to go with this series? We’ll take a look at:

  • The importance of interpreting the validity and reliability of data
  • How test proliferation and commoditization lowers user expectations
  • Why a poor understanding and use of job analysis guts the assessment process
  • What both HR and operational managers need to know about workplace behavior
  • What test users should be able to do with technology and data analysis
  • How weak command and control can cause unnecessary problems and undermine the potential value of assessment

Stay with us and let us know your experiences.

__________________________________________________________________________13259f4For more than forty years, Frank Gump has been helping corporations become more productive and profitable by helping management teams identify and hire top performers and manage them most effectively. Developed and refined through extensive experience in more than 1200 organizations in the United States, Canada, England, and Australia, ADGI’s Organizational Management System (OMS) is a finely calibrated, technologically advanced decision-making process offering the potential for enormous payback. Contact ADGI for more insight and connect with Frank on LinkedIn. Follow ADGI on Twitter @ADGIGroup. Like ADGI on Facebook and follow us on Google+.