Finding The Right Match Can Be Tricky: The Effectiveness of Various Candidate Selection Methods May Shock You!

Key Points:

Selection tools are quite different in their ability to predict performance than previously thought. Some less, some more.
Overall (rather than contextualized) personality measures and years of job experience, are not meaningful predictors of performance.
Structured Interviews, Job Knowledge, Biodata, Work Samples, Cognitive Ability and Integrity Tests are useful assessment methods and criteria.

Hiring the right people is crucial to the success of your business. This is becoming more and more challenging, however, as we are seeing changes in the labor market. As organizational structures and job duties become more complex, the jobs of recruiters and hiring managers do too.

On average, recruiters look at 250 applications for each corporate position. In addition, a 2021 Recruiter Nation report shows that 21% of recruiters feel overwhelmed with the number of applications they need to review. They also understand the importance of effective processes, with 48% focusing on improving the quality of their hires. This is because securing the right talent can improve retention, productivity, and your organizational culture. Although hiring practices like resumes and interviews have been around for centuries, researchers like Jennifer Platt argue that it’s time to refine our processes to better serve both employers and employees.

Paul Sackett and Charlene Zhang (2022) conducted a review to understand whether selection procedures accurately predict job performance. That is the ultimate goal when selecting from candidates, right? We want to rely on these tools to help us hire the best people for the role based on their knowledge, skills, abilities, and fit. The authors collected meta-analyses, which are papers using statistics to synthesize the results of multiple individual studies, related to this topic. Sackett and his team then synthesized the findings of over 1000 studies, providing a complete overview of the effectiveness of over 20 common selection tools.

Their work builds on previous work (published in 1998) by Schmidt and Hunter, although we know how much can change in 25 years. The goal was to consider a variety of positions in many industries to get a general picture of how to effectively screen candidates in the modern labor market. The ranking of the selection primarily aligns with those from the late nineties despite using new methods and adding many more research studies to their review. The main difference is that these authors found that almost all tools were less able to predict future performance than we may have believed. So is that it, is it all doom and gloom and we have no way of predicting performance? Well, not quite. There are still quite a few common approaches to selection that are useful.

Sackett and Zhang’s Top 5 Selection Tools!

Structured Interviews

Structured interviews, a technique where candidates are assessed using the same interview questions asked in the same way, via the same order, are the most predictive of future job performance. A recent survey by Breezy HR found that structured interviews were one of the most common application practices, but only 67% of companies report using them. As we all know most application processes overwhelmingly include an interview however, it is important we rely on structured processes rather than unstructured ones.

Keep in mind that unstructured interviews have long been used by hiring managers due to their flexibility and candidness, however, they are less than half as predictive as structured interviews and can put companies at risk of being biased in their decisions.

Job Knowledge

Job knowledge is a close second for predicting future performance which indicates that candidates with a stronger understanding of the responsibilities and skills related to a specific role are more likely to perform well. By selecting candidates with job knowledge, we can help ensure our candidates are aware of role responsibilities and we can be sure that their relevant skills can make them a stronger employee who we can train to be the perfect fit.

Biodata

Empirically selected biodata is suggested to be another strong factor for consideration. This includes examples of past events and behaviors to showcase their experiences, personal attributes, interests, and abilities. Though single pieces of information are less useful, using this information to look at patterns of the candidates’ behavior can help us predict whether candidates are the right choice.

Work Samples and Assessment Centers

Work samples and assessment centers are both valuable tools for predicting job performance. While work samples may not be as potent as some other methods, they still offer significant insights. Employees can exhibit their skills in a concrete and tangible manner through work samples. For instance, an artist’s portfolio or a professor’s display of high cognitive abilities directly showcases the fundamental skills pertinent to their roles. On the other hand, assessment centers utilize simulations, interviews, and various tests to evaluate behaviors and capabilities, providing a comprehensive understanding of candidates’ competencies.

Personal Characteristics and Intelligences

Personality traits, integrity tests, cognitive ability, and emotional intelligence can be crucial factors in gauging potential job performance. The study’s authors highlight that cognitive ability and integrity tests, which assess an applicant’s honesty, trustworthiness, and reliability, demonstrate a comparable predictive capacity to work samples. These attributes are pertinent in determining how well an individual might perform in a given role. Additionally, contextually defined personality traits can be useful as well. This distinction was found to be important as it’s important to capture people’s typical personality at work (known as contextually defined personality) rather than their personality in general (also known as overall personality). Cognitive ability, contextually defined personality traits, emotional intelligence, and integrity are all predictive of a candidate’s overall suitability for a position when properly incorporated into effective selection systems. Evaluating these traits can enhance the accuracy of hiring decisions and lead to more successful outcomes in the long run. It’s interesting to note, here, that despite the controversy that surrounds emotional intelligence, that personality-based measures of emotional intelligence seem to be a slightly better predictor than conscientiousness – formerly among the top contenders in the prediction of performance.

Situational Judgement Tests

The last one on the list is situational judgment tests. Situational judgment tests are often included in structured interviews as questions asking what a candidate would do in a given situation to assess candidates’ knowledge and behaviors; however they also have use cases outside of interviews. Sackett et. al. (2022) found that situational judgment tests are useful in assessing candidates’ knowledge and behavioral tendencies, emotional intelligence and vocational tendencies.

What About Others?

The study did not have enough information to determine the effectiveness of other common factors like years of education, peer ratings, and reference checks. But most importantly the study found that the overall personality (personality in a general context) rather than contextualized personality measures (measures of people’s traits specific to the context in which they work) , as well as years of job experience, are not likely to be meaningful predictors of performance although they have benefits in some circumstances, it was found that selection methods like unstructured interviews, overall personality traits of conscientiousness, agreeableness and openness at work were unlikely to be predictive in all circumstances. However, most informed evidence-based practitioners would recommend against using a single personality traits in the selection of virtually all candidates. Context matters. It’s much better to use personality traits that are contextualized in terms of how people behave specifically in a work context and that are justified for use in hiring someone for a specific role in question. This means that traits should be specifically justified on the basis of a job analysis that justifies their inclusion in your specific selection context (for example, for a data analyst position).

What are the take-aways for you and your practice?

Effective recruitment is pivotal in today’s evolving labor market with its complex structures. Hiring effectively requires you treat the recruitment and selection process as professionally as you hope candidates would treat the job if they were hired.
Despite reduced predictability in many tools, structured interviews (but not unstructured interviews) and job knowledge assessments stand out.
Biodata and work samples offer valuable insights for gauging candidates’ potential.
Cognitive ability, integrity, and contextually defined personality traits enhance suitability predictions as well. Practitioners should contextualize personality traits based on role-specific justifications for improved selection outcomes.
Situational judgement tests may contribute to evaluating knowledge, behaviors, and emotional intelligence in an applied context.
Conventional approaches like unstructured interviews and overall personality traits are less predictive.

The contemporary recruitment landscape demands an astute understanding of selection tools and strategies. Sackett and Zhang’s comprehensive assessment shines a light on the evolving efficacy of these tools. While some well-established methods exhibit reduced predictive power, the study offers clear recommendations. Structured interviews, job knowledge evaluations, biodata analysis, work samples, cognitive ability, integrity assessments, and situational judgement tests emerge as reliable indicators of performance potential. By aligning with these evidence-based findings, organizations can better navigate the complexities of modern recruitment, leading to more precise and successful hiring outcomes in an ever-changing job market.

Trustworthiness score:

This meta-analysis was actually a meta-meta-analysis (also described as a meta-analysis of meta-analyses). Therefore, it’s likely to be a highly rigorous design. Therefore the trustworthiness of the study is at minimum moderate (80%). This means there is a 20% chance that alternative explanations for the effect found are possible.

Learn how we critically appraise studies to assign them a Trustworthiness Score.

We aim to provide you only the best available scientific evidence to inform your decisions.

Did you like this evidence summary? Share it with your network by clicking on the buttons below!

Follow us on LinkedIn, X (formerly Twitter), and Instagram, and subscribe to our newsletter to receive the most trustworthy scientific research summarized and practical recommendations for work and life in a wide variety of digestible content pieces!

References

Source: Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040–2068. https://doi.org/10.1037/apl0000994

Reference: Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274. https://doi.org/10.1037/0033-2909.124.2.262