Bias creeps into reference checks, so is it time to ditch them?
2018/01/08 Leave a comment
Hadn’t thought of this aspect of bias in reference checks. When hiring in government, I was always conscious of the selection bias in the references submitted – people generally do not submit negative references! When asked if I was willing to be a reference, I would flag if I had any issues that I would have to include in the reference:
As much as we’d like to think we’ve refined the hiring process over the years to carefully select the best candidate for the job, bias still creeps in.
Candidates who come from privileged backgrounds are more able to source impressive, well-connected referrers and this perpetuates the cycle of privilege. While the referrer’s reputation and personal clout make up one aspect of the recommendation, what they actually say – the content – completes the picture.
Research shows gender bias even invades in the content of recommendations. In this study female applicants for post-doctoral research positions in the field of geoscience were only half as likely as their male counterparts to receive excellent (as opposed to just good) endorsements from their referees. Since it’s unlikely that of the 1,200 recommendation letters analysed, female candidates were less excellent than the male candidates, it means something else is going on.
A result like this may be explained by the gender role conforming adjectives that are used to describe female versus male applicants. Women are more likely to be observed and described as “nurturing” and “helpful”, whereas men are attributed with stronger, more competence-based words like “confident” and “ambitious”. This can, in turn, lead to stronger recommendations for male candidates.
Worryingly, in another study similar patterns emerged in the way black versus white, and female versus male, medical students were described in performance evaluations. These were used as input to select residents.
In both cases the members of minority groups were described using less impressive words (like “competent” versus “exceptional”), a pattern that was observed even after controlling for licensing examination scores, an objective measure of competence.
Recommendations aren’t good predictors of performance
Let’s put the concerns about bias aside for a moment while we examine an even bigger question: are recommendations actually helpful, valid indicators of future job performance or are they based on outdated traditions that we keep enforcing?
Even back in the 90s, researchers were trying to alert hiring managers to the ineffectiveness of this as a tool, noting some major problems.
The first problem is leniency, referees are allowed to be chosen by the candidate and tend to be overly positive. The second is too little knowledge of the applicant, as referees are unlikely to see all aspects of a prospective employees’ work and personal character.
Reliability is another problem. It turns out there is higher agreement between two letters written by the same referee for different candidates, than there is for two letters (written by two different referees) for the same candidate!
There is evidence that people behave in different ways when they are in different situations at work, which would reasonably lead to different recommendations from various referees. However, the fact that there is more consistency between what referees say about different candidates than between what different referees say about the same candidate remains a problem.
The alternatives to the referee
There are a few initiatives that are currently being used as alternatives to standard recruitment processes. One example is gamification – where candidates play spatial awareness or other job-relevant games to demonstrate their competence. For example, Deloitte has teamed up with software developer, Arctic Shores, for a fresh take on recruitment in an attempt to move away from the more traditional methods of recruitment.
However, gamification is not without its flaws – these methods would certainly favour individuals who are more experienced with certain kinds of video games, and gamers are more likely to be male. So it’s a bit of a catch-22 for recruiters who are introducing bias through a process designed to try to eliminate bias.
If companies are serious about overcoming potential bias in recruitment and selection processes, they should consider addressing gender, racial, economic and other forms of inequality. One way to do this is through broadening the recruitment pool by making sure the language they use in position descriptions and jobs ads is more inclusive. Employers can indicate flexible work options are available and make the decision to choose the minority candidates when they are equally qualified as other candidates.
Another option is to increase the diversity of the selection committee to add some new perspectives to previously homogeneous committees. Diverse selectors are more likely to speak up about and consider the importance of hiring more diverse candidates.
Job seekers could even try running a letter of reference through software, such as Textio, that reports gender bias in pieces of text and provides gender-neutral alternatives. But just as crucial is the need for human resources departments to start looking for more accurate mechanisms to evaluate candidates’ competencies.
via Bias creeps into reference checks, so is it time to ditch them?
Multicultural Meanderings 