Blog

Using AI to detect and reduce bias in hiring processes

Unconscious bias can put applicants at a disadvantage and exclude suitable people. Diverse workforces are 35% more likely to achieve better business results. AI can support fairer processes if data, assessments and outcomes are reviewed regularly.

The problem:

In studies, identical CVs received callback rates that differed by up to 50% depending on the ethnic origin or gender attributed to the applicant. Bias can occur at any point from candidate sourcing to the final selection.

Understanding bias in recruiting

Unconscious bias shows up in different patterns. Those who recognise these patterns can develop specific procedures to reduce bias.

Affinity bias

During selection, preference is given to candidates whose background, interests or communication style resemble one’s own.

Confirmation bias

Information that confirms the first impression is given more weight. Contradictory evidence, by contrast, is passed over.

Halo effect

A single positive trait, such as a prestigious education, shapes the entire assessment. Other important factors are given too little weight as a result.

Vague ideas of cultural fit

Similarity is mistaken for "fit". People with different ways of thinking or a different background are considered less often as a result.

How AI can support checks for bias

AI can reveal signs of bias that are overlooked in manual review. There are three areas of use; their value depends on the data, the criteria and ongoing monitoring.

Anonymised CV screening

In anonymised pre-screening, the AI removes information that allows conclusions to be drawn about a person before a human reviews the application. This includes names, addresses, graduation years and other demographic details.

Impact:

Anonymised pre-screening leads to 30% more diverse applicant groups at the interview stage and improves quality metrics by 25%.

Structured assessment criteria

Defined criteria are intended to ensure that all applicants are assessed by the same standard. To achieve this, weightings and standards must be documented in advance and changes during the process must be controlled.

  • Weight and assess skills against the job requirements.
  • Check experience for its relevance to the tasks instead of merely counting years of service.
  • Evaluate work samples against defined criteria.

Checking language for bias

AI can check job descriptions and interview feedback for biased language. Such wording can deter applicants from different backgrounds or point to bias in the assessment.

Problematic wording:

  • "Aggressive" or "dominant"
  • "Cultural fit" without a definition
  • "Native speaker"
  • "Entry-level candidate"

Better alternatives:

  • "Results-oriented" or "strategic"
  • "Values alignment" with specifics
  • "Excellent communication skills"
  • "3+ years of experience"

The challenge of AI training data

AI can adopt bias from its training data. Historical hiring decisions are therefore not suitable as a benchmark for future recommendations without review.

The Amazon example:

Amazon abandoned a recruiting tool after it was found to discriminate against women. The system had been trained on CVs from a 10-year period, most of which came from men. It rated terms such as “women’s” negatively, for example in “captain of the women’s chess club”.

Historical decisions must be checked for bias before they serve as a basis for training.

Checking AI processes for bias

Review your training data

Before using AI, check historical hiring data for recurring bias. Remove biased data entries or adjust their weighting.

Ask: Who was hired? Who was rejected? Were there demographic patterns in the outcomes?

Regularly check recommendations for bias

Check the AI recommendations for disparate impact every month. The 80% rule from the EEOC context serves as the benchmark.

If the selection rate of one group is below 80% of the rate of another group, the cause should be investigated.

Different perspectives in the development team

Different perspectives in the development team help to identify possible bias when designing and testing the systems.

Explainable AI

Recommendations should include comprehensible reasons so that those responsible can review the grounds for selection and rejection.

Human oversight

Trained decision-makers should review the recommendations and make the final hiring decision themselves.

Measuring success

Use these metrics to review the measures:

Diversity of applicant groups

Record the composition of the applicant groups at every stage of hiring.

Disparate impact

Compare the selection rates of the different groups with one another.

Quality of hire

Compare the job performance of different groups after hiring.

The business case

Studies show the following results for companies with above-average diversity:

  • 19% higher innovation revenue (BCG)
  • 35% higher financial returns (McKinsey)
  • 70% greater chance of capturing new markets (Harvard)
  • 2.3x higher cash flow per employee over 3 years (Josh Bersin)

Review AI recommendations continuously

AI is not automatically free of bias. It requires clear criteria, human oversight and ongoing reviews. Consistent procedures should be combined with professional judgement and different perspectives so that more suitable applicants get a fair chance.

Topics: Diversity, equity and inclusion (DEI) · AI ethics · Legal requirements

About the author

Nico Dorn

Nico Dorn is CEO and CTO of Talenty.ai. He holds a degree in computer science and is responsible for technical development and the integration of AI features.

Discuss your selection process

Discuss which criteria are relevant to your vacancies and how your team can review assessments and possible bias.

Book a demo
Book a demo