You know the process. You update your resume, apply for the job, pray the algorithm is on your side, then breathe and repeat. Finding a job has become an exhausting cycle that increasingly shuts out human interaction. Today’s market relies on machine learning to identify “the right” candidates long before a human even reads your name. And now, a new Stanford University study has found that these AI-powered screening tools aren’t just the bane of every applicant’s existence; they are invisibly taking discrimination to the next level.
For Black job seekers, one rejection used to mean one company said no. Discrimination concerns often started with wondering if your name sounded “too ethnic” or your zip code was a bit too urban. Before the arrival of artificial intelligence, rejection was largely at an individual level. Now more than 90 percent of companies globally use some form of AI for initial candidate screening, according to the World Economic Forum.
What Stanford Researchers Found
Stanford University Human-Centered Artificial Intelligence (HAI) recently published a study that it called “the first large-scale study of hiring algorithms in the wild.” Their goal was to test whether an “algorithmic monoculture” — which results when different organizations rely on the same algorithms for decision-making — limits job opportunities. What they found uncovered patterns in hiring tools’ algorithms that are spreading racial bias and doing so at scale.
Stanford researchers analyzed more than 4 million job applications from 3.4 million people across 156 employers, and 1,700 job postings, using the same AI hiring vendor. They discovered that AI tools essentially locked out nearly 26 percent of Black candidates, trapping them in a loop of systemic rejection.
The researchers also saw negative effects for Asian candidates and explained what this rejection means. “If the AI had recommended Black and Asian candidates at the same rate as it recommended the most-favored group (typically white applicants), 40,000 more of their applications would have advanced to the next stage of hiring,” they wrote.
They also found that the algorithm was less likely to recommend Black candidates for jobs in sectors like finance but more likely to recommend them for jobs like warehouse jobs. And the tricky thing, according to researchers, is that both patterns “cancel each other out,” making it “seem like there is no discrimination.” The researchers tied the patterns to market concentration, noting that when industries rely on a single vendor, “it may be more likely that candidates are shut out.”
Why These Systems Can Produce Biased Results
The Stanford paper does not specify which inputs caused the disparities. However, a 2024 American Bar Association paper explained that when AI training data overrepresents or underrepresents certain groups, it can lead to biased outcomes, such as AI hiring tools rejecting qualified candidates due to mislabeled data or existing inequalities.
The researchers identified three characteristics that should never coexist in systems used to make important decisions, but often do in AI-powered screening tools: widespread use, high-stakes consequences and a lack of public transparency. While research is beginning to shed light on the effects of AI hiring tools, the authors note that many of the impacts remain largely unknown. They argue that independent research into algorithm-based hiring is essential for developing evidence-based AI policy and improving accountability.
