LLMs Form Novel Hiring Stereotypes in Study
An ICML 2026 study found that frontier LLMs developed occupation-linked stereotypes among fictional groups during repeated hiring decisions, despite equal success rates. The models' average stratification exceeded the human baseline, and OpenAI's o3 scored 1.83 on a scale capped at 2. The result is a controlled warning about feedback loops, not evidence from a deployed hiring system.
Sequential decisions are the central risk
For practitioners, this study is most relevant as a warning about feedback-driven decision workflows, rather than as a direct measurement of a deployed applicant-tracking system. Industry context: systems that repeatedly make allocations, receive outcome feedback, and use group-correlated information can amplify early random variation into persistent patterns. That makes one-shot prompt tests insufficient for evaluating tools used across repeated hiring, ranking, triage, or recommendation decisions.
MIT Technology Review reports that researchers at Princeton University and the University of Chicago adapted a hiring game previously used with human participants. They asked LLMs, including ChatGPT, Claude, and Gemini, to choose candidates for 20 occupations, such as doctor, lawyer, child-care aide, and janitor.
Each round presented four candidates, one from each fictional ethnic group: Tufa, Aima, Reku, and Weki. After each selection, the model received feedback on whether the candidate succeeded, then moved to a new opening. According to MIT Technology Review, every candidate was equally likely to succeed in every role, and the models were instructed to maximize successful hires over 40 rounds.
Models segregated groups by occupation
MIT Technology Review reports that the models began assigning candidates from different groups to different jobs based on early hiring outcomes. The publication describes an example in which a model, after learning that an Aima candidate failed as a doctor, avoided hiring Aima candidates as doctors and instead selected them for janitorial roles.
The study compared the model results with results from the prior human experiment using a segregation scale where 2 indicates complete confinement of each group to a distinct job niche. MIT Technology Review reports a human score of 0.84 and model scores roughly 65% higher; OpenAI's reasoning model o3 recorded 1.83, close to the maximum value.
Gizmodo quotes the researchers: "LLMs can spontaneously develop novel social biases about artificial demographic groups even when no inherent differences exist." The researchers added that the findings raise questions about how these systems could shape society over time.
Implications for evaluation
For practitioners, the experiment supports testing model behavior across multi-round simulations with controlled ground truth, not solely through static fairness benchmarks. Observed patterns in comparable sequential systems suggest that evaluation should measure whether outcome feedback changes selection rates, occupational assignment, or error rates across protected and proxy groups over time.
Industry context
the study used invented groups and a constrained game, so its reported scores do not establish the prevalence or magnitude of discrimination in real hiring deployments. Its value is narrower but technically important: it isolates a mechanism through which a model's decisions can produce group stereotypes without needing pre-existing differences among candidates.
Key Points
- 1Researchers found that LLMs formed occupation-linked stereotypes among fictional groups despite every candidate having an equal probability of success.
- 2For practitioners, sequential simulations can expose feedback-loop bias that single-prompt fairness tests may fail to detect.
- 3MIT Technology Review reports OpenAI's o3 scored 1.83 on a segregation scale where 2 indicates complete occupational separation.
Scoring Rationale
The study identifies a technically relevant failure mode for LLMs used in repeated selection or allocation workflows. Its simulated design limits direct conclusions about production hiring systems, but the results are important for fairness evaluation and governance practitioners.
Sources
Primary source and supporting public references used for this report.
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.
Try 250 free problems
