Evaluation of LLMs Biases Towards Elite Universities: A Persona-Based Exploration

Elite universities are a dream destination for not just students but also top employers who get a supply of amazing talents. When we hear about top universities, the first thing that comes to mind is their academic rigor, prestigious reputation, and highly successful alumni. However, society at large is not just represented by a few elite universities, but several others. We have seen several examples where many, even without formal education, built big businesses. There are various instances in which several people, however talented, couldn't make it to top elite universities because of several resource constraints. For recruitment of candidates, we do see candidates from a few elite universities well represented in top technology companies. However, we found during our study that LLMs go overboard in representing that. This study investigates whether popular LLMs exhibit bias towards elite universities when generating personas for technology industry professionals. We employed a novel persona-based approach to compare the educational background predictions of GPT-3.5, Gemini, and Claude 3 Sonnet with actual data from LinkedIn. The study focused on various roles at Microsoft, Meta, and Google, including VP Product, Director of Engineering, and Software Engineer. We generated 432 personas across the three LLMs and analyzed the frequency of elite universities (Stanford, MIT, UC Berkeley, and Harvard) in these personas compared to LinkedIn data. Results showed that LLMs significantly overrepresented elite universities, with 72.45% of generated personas featuring these institutions, compared to only 8.56% in the actual LinkedIn data. ChatGPT 3.5 exhibited the highest bias, followed by Claude Sonnet 3, while Gemini performed best. This research highlights the need to address educational bias in LLMs and suggests strategies for mitigating such biases in AI-driven recruitment processes.