Measuring Linguistic Bias in LLMs Across I.N.P.S. (Italian, Neapolitan, Parmesan and Sicilian) Dialect Corpora

Published in 4th International Conference on Frontiers of Artificial Intelligence, Ethics, and Multidisciplinary Applications (FAIEMA 2026), 2026

Recommended citation: M. Ullasci, M. Rondina, R. Coppola, F. Giobergia, A. De Cesare, M. Di Gregorio, G. Giordano, A. L. Maddaloni, S. Mantione, and L. Calò. 2026. Measuring Linguistic Bias in LLMs Across I.N.P.S. (Italian, Neapolitan, Parmesan and Sicilian) Dialect Corpora. In FAIEMA '26: Proceedings of the 4th International Conference on Frontiers of Artificial Intelligence, Ethics, and Multidisciplinary Applications. https://doi.org/10.1145/3794786.3830782

Abstract

Large Language Models (LLMs) encode and reproduce social biases rooted in their training data, including biases associated with linguistic varieties. This work investigates whether GPT-4.1 mini exhibits stereotypes towards speakers of three Italian dialects - Neapolitan, Parmesan and Sicilian - compared to standard Italian, through three experiments covering job assignment, personality trait attribution and character scoring. Results show that the model systematically favours standard Italian across all three tasks, associating dialect speakers with lower-prestige occupations and negative social traits. Additionally, southern varieties are penalised more consistently than the northern one, reproducing patterns associated with anti-meridionalism, a historically grounded form of discrimination against southern Italians. Mitigation strategies based on role prompting and multistep critique pipelines partially reduce, but do not eliminate the observed bias. These findings suggest that dialectal bias is intrinsic to LLMs trained on data that reflect existing social hierarchies, extending beyond specific languages and cultures.