Carlos Castillo
[introductory] Algorithmic Fairness in High-Risk AI Applications
Summary
The European AI Act defines a number of application areas of AI as “high-risk”: these include all applications involving security and critical infrastructure, and those dealing with access to justice, employment, education, and essential services. This course will summarize key concepts, frameworks, and standards for anti-discrimination in high-risk applications, with an emphasis on AI-powered decision support tools developed to be used by social services.
Syllabus
- Definitions: algorithm, discrimination
- Legal framework: the AI Act
- Decision support tools in high-risk scenarios
- Application 1: recidivism risk assessment
- Application 2: algorithmic hiring
- Case study
References
Marzieh Karimi-Haghighi, Carlos Castillo, Songül Tolan, Kristian Lum: Effect of Conditional Release on Violent and General Recidivism: A Causal Inference Study. Journal of Experimental Criminology. Springer, 2023.
Manuel Portela, Carlos Castillo, Songül Tolan, Marzieh Karimi-Haghighi, Antonio Andrés Pueyo. A Comparative User Study of Human Predictions in Algorithm-Supported Recidivism Risk Assessment. Artificial Intelligence and Law. Springer, 2024.
Alessandro Fabris, Clara Rus, Jorge Saldivar, Anna Gatzioura, Asia Biega, Carlos Castillo. The Impact of Ranking Interventions and Task-Level Factors in Recruitment Interfaces for Shortlisting. Pre-print, 2025.
Pre-requisites
Basic knowledge of machine learning methods would facilitate the learning.
Short bio
Carlos Castillo (they/them) is an ICREA Research Professor at Universitat Pompeu Fabra in Barcelona, where they lead the Social and Responsible Computing research group. They are a web miner with a background in information retrieval and have been influential in the areas of crisis informatics, web content quality and credibility, and adversarial web search. They are a prolific, highly cited researcher who has co-authored over 110 publications in top-tier international conferences and journals, receiving two test-of-time awards, five best paper awards, and two best student paper awards. Their works include a book on Big Crisis Data, as well as monographs on Information and Influence Propagation, and Adversarial Web Search.
