×

Social Stereotypes in Image Tagging Algorithms

Research / Pillars & Groups / Communications & Artificial Intelligence / fAIre / Internships / Social Stereotypes in Image Tagging Algorithms
17 January. 2021

Image analysis algorithms have been proven to reflect social stereotypes in their outputs. In our earlier work, we used standardized images of people to systematically examine how the gender and race of the person depicted, as well as the background of the image, affect the outputs of image tagging algorithms. We found evidence for gender-based stereotypes, especially those relating to occupation, in the tags used to describe the content of the images. There is an opportunity to extend this work, to look closer at race and/or class-based stereotypes, as well as social stereotypes in other contexts. Another option is to replicate the work using multiple images of a single type of context (e.g. different baby nurseries). 

For more information: 

  1. To "See" is to Stereotype: Image Tagging Algorithms, Gender Recognition, and the Accuracy–Fairness Trade-off. Pınar Barlas, Kyriakos Kyriakou, Olivia Guest, Styliani Kleanthous, and Jahna Otterbacher. 2020. Presented at CSCW ‘20; upcoming publication in Proc. ACM Hum.-Comput. Interact. (V4, CSCW3, #232). -- PDF 

  1. Fairness in Proprietary Image Tagging Algorithms: A Cross-Platform Audit on People Images. Kyriakos Kyriakou, Pınar Barlas, Styliani Kleanthous, and Jahna Otterbacher. 2019. In Proceedings of the Thirteenth International AAAI Conference on Web and Social Media (ICWSM ‘19). -- PDF 

Required Skills
One or more of the following: knowledge of theories and experiments in the social sciences -- particularly (social) psychology, sociology, media studies or similar; experience with computer vision algorithms; quantitative analysis; knowledge of/experience with algorithmic audits. 
Skills Level
Intermediate
Objectives
The aim of the project is to extend previous work in the area. The findings are expected to demonstrate which social stereotypes may appear in image tagging algorithms, and how. The project is highly interdisciplinary: the intern will improve their existing skills, while getting exposure to skills and areas they may not be familiar with. Depending on their skills and availability, the intern may contribute to one or most of the steps, from literature review to experiment setup, data analysis, and/or writing the paper. 

Expected deliverables: One or more from: literature review; dataset of images and tags collected from image tagging algorithms; (part of) a research paper