17 January. 2021
Vision-based Cognitive Services (CogS) have become crucial in a wide range of applications, from real-time security and social networks to smartphone applications. Many services focus on analyzing people's images. When it comes to facial analysis, these services can be misleading or even inaccurate, raising ethical concerns such as the amplification of social stereotypes. In previous work, we analyzed popular Image Tagging CogS that infer emotion from a person's face, considering whether they perpetuate racial and gender stereotypes concerning emotion. By comparing both CogS and Human-generated descriptions on a set of controlled images, we highlighted the need for transparency and fairness in CogS. In particular, we documented evidence that CogS may actually be more likely than crowdworkers to perpetuate the stereotype of the "angry black man" and often attribute black race individuals with "emotions of hostility".
Our previous research was limited to the White and Black Race individuals. In the current research study, we are planning to extend this effort by analyzing further emotion datasets which include other races such as Asian, Latinx, etc. Another aim is to refine and reconduct the previous research to inspect the Image Analysis Prejudice Evolution in Computer Vision Algorithms through time and investigate if the Emotion Stereotypes found still persist or even minimized or amplified.
For more information, see our related work:
-
Kyriakou, K., Kleanthous, S., Otterbacher, J., & Papadopoulos, G. A. (2020, July). Emotion-based Stereotypes in Image Analysis Services. In Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization(pp. 252-259). - PDF
-
Kyriakou, K., Barlas, P., Kleanthous, S., & Otterbacher, J. (2019, July). Fairness in proprietary image tagging algorithms: A cross-platform audit on people images. In Proceedings of the International AAAI Conference on Web and Social Media(Vol. 13, pp. 313-322). - PDF
The intern must be an advanced university student (3rd year of studies or more) in Computer Science or related fields of study.
The successful applicant must have good knowledge in:
-
Python programming language
-
Image Processing or Computer Vision or Machine Learning or Deep Learning (most preferably in Python frameworks, tools & libraries)
-
RESTful APIs/Services
Any prior knowledge or experience on the Psychology perspective of Emotion or in Emotion Analysis will be considered as a plus.
Intermediate
The project aims to develop python tools for extending the study into further datasets and use the groups’ prior research tools to study the behavior of computer vision services such as Google Vision, Amazon Rekognition, IBM Watson, Microsoft FACE, Clarifai and Imagga. The project will involve work in data collection and analysis.
Expected deliverables:
The successful applicant will discuss with the TAG MRG team about the final expected deliverables. Although some main expected deliverables would be the following:
The team will ensure that the successful applicant will have the opportunity to get involved in a high-quality research environment and expert team, while also aiming to generate a paper for submission in a competitive Conference, Workshop, or Journal on the related topic. The publication will include the successful applicant’s (intern) name as one of the authors of the current work.