News article by Concerned Researchers.
Published on Medium.
Excerpt:
Over the past few months, there has been increased public concern over the accuracy and use of new face recognition systems. A recent study conducted by Inioluwa Deborah Raji and Joy Buolamwini, published at the AAAI/ACM conference on Artificial Intelligence, Ethics, and Society, found that the version of Amazon’s Rekognition tool which was available on August 2018, has much higher error rates while classifying the gender of darker skinned women than lighter skinned men (31% vs. 0%). In response, two Amazon officials, Matthew Wood and Michael Punke, wrote a series of blog posts attempting to refute the results of the study. In this piece we highlight several important facts reinforcing the importance of the study and discussing the manner in which Wood and Punke’s blog posts misrepresented the technical details for the work and the state-of-the-art in facial analysis and face recognition.
- There is an indirect or direct relationship between modern facial analysis and face recognition (depending on the approach). So in contrast to Dr. Wood’s claims, bias found in one system is cause for concern in the other, particularly in use cases that could severely impact people’s lives, such as law enforcement applications.
- Raji and Buolamwini’s study was conducted within the context of Rekognition’s use. This means using an API that was publicly available at the time of the study, considering the societal context under which it was being used (law enforcement), and the amount of documentation, standards and regulation in place at the time of use.
- The data used in the study can be obtained through a request to https://www.ajlunited.org/gender-shades for non commercial uses, and has been replicated by many companies based on the details provided in the paper available at http://gendershades.org/.
- There are no laws or required standards to ensure that Rekognition is used in a manner that does not infringe on civil liberties.
We call on Amazon to stop selling Rekognition to law enforcement. [ . . . ]
Signed by Concerned Researchers
- Ali Alkhatib, Stanford University
- Noura Al Moubayed, Durham University
- Miguel Alonso Jr, Florida International University
- Anima Anandkumar, Caltech (formerly Principal Scientist at AWS)
- Akilesh Badrinaaraayanan, MILA/University of Montreal
- Esube Bekele, National Research Council fellow
- Yoshua Bengio, MILA/University of Montreal
- Alex Berg, UNC Chapel Hill
- Miles Brundage, OpenAI; Oxford; Axon AI Ethics Board
- Dan Calacci, Massachusetts Institute of Technology
- Pablo Samuel Castro, Google
- Stayce Cavanaugh, Google
- Abir Das, IIT Kharagpur
- Hal Daumé III, Microsoft Research and University of Maryland
- Maria De-Arteaga, Carnegie Mellon University
- Mostafa Dehghani, University of Amsterdam
- Emily Denton, Google
- Lucio Dery, Facebook AI Research
- Priya Donti, Carnegie Mellon University
- Hamid Eghbal-zadeh, Johannes Kepler University Linz
- El Mahdi El Mhamdi, Ecole Polytechnique Fédérale de Lausanne
- Paul Feigelfeld, IFK Vienna, Strelka Institute
- Jessica Finocchiaro, University of Colorado Boulder
- Andrea Frome, Google
- Field Garthwaite, IRIS.TV
- Timnit Gebru, Google
- Sebastian Gehrmann, Harvard University
- Oguzhan Gencoglu, Top Data Science
- Marzyeh Ghassemi, University of Toronto, Vector Institute
- Georgia Gkioxari, Facebook AI Research
- Alvin Grissom II, Ursinus College
- Sergio Guadarrama, Google
- Alex Hanna, Google
- Bernease Herman, University of Washington
- William Isaac, Deep Mind
- Phillip Isola, Massachusetts Institute of Technology
- Alexia Jolicoeur-Martineau, MILA/University of Montreal
- Yannis Kalantidis, Facebook AI
- Khimya Khetarpal, MILA/McGill University
- Michael Kim, Stanford University
- Morgan Klaus Scheuerman, University of Colorado Boulder
- Hugo Larochelle, Google/MILA
- Erik Learned-Miller, UMass Amherst
- Xing Han Lu, McGill University
- Kristian Lum, Human Rights Data Analysis Group
- Michael Madaio, Carnegie Mellon University
- Tegan Maharaj, Mila/École Polytechnique
- João Martins, Carnegie Mellon University
- El Mahdi El Mhamdi, Ecole Polytechnique Fédérale de Lausanne
- Vincent Michalski, MILA/University of Montreal
- Margaret Mitchell, Google
- Melanie Mitchell, Portland State University and Santa Fe Institute
- Ioannis Mitliagkas, MILA/University of Montreal
- Bhaskar Mitra, Microsoft and University College London
- Jamie Morgenstern, Georgia Institute of Technology
- Bikalpa Neupane, Pennsylvania State University, UP
- Ifeoma Nwogu, Rochester Institute of Technology
- Vicente Ordonez-Roman, University of Virginia
- Pedro O. Pinheiro
- Vinodkumar Prabhakaran, Google
- Parisa Rashidi, University of Florida
- Anna Rohrbach, UC Berkeley
- Daniel Roy, University of Toronto
- Negar Rostamzadeh
- Kate Saenko, Boston University
- Niloufar Salehi, UC Berkeley
- Anirban Santara, IIT Kharagpur (Google PhD Fellow)
- Brigit Schroeder, Intel AI Lab
- Laura Sevilla-Lara, University of Edinburgh
- Shagun Sodhani, MILA/University of Montreal
- Biplav Srivastava
- Luke Stark, Microsoft Research Montreal
- Rachel Thomas, fast.ai; University of San Francisco
- Briana Vecchione, Cornell University
- Toby Walsh, UNSW Sydney
- Serena Yeung, Harvard University
- Yassine Yousfi, Binghamton University
- Richard Zemel, Vector & University of Toronto
List retrieved February 29, 2020