Amazon dubbed a study “misleading” after it claimed that their facial recognition tool was racial and gender-biased.
The research published by the Massachusetts Institute of Technology compared similar tools from five companies including Microsoft and IBM.
It had found that none of the companies could accurately read and identify faces. The Amazon Recognition tool performed the worst when it came to recognising females with darker skin.
Pointing to racial bias, it said both Amazon and their facial recognition competitors, Kairos and Microsoft scored at 100% when analysing white-skinned men. But the picture looked quite different when it came to analysing people of colour.
The study detected Amazon was having an error rate of 31% when identifying images of non-white women. Whereas, a facial recognition competitor, Kairos and IBM saw 22.5% and 17% inaccuracy rates, respectively.
Amazon’s general manager of artificial intelligence at AWS, Dr Matt Wood addressed the criticism by stating that the study is “misleading and drew false conclusions.”
In his blog aimed at clarifying “several misperceptions and inaccuracies”, he pointed to the differences in facial recognition and facial analysis.
He said that Amazon Recognition is designed to “match faces that appear similar” to authenticate an individuals’ identity. Therefore, to use facial recognition to gauge the accuracy of facial recognition is “ill-advised, as it’s not the intended algorithm for that purpose.”
“The answer to anxieties over new technology is not to run ‘tests’ inconsistent with how the service is designed to be used and to amplify the test’s false and misleading conclusions through the news media. The research paper in question does not use the recommended facial recognition capabilities, does not share the confidence levels used in their research, and we have not been able to reproduce the results of the study. We are eager to continue to work with researchers, academics, and customers, to continuously improve as we evolve this important technology.”
– Dr Matt Wood, general manager of artificial intelligence, AWS