Amazon’s facial recognition business will mark a significant setback this week if privacy and civil liberties advocates and shareholders get their way.
"A unified chorus of civil rights orgs AI experts, customers, employees, academics, and members of Congress has warned about the dangers of government face surveillance.
Against these voices stands Amazon." https://t.co/jTkrXx0wl0
— Matt Cagle (@Matt_Cagle) May 20, 2019
This year in January, shareholders proposed a resolution to limit the sale of Amazon’s facial recognition tech called Rekognition to law enforcement and government agencies. The technology was found to be biased and inaccurate and is regarded as an enabler of racial discrimination of minorities. Rekognition, which runs image and video analysis of faces, has been sold to two states so far, and Amazon has also pitched it to Immigration and Customs Enforcement.
The first proposal asked the Board of Directors to stop sales of “Rekognition” — Amazon’s face surveillance technology — to the government. The second demands an independent review of its human and civil rights impacts, particularly for people of color, immigrants, and activists, who have always been disproportionately impacted by surveillance. ACLU in its letter backs the measures and calls on for shareholders to pass these resolutions.
Amazon’s non responsiveness and its failure to act
ACLU’s letter will be presented at Amazon’s annual shareholder meeting on Wednesday. There ACLU plans to accuse Amazon of “failing to act responsibly” by refusing to stop the sale of the technology to the government.
“Amazon has stayed the course,” states the letter. “Amazon has heard repeatedly about the dangers to our democracy and vulnerable communities about this technology but they have refused to acknowledge those dangers, let alone address them”.
“This technology fundamentally alters the balance of power between government and individuals, arming governments with unprecedented power to track, control, and harm people,” as per the letter. “It would enable police to instantaneously and automatically determine the identities and locations of people going about their daily lives, allowing government agencies to routinely track their own residents. Associated software may even display dangerous and likely inaccurate information to police about a person’s emotions or state of mind.”
“As shown by a long history of other surveillance technologies, face surveillance is certain to be disproportionately aimed at immigrants, religious minorities, people of color, activists, and other vulnerable communities,” the letter added.
“Without shareholder action, Amazon may soon become known more for its role in facilitating pervasive government surveillance than for its consumer retail operations,” it read.
Facial recognition has become one of the most hot-button topics in privacy in years. Recently Amnesty International in Canada raised serious privacy issues on the Google’s Sidewalk Labs project in Toronto. There was concerns around the project’s potential to normalize mass surveillance and is a threat to human rights.
While Amazon Rekognition, a cloud-based facial recognition system, remains in its infancy, it is yet one of the most prominent systems available. But critics say the technology is flawed. Exactly a year prior to this week’s shareholder meeting, the ALCU first raised “profound” concerns with Rekognition and its installation at airports, public places and by police. Since then, the technology was shown to struggle to detect people of color. In its tests, the system struggled to match 28 congresspeople who were falsely matched in a mugshot database who had been previously arrested.
This latest move is a concerted effort by dozens of shareholders and investment firms, tech experts and academics, privacy and rights groups and organizations who decry the use of facial recognition technology.
In the month of March, top AI researchers including this year’s winner of the Turing Award, Yoshua Bengio also issued a joint statement calling on Amazon Web Services to stop all sales of its Rekognition facial-recognition technology to law enforcement agencies.
There has been a pushback even from government. Several municipalities have rolled out surveillance-curtailing laws and ordinances this year. San Francisco last week became the first major U.S. city government to ban the use of facial recognition. In April, the Oakland Privacy Advisory Commission released 2 key documents, an initiative to protect Oaklanders’ privacy namely, Proposed ban on facial recognition and City of Oakland Privacy Principles.
“Amazon leadership has failed to recognize these issues,” said the ACLU’s letter. “This failure will lead to real-life harm.” The ACLU said shareholders “have the power to protect Amazon from its own failed judgment.”
Miles Brundage, a Research scientist at OpenAI commented that every facial recognition expert agrees Amazon is behaving irresponsible and yet it is continuing. He says Amazon is making a big misguided bet on first mover advantages in this area.
*Misguided not just ethically but practically: I don't think there are big first mover advantages in FR, for various reasons (lack of a single platform for all applications, local police elections make any relationship-building potentially short-lived given controversies, etc.)
— Miles Brundage (@Miles_Brundage) May 20, 2019
Amazon has pushed back against the claims by arguing that the technology is accurate and has largely criticized how the ACLU conducted its tests using Rekognition.
Amazon said shareholders’ proposals on the civil rights threats of facial surveillance “raise only conjecture and speculation” about possible misuse of the technology.
At least a few large institutional investors disagree.https://t.co/DCrDEMcGBj
— Natasha Singer (@natashanyt) May 20, 2019
Amazon states, “ Machine learning is a very valuable tool to help law enforcement agencies, and while being concerned it’s applied correctly. We should not throw away the oven because the temperature could be set wrong and burn the pizza. It is a very reasonable idea, however, for the government to weigh in and specify what temperature (or confidence levels) it wants law enforcement agencies to meet to assist in their public safety work.”
It is interesting that Amazon compares facial recognition tech with an oven to justify its decision to sell the tech while transferring the accountability to the buyer and user of the tech. Unlike an oven which has a narrow range of applications and clear liabilities for misuse of such a device, to say willfully cause harm to another person, facial recognition applications are wide and varied and no clear regulations or accountability boundaries exist for harms caused using it.
In the absence of clear laws and regulations, enabling state surveillance and control with facial recognition products that are used by law enforcement and governing bodies, can set a dangerous precedent and curtail individual freedoms and exacerbate socio-economic inequalities. Hence, it is not only important but essential for civil liberty groups like ACLU to urge Amazon and its shareholders to act quick.