Amazon Is Pushing Facial Technology That a Study Says Could Be Biased

Of note. These kinds of studies are important to expose the bias inherent in some corporate facial recognition systems:

Over the last two years, Amazon has aggressively marketed its facial recognition technology to police departments and federal agencies as a service to help law enforcement identify suspects more quickly. It has done so as another tech giant, Microsoft, has called on Congress to regulate the technology, arguing that it is too risky for companies to oversee on their own.

Now a new study from researchers at the M.I.T. Media Lab has found that Amazon’s system, Rekognition, had much more difficulty in telling the gender of female faces and of darker-skinned faces in photos than similar services from IBM and Microsoft. The results raise questions about potential bias that could hamper Amazon’s drive to popularize the technology.

In the study, published Thursday, Rekognition made no errors in recognizing the gender of lighter-skinned men. But it misclassified women as men 19 percent of the time, the researchers said, and mistook darker-skinned women for men 31 percent of the time. Microsoft’s technology mistook darker-skinned women for men just 1.5 percent of the time.

A study published a year ago found similar problems in the programs built by IBM, Microsoft and Megvii, an artificial intelligence company in China known as Face++. Those results set off an outcry that was amplified when a co-author of the study, Joy Buolamwini, posted YouTube videos showing the technology misclassifying famous African-American women, like Michelle Obama, as men.

The companies in last year’s report all reacted by quickly releasing more accurate technology. For the latest study, Ms. Buolamwini said, she sent a letter with some preliminary results to Amazon seven months ago. But she said that she hadn’t heard back from Amazon, and that when she and a co-author retested the company’s product a couple of months later, it had not improved.

Matt Wood, general manager of artificial intelligence at Amazon Web Services, said the researchers had examined facial analysis — a technology that can spot features such as mustaches or expressions such as smiles — and not facial recognition, a technology that can match faces in photos or video stills to identify individuals. Amazon markets both services.

“It’s not possible to draw a conclusion on the accuracy of facial recognition for any use case — including law enforcement — based on results obtained using facial analysis,” Dr. Wood said in a statement. He added that the researchers had not tested the latest version of Rekognition, which was updated in November.

Amazon said that in recent internal tests using an updated version of its service, the company found no difference in accuracy in classifying gender across all ethnicities.

The M.I.T. researchers used these and other photos to study the accuracy of facial technology in identifying gender.

With advancements in artificial intelligence, facial technologies — services that can be used to identify people in crowds, analyze their emotions, or detect their age and facial characteristics — are proliferating. Now, as companies begin to market these services more aggressively for uses like policing and vetting job candidates, they have emerged as a lightning rod in the debate about whether and how Congress should regulate powerful emerging technologies.

The new study, scheduled to be presented Monday at an artificial intelligence and ethics conference in Honolulu, is sure to inflame that argument.

Proponents see facial recognition as an important advance in helping law enforcement agencies catch criminals and find missing children. Some police departments, and the Federal Bureau of Investigation, have tested Amazon’s product.

But civil liberties experts warn that it can also be used to secretly identify people — potentially chilling Americans’ ability to speak freely or simply go about their business anonymously in public.

Over the last year, Amazon has come under intense scrutiny by federal lawmakers, the American Civil Liberties Union, shareholders, employees and academic researchers for marketing Rekognition to law enforcement agencies. That is partly because, unlike Microsoft, IBM and other tech giants, Amazon has been less willing to publicly discuss concerns.

Amazon, citing customer confidentiality, has also declined to answer questions from federal lawmakers about which government agencies are using Rekognition or how they are using it. The company’s responses have further troubled some federal lawmakers.

“Not only do I want to see them address our concerns with the sense of urgency it deserves,” said Representative Jimmy Gomez, a California Democrat who has been investigating Amazon’s facial recognition practices. “But I also want to know if law enforcement is using it in ways that violate civil liberties, and what — if any — protections Amazon has built into the technology to protect the rights of our constituents.”

In a letter last month to Mr. Gomez, Amazon said Rekognition customers must abide by Amazon’s policies, which require them to comply with civil rights and other laws. But the company said that for privacy reasons it did not audit customers, giving it little insight into how its product is being used.

The study published last year reported that Microsoft had a perfect score in identifying the gender of lighter-skinned men in a photo database, but that it misclassified darker-skinned women as men about one in five times. IBM and Face++ had an even higher error rate, each misclassifying the gender of darker-skinned women about one in three times.

Ms. Buolamwini said she had developed her methodology with the idea of harnessing public pressure, and market competition, to push companies to fix biases in their software that could pose serious risks to people.

Ms. Buolamwini, who had done similar tests last year, conducted another round to learn whether industry practices had changed, she said.CreditTony Luong for The New York Times

“One of the things we were trying to explore with the paper was how to galvanize action,” Ms. Buolamwini said.

Immediately after the study came out last year, IBM published a blog post, “Mitigating Bias in A.I. Models,” citing Ms. Buolamwini’s study. In the post, Ruchir Puri, chief architect at IBM Watson, said IBM had been working for months to reduce bias in its facial recognition system. The company post included test results showing improvements, particularly in classifying the gender of darker-skinned women. Soon after, IBM released a new system that the company said had a tenfold decrease in error rates.

A few months later, Microsoft published its own post, titled “Microsoft improves facial recognition technology to perform well across all skin tones, genders.” In particular, the company said, it had significantly reduced the error rates for female and darker-skinned faces.

Ms. Buolamwini wanted to learn whether the study had changed overall industry practices. So she and a colleague, Deborah Raji, a college student who did an internship at the M.I.T. Media Lab last summer, conducted a new study.

In it, they retested the facial systems of IBM, Microsoft and Face++. They also tested the facial systems of two companies that were not included in the first study: Amazon and Kairos, a start-up in Florida.

The new study found that IBM, Microsoft and Face++ all improved their accuracy in identifying gender.

By contrast, the study reported, Amazon misclassified the gender of darker-skinned females 31 percent of the time, while Kairos had an error rate of 22.5 percent.

Melissa Doval, the chief executive of Kairos, said the company, inspired by Ms. Buolamwini’s work, released a more accurate algorithm in October.

Ms. Buolamwini said the results of her studies raised fundamental questions for society about whether facial technology should not be used in certain situations, such as job interviews, or in products, like drones or police body cameras.

Some federal lawmakers are voicing similar issues.

“Technology like Amazon’s Rekognition should be used if and only if it is imbued with American values like the right to privacy and equal protection,” said Senator Edward J. Markey, a Massachusetts Democrat who has been investigating Amazon’s facial recognition practices. “I do not think that standard is currently being met.”

Source: Amazon Is Pushing Facial Technology That a Study Says Could Be Biased

Amazon’s Algorithm Has an Anti-Semitism Problem

As we are also seeing in the ongoing Facebook scandals, tech hasn’t managed to include ethical and moral considerations in its business models and algorithms:

In the wake of the Pittsburgh massacre, I invited my followers on Twitter to send me their inquiries about anti-Semitism. The response was overwhelming, and ran the gamut from questions about specific aspects of the prejudice to requests for advice on how to help fight it. One reader asked for more information about the Rothschild family, the Jewish banking dynasty that is a favorite bogeyman of anti-Semites and is typically used as a stand-in for the Jewish conspiracy that purportedly controls world affairs. He explained that some in his circle of friends regularly make bigoted remarks about the Rothschilds and their vast power and he wanted to set them straight. I’d written a report about the Rothschilds in school years ago, but figured there was probably better, more up-to-date material out there. So like anyone else, I went to Amazon.com and plugged in “history of rothschilds.” To my surprise, this is what I got:

amazonrothschilds
Rather than direct me to serious scholarship on the Rothschilds, like historian Niall Ferguson’s multivolume history on the family, Amazon first recommended blatantly bigoted content.

This isn’t the only instance of Amazon’s algorithm feeding intellectually bankrupt content to intellectually curious readers regarding fraught subjects. A search for “who did 9/11” yields this book as the #1 search result:

amazon911

As the book’s own blurb notes, its author, Nick Kollerstrom, is a “longtime member of Britain’s 9/11 truth group.” Among other conspiracies, the book contains an entire chapter entitled “9/11 and Zion” which blames the attack on the Jews. (Kollerstrom also happens to be a Holocaust denier who infamously declared, “Let us hope the schoolchildren visitors are properly taught about the elegant swimming pool at Auschwitz, built by the inmates, who would sunbathe there on Saturday and Sunday afternoons while watching the water polo matches.”)

Similarly, if one searches for “Jews and the slave trade,” the second, fourth, and fifth results are not scholarship on the subject, but notoriously anti-Semitic publications from Louis Farrakhan’s Nation of Islam. Farrakhan has worked for years to mainstream the baseless anti-Semitic conspiracy theory that Jews were behind the African slave trade.

(All searches above were done logged out from Amazon while incognito on Chrome, to ensure that the search results were the default ones, and not influenced by the specific user or their past search history.)

This isn’t Amazon’s first run-in with anti-Semitism concerns. Back in 2000, the company came under fire for stocking The Protocols of the Elders of Zion, one of the most influential anti-Semitic tracts in history. At the time, while firmly distancing itself from the bigoted content, Amazon insisted that it would not remove it from the catalog, because the company does not censor books. Today, copies of The Protocols on Amazon carry a cautionary message from the Anti-Defamation League and the following disclaimer from the company:

As a bookseller, Amazon strongly believes that providing open access to written speech, no matter how hateful or ugly, is one of the most important things we do. And because we think the best remedy for offensive speech is more speech, we also make available to readers the ability to make their own voices heard and express their views about this and all our titles in reviews and ratings.

It’s a reasonable defense. But it does not cover Amazon’s algorithm prioritizing the bigoted books over the legitimate ones.

The problem here is not that Amazon sells anti-Semitic material. The problem here is not that Amazon is trying to be anti-Semitic. It’s that the company is ignorant of anti-Semitic ideas, and so has not trained its algorithm to discount them. If a human librarian were asked about the Rothschilds, 9/11, or Jews and the slave trade, they would know how to distinguish between conspiratorial rantings and genuine documentation. They would also likely be aware of the anti-Semitic canards swirling around the subjects, and would steer interested readers away from them. Amazon’s vaunted search engine, perfectly tuned to maximize sales and the user’s shopping experience, has no such cultural competency.

Beyond the moral failure, these results also represent a straightforward professional failure. When a person searches for “history of rothschilds,” they are looking for historical information on the family, not a book featuring a shadowy figure squeezing blood out of globe. A bookselling algorithm that feeds readers misinformation is a broken bookselling algorithm.

If big tech companies like Amazon, Facebook, and Google want to get serious about combating online hate and misinformation, they need to start developing cultural competency on bigotry—and fast. They need not just coding experts working on their algorithms, but anti-hate experts who can flag conspiratorial currents. After all, it’s impossible for computers to identify a prejudice if they don’t know what it looks like. It’s about time we started teaching them.

Source: Amazon’s Algorithm Has an Anti-Semitism Problem

This is the Jeff Bezos playbook for preventing Amazon’s demise – Recode

Lots of interesting insights into Bezos’ thinking.

The one I found the most interesting, given my government background, was on the need for quick decision-making and related implications (impossible in government context given risk concerns but nevertheless helpful as a prompt to think more deeply about government processes):

High-Velocity Decision Making

Day 2 companies make high-quality decisions, but they make high-quality decisions slowly. To keep the energy and dynamism of Day 1, you have to somehow make high-quality, high-velocity decisions. Easy for start-ups and very challenging for large organizations. The senior team at Amazon is determined to keep our decision-making velocity high. Speed matters in business – plus a high-velocity decision-making environment is more fun too. We don’t know all the answers, but here are some thoughts.

First, never use a one-size-fits-all decision-making process. Many decisions are reversible, two-way doors. Those decisions can use a light-weight process. For those, so what if you’re wrong? I wrote about this in more detail in last year’s letter.

Second, most decisions should probably be made with somewhere around 70% of the information you wish you had. If you wait for 90%, in most cases, you’re probably being slow. Plus, either way, you need to be good at quickly recognizing and correcting bad decisions. If you’re good at course correcting, being wrong may be less costly than you think, whereas being slow is going to be expensive for sure.

Third, use the phrase “disagree and commit.” This phrase will save a lot of time. If you have conviction on a particular direction even though there’s no consensus, it’s helpful to say, “Look, I know we disagree on this but will you gamble with me on it? Disagree and commit?” By the time you’re at this point, no one can know the answer for sure, and you’ll probably get a quick yes.

This isn’t one way. If you’re the boss, you should do this too. I disagree and commit all the time. We recently greenlit a particular Amazon Studios original. I told the team my view: debatable whether it would be interesting enough, complicated to produce, the business terms aren’t that good, and we have lots of other opportunities. They had a completely different opinion and wanted to go ahead. I wrote back right away with “I disagree and commit and hope it becomes the most watched thing we’ve ever made.” Consider how much slower this decision cycle would have been if the team had actually had to convince me rather than simply get my commitment.

Note what this example is not: it’s not me thinking to myself “well, these guys are wrong and missing the point, but this isn’t worth me chasing.” It’s a genuine disagreement of opinion, a candid expression of my view, a chance for the team to weigh my view, and a quick, sincere commitment to go their way. And given that this team has already brought home 11 Emmys, 6 Golden Globes, and 3 Oscars, I’m just glad they let me in the room at all!

Fourth, recognize true misalignment issues early and escalate them immediately. Sometimes teams have different objectives and fundamentally different views. They are not aligned. No amount of discussion, no number of meetings will resolve that deep misalignment. Without escalation, the default dispute resolution mechanism for this scenario is exhaustion. Whoever has more stamina carries the decision.

I’ve seen many examples of sincere misalignment at Amazon over the years. When we decided to invite third-party sellers to compete directly against us on our own product detail pages – that was a big one. Many smart, well-intentioned Amazonians were simply not at all aligned with the direction. The big decision set up hundreds of smaller decisions, many of which needed to be escalated to the senior team.

“You’ve worn me down” is an awful decision-making process. It’s slow and de-energizing. Go for quick escalation instead – it’s better.

So, have you settled only for decision quality, or are you mindful of decision velocity too? Are the world’s trends tailwinds for you? Are you falling prey to proxies, or do they serve you? And most important of all, are you delighting customers? We can have the scope and capabilities of a large company and the spirit and heart of a small one. But we have to choose it.

Source: This is the Jeff Bezos playbook for preventing Amazon’s demise – Recode

Amazon-Hachette Dispute: Amazon’s Self-Serving Messaging

Hadn’t been following this dispute that much but when Amazon sent me the letter below, pretending to be on the side of the angels, I reacted in my response to them below:

I am sorry, but this letter and its request, is self-serving and outrageous.

I say this as someone who has both bought and published with Amazon.
I will be contacting the publishers telling them I do not support Amazon’s position as detailed in your letter.
Cloaking your corporate interests in consumer-friendly language, neglecting the content creation aspects of publishing, and shamefully invoking Orwell, is an extremely cynical move.
Will be sharing your letter and my response on my blog.

The NY Times points out that Amazon has misrepresented Orwell:

But Orwell then went on to undermine Amazon’s argument much more effectively than Hachette ever has. “It is of course a great mistake to imagine that cheap books are good for the book trade,” he wrote. “Actually it is just the other way about … The cheaper books become, the less money is spent on books.”

Instead of buying two expensive books, he says, the consumer will buy two cheap books and then use the rest of his money to go to the movies. “This is an advantage from the reader’s point of view and doesn’t hurt trade as a whole, but for the publisher, the compositor, the author and the bookseller, it is a disaster,” Orwell wrote.

The real problem, the writer argued in an essay a decade later, “Books v. Cigarettes,” was with the books themselves. They had a hard time competing against other media — a point people are still making in 2014.

“If our book consumption remains as low as it has been,” he wrote, “at least let us admit that it is because reading is a less exciting pastime than going to the dogs, the pictures or the pub, and not because books, whether bought or borrowed, are too expensive.”

Bits Blog: Dispute Between Amazon and Hachette Takes an Orwellian Turn

The original letter:

Dear KDP Author,

Just ahead of World War II, there was a radical invention that shook the foundations of book publishing. It was the paperback book. This was a time when movie tickets cost 10 or 20 cents, and books cost $2.50. The new paperback cost 25 cents – it was ten times cheaper. Readers loved the paperback and millions of copies were sold in just the first year.

With it being so inexpensive and with so many more people able to afford to buy and read books, you would think the literary establishment of the day would have celebrated the invention of the paperback, yes? Nope. Instead, they dug in and circled the wagons. They believed low cost paperbacks would destroy literary culture and harm the industry (not to mention their own bank accounts). Many bookstores refused to stock them, and the early paperback publishers had to use unconventional methods of distribution – places like newsstands and drugstores. The famous author George Orwell came out publicly and said about the new paperback format, if “publishers had any sense, they would combine against them and suppress them.” Yes, George Orwell was suggesting collusion.

Well… history doesn’t repeat itself, but it does rhyme.

Fast forward to today, and it’s the e-book’s turn to be opposed by the literary establishment. Amazon and Hachette – a big US publisher and part of a $10 billion media conglomerate – are in the middle of a business dispute about e-books. We want lower e-book prices. Hachette does not. Many e-books are being released at $14.99 and even $19.99. That is unjustifiably high for an e-book. With an e-book, there’s no printing, no over-printing, no need to forecast, no returns, no lost sales due to out of stock, no warehousing costs, no transportation costs, and there is no secondary market – e-books cannot be resold as used books. E-books can and should be less expensive.

Perhaps channeling Orwell’s decades old suggestion, Hachette has already been caught illegally colluding with its competitors to raise e-book prices. So far those parties have paid $166 million in penalties and restitution. Colluding with its competitors to raise prices wasn’t only illegal, it was also highly disrespectful to Hachette’s readers.

The fact is many established incumbents in the industry have taken the position that lower e-book prices will “devalue books” and hurt “Arts and Letters.” They’re wrong. Just as paperbacks did not destroy book culture despite being ten times cheaper, neither will e-books. On the contrary, paperbacks ended up rejuvenating the book industry and making it stronger. The same will happen with e-books.

Many inside the echo-chamber of the industry often draw the box too small. They think books only compete against books. But in reality, books compete against mobile games, television, movies, Facebook, blogs, free news sites and more. If we want a healthy reading culture, we have to work hard to be sure books actually are competitive against these other media types, and a big part of that is working hard to make books less expensive.

Moreover, e-books are highly price elastic. This means that when the price goes down, customers buy much more. We’ve quantified the price elasticity of e-books from repeated measurements across many titles. For every copy an e-book would sell at $14.99, it would sell 1.74 copies if priced at $9.99. So, for example, if customers would buy 100,000 copies of a particular e-book at $14.99, then customers would buy 174,000 copies of that same e-book at $9.99. Total revenue at $14.99 would be $1,499,000. Total revenue at $9.99 is $1,738,000. The important thing to note here is that the lower price is good for all parties involved: the customer is paying 33% less and the author is getting a royalty check 16% larger and being read by an audience that’s 74% larger. The pie is simply bigger.

But when a thing has been done a certain way for a long time, resisting change can be a reflexive instinct, and the powerful interests of the status quo are hard to move. It was never in George Orwell’s interest to suppress paperback books – he was wrong about that.

And despite what some would have you believe, authors are not united on this issue. When the Authors Guild recently wrote on this, they titled their post: “Amazon-Hachette Debate Yields Diverse Opinions Among Authors” (the comments to this post are worth a read).  A petition started by another group of authors and aimed at Hachette, titled “Stop Fighting Low Prices and Fair Wages,” garnered over 7,600 signatures.  And there are myriad articles and posts, by authors and readers alike, supporting us in our effort to keep prices low and build a healthy reading culture. Author David Gaughran’s recent interview is another piece worth reading.

We recognize that writers reasonably want to be left out of a dispute between large companies. Some have suggested that we “just talk.” We tried that. Hachette spent three months stonewalling and only grudgingly began to even acknowledge our concerns when we took action to reduce sales of their titles in our store. Since then Amazon has made three separate offers to Hachette to take authors out of the middle. We first suggested that we (Amazon and Hachette) jointly make author royalties whole during the term of the dispute. Then we suggested that authors receive 100% of all sales of their titles until this dispute is resolved. Then we suggested that we would return to normal business operations if Amazon and Hachette’s normal share of revenue went to a literacy charity. But Hachette, and their parent company Lagardere, have quickly and repeatedly dismissed these offers even though e-books represent 1% of their revenues and they could easily agree to do so. They believe they get leverage from keeping their authors in the middle.

We will never give up our fight for reasonable e-book prices. We know making books more affordable is good for book culture. We’d like your help. Please email Hachette and copy us.

Hachette CEO, Michael Pietsch: Michael.Pietsch@hbgusa.com

Copy us at: readers-united@amazon.com

Please consider including these points:

– We have noted your illegal collusion. Please stop working so hard to overcharge for ebooks. They can and should be less expensive.
– Lowering e-book prices will help – not hurt – the reading culture, just like paperbacks did.
– Stop using your authors as leverage and accept one of Amazon’s offers to take them out of the middle.
– Especially if you’re an author yourself: Remind them that authors are not united on this issue.

Thanks for your support.

The Amazon Books Team

P.S. You can also find this letter at www.readersunited.com

Professional and Non Partisan Thoughts on Renewing the Public Service: What Government Can Learn From Amazon

Good post on the potential for innovation. Getting decisions to do this, and implementation are another matter, as some of the citizen-centred service, service strategy and web work in the early days of Service Canada illustrate.

Professional and Non Partisan Thoughts on Renewing the Public Service: What Government Can Learn From Amazon.