Using Algorithms to Determine Character – The New York Times

2015/07/27 Leave a comment

Good piece on the increasing use of algorithms in granting loans and in the workplace, and the potential for (and limits to – see emphasized text) reducing bias:

Mr. Merrill, who also has a Ph.D. in psychology (from Princeton, in case Mr. Gu wants to lend him money), thinks that data-driven analysis of personality is ultimately fairer than standard measures.

“We’re always judging people in all sorts of ways, but without data we do it with a selection bias,” he said. “We base it on stuff we know about people, but that usually means favoring people who are most like ourselves.” Familiarity is a crude form of risk management, since we know what to expect. But that doesn’t make it fair.

Character (though it is usually called something more neutral-sounding) is now judged by many other algorithms. Workday, a company offering cloud-based personnel software, has released a product that looks at 45 employee performance factors, including how long a person has held a position and how well the person has done. It predicts whether a person is likely to quit and suggests appropriate things, like a new job or a transfer, that could make this kind of person stay.

It also characterizes managers as “rainmakers” or “terminators,” depending on how well they hold talent. Inside Workday, the company has analyzed its own sales force to see what makes for success. The top indicator is tenacity.

“We all have biases about how we hire and promote,” said Dan Beck, Workday’s head of technology strategy. “If you can leverage data to overcome that, great.”

People studying these traits will be encouraged to adopt them, he said, since “if you know there is a pattern of success, why wouldn’t you adopt it?”

In a sense, it’s no different from the way people read the biographies of high achievers, looking for clues for what they need to do differently to succeed. It’s just at a much larger scale, based on observing everybody.

There are reasons to think that data-based character judgments are more reasonable. Jure Leskovec, a professor of computer science at Stanford, is finishing up a study comparing the predictions of data analysis against those of judges at bail hearings, who have just a few minutes to size up prisoners and decide if they could be risks to society. Early results indicate that data-driven analysis is 30 percent better at predicting crime, Mr. Leskovec said.

“Algorithms aren’t subjective,” he said. “Bias comes from people.”

That is only true to a point: Algorithms do not fall from the sky. Algorithms are written by human beings. Even if the facts aren’t biased, design can be, and we could end up with a flawed belief that math is always truth.

Upstart’s Mr. Gu, who said he had perfect SAT scores but dropped out of Yale, wouldn’t have qualified for an Upstart loan using his own initial algorithms. He has since changed the design, and he said he is aware of the responsibility of the work ahead.

“Every time we find a signal, we have to ask ourselves, ‘Would we feel comfortable telling someone this was why they were rejected?’ ” he said.

Using Algorithms to Determine Character – The New York Times.

Filed under Multiculturalism Tagged with ,

When Algorithms Discriminate – The New York Times

2015/07/13 Leave a comment

Given that people have biases, not surprising that the algorithms created reflect some of these biases:

Algorithms, which are a series of instructions written by programmers, are often described as a black box; it is hard to know why websites produce certain results. Often, algorithms and online results simply reflect people’s attitudes and behavior. Machine learning algorithms learn and evolve based on what people do online. The autocomplete feature on Google and Bing is an example. A recent Google search for “Are transgender,” for instance, suggested, “Are transgenders going to hell.”

“Even if they are not designed with the intent of discriminating against those groups, if they reproduce social preferences even in a completely rational way, they also reproduce those forms of discrimination,” said David Oppenheimer, who teaches discrimination law at the University of California, Berkeley.

But there are laws that prohibit discrimination against certain groups, despite any biases people might have. Take the example of Google ads for high-paying jobs showing up for men and not women. Targeting ads is legal. Discriminating on the basis of gender is not.

The Carnegie Mellon researchers who did that study built a tool to simulate Google users that started with no search history and then visited employment websites. Later, on a third-party news site, Google showed an ad for a career coaching service advertising “$200k+” executive positions 1,852 times to men and 318 times to women.

The reason for the difference is unclear. It could have been that the advertiser requested that the ads be targeted toward men, or that the algorithm determined that men were more likely to click on the ads.

Google declined to say how the ad showed up, but said in a statement, “Advertisers can choose to target the audience they want to reach, and we have policies that guide the type of interest-based ads that are allowed.”

Anupam Datta, one of the researchers, said, “Given the big gender pay gap we’ve had between males and females, this type of targeting helps to perpetuate it.”

It would be impossible for humans to oversee every decision an algorithm makes. But companies can regularly run simulations to test the results of their algorithms. Mr. Datta suggested that algorithms “be designed from scratch to be aware of values and not discriminate.”

“The question of determining which kinds of biases we don’t want to tolerate is a policy one,” said Deirdre Mulligan, who studies these issues at the University of California, Berkeley School of Information. “It requires a lot of care and thinking about the ways we compose these technical systems.”

Silicon Valley, however, is known for pushing out new products without necessarily considering the societal or ethical implications. “There’s a huge rush to innovate,” Ms. Mulligan said, “a desire to release early and often — and then do cleanup.”

When Algorithms Discriminate – The New York Times.

Filed under discrimination Tagged with , ,

Peer reviewer tells female biologists their study would be better if they worked with men

2015/05/05 Leave a comment

While I have a general preference for mixed teams (and most of the evidence I have seen supports mixed teams), this is taking it too far. But given the subject of the paper (sexism), one can see the possibility of bias.

However, the peer review should focus on the substance and the assumptions of the study, rather than the gender of the authors:

“I read it through a couple of times trying to figure out whether it was a joke,” Head tells As It Happens guest host Tom Harrington. “[When I showed it to my colleagues], both male and female, they were unanimously outraged. It confirmed what I initially thought… The tone was completely condescending and the sexist comments were peppered throughout the review. I don’t know what they were trying to achieve, really.”

“It would probably also be beneficial to find one or two male biologists to work with (or at least obtain internal peer review from, but better yet as active co-authors), in order to serve as a possible check against interpretations that may sometimes be drifting too far away from empirical evidence into ideologically biased assumptions.”

– Excerpt from anonymous peer review

Ironically, their paper was about sexism. Head and Ingleby conducted a survey of 244 biology PhD students and found that women had worse job prospects than their male colleagues, possibly due to gender bias.

“We initially sent an appeal to the journal when we first received the review back,” she says. “We thought it was taking them too long to respond — all we received from them was a form letter apologizing for the delay. But really, this is an open-and-shut case. We couldn’t see why it was taking so long, and we didn’t want to see this swept under the carpet.”

Head and Ingleby decided to share excerpts of their review on Ingleby’s Twitter account. It went viral.

“Everyone paid attention it seemed,” she says with a laugh. “My co-author posted the tweets just before I went to bed at 11 p.m. Australian time. I woke up the next morning and Science magazine had covered the Twitter storm… it’s been really crazy, the response.”

In less than 24 hours, PLOS ONE issued a statement of apology and announced their appeal was in process.

“PLOS regrets the tone, spirit and content of this particular review. We take peer review seriously and are diligently and expeditiously looking into this matter. The appeal is in process. PLOS allows Academic Editors autonomy in how they handle manuscripts, but we always follow up if concerns are raised at any stage of the process. Our appeals policy also means that any complaints of the review process can be fully addressed and the author given opportunity to have their paper re-reviewed.”

– PLOS One statement

Peer reviewer tells female biologists their study would be better if they worked with men – Home | As It Happens | CBC Radio.

Filed under Other Tagged with , ,

Research Shows White Privilege Is Real

2015/02/26 Leave a comment

More examples of unconscious bias at work:

A field experiment about who gets free bus rides in Brisbane, a city on the eastern coast of Australia, shows that even today, whites get special privileges, particularly when other people aren’t around to notice.

The bus study underscores this point. Drivers were more likely to let testers ride free when there were fewer people on the bus to observe the transaction. And the drivers themselves were probably not aware that they were treating minorities differently. When drivers, in a questionnaire conducted after the field test, were shown photographs of the testers and asked how they would respond, hypothetically, to a free-ride request, they indicated no statistically significant bias against minorities in the photos (86 percent said they would let the black individual ride free).

Of course, unconscious bias might play out differently in the United States than in Australia. But research in America, too, suggests that decision makers use discretion to bestow benefits in a discriminatory fashion. For example, a recent study of 22 law firms by Arin N. Reeves, a lawyer and sociologist, found that partners were less critical of a junior lawyer’s draft memo if they were told the lawyer was white than if they were told the lawyer was black.

What does white privilege mean today? In part, it means to live in the world while being given the benefit of the doubt. Have you ever been able to return a sweater without a receipt? Has an employee ever let you into a store after closing time? Did a car dealership take a little extra off the sticker price when you asked? When’s the last time you received service with a smile?

White privilege doesn’t (usually) operate as brazenly and audaciously as in the Eddie Murphy joke, but it continues in the form of discretionary benefits, many of them unconscious ones. These privileges are hard to eradicate, but essential to understand.

Research Shows White Privilege Is Real – NYTimes.com.

Filed under Multiculturalism Tagged with , , ,

James Comey, FBI director, gives frank talk on policing and race

2015/02/14 Leave a comment

More on unconscious bias, assumptions and instinctive reactions:

The deaths of Michael Brown in Ferguson, Mo., and Eric Garner in New York, at the hands of white police officers, as well as the more recent slayings of two New York police officers, have raised difficult issues on both sides of the debate, Comey said.

One is that police officers who work in neighbourhoods where most street crime is committed by young black men may hold unconscious biases and be tempted to take what he called “lazy mental shortcuts” in dealing with suspicious situations.

That means officers may be influenced by feelings of “cynicism,” relying on assumptions they should not make and complicating the “relationship between police and the communities they serve,” he said.

‘The two young black men on one side of the street look like so many others that officer has locked up. Two white men on the other side of the street — even in the same clothes — do not. The officer does not make the same association about the two white guys, whether that officer is white or black, and that drives different behaviour.’

But another truth, he said, is that minorities in poor neighbourhoods too often inherit a “legacy of crime and prison,” a cycle he said must be broken to improve race relations with police.

Comey contended that everyone, regardless of background or colour, carries around biases.

“I am reminded of the song from the Broadway hit, Avenue Q — Everyone’s a Little Bit Racist …”

“But if we can’t help our latent biases, we can help our behaviour in response to those instinctive reactions, which is why we work to design systems and processes that overcome that very human part of us all,” he added.

James Comey, FBI director, gives frank talk on policing and race – World – CBC News.

Filed under Racism Tagged with , ,

Is the Professor Bossy or Brilliant? Much Depends on Gender – NYTimes.com

2015/02/10 Leave a comment

Gender bias universities

Frequency of word “genius” in RatemyProfessor

Interesting study on bias, this time in the university setting:

Studies have also shown that students can be biased against female professors. In one, teachers graded and returned papers to students at the exact same time, but when asked to rate their promptness, students gave female professors lower scores than men. Biases cut both ways — teachers have also been found to believe girls are not as good in math and science, even when they perform similarly to boys.

Mr. Schmidt, who made the chart as part of a project called Bookworm for searching and visualizing large texts, said he was struck by “this spectrum from smart to brilliant to genius, where each one of those is more strongly gendered male than the previous one was.” He was also surprised that relatively few people commented on female professors’ clothing or looks, which he had expected to be the case.

Another surprise, he said, was Shakespeare — apparently many more men than women teach it in English departments.

Men are more likely to be described as a star, knowledgeable, awesome or the best professor. Women are more likely to be described as bossy, disorganized, helpful, annoying or as playing favorites. Nice or rude are also more often used to describe women than men.

Men and women seemed equally likely to be thought of as tough or easy, lazy, distracted or inspiring.

Interestingly, women were more likely to be described in reviews as role models. Mr. Schmidt notes that the reviews are anonymous, so he doesn’t know the gender of reviewers. It could be that more female students describe female professors as role models than men do when describing men or women.

Is the Professor Bossy or Brilliant? Much Depends on Gender – NYTimes.com.

Filed under Other Tagged with , , , ,

Sex differences in academia: University challenge | The Economist

2015/01/20 1 Comment

Interesting analysis of the some of the unconscious beliefs and habits that may undermine efforts to increase diversity within STEM disciplines:

All this raises interesting and awkward questions. It may be unpalatable to some, but the idea that males and females have evolved cognitive differences over the course of many millions of years, because of the different interests of the sexes, is plausible. That people of different races have evolved such differences is far less likely, given the youth of Homo sapiens as a species. Prejudice thus seems a more plausible explanation for what Dr Leslie and Dr Cimpian have observed. But prejudice can work in subtle ways.

It could indeed be that recruiters from disciplines which think innate talent important are prejudiced about who they select for their PhD programmes. It could instead, though, be that women and black people themselves, through exposure to a culture that constantly tells them (which research suggests it does) that they do not have an aptitude for things like maths and physics, have come to believe this is true.

If that is the case (and Dr Leslie and Dr Cimpian suspect it is), it suggests that a cultural shift in schools and universities, playing down talent and emphasising hard work, might serve to broaden the intake of currently male-dominated and black-deficient fields, to the benefit of all.

Sex differences in academia: University challenge | The Economist.

Filed under Multiculturalism Tagged with , ,

The Science of Why Cops Shoot Young Black Men

2015/01/05 4 Comments

Good in-depth article on the psychology and neurology of subconscious bias and how it is part of our automatic thinking and sorting:

Science offers an explanation for this paradox—albeit a very uncomfortable one. An impressive body of psychological research suggests that the men who killed Brown and Martin need not have been conscious, overt racists to do what they did (though they may have been). The same goes for the crowds that flock to support the shooter each time these tragedies become public, or the birthers whose racially tinged conspiracy theories paint President Obama as a usurper. These people who voice mind-boggling opinions while swearing they’re not racist at all—they make sense to science, because the paradigm for understanding prejudice has evolved. There “doesn’t need to be intent, doesn’t need to be desire; there could even be desire in the opposite direction,” explains University of Virginia psychologist Brian Nosek, a prominent IAT researcher.”But biased results can still occur.”

The IAT is the most famous demonstration of this reality, but it’s just one of many similar tools. Through them, psychologists have chased prejudice back to its lair—the human brain.

Were not born with racial prejudices. We may never even have been “taught” them. Rather, explains Nosek, prejudice draws on “many of the same tools that help our minds figure out whats good and whats bad.” In evolutionary terms, its efficient to quickly classify a grizzly bear as “dangerous.” The trouble comes when the brain uses similar processes to form negative views about groups of people.

But here’s the good news: Research suggests that once we understand the psychological pathways that lead to prejudice, we just might be able to train our brains to go in the opposite direction.

The Science of Why Cops Shoot Young Black Men | Mother Jones.

And yes, I did take the Implicit Association Test (also available at UnderstandingPrejudice.org) and scored just as miserably as the Chris Mooney, the author of this article. Very sobering, and I encourage all to take it.

Filed under Multiculturalism, Racism Tagged with , , , ,

Can You Overcome Inbuilt Bias?

2014/08/20 Leave a comment

Interesting psych experiment, showing that appealing to higher motives less effective than more targeted tasking to reduce implicit biases:

Interestingly, most of the successful interventions were explicit about what they were trying to achieve and why. It’s important to remove the taboos around workplace discrimination and to educate people that bias is natural – what matters is that it doesn’t influence behavior. But worryingly, the majority of the successful interventions both associated black people with positive attributes and white people with negative attributes, reversing the natural direction of the white participants’ bias. Clearly reducing workplace bias by encouraging negativity towards a different group is not a solution.

The results of this comparison also raise an interesting question about the means of change and the outcome it achieves. Interventions which appealed to participants’ moral, conscious beliefs didn’t work, while those which targeted specific task behaviors – e.g. responding faster when black was paired with good – did. Some may argue that these interventions addressed the symptoms and not the cause. But in the workplace, when the ‘symptoms’ of implicit bias include unconsciously excluding and ostracizing others, addressing these behaviors may be a more effective use of time and resources than trying and failing to change the underlying beliefs which cause them.

It’s a tricky, emotive subject, but as more organizations wake up to the damaging consequences of implicit bias in terms of workforce engagement and performance, we can only hope for more research to shed light on how best to overcome it.

Can You Overcome Inbuilt Bias?.

Filed under Multiculturalism Tagged with , , ,

Bias-Free Hiring: Interview questions not to ask

2014/05/29 Leave a comment

An interesting but somewhat frustrating checklist of what to ask and what not to ask in interviewing candidates, as bias-free as possible. All too familiar to those of us in government, where the guidelines below are followed religiously and yet are deeply unsatisfying given the over-scripting that occurs. Sometimes it works out fine, sometimes less so:

  • Use the job description to identify the essential skills and abilities needed for the job. Determine which of these skills and abilities are best assessed through a written or practical test, through an interview, and from reference checks. From there, interview questions should be developed and clearly linked to the skills and abilities needed to do the job.
  • Develop the responses which you will look for in the candidates’ responses.
  • Attach a score to each question.
  • Use an interview panel when interviewing. Require each interviewer to write down each candidates’ responses to each question.
  • Ask each candidate the same questions.
  • After each interview, have the interview panel discuss the candidate’s responses and come to an agreed score for each question.

Bias-Free Hiring: Interview questions not to ask.

Filed under Government Tagged with ,

← Older posts

Newer posts →