Bias Is a Big Problem. But So Is ‘Noise.’

Useful discussion in the context of human and AI decision-making. AI provides greater consistency (less noise or variability than humans) but with the risk of bias being part of the algorithms, and the importance of distinguishing the two when assessing decision-making:

The word “bias” commonly appears in conversations about mistaken judgments and unfortunate decisions. We use it when there is discrimination, for instance against women or in favor of Ivy League graduates. But the meaning of the word is broader: A bias is any predictable error that inclines your judgment in a particular direction. For instance, we speak of bias when forecasts of sales are consistently optimistic or investment decisions overly cautious.

Society has devoted a lot of attention to the problem of bias — and rightly so. But when it comes to mistaken judgments and unfortunate decisions, there is another type of error that attracts far less attention: noise.

To see the difference between bias and noise, consider your bathroom scale. If on average the readings it gives are too high (or too low), the scale is biased. If it shows different readings when you step on it several times in quick succession, the scale is noisy. (Cheap scales are likely to be both biased and noisy.) While bias is the average of errors, noise is their variability.

Although it is often ignored, noise is a large source of malfunction in society. In a 1981 study, for example, 208 federal judges were asked to determine the appropriate sentences for the same 16 cases. The cases were described by the characteristics of the offense (robbery or fraud, violent or not) and of the defendant (young or old, repeat or first-time offender, accomplice or principal). You might have expected judges to agree closely about such vignettes, which were stripped of distracting details and contained only relevant information.

But the judges did not agree. The average difference between the sentences that two randomly chosen judges gave for the same crime was more than 3.5 years. Considering that the mean sentence was seven years, that was a disconcerting amount of noise.

Noise in real courtrooms is surely only worse, as actual cases are more complex and difficult to judge than stylized vignettes. It is hard to escape the conclusion that sentencing is in part a lottery, because the punishment can vary by many years depending on which judge is assigned to the case and on the judge’s state of mind on that day. The judicial system is unacceptably noisy.

Consider another noisy system, this time in the private sector. In 2015, we conducted a study of underwriters in a large insurance company. Forty-eight underwriters were shown realistic summaries of risks to which they assigned premiums, just as they did in their jobs.

How much of a difference would you expect to find between the premium values that two competent underwriters assigned to the same risk? Executives in the insurance company said they expected about a 10 percent difference. But the typical difference we found between two underwriters was an astonishing 55 percent of their average premium — more than five times as large as the executives had expected.

Many other studies demonstrate noise in professional judgments. Radiologists disagree on their readings of images and cardiologists on their surgery decisions. Forecasts of economic outcomes are notoriously noisy. Sometimes fingerprint experts disagree about whether there is a “match.” Wherever there is judgment, there is noise — and more of it than you think.

Noise causes error, as does bias, but the two kinds of error are separate and independent. A company’s hiring decisions could be unbiased overall if some of its recruiters favor men and others favor women. However, its hiring decisions would be noisy, and the company would make many bad choices. Likewise, if one insurance policy is overpriced and another is underpriced by the same amount, the company is making two mistakes, even though there is no overall bias.

Where does noise come from? There is much evidence that irrelevant circumstances can affect judgments. In the case of criminal sentencing, for instance, a judge’s mood, fatigue and even the weather can all have modest but detectable effects on judicial decisions.

Another source of noise is that people can have different general tendencies. Judges often vary in the severity of the sentences they mete out: There are “hanging” judges and lenient ones.

A third source of noise is less intuitive, although it is usually the largest: People can have not only different general tendencies (say, whether they are harsh or lenient) but also different patterns of assessment (say, which types of cases they believe merit being harsh or lenient about). Underwriters differ in their views of what is risky, and doctors in their views of which ailments require treatment. We celebrate the uniqueness of individuals, but we tend to forget that, when we expect consistency, uniqueness becomes a liability.

Once you become aware of noise, you can look for ways to reduce it. For instance, independent judgments from a number of people can be averaged (a frequent practice in forecasting). Guidelines, such as those often used in medicine, can help professionals reach better and more uniform decisions. As studies of hiring practices have consistently shown, imposing structure and discipline in interviews and other forms of assessment tends to improve judgments of job candidates.

No noise-reduction techniques will be deployed, however, if we do not first recognize the existence of noise. Noise is too often neglected. But it is a serious issue that results in frequent error and rampant injustice. Organizations and institutions, public and private, will make better decisions if they take noise seriously.

Daniel Kahneman is an emeritus professor of psychology at Princeton and a recipient of the 2002 Nobel Memorial Prize in Economic Sciences. Olivier Sibony is a professor of strategy at the HEC Paris business school. Cass R. Sunstein is a law professor at Harvard. They are the authors of the forthcoming book “Noise: A Flaw in Human Judgment,” on which this essay is based.


Sunstein: In Politics, Apologies Are for Losers


Suppose that a public figure has said or done something that many people consider offensive, outrageous or despicable — for example, lied about his military service or insulted people’s religious convictions. Should he apologize?

Let’s assume that his goal is not to be a good person, but only to improve his standing — to increase the chance that he will be elected, get confirmed by the Senate or keep his job.

Recent evidence converges on a simple answer: An apology is a risky strategy.

A case in point, now receiving reconsideration as a result of recent reporting in The New Yorker about the allegations against Al Franken. In response to claims of inappropriate physical contact with several women, Mr. Franken, then a member of the Senate, publicly apologized. But the apology did not appear to do him much good, and it might have fanned some flames. Soon after apologizing, he was forced to resign.

Mr. Franken’s post-apology experience may not be so exceptional. According to recent surveys that I have conducted, apologies do not increase support for people who have said or done offensive things.

Yes, Donald Trump is making xenophobia acceptable: Cass Sunstein and “preference falsification”

Good piece and interesting study cited by Sunstein:

In the U.S. and Europe, many people worry that if prominent politicians signal that they dislike and fear immigrants, foreigners and people of minority religions, they will unleash people’s basest impulses and fuel violence. In their view, social norms of civility, tolerance and respect are fragile. If national leaders such as U.S. President Donald Trump flout those norms, they might unravel.

The most careful work on this general subject comes from Duke University economist Timur Kuran, who has studied the topic of “preference falsification.” In Kuran’s view, there is a big difference between what people say they think and what they actually think. Sometimes for better or sometimes for worse, people’s statements and actions are inhibited by prevailing social norms. When norms start to disintegrate, we can see startlingly fast alterations in what people say and do.

Kuran’s leading example is the collapse of communism in Eastern Europe, which, he says, was long sustained by the widespread misconception that other people supported communism. Once prominent citizens started to announce, in public, that they abhorred communism, others felt freer to say that they abhorred it too, and regimes were bound to collapse.

Kuran’s theory can be applied broadly. Writing in the late 1990s, he predicted the backlash against affirmative action programs, contending that a lot of people opposed such programs even though they weren’t saying so. Millions of people favoured same-sex marriage before they felt free to announce that they did. When professors keep quiet after left-wing students shut down conservative speakers, it may not be because they approve; they might be capitulating to social norms on campus. There is a strong taboo on anti-Semitism, which limits its public expression.

It’s hard to test these kinds of ideas rigorously, but in an ingenious new paper, a team of economists has done exactly that.

Leonardo Bursztyn of the University of Chicago, Georgy Egorov of Northwestern University and Stefano Fiorin of the University of California at Los Angeles designed an elaborate experiment to test whether Trump’s political success affects Americans’ willingness to support, in public, a xenophobic organization. They find that it does — big-time. It’s a little finding with big implications.

The experiment is pretty complicated, so please bear with me. Two weeks before the election, Bursztyn and his colleagues recruited 458 people from eight states that the website Predictwise said that Trump was certain to win (Alabama, Arkansas, Idaho, Nebraska, Oklahoma, Mississippi, West Virginia and Wyoming). Half the participants were told that Trump would win. The other half received no information about Trump’s projected victory.

All participants were then asked an assortment of questions, including whether they would authorize the researchers to donate $1 to the Federation for American Immigration Reform, accurately described as an anti-immigrant organization whose founder has written, “I’ve come to the point of view that for European-American society and culture to persist requires a European-American majority, and a clear one at that.” If participants agreed to authorize the donation, they were told that they would be paid an additional $1.

Here’s where things get interesting. Half the participants were assured that their decision to authorize a donation would be anonymous. The other half were given no such assurance. On the contrary, they were told that members of the research team might contact them, thus suggesting that their willingness to authorize the donation could become public.

For those who were not informed about Trump’s expected victory in their state, giving to the anti-immigration group was a lot more attractive when anonymity was assured: 54 per cent authorized the donation under cover of secrecy as opposed to 34 per cent when the authorization might become public. But for those who were informed that Trump would win, anonymity didn’t matter at all. When so informed, about half the participants were willing to authorize the donation regardless of whether they received a promise of anonymity.

As an additional test, Bursztyn and his colleagues repeated their experiment in the same states during the first week after Trump’s election. They found that Trump’s victory also eliminated the effects of anonymity — again, about half the participants authorized the donation regardless of whether the authorization would be public.

The upshot is that if Trump had not come on the scene, a lot of Americans would refuse to authorize a donation to an anti-immigrant organization unless they were promised anonymity. But with Trump as president, people feel liberated. Anonymity no longer matters, apparently because Trump’s election weakened the social norm against supporting anti-immigrant groups. It’s now OK to be known to agree “that for European-American society and culture to persist requires a European-American majority, and a clear one at that.”

Nothing in these findings demonstrates that Trump’s election is leading to an erosion of social norms against incivility and hatred, let alone against violence. But they’re suggestive. Sometimes people don’t say what they think, or do as they like, because of their beliefs about the beliefs of their fellow citizens. A nation’s leader can give strong signals about those beliefs — and so diminish the effects of social norms that constrain ugly impulses.

Source: Yes, Donald Trump is making xenophobia acceptable | Toronto Star

ICYMI: The Choice Explosion – The New York Times

Interesting insights on decision-making in the book, Decisive, by  Chip and Dan Heath:

It’s becoming incredibly important to learn to decide well, to develop the techniques of self-distancing to counteract the flaws in our own mental machinery. The Heath book is a very good compilation of those techniques.

For example, they mention the maxim, assume positive intent. When in the midst of some conflict, start with the belief that others are well-intentioned. It makes it easier to absorb information from people you’d rather not listen to.

They highlight Suzy Welch’s 10-10-10 rule. When you’re about to make a decision, ask yourself how you will feel about it 10 minutes from now, 10 months from now and 10 years from now. People are overly biased by the immediate pain of some choice, but they can put the short-term pain in long-term perspective by asking these questions.

The Heaths recommend making deliberate mistakes. A survey of new brides found that 20 percent were not initially attracted to the man they ended up marrying. Sometimes it’s useful to make a deliberate “mistake” — agreeing to dinner with a guy who is not your normal type. Sometimes you don’t really know what you want and the filters you apply are hurting you.

They mention our tendency to narrow-frame, to see every decision as a binary “whether or not” alternative. Whenever you find yourself asking “whether or not,” it’s best to step back and ask, “How can I widen my options?” In other words, before you ask, “Should I fire this person?” Ask, “Is there any way I can shift this employee’s role to take advantage of his strengths and avoid his weaknesses?”

The explosion of choice means we all need more help understanding the anatomy of decision-making. It makes you think that we should have explicit decision-making curriculums in all schools. Maybe there should be a common course publicizing the work of Daniel Kahneman, Cass Sunstein, Dan Ariely and others who study the way we mess up and the techniques we can adopt to prevent error.

Source: The Choice Explosion – The New York Times

What conservatives really care about

Interesting perspective by Sunstein of Jonathan Haidt’s assessment of the similarities and differences between the values of  liberals and conservatives:

In his later work, Haidt has rightly emphasized a sixth moral foundation, one that conservatives and liberals both respect, but that they understand differently: liberty. He finds that conservatives are more likely to emphasize the right to be let alone, while liberals emphasize the rights of vulnerable groups, such as racial minorities, whose freedom requires (in their view) government support. Nonetheless, the biggest and most consistent partisan differences involve loyalty, authority and sanctity.

Haidt’s central claim is that across partisan lines, people often fail to understand one another, because a moral concern that strongly motivates one group may be obscure or unintelligible to another. Democrats are wrong to be puzzled when rural and working-class Americans turn out to favor Republicans. There is no puzzle here, because Republicans are more likely to speak to their deepest moral commitments.

These claims are arresting, but it’s not clear that they are entirely right. Insofar as liberals focus on the environment, they are often motivated by ideas about the sanctity of nature. More than conservatives, liberals appear disgusted by cigarette smoking. Nor are they indifferent to loyalty: If a civil rights leader publicly opposed affirmative action, or if a prominent Democrat broke with the party on health care or climate change, many liberals would feel a sense of betrayal. Conservatives may be more likely to emphasize loyalty in the abstract, but in concrete cases, everyone cares about that virtue.

That said, Haidt’s general conclusions are founded on evidence, not speculation, and he has compiled a mountain of evidence to support his conclusions. There’s a big lesson here for those who aspire to public office, including the White House: If they neglect the values of loyalty, authority and sanctity, they’re not going to speak to the moral commitments of a large segment of the American electorate.

Suspect this also holds true for Canada.

What conservatives really care about

Extremism loves company: Sunstein

Cass Sunstein on the psychology of radicalization.

What nudge does he suggest to reduce polarization and extremism?

Why does group polarization occur? The first answer involves information. Suppose that most group members begin by thinking that some religious group, leader or nation is evil. If so, they will hear a lot of arguments to that effect. As they absorb them, they will be inclined to move toward a more extreme version of their initial judgment.

People also care about their reputations, so some group members will adjust their positions in the direction of the dominant view. A disturbing implication is that if group members listen only to one another, and if most of them have extremist tendencies, the whole group might well march toward greater radicalism and even brutality.

Writing in 1998, Russell Hardin, a political scientist at New York University, drew attention to the “crippled epistemology of extremism,” by which he meant to emphasize how little extremists know. Focused on Islamic fundamentalists, Hardin was concerned about what happens “when the fanatic is in a group of like-minded people, and especially when the group isolates itself from others.”

In the years ahead, the international effort to combat violent extremism will sometimes require force, and it will sometimes require economic pressure. But it will succeed only if it disrupts recruitment and radicalization by enclaves of like-minded people.

Bloomberg’s Cass R. Sunstein (pay wall)

Nudges vs. Shoves by Cass R. Sunstein

For those interested in public policy and nudges, good discussion by Cass Sunstein on the benefits of nudges, which preserve choice, to mandatory measures. Another instrument in the public policy toolkit.

Dry abstract below:

Behavioral findings, demonstrating human errors, have led some people to favor choice-preserving responses (“nudges”), and others to favor mandates and bans. If people’s choices lead them to err, it might seem puzzling, or even odd, to respond with solutions that insist on preserving freedom of choice. But mandates have serious problems of their own, even in the face of behavioral market failures. Mandates might not be able to handle heterogeneity; they might reflect limited knowledge on the part of public officials or the interests of powerful private groups; and they override freedom, potentially producing welfare losses and insulting individual dignity. It is true that in some cases, a behavioral market failure (such as a self-control problem) might justify a mandate on social welfare grounds, but on those very grounds, it makes sense to begin by examining choice-preserving approaches, which are far less intrusive and often highly effective.

Nudges vs. Shoves by Cass R. Sunstein :: SSRN.