Racial bias exists in five-star ratings for gig workers, study shows. Thumbs up/thumbs down scale would fix that

Small but significant difference and impact:

Most of us do it, sometimes daily. After ordering a ride, getting a meal delivered or hiring someone for home repairs through an app, we’re asked to rate the service – often on a five-star scale.

Ratings are intended to be a fair way to reward good work and ensure those who provide exceptional service get more business.

“We want to ensure that the [rating] system allows shoppers’ effort to shine,” John Adams, vice-president of product at Instacart, states on the delivery company’s website. 

But a study published recently in the science journal Nature suggests otherwise. It found these seemingly neutral five-star systems harbour subtle but measurable racial bias that disappears when a “thumbs-up” or “thumbs-down” rating system is used.

Using data from an unnamed home-services app operating in Canada and the United States that had been using a five-star scale, researchers found a statistically significant difference between ratings given to white and non-white workers. After analyzing tens of thousands of reviews, the study showed white workers received an average rating of 4.79 stars, while non-white workers averaged 4.72.

That 0.07-point gap may seem trivial, but co-author Katherine DeCelles, a professor of organizational behaviour at the University of Toronto’s Rotman School of Management, says it has real financial consequences. Because many apps rely on customer ratings to determine which workers are recommended most often, non-white workers were found to earn 91 cents for every dollar their white counterparts made for the same work.

The researchers attribute part of the disparity to subtle and often unconscious bias. For instance, a customer giving a racial minority worker who performs well four stars, instead of five, “does not challenge the customer’s self-image as non-prejudiced, since four stars can still be seen as a positive rating,” according to the report….

Source: Racial bias exists in five-star ratings for gig workers, study shows. Thumbs up/thumbs down scale would fix that