Have you ever given an Uber driver five stars who didn’t deserve it? If you’ve ever taken any ride-hailing service, the answer is probably yes.
Uber asks riders to give their drivers a rating of one to five stars at the end of each trip. But very few people make use of this full scale. That’s because it’s common knowledge among Uber’s users that drivers need to maintain a certain minimum rating to work, and that leaving anything less than five stars could jeopardize their status.
Drivers are so concerned about their ratings that one Lyft driver in California last year posted a translation of the five-star system in his car, to educate less savvy passengers. Next to four stars he wrote: “This driver sucks, fire him slowly; it does not mean ‘average’ or above ‘average.’” In a tacit acknowledgement of this, Uber said in July that it would make riders add an explanation when they awarded a driver less than five stars.
How did Uber’s ratings become more inflated than grades at Harvard? That’s the topic of a new paper, “Reputation Inflation,” from NYU’s John Horton and Apostolos Filippas, and Collage.com CEO Joseph Golden. The paper argues that online platforms, especially peer-to-peer ones like Uber and Airbnb, are highly susceptible to ratings inflation because, well, it’s uncomfortable for one person to leave another a bad review.
The somewhat more technical way to say this is that there’s a “cost” to leaving negative feedback. That cost can take different forms: It might be that the reviewer fears retaliation, or that he feels guilty doing something that might harm the underperforming worker. If this “cost” increases over time—i.e., the fear or guilt associated with leaving a bad review increases—then the platform is likely to experience ratings inflation.
The paper focuses on an unnamed gig economy platform where people (“employers”) can hire other people (“workers”) to do specific tasks. After a job is completed, employers can leave two different kinds of feedback: “public” feedback that the worker sees, and “private” reviews and ratings that aren’t shown to the worker or other people on the platform. Over the history of the platform, 82% of people have chosen to leave reviews, including a numerical rating on a scale from one to five stars.
In the early days of the platform in 2007, the average worker score was pretty, well, average at 3.74 stars. Over time that changed. The average score rose by 0.53 stars over the course 2007. By May 2016, it had climbed to 4.85 stars.
People were more candid in private. The platform introduced its option to leave private feedback in April 2013. From June 2014 to May 2016, the period studied in the paper, about 15% of employers left “unambiguously bad private feedback” but only 4% gave a public rating of three stars or less. They were also more candid in written comments, possibly because written comments are less directly harmful to the worker than a low numerical score.
Then, in March 2015, the platform decided to release private ratings in batches to workers. In other words, a private review wasn’t totally private anymore, and leaving a negative one could cause harm. The result was immediate: Bad feedback became scarce and imperfect scores were reserved for truly poor experiences. If the trend continued, the authors estimated that the average private rating would be the highest possible score in seven years.
This, again, is similar to what has happened on Uber and other ride-hailing platforms. In the early days, riders left a range of reviews, but it didn’t take long for the default to become five stars, with anything else reserved for extreme cases of hostile conduct or reckless driving. “I took a ride in a car as grimy and musty-smelling as a typical yellow cab,” Jeff Bercovici recalled for Forbes in August 2014. “I only gave the driver three out of five stars. Just kidding. I gave him five stars, of course. What do you think I am, a psychopath?”
Services are different from products. Someone who feels guilty leaving a bad review for another person probably won’t share those concerns about posting a negative review of a toaster. It’s the personal element that gives us pause. A separate, forthcoming study on online reputations found that the number of users leaving negative feedback on a travel review website decreased after hotels started replying to the critiques, despite no change in hotel quality.
The problem is particularly acute on “sharing” economy platforms because companies like Uber, which regard their workers as independent contractors instead of employees, use ratings riders provide to manage their workforces at arm’s length. These ratings systems ask customers to make tough decisions about whether workers are fit to be on the platform, and live with the guilt if they’re not. Put another way: On-demand platforms are offloading their guilt onto you. Five stars for all!