Review Inflation: What Happens When Everyone Has 5 Stars?

5 stars

There’s plenty of data that show consumers rely heavily on ratings and reviews to help them make purchase decisions. Dozens of surveys and studies affirm this.

Businesses are starting to become much more sophisticated about online reviews as well. They understand the importance of these consumer trends and are increasingly asking customers provide feedback and review them on Yelp, TripAdvisor and other key local sites.

Tools such as GetFiveStars, Refgo and Customer Lobby help local businesses obtain and syndicate consumer reviews. Amazon and vertical sites such as OpenTable and Hotels.com solicit reviews from verified buyers.

But what happens when the tools and growing sophistication of local businesses results in what might be called “review inflation”?

Five star reviews

This morning I was doing research on local electricians (for myself) and came across the above page on Yelp. There were at least 10 listings that had five stars and a comparable number of reviews. I had to go to the second page before finding an electrician that wasn’t as highly rated.

Either these are all great local businesses or these businesses have gotten very good at generating positive reviews — or both. I’m not suggesting that these reviews are fraudulent. However Yelp becomes less useful when there’s less variation among ratings.

A user has to spend a lot more time reading reviews in depth or go to “secondary criteria” or filters to make a decision. Of course I could pick three electricians, call and get price quotes, etc. That’s the “old school” approach.

This “review inflation” as I’m calling it won’t happen across all business categories. But it will likely be more common over time. The best and most digitally savvy local businesses will have mostly positive reviews, making it hard for consumers to use sites like Yelp to differentiate them quickly.

This becomes a potentially bigger problem for mobile users unless proximity becomes the chief variable after business quality.

What are your thoughts on this? Do you believe:

  • Over time more businesses will be favorably reviewed as I’ve suggested?
  • If that happens, it will diminish the value of review sites like Yelp?
  • That more “selectively solicited” reviews will start to game the system?

You can follow any responses to this entry through the comments feed.

25 Responses to “Review Inflation: What Happens When Everyone Has 5 Stars?”

  1. Tom Lipscomb says at

    So today…  Reviews are used a lot.  Reviews influence buyers.  Reviews are part of the process by which we procure products and services.  They are the new “Word of Mouth” and they are not going away.  
    As a company we look at tens of thousands of reviews, and you can see both small businesses and large brands that obviously are trying to game the system, but these people will ultimately fail.  
    I think this will lead to a consolidation of review sites based on their ability to screen reviews.  It is more common that the sophisticated review sites are validating that customers are real through credit card transactions, and a host of advanced algorithms.  I see reviews that pop up and disappear all the time that are obviously bogus (note: not always by the business, often someone or some organization trying to tarnish or discredit a brand).
    The ecosystem of reviews will continue to evolve and if the sensitivity of the standard 5 Star Likert Scale we use today ever loses it’s ability to be sensitive enough to differentiate, it will evolve.
    Digital dimensions of commerce (such as reviews) influence literally trillions in sales revenues.  If today’s Five Star System become inadequate someone will build a better one.  In reality the ultimate demise of todays ratings and reviews system will be the result of someone building a better one, because both the value and adoption of this market mechanism is way past critical mass! 

  2. Greg Sterling says at

    Agree that verified purchasers will take over as booking/payments in local take hold broadly. Think Yelp will ultimately go down this path as transactions become more prevalent.

  3. Tom says at

    Verified purchasers are not necessarily more trustworthy.

  4. Scott Barnett says at

    Greg, 

    A few items are at play here:

    (1) You’re in a fairly dense area and also in the Bay Area where Yelp is fairly ubiquitous. Imagine being in a suburb in Indiana where they might be 5 Yelp reviews total. 

    (2) There needs to be “context” around what people are reviewing. 5 stars isn’t enough to distinguish the nuances of a business. 

    (3) I’m assuming the reviewers in your example are all anonymous to you. What if you knew some of the people giving the review – would that matter/help?

    (4) What about the responsiveness of the business owner? Would somebody who interacted with their customers influence you?

    Did you look at anything else other than Yelp? Angie’s List, G+, etc? I’m guessing they suffer from the same issue(s), but don’t want to assume…

  5. Tony Leary says at

    Hey Greg, excellent observation. I’ve recently found myself pondering the effectiveness of 5 star ratings as well. I wrote a little rant on my personal blog that “5 Star Ratings are Bullshit”.

    For quite some time, I would usually give a rating of 4 stars. On almost everything. For someone to get 5 stars from me, I would have to have been thoroughly impressed or in a really good mood. 2 and 3 stars required way too much thought and 1 star would have to be down right despicable service with no effort for remediation.

    But recently with a few services like Pandora and thumbs up or down, I think that’s suitable for most everything. It really simplifies it to “did you like this, yes or no?”. Then an average can be calculated to give people a percentage of how much a service is liked. That’s it.

  6. Street Fight Daily: Microsoft’s Square Competitor, Moovit Raises $50M | Street Fight says at

    […] Inflation: What Happens When Everyone Has 5 Stars? (Screenwerk) Greg Sterling: There’s plenty of data that show consumers rely heavily on ratings and reviews to […]

  7. Greg Sterling says at

    Clearly in areas with less review density this “problem” won’t exist. Yes, agree with your “2” — that’s the point: when the high level criteria/evaluation becomes ineffective you need to turn to other things but they need to be available “at a glance” to accommodate the mobile use case. 3: this is the “Facebook advantage” that has yet to be fully deployed or exploited in a local product. Others have tried to “go there” w/o success in the past.

  8. Greg Sterling says at

    Foursquare uses a numerical rating system that is derived from an algorithm I believe.People need both quantitative and qualitative information, though it depends on the category.

  9. Malcolm Lewis says at

    The answer to review inflation is trust. One review from my friend Greg Sterling is worth 50 reviews from people I don’t know or trust. Review sites like Yelp need to evolve so that users can search for the subset of all businesses that are those recommended by people they know and trust. Of course, I’m biased.

  10. OHenry says at

    If  only there was a way to give this post six stars…

  11. Greg Sterling says at

    Soon there will be 🙂

  12. Greg Sterling says at

    Indeed. However my taste and yours may not always overlap. In theory any review site can achieve this through Facebook Connect (or whatever it’s called now).

  13. Malcolm Lewis says at

    Agreed re taste, but since I know you, I’ll have a good idea if we share the same tastes. And while taste is important for some local businesses, like restaurants, where the experience is more subjective, it’s less so for services like a handyman or plumber. Regardless, since I know you, I can call you up and get more info on your experience. 

    Agree FB makes it somewhat possible, but it hasn’t really happened. I feel there is an opportunity for someone new to come into the space and lead with “recommendations and reviews from people you know and trust.” And trust is not necessarily based on friendship. I might trust restaurant recommendations from a stranger who appears to like the majority or restaurants I like, and dislike the majority of restaurants I dislike.

  14. Greg Sterling says at

    I agree. It’s tough because you need that social data and very few have that or have the capacity to generate it at scale. It’s the “cold start” problem.

  15. Ted Paff says at

    Over time (and this will take a while in some markets) very good reviews will be far more common among the local leaders in non-“experiential” industries.  This will greatly diminish the utility of Yelp in the use-case you described: to winnow a list of potential service providers based on rating.  

    This is not a function of review fraud or gaming the system or even selective solicitation.  Review inflation is a function of the tendencies toward: i) a more representative sample population over time and ii) binary grading habits of most reviewers (i.e. 1 or 5 stars). Selective  solicitation is much less of a driver here than you might imagine. 

  16. Greg Sterling says at

    Ted: What do you think the future holds if more 5-star reviews occur?

  17. Ted Paff says at

    @Greg: Big question 🙂

    I think you and Malcolm are right in that a new reviews model will emerge but those models will not get around that underlying truth which is that most people are satisfied with the service they received most of the time (again, in non-experiential businesses).  I think consumers will use the content more than the ratings. 

  18. Greg Sterling says at

    All this much be capable of being “consumed” on mobile devices, which makes long-form content a challenge.

  19. Danial Ahchow says at

    Greg, I find it interesting that you were still using the old school “Directory” style of site to find a local business. Given all of the reporting you do in this space, I would have thought that you would have used a more modern approach such as Thumbtack, Red Beacon, etc.

    Thoughts? Would you consider yourself typical of most consumers on this front? What other methods did you use to find (and eventually hire) an electrician?

  20. Greg Sterling says at

    Have used Thumbtack in the past and found experience unsatisfactory. Same with RedBeacon. Yelp’s content still superior in most cases to others, notwithstanding this article.

  21. Scott Barnett says at

    Danial,

    I’m curious, what makes you say that sites like Thumbtack / RedBeacon are more modern than a Directory listing site? What makes it more modern?

  22. Greg Sterling says at

    To the extend that e-commerce or transactions are involved vs. pure listings I suppose.

  23. Chuck says at

    If people don’t get five star service, I expect lower ratings to show up over time.

  24. Nathan Payne says at

    I think over time the overall “best” service will float to the top and consistently have 5 star ratings. I would prefer a more tailored search and rating through. 

    For example if I am searching for a specific type of plumber service. I want to know out of those 75 reviews which were for installing a boiler, or bath etc. 

    I would doubt the “best”, or should I say top rated, companies are honestly the top rated for every service they provide… 

  25. David Dacromi says at

    Reviews have become a bit of a cornerstone in social proof and now ultimately are a factor in rankings. It’s impossible to police correctly, but the larger companies effectively have a monopoly on reviews while they have the budget to inflate them, leading to a slightly unfair environment for newer local businesses. And until a true intelligent algorithm is developed for the platforms unfortunately they always be gamed.

  26. Greg Sterling says at

    Smaller companies/SMBs are in some ways in a stronger position. The issue is “review acquisition.” How do they ethically “ask” for reviews and keep them coming.

Leave a Reply