How Big a Problem Is ‘Review Spam’?

Bonnie Wang from Demandforce riffs on the idea of fake reviews or review spam raised in a recent NY Times article:

“For $5, I will submit two great reviews for your business,” offered one entrepreneur on the help-for-hire site Fiverr, one of a multitude of similar pitches. On another forum, Digital Point, a poster wrote, “I will pay for positive feedback on TripAdvisor.” A Craigslist post proposed this: “If you have an active Yelp account and would like to make very easy money please respond.”

As reviews have become essential to online consumers — and now a factor in search rankings — review fraud or review spam has increased, as suggested by the NYT article. Wang says that Demandforce, which solicits consumer reviews on behalf of clients, uses an Amazon-like certification system to verify to the public that the reviews are legit:

Here at Demandforce, we offer a similar feature called certified reviews (see image on the right), which are reviews written by our clients’ real customers. We’re able to achieve this through our technology that allows us to read the reviewers’ last visit date to our clients’ business.

The fact that Demandforce solicits reviews on behalf of its SMB clients is one of the interesting dimensions of its CRM services.

But I’m interested to hear whether you think “review spam” or review fraud is a major problem or one that exists at the margins?

You can follow any responses to this entry through the comments feed.

26 Responses to “How Big a Problem Is ‘Review Spam’?”

  1. Suresh says at

    I think it’s a growing problem Greg. We have a reviews service in the real estate space with a Verification process similar to Demandforce’s. In real estate the stakes are high, the volume of annual reviews relatively low, and the competition is fierce; we reject around 1 out of 45 reviews due to agents’ attempts to ‘game’ the system.

    The problem’s made worse because the motive for many sites that accept reviews is not quality, they just want to grow and monetize traffic.

  2. Yaro says at

    Just looking at the graphic you have above…

    Under normal circumstances, you would expect a normal/gaussian distribution with very few reviews at 5 and 1 star and most of them clustering in the middle ie reflecting the actual experience of real customers across various businesses. Instead, the distribution is completely inverted.

    Of course, in part this inversion is due to really pissed off people tending to be more likely to go and leave a negative review and thus inflate that part of the curve. But, I find it unlikely that extremely pleased customers are likely to inflate the 5 star part by as much.

    So just based on the data, it’s pretty clear that the problem exists and that it’s creating an unrealistic representation of consumer experiences as reflected by the inverted distribution of reviews.

  3. Greg says at

    Yaro the review snapshot above is for Yelp itself ironically. 

  4. Greg says at

    Suresh:

    One of the interesting things to consider is whether certification/verification systems will need to be more common. Social search or Facebook Connect-based filtering has thus far been one indirect way to deal with “review spam” by prioritizing or ranking Friends’ reviews higher. 

  5. Suresh says at

    Agreed, Facebook Connect is good; however it does impose a hurdle on the reviewer and consumer.

  6. Jim M. says at

    That distro is quite normal though I’d say that the Yelp folks are slightly more negative than those who rate products (we’ve got a database of over 16mm product reviews). We solicit (via our clients) mostly via email survey and so you tend to drown out the pissed off customers and the ballot box stuffers. I think expecting that 3 stars will be the average on a 5 star scale is the same as assuming that consumers make rational decisions.

  7. Greg Sterling says at

    Suresh: Wasn’t suggested that Facebook is a cure for review spam; it’s just one way to filter out some of it indirectly.

  8. Yaro says at

    Jim, if that distribution is normal, then the consumers themselves aren’t leaving reviews that are representative of actual experiences or there’s other biases. Most consumer-business interactions fall in the range of “average” – not exceptional and not poor/negative.

    Perhaps there’s sampling/self-selection bias or the consumers themselves are implicitly interpreting the scale where a “5” is normal-type service.

  9. Mike Blumenthal says at

    Review spam is a growing problem. I am seeing it everywhere from small town insurance agents to car dealers using services that, unlike Demand Force, will key in customer comments from comment cards and make up users. 

  10. Mike says at

    The lack of midrange/average reviews might also be due to folks just assuming average = expected = no need to expend the effort to comment on service as expected; reserving the effort to comment for either exceptionally good or bad service. 

  11. Jim M says at

    What I see a lot is that someone gives a product 4 stars but if you read the review, it feels more like 3 stars. Note that our experience in France is that the average rating is lower. Partly, I think it’s a cultural grade inflation in America. In low review situations there’s room to game the system. Product reviews are more temporal than services in the sense that most products are sunsetted after 3-9 months. Whereas, when my kids’ preschool gets dinged on SavvySource, that’s more or less forever. When that happened, I just encouraged the other parents (the silent positive majority) to speak and drown out the curmudgeons (though the existence of negative reviews solidifies the trust of the review reading masses…so deletion is not the answer). Review volume is the key to drowning out the fakers and manipulators (as well as human moderation and algorithmic signals). If you’ve been in business for more than a year then you’ve got a silent positive majority you need to wake up.

  12. Greg says at

    A concentration of 3-star reviews is not that helpful to the consumer. When all hotels, dentists and others look the same in terms of quality users have to “go somewhere else” for advice and help. Some sort of system like the demandforce approach that certifies reviews are from actual patrons and clients is potentially desirable. However from a search engine point of view it’s challenging to police and implement. 

    Livingsocial is collecting reviews after people use vouchers. In the end this is a problem that will have to be addressed in some way structurally or systemically. 

  13. SH says at

    Greg –

    Is review spam an issue?
    Uh… yeah. It’s probably the biggest issue a review site faces (what else would matter more, site design?)

    Not to be a dork but since you’re using a Yelp rating distribution graph: http://officialblog.yelp.com/2011/08/why-consumers-find-yelp-useful.html

  14. Greg Sterling says at

    SH:

    Question wasn’t whether its AN issue but how pervasive is it. Re the Yelp review graph, I just pulled it as a visual. I wasn’t trying to “dis” Yelp.

  15. SH says at

    @Greg,

    Sorry, I meant to imply that it’s clearly very pervasive and that’s why review sites (Google Places, Yelp, presumably others) implement filtering algorithms in the first place.

    Solicited reviews & bogus “paid” reviews are like the Black Hat SEO wars of search sites.

    IMHO, the review sites that are better at staying a step ahead in this arms race will win, and the lenient/unfiltered sites will lose. Just like how Google’s superior search result filtering keeps them in the lead of search.

  16. Greg says at

    Yelp’s algorithm has been somewhat effective in filtering out spam or fraud but it also catches real reviews and that both mystifies and infuriates people. 

  17. MiriamEllis says at

    This is a good topic, Greg. Unfortunately, I see so much review spam, it starts to make all reviews seem questionable. And this growing lack of credibility is exactly how spammers are shooting themselves in the foot, in my opinion.

  18. SNV says at

    Massive. A recent deep dive survey into the “visitor” posted reviews of a location and category specific section of a major travel site showed that not only are the reviews patently fabricated but so much so as to be obvious.

  19. How Problematic Has Review Spam Become? | CopyLocal says at

    […] Local SEOs in the business, Greg Sterling, has posted and article worthy of your time: How Big A Problem Is ‘Review Spam’. Here at CopyLocal.com, we deal specifically with helping local business owners to respond in a […]

  20. Rich says at

    I saw some real estate sites called http://www.Realcircle.com and zillow.com with most reviews being positive. Where are the bad reviews?

  21. DB says at

    This is somewhat off topic, but many customers are upset about the fact that they have great reviews on Yelp or Angie’s List that they worked hard to procure, but when you visit other sites, Google Places namely (due to the recent review changes). They have to go back and solicit new reviews from clients.  Very frustrating and time consuming, which could then lead to cut and paste reviews or utilizing paid services.  

  22. Greg Sterling says at

    DB:

    Very interesting. Google dropped third party publisher reviews from Places because of controversy around what the publishers felt was Google benefiting from their content.

  23. Garrett R says at

    Demandforce may credential its leads, but my understanding is that the system has a different problem – the owner of the establishment can screen the reviews and choose not to have negative reviews posted. 

  24. Greg Sterling says at

    Garrett:
    I was told that they don’t allow that kind of screening. But perhaps they do. I know their clients do request negative reviews be removed however.

  25. Rich says at

    I think the distribution looks normal, while we would expect the number of reviews in the entire world to be normal theoretically, it’s important to consider that people leave a 5 if they are really impressed and a 1 if they are angry, rarely leaving a review if their experience was average. Equally, a good product normally garners a string of 5* ratings and a bad product/service a string of 1* and the product doesn’t vary much by person, e.g. if you buy a great phone, it is a great phone so it deserves a 5, unless there is a heterogeneous end product for each user.

    Overall however, the major problem is that people don’t review an average service, unless they are hardcore reviewers.

  26. Mario Lovo says at

    As what I have search about this issue, this will give us large amount of negative effect, the term is applied to similar abuses in other media. we should not be afraid in these subject matter, we should be careful before doing step to be act. By the way Yelp has also offer user review where you can also rely on this problem.

  27. Thaniel says at

    Yes but Dreamforce does not allow the public to add reviews. So if your e-mail is not on file with the business AND they don’t decide to e-mail you, your opinion will not be counted. Have a bad experience and plan to never go back to a business. Odds are this will not be captured. That’s why the bulk of the dreamforce reviews are so great. The savy business can control who gets the review. (Make someone unhappy just go into their customer record and delete their e-mail)

Leave a Reply