tl:dr The existing flaws are dizzying and likely beyond rescue.
I’ll preface this post by saying that I have no new hard evidence to present here. There’s no conclusive DNA test, no bloody murder weapon, and no one caught red-handed with 12 kilos of heroin shoved up their butts or however that works.
As is the case with so many unchallenged crimes and injustices these days, much of the evidence that hotel user-generated reviews are of little or no value is in plain sight. And after so much time right in our faces, pointing out this conspicuous failing has little leap-into-action effect. It’s like shouting “that guy cut in line!” at the movies when everyone can clearly see the guy cutting in line. Most people think “Eh, I already knew about that guy cutting in line. Maybe an assistant manager will intervene. Next controversy, please.”
However, I have recent personal experience that, for the umpteenth time, reinforced exactly why the current system is all but hopeless. So, even though this probably isn’t breaking news to anyone, here I go nonetheless.
I am, of course, biased. I was a travel writer for 13 years. I’ve written (or edited/updated) hundreds if not thousands of hotel reviews, mainly short, tidy blurbs for Lonely Planet guidebooks, but also longer, in-depth reviews for magazines and consumer traveler websites. I’m all too aware that writing accurate, sober and useful reviews based on firsthand experience, paired with a full understanding of what should be reasonably expected from a hotel stay takes a fair amount of experience. Thus, I was skeptical of promotion and marketing on the strength of user-generated reviews from the outset.
In the early days, before it became a game of three-dimensional chess with bluffing, the problem was that hotel reviews written by professionals were being given the same weight as reviews written by “the beach was too sandy” caliber of package vacation idiots.
When the “I don’t need to buy guidebooks anymore because I have the internet” people started talking trash, one of their arguments against guidebooks was that they could find information about hotels for free. I wrote far too many words on this very topic more than a decade ago.
“Here’s why I take online hostel reviews with a grain of salt: of the 80 people that wrote those reviews, 25 were dimwit travel first-timers (not that there’s anything wrong with that, we’ve all been there, but I don’t trust them to write a dependable hostel review), 15 were partying drunks, 10 were unreasonable assholes and five were straight up schizophrenics. Also, a few of the most glowing reviews were probably written by the hostel owner or employees (and a few of the most negative by the competition across the street).
So, out of 80-odd reviews I have to assume only about 20 people have the capacity to write a fair hostel review, and of that 20, probably only 10 are looking for the same things I’m looking for in a hostel. All the rest are just skewing the results of that quick-glance rating system those web sites prominently feature.”
The only thing that’s changed about this situation in the past 11 years is we’re now aware that even more hotels than we previously thought are stuffing the box with fake positive reviews, while firing off reviews trashing the hotel down the street.
Another facet of user-generated reviews that not only hasn’t changed, but gotten worse in the past decade is human nature and the easy, go-to option of internet vengeance. People were then, and still are, 10,345% more likely to go online to write a review (or tweet or post on Facebook) when they are super pissed off about something. At least those results-skewing reviews are legit, if not entirely fair. At the other end of the spectrum, the hysterically positive reviews don’t do everyday travelers much good either, especially when they’re written while the reviewer is still staying at the hotel, incentive-ized by a free breakfast. The segment we need to hear from – sane, unbiased people who just enjoyed an unremarkable stay – is vanishingly rare.
When hotels shouted enough about having no recourse to negative, fake reviews, travel websites responded by starting review moderation, which successfully made writing fake negative reviews slightly more time consuming. But they also went over the line into policing legitimate negative reviews. Then Booking.com went way over the line, making it delightfully easy for hotels to have unchallenged veto power over negative reviews.
Booking only allows people who stayed at the hotel in question, reserved through their site, obvs, to leave a review. Sounds reasonable, right? But as I learned earlier this year, hotel owners can easily abuse this “feature” to block negative reviews by simply going into their dashboard and marking the guest as a no-show. Now the aggrieved guest can’t leave a review because they (allegedly) never stayed at the hotel.
After a disastrous stay at what turned out to be an illegal hotel run by grifters called Barracuda Apartments in Playa Samara, Costa Rica, I submitted a review of my experience on Booking, but it was rejected, because the shitface owner marked us as a no-show. Despite having stayed there three days, with a receipt to prove it, Booking said it was impossible to publish my review, because the owner said we never stayed there.
When I asked Booking about what, if any, recourse I had about this unbelievably one-sided approach to review moderation, they said “leave a review at TripAdvisor.” So, be advised that Booking recommends you use TripAdvisor if you’re seeking (ostensibly) more accurate user-generated reviews. Clearly, these are not the Stephen Hawkings of the hotel review sites world. More like the Stephen Millers.
As this was possibly the worst hotel experience I’d had since becoming a travel writer (that didn’t involve bed bugs), and still determined to have my voice heard, I went to TripAdvisor as directed and left a review about Barracuda Apartments. After three weeks the review disappeared. In several email exchanges with TripAdvisor’s team of moderators, who have the unenviable task of interpreting reviews that are frequently not in their native languages, I never got a proper explanation for why my review was taken down.
Anyone who’s gone through this maddening exercise is likely familiar with the canned TripAdvisor response to disputed reviews: “We use an automated filtering system that pulls aside reviews that may require special attention or which are flagged as possibly violating our review guidelines.” And when one inquires about why exactly their review was deleted, so they can have another shot at writing a review that stays posted, “…our process is proprietary to our business, unfortunately I can’t share any additional details about why your review was filtered.”
Allow me to run that bullshit response through the Truth-O-Meter:
“We have no fucking idea what we’re doing. It’s completely random. We lost control of our moderation system within months of launching it and we’ve spent the past 19 years trying to cover it up. We can’t admit that, obviously, because then people will know that our reviews are essentially useless and they might (but probably not, because who are we kidding?) give their business to our competitors.”
Though, with Booking happily sending their customers to TripAdvisor, I don’t think TripAdvisor needs to worry about losing customers – at least not to Booking.
The issues of arbitrarily moderated, user-generated review sites are now, and have always been, damaging for both travelers and proprietors. Experienced travelers have little faith that reviews are trustworthy, especially with stories like “My derelict pub still got TripAdvisor reviews,” “One in three TripAdvisor reviews are FAKE” and “I Made My Shed the Top-Rated Restaurant on TripAdvisor” coming out on a regular basis. Equally, proprietors have to sink unnecessary time and resources into combating fake negative reviews, never mind the wildly unfair ones from people who lose their shit when Wonder Bread and Pop Tarts aren’t on the breakfast table in Sicily.
Though getting a reply from TripAdvisor that isn’t part of their “Customer Service Corporate Babble Copy/Paste Replies That You Better Not Edit or Amend Or You’re Fired Handbook” is virtually unheard of, they’re apparently aware that they could eventually start losing revenue due to their zero transparency problem.
The torrent of critical articles about fake reviews and accusations of “’hugely suspicious’ patterns of comments from contributors” by the consumer group Which? (work that, among other things, resulted in one hotel in Jordan being stripped of 730 five-star reviews!), prompted TripAdvisor to release its first “transparency report” in their 19 years online. Among the information it divulged, was that they supposedly intercepted one million fake reviews among the 66 million reviews that were submitted in 2018.
In fact, the report was little more than a long-form version of the useless, canned emails they send to abused vacationers. It explained, at length, that all their submitted reviews are filtered through their proprietary fraud detection technology, and those that are flagged, reportedly 2.7 million reviews in 2018 alone, are then picked over by their moderators. I’m having a difficult time believing that 2.7 million reviews can be thoughtfully critiqued by what I assume is a beleaguered team of moderators in India and the Philippines earning something like $12 a day.
The report further claims that “79% of the reviews were assessed within six hours of being posted, and only 4.7% of reviews were rejected before or after the content had been posted.” Then they pivot to defending themselves against the big meanies pointing out the many flaws in their moderation process. “These third parties do not have access to the key technical data necessary to determine whether or not a review is fraudulent. We do – and with this report, for the first time ever, we want to provide definitive insights into the details and data behind our extensive content moderation efforts.”
Well, third parties don’t have access to the key technical data behind my extensive hatred of pickles, but I still have to take shit from people who I have to assume have only three functioning taste buds and a deviated septum. So, if TripAdvisor doesn’t like people pointing out the plainly obvious defects in their moderation system, they can take their ball, go home and let a new player take on the effort of hosting hotel reviews that are actually of use to prospective travelers and fair to proprietors. The closest thing we have to that at the moment is Oyster.
You may be thinking to yourself, “But Leif, with all that’s going on in the world right now, is this really an issue worthy of 2,200 words? What’s the point in moaning about this when, as you’ve suggested, we’ll all be using bug-based protein bars as currency in 30 years?”
This is true. I’ll be shocked if leisure travel still even exists on a grand scale 30 years from now, never mind weeding through the crossfire of fake reviews by feuding B&Bs. But in the meantime, people are being screwed, lied to and fucked with. This subterfuge and denial is separating us from our hard-earned money in the precious few decades we have left before fuck knows what happens to society.
There’s already not enough enjoyment in life these days without having to deal with strategically dishonest travel reviews ruining our vacations – and sometimes the revenue of perfectly good hotels. The travel industry needs to shine a light on this issue right now, perhaps during breaks from outfitting a bulletproof, gun-mounted, Mad Max death truck for safe passage to/from grandma’s house three towns over on MinWisWa Independence Day—or whatever Minnesota, Wisconsin and Iowa decide to call themselves when they form a new country after America splinters in 2048.