“Guinness isn’t the highest rated stout anymore,” declared a headline in The Drinks Business.¹ “New data,” they said, reveals “which brand comes out on top.” I soon spotted that the source of their fresh numbers was Untappd, the beer logging and rating app — hardly a representative sample² — so I knew this wasn’t going to be rigorous. But it’s always fun to poke around with these things, so I took a closer look. I didn’t expect things to get so weird and worrying.

The headline immediately invites the question: was it ever the “highest rated” stout — on Untappd, specifically? Oddly, the article never actually gets around to saying so. Some time after publication — and a few raised eyebrows on social media — the title was amended,³ quietly dropping that claim but proceeding otherwise as before.

Guinness is certainly the most rated stout on the platform, with orders of magnitude more check-ins than anything else discussed in the piece.⁴ I asked Untappd if they knew of any time it had the #1 spot by average rating. They observed that since it was one of the first thousand beers added to the system, it was “highly likely” it was “top-rated in [its] category at some point” but they didn’t have any “historical ‘snapshot’ data” so there was no way to “determine if/when this may have occurred.” Given the userbase, my money would be on it being a status held briefly and long ago (if ever) but we just don’t know.

Untappd also told me they had nothing to do with the “analysis” behind the article. So I spent a few minutes with the Top Rated section of the website, just as whoever put together this report presumably did, tried to recreate their dataset,⁵ and it looks like they’ve only included stouts from the U.K. & Ireland.⁶ TDB is a London-based publication, so maybe skipping the American breweries that dominate the worldwide rankings makes sense. But they don’t say that’s what they’re doing.

There’s a similar not-quite-explicit filter at work when they claim “the analysis shows that, rather than pushing higher alcohol or novelty flavours, most of the best-rated stouts sit between 4.4% ABV and 4.9% ABV.” That’s only true of the Irish Dry Stout subcategory specifically, almost by definition — insanely high-strength beers take the top spots for every other kind of stout, as indeed they often do across Untappd for other styles. But the article slips a little in its focus between stouts more generally and those like Guinness.

Enough oddities accumulated that I wanted to ask more about the actual thinking and motivation behind the whole thing. Lengthy quotes (about 40% of the whole text) were attributed to the piece’s only interviewee, the owner of a light industrial firm somewhat adjacent to the brewing industry. Which was itself another oddity, but who am I to judge, when I do deep dives into beer awards data and tally up relevant references in the crossword from here on my couch? So I emailed him.

He was surprised, and frustrated. He didn’t know the article existed, was never interviewed for it or anything like it, and didn’t endorse any of the ‘insights’ bearing his name. He just “enjoy[s] a pint of the black stuff occasionally at the weekend,” like a normal person. He traced the cause to the marketing firm his company used, and “asked for it to be removed.”

But TDB didn’t take it down. They simply switched the name on all those quotes (but missed one),⁷ re-assigning them to a different man at a different light industrial firm, also somewhat adjacent to the brewing industry. No note about an edit, no softening of the evidently misleading phrasing like “speaking to TDB…” For completeness, I did reach out to the new puppeteered person to see what they thought about it all, but didn’t hear back.

Someone from that marketing firm — Add People, in Manchester — did get in touch, referenced the analysis as “our data,” and said they were “happy to answer [my] questions fully,” which was nice. But first they wanted to know if I was asking out of personal curiosity or for “any particular publication” so I explained that I might write something up here, and… then they ghosted me. That was a month ago.

And while TDB do run “partner content” branded as such,⁸ I’ve never seen such a tag attached to the ‘Highest Rated Stouts’ piece. Add People are obviously getting paid to put the list together and fabricate the quotes but I’m unclear whether they are in turn paying to get their client’s name on TDB’s website — and then suggest a backup client, if the first one isn’t happy. Honestly, I kind of hope TDB are getting paid for the placement. This isn’t worth the compromises otherwise, judged as “content.”

The following week — just a I was hearing that the original article’s main character was an unwilling ventriloquist’s dummy — TDB ran an interview with The Kernel Brewery’s founder by Jessica Mason, who also has the byline on the Guinness piece. It’s much more interesting, and was shared around a bit, but I realised I was vaguely wondering if it ever happened.

TDB were willing to fudge the idea of “speaking to” someone before, after all. That “top 26” stouts list — which gets linked to — was all but rigged to make The Kernel dominate it, and here they are featured again. Is this all an ongoing campaign of elaborately orchestrated fiction? I mean… probably not. But this is the corrosive problem of breaching trust and cutting corners.

A lot of folks are (rightly) concerned about — and pushing back against — machine-generated slop. There are still plenty of people who go to work and make it by hand.

---

1. Who style themselves as “the drinks business” — without the capital letters — for some reason. But that just makes to hard to tell if you’re referring to the publication or emphatically saying something about the beverage industry as a whole. Which is a thing I like to do sometimes. So I’m not going to follow their lead. Also, if you clicked through and noticed that the seventh word is missing; we’ll get to that.
2. See also @pilsnerish for collections of absolutely bonkers idiosyncratic approaches ratings (e.g., here and here).
3. The original is preserved in the blessed Wayback Machine, and indeed still reflected in the URL. We’ll have cause to come back to it again soon.
4. At time of writing, around a million — though now that I notice the disparity between the website and the app’s totals, I have another reason to worry about their data .
5. For transparency’s sake, here’s mine. I collected the top-rated beers in the “Stout — Irish Dry” category (which were almost all American) and added the top ten from England, Scotland, Wales, Northern Ireland, and the Republic of Ireland. For extra context, I also grabbed the top-rated beer in the other “Stout” subcategories.
6. At least, I hope they didn’t forget about the Republic of Ireland. The only way to get something resembling the “top 26″(‽) they describe — or the “top 10” actually in the article — was to exclude it from my data. Which would be madness, since that’s where Guinness comes from. Let’s provisionally assume that Untappd’s data shifted between their snapshot and mine.
7. Again, the original text is archived in the marvellous Wayback Machine.
8. But come on; ditch the weasel word. Say it’s “sponsored” or stop doing it, if you’re embarrassed.