Medals and math V — close calls, clean sweeps, and other countries

A bottle of 8 Wired's 'Wild Feijoa' on the bench out the front of my house
My apparently-now-traditional way to toast the Champion; on the bench outside my house

So. Beer awards, again. (And belatedly, again.) The announcement that entries were open for the 2024 Brewers Guild of NZ Awards, together with the fact I was at the presentation dinner for the Australian competition last week,1 plus the chance to drink a bunch more New Zealand beer than usual at The Catfish recently have all combined to spur me into finally publishing the number-crunching I did for last year’s BGONZAs. As always, there’s some interesting details in here that are easily overlooked if you don’t do a little elementary statistics, and plenty of trends and quirks to keep in mind while anticipating doing it all over again in August.

My method remains the same: I’ve worked out each brewery’s medal percentage (MPC: how many of their beers won a medal of any kind) and their points per entry (PPE: adding 3 for gold, 2 for silver, 1 for bronze, then dividing by number of beers submitted). Bigger numbers = better performances; hopefully both in relatively-intuitive senses like a batting average or golds per capita at the Olympics (as New Zealanders always insist on calculating). 2023’s overall MPC was 68.7% and PPE hit 1.17, both the highest I’ve seen — and for breweries entering ten or more beers (the focus of the table2 below), those rise slightly to 71% and 1.24.

Performance at the 2023 BGONZAs, by breweries entering ten or more beers (or winning a Championship title)

n = number of entries, MPC = medal percentage, PPE = points per entry, G/S/B = individual medals

8 Wired (Overall Champion)51002.20221
Garage Project (Champion Large Brewery)261002.1910115
Three Boys121001.67246
Choice Bros131001.54157
Shining Peak (Champion Small)1794.12.00583
Workshop Brewing (Champion Micro)683.31.83221
Good George1283.31.75352
Beer Baroness1883.31.33258
North End10801.40224
Sprig + Fern2176.21.14169
Rhyme and Reason1172.71.55332
Ballast Point (Champion International)771.41.43131
Double Vision1266.71.00206
Behemoth (Champion Medium Brewery)5263.51.1961710
D.B. (incl. subsidiaries)10601.00123
Three Sisters25561.00275
Deep Creek16501.06332
Fork & Brewer1546.60.60025

8 Wired seemed to get the bulk of the post-awards coverage,3 with the latest Wild Feijoa vintage winning competition-wide ‘Best In Show’ and the brewery earning the title of ‘Overall Champion’ with a record-high PPE (which isn’t what the Guild calls it, but that’s basically what it is) and medals for all their entries along the way. I don’t want to take away from any of those achievements — I’m a huge fan of the brewery to the extent I’d buy it right now if I could, and that Champion beer in particular is enough of a cult favourite among my friends over here in Melbourne that I’ll usually get pinged in a group chat if it goes on tap somewhere — but I do want to draw more attention two things; one that needs fixing, and one that doesn’t.

The problem is the phrasing of that title, “Overall Champion New Zealand Brewery”. Given that four other local breweries are also named “Champion” of their respective production size categories, that’s extremely confusing and strongly implies that 8 Wired have in some sense bested those four “overall” — as in, on the same terms, implying that the others are somehow runners-up of different sizes. I think that’s borne out in the disproportionate coverage. What they’ve actually done is win what used to be called “Champion Exhibitor”, calculated in a completely different way than the other headline awards (which technically compare the scores of each entrant’s top four beers, but usually amount to a simple question of “who had the most gold medals?”). I like this category, I am — obviously, with years of spreadsheets to prove it — an advocate for applauding this “high batting average” sense of success, but it needlessly muddies things to describe the trophy this way. The Guild should revert the title, or come up with something new.4

A can of 8 Wired 'Hopwired' IPA in front of a laptop with a spreadsheet on screen
Spreadsheeting is thirsty work and this classic from the Champion Exhibitor helped a lot (even though it wasn’t in the running, itself)

The other thing is to take a moment to appreciate is just how close it was. 8 Wired’s PPE of 2.20 is the highest I can find in six competitions worth of data,5 but Garage Project’s works out as 2.1923[…], a 0.35% difference, despite entering five times as many beers and the extra “risk” that brings. The Guild’s rules anticipate using trophies as to break a tie, but both breweries here had the same number of those, and I’m left wondering what really counts as a tie. If you’re computing averages, you’ll often get a long tail of decimals and I don’t know how far from whole numbers is too far to be sensible in this context. If you felt that Garage Project “deserved” the “Overall” title, here, there’s a few post hoc changes you could suggest for the rules that would’ve given it to them, like upping the minimum number of entries to qualify.6 But alternate rules would generate their own weird little edge cases some other time. 8 Wired won by the narrowest of margins, but they did win, and their performance was a masterclass in a perfectly praiseworthy way to do well at awards: laser focus and an extremely high degree of self-awareness.

What Garage Project did achieve was pretty incredible anyway. They definitively won Champion Brewery in their size class, and would’ve handsomely beaten everyone else anyway if there were no production-based tiers, or even if the Lion mothership and its better-performing subsidiaries (Emerson’s and Panhead) entered as one unit. This was their third year in a row winning their division, and each year they’ve done it they’ve scored better and better MPC and PPE — now maxing out the former at 100%, with twice as many entries as anyone else who managed a perfect run in this competition, or any previous round. Their oak-aged spontaneous-ferment beer ‘Chance, Luck & Magic’ 2020 won its third consecutive trophy,7 evidently ageing more than gracefully, even as its younger vintages also score well (in New Zealand, and also at the AIBAs here and the World Beer Cup in the USA) — helping cement the Wild Workshop (where GP make their weird ferment and/or barrel aged stuff) as a real force. Indeed, had the Workshop and the Garage contested the awards as separate entities, they would’ve come very close to both being Champions.8

The standard among the variously-sized Champions was a huge improvement on what I observed with the previous year’s competition. Shining Peak9 missed out on a perfect MPC by just one beer, but their result of 94.1% stands in stark contrast to Three Sisters’ run in the “Small” tier in 2022 with only 36.8%. Workshop Brewing10 should also be very proud of the company they’re in on that first page of my table, and they likewise recorded scores that were well above the previous winning “Micro” brewery. Beyond the title-winning Champions, there was a higher number of breweries with MPCs in the A+ range, with Choice Bros and Three Boys notable for their perfect runs, and Heyday worthy of special mention despite falling just outside the scope of my table with 9 medals from 9 entries — thereby scoring 100% two years running, which I believe is unprecedented.

The exception was Behemoth, who again won Champion Medium Brewery, but with a performance that was essentially flat compared to the year before (MPC slightly down, PPE slightly up) — failing to catch the general trend upwards and falling well short of their title-holding peers. Here, they exemplify the other strategy for doing well at beer awards: throw everything at the wall and hope that enough will stick.11 Indeed, if you’d asked me last this time last year, I’d have described that “scattergun” approach and the “precision” path of 8 Wired and Workshop as the two viable strategies; Garage Project and Shining Peak together forged a third way, the “playing on hard mode” of simultaneously doing more and doing better. Admittedly, with their MPC in the 60s, Behemoth aren’t on the level of spam Boston Beer / Sam Adams once wagered on for their serial wins in the “International” category,12 but this was the third consecutive year for them entering by far the most beers in the competition. Since there’s no penalty for an entry that fails, this is a reliable (if expensive) road to getting some silverware to brag about.

A Ballast Point branded pint glass in extreme close-up
A very close examination of a Ballast Point beer, from back when it was definitely imported (and extremely influential on the fledgling NZ “craft” scene)

Which brings me to my final observation. The Guild heralded the reintroduction of an international component to the awards,13 but it looks like only two overseas breweries took them up on the invitation — Ballast Point from the USA (who went on to win, with respectable MPC and PPE), and Vietnam’s Heart of Darkness — and both of those have close commercial (and contract-brewing)14 arrangements with Behemoth. Weirdly, and contrary to other Championship classes,15 beers brewed in NZ under contract for an overseas brand can count towards this award. I can’t think of any good reason why this would be. If Kirin were named “Champion International Brewery” entirely on the basis of beers brewed in Auckland by their NZ subsidiary Lion, I think the weirdness would be obvious. Despite it being relevant to a beer’s eligibility for most Championship classes, the Guild don’t make the contract status of entries public — and declined to when I asked, curious about this result.16 That needs to change. But whether or not they exploited this loophole (to more-easily get beer in better condition in front of the judges), Behemoth will get most of the benefit from the trophy with Ballast Point’s name on it.

More importantly though, two non-local entrants in a field of 79 is grim. By contrast, the AIBAs this year had 157 overseas entrants (including 20 from NZ), more than half the total number of local (i.e., Australian) breweries in the running. Meanwhile, not a single Australian brewery was moved to send their beer to the BGONZAs, despite the physical proximity and the close ties between the two countries.17 It demeaned the awards in the old days when the “International” trophy was handed to Sam Adams year after year despite their truly half-assed performances, and it does them no credit now to pretend there’s any real international component, here, if only a tiny number of overseas breweries put themselves forward — and even those come by way of a local company stuffing the ballot. Unless there’s a dramatic improvement this year, the Guild should knock this idea on the head and accept that this is an event for the local New Zealand industry. That’s a perfectly fine thing to be.

And… that’s enough about a months-old set of numbers. As always, if you notice something else, let me know. Otherwise, I’ve made a list of things to keep an eye out for next time — and maybe I’ll have to see if there’s any fun to be had some day with the much-larger AIBA dataset… [Addendum, 23 July: after a bunch of laborious data entry, I did indeed have some fun with the AIBAs.]

  1. Many thanks to Matt Kirkegaard (him of already-sadly-missed Brews News) for the last-minute ticket at a time where I really needed to reconnect with my industry — and especially some of my favourite people in it (like, well, him; even in “retirement” he still counts so long as he haunts the minds of everyone in this business who is about to do something stupid).
  2. Perpetual thanks to the extremely reliable TablePress plugin.
  3. Dominating industry reporting in New Zealand and Australia, relatively niche business news, and the normal-person news.
  4. The main ‘about the awards’ page on the Guild website just uses “Champion New Zealand Brewing Company”, but the linked entry guide still uses the “Overall” version of the title. Still time to fix it, though. (It also seems they’re determined to switch the acronym to NZBAs instead of BGONZAs, which I will resist as best I can in my own petty little way.)
  5. Well… Brood Fermentation in 2022, Wilderness Brewing in 2019, plus Bootleg and Renaissance in 2017, each managed the nifty trick of submitting a single beer and winning a gold with it for a perfect PPE of 3.0, but multiple entries have always been required, here. Mysteriously, Beer Baroness also scored 2.20 in 2019 (also with 5 beers) but the Champion Exhibitor that year was Liberty on 2.18. I’m not sure why, and assume some now-forgotten quirk of the rules kicked in.
  6. It’s currently 4, across at least 3 style classes. I initially wondered if it should be as high as ten, and lots of breweries regularly surpass that number. But entering each beer costs two hundred dollars (plus the stock itself, plus time to figure out what to enter and in what category), and I now think I’d rather see the category accessible and hotly contested.
  7. I’m still keeping an eye out for any BGONZA beers that would earn an equivalent of the AIBA’s ‘Consistency of Excellence’ honour, and this probably doesn’t quite count since it’s the same beer. It’s a different kind of consistent excellence, though. I was sad to see Lion discontinue Rheineck just when it was on target to the be first. That would’ve freaked out some snobs, though of course it’s possible (probable, even) that they’ve just rebranded “the liquid” (as they say, ugh) and it’s in the results, somewhere, with an underappreciated gold of its own.
  8. Depending on how big the WW’s output is. If they’re “Small”, the exact scores of their best gold-medal-winning beers would’ve been up against those from Shining Peak, and those numbers aren’t made public. But if they’re “Micro” (<50,000L), they’d have taken it easily.
  9. See Jed Soane’s lovely recent profile if the name is (understandably) unfamiliar.
  10. A small brewery in Raglan, not to be confused with the previously-praised Wild Workshop in Wellington. As I noted last time, beer is full of weird little clusters of reused words. It gets confusing (or aggravating) sometimes.
  11. In fairness, at last week’s AIBAs they and Garage Project were both again the Champion Medium and Large breweries (but the “International” ones, this time), and Behemoth’s performance there was comparable to GP’s.
  12. A mere 18% in 2017 — likely the single most-surprising discovery of my initial data-crunching and the best sign I was on to something, here — and a scarcely-better 21.1% in 2018.
  13. The media release says “with the disruption of the past few years behind us, it seemed a good time to reintroduce the International category”, but the disappearance of it was actually pre-pandemic, though I can’t now find any announcement of the original reasoning for ending it.
  14. See this announcement on Brews News, and the fact there are BP and HoD subsections in the “Shop by Brand” menu of their online store (though the latter is currently empty).
  15. i.e., if a local brewery has a beer brewed under contract — in NZ or overseas — it cannot count towards a Championship win. (Though it could for the “Overall” award. It’s a mess and a muddle.)
  16. They do make the packaging format public (which is interesting trivia, if not determinative of anything), so we do know that 6 of 7 BP beers and 3 of 4 from HoD were submitted in keg, not bottle or can. I’m not sure if that suggests local production, but I shouldn’t have to speculate.
  17. The head judge of the NZ awards is Australian, and so is at least one other on the panel; breweries here are sending their staff, but not their product.

One thought on “Medals and math V — close calls, clean sweeps, and other countries”

Have at it: