Critics are supposed to help consumers first and foremost. So, “fairness” should be measured primarily using that metric.
Critics need to get paid well and have their reasonable expenses covered. Nothing fosters corruption and conflicts of interests more than struggling to make ends meet. Professionals need to be treated as such.
3.I have said many, many, many times that technical data is essential to the best reviews. “Things like Brix, malic and lactic acid at picking, TA, ABV, RS, some other phenolic ripeness indicators, potassium, TDS, picking dates, wood regimens, etc.” And that we “should pressure paid critics to institute a rule. Published scores are hard-capped at 94 [or its equivalent under the chosen score range] unless the winemaker provides the technical data for publication. AFAIK you can’t chaptalize, acidify, deacidify or water-back a wine without it becoming obvious through the chem numbers. Of course, that doesn’t mean a critic must ding a wine for doing one of those. But the critic should state it was done, even if for good reason. A system like that would cover the critic’s back and would help consumers decide beyond the language of the critic or the score.” That is language I had used before and becomes more essential today. It’s harder to claim a critic is being unduly biased when the technical data is published with the review.
I don’t think pay-by-view is an economically feasible model.
I think the best solution is a nonprofit, perhaps modeled on the golden age of the Consumers Union and Consumer Reports. The members there actually elect its president, somewhat like a cooperative. But regardless of the model, it should have a board made up of consumers and accountable to consumers.
Unlike Consumer Reports, the organization would need to accept freebies from industry, and some level of bridge-building is to be expected. But, a report would be produced disclosing everything that could be construed as a potential conflict of interest (even when innocent or indispensable).
Criteria for scoring should be clear, though, of course, honest subjectivity is unavoidable and perfectly acceptable.
Critics performance should be periodically reviewed by the board with an eye to avoiding score-creep and to keeping scores in line with what knowledgeable consumers are finding.
Critics should endeavor to introduce their palates to readers.
Critics should be periodically available to answer questions from readers.
EDIT: I forgot one thing. One way to potentially ethically monetize it is to do something that both Wine Enthusiast and James Suckling do: sell paraphernalia, wine glasses, etc.
Those are my thoughts on the matter for now, and I request all of your thoughts from your perspectives.
This is a solution in search of a problem It reminds me a lot of “blockchain will solve it” type of posts.
Wine criticism is an art, not a science. Ultimately, find a critic whose palate you like and respect and whose business practices you think are honest and follow them (or a couple of others). Imposing a multitude of criteria is utterly hopeless because the industry is rife with conflict - the only question is how you manage it and how you disclose it. Wine critics get free samples, buy wines from producers, attend dinners, with producers, make wine, accept free things, etc. It runs an entire gamut. How much of this they disclose and how much of it a consumer is willing to put up with is up to that consumer.
For example, since Johan waded into the fray, he is very good at promotion, but really bad on specifics. When pressed on independence he blithely dismissed very basic conflicts as non-issues (we will not live like hermits) and issues vague and prevaricating responses. Now, for some people, perhaps those are sufficient. To me, they’re very obviously not - I have absolutely no clue what equal items of value Johan brings to dinners at Lafite when they pour wines from the 19th century. So I would never subscribe to a publication like that. Similarly, some people on this forum were very unhappy at Galloni’s early notification and pulled their membership. It’s about one’s comfort with the level of conflict management. But it’s all within grey areas.
And, once again, to be clear, I do not think all conflicts are inherently problematic. I do not find free barrel samples or review bottles problematic, I think disclosure cures a lot of ills, etc.
Oh, and one point specifically:
This is just wrong. Critics should not become lab reporters for the benefit of a few people who have taken a WSET course and know some technical terms. There are so many problems with this approach I don’t even know where to start. But a few things:
The average wine consumer couldn’t care less about these. Even the average high end wine consume doesn’t care. A wine review is meant to convey the critic’s opinion of the wine, not the underlying numbers. It’s not an equities report.
Numbers without context are not meaningful. Unless you make wine or are sufficiently technically experienced, knowing the Brix level is not especially useful. Even with sufficient technical knowledge, without the ability to ask enough questions about the winemaking process these numbers will not provide meaningful information. Like my research report comment, there is no “value investing” in wine based on the numbers. That’s not how wine works.
As for the information not being meaningful, a simple example - wood regimen. If a certain Domaine uses 30-50% new oak on a cuvee, that does not necessarily impart meaningful information unless you know the oak regimen. The cooperage, the number of coopers, which barrels went into the final blends, what kind of toasting the barrels had, etc. For example, Mugneret-Gibourg work with at least 4 cooperages on most cuvees and will then blend the final product, so when I visit and ask “what is the oak regimen” they happily tell me its @50% (a bit more iirc), but that piece of information hasn’t actually made me a smarter consumer of the wines. Unless you’re familiar between the toasting on Freres, Billon* and the other coopers they use, having this information is almost entirely academic. It’s surface level information that doesn’t meaningfully inform the reader.
Providing this information would take a lot of time! Consider your favorite wine critic - would their work product be improved if they had to spend extra time acquiring and then providing all this information?
There’s no reason to believe tech sheets will be accurate. That is clear based on Barolo producers who now claim despite all obvious claims to the contrary that they didn’t use barriques in the late 90s. I see tech sheets on the internet for the same wine from different sources that are different, because even winemakers don’t always remember exactly what they did. It’s not an exact science.
There are many other reasons why this is a terrible idea (like the implication that critics should ferret out the perfectly normal practice of chaptalization), but there’s only so much time.
*I don’t recall if they use Billon, just throwing out random tonnelieres.
Thanks for doing this, @Guillermo_M. I think you raise some valid points - I like the idea on being more data driven (point #3), although that’s likely my own bias coming out, as data science and analytics is my profession! That said, beyond a certain number of core metrics, I also don’t mind a degree of ambiguity if that helps protect some level of a winemaker’s own IP (just spit-balling here…)
Isn’t the Consumer Reports model essentially what we have already though (e.g., subscription based like Wine Spectator, et. al.)? In which case I would wonder if the actual business model isn’t broken but rather the degree of oversight and transparency perhaps (to your point #6).
love the blockchain analogy, although I wouldn’t go quite so far as to say what we’re discussing above is a solution looking for a problem but that’s own my opinion.
Really? That’s why I had to pull it from an active thread of people complaining about the problem?
Anyway, I don’t understand why anyone would be a fan of opacity. If you think the information is useless, don’t use it. It’s really that simple. I think it’s useful and the more there is the more useful it will become.
I will answer some of your points.
Yes. It will take time. But it doesn’t have to be entered by the critics themselves. Those are things an office staff can do, or even the producer itself when sending the wines for review, as, for example, Wine Spectator already requests they do for many data points.
From their website:
How do I submit a wine?
Each submission consists of two bottles of wine and a completed Sample Information Form. Please contact the appropriate office to receive the form that corresponds to your wine type/region.
All samples must have U.S. Alcohol and Tobacco Tax and Trade Bureau (TTB)-approved front and back labels on them. We will not accept wines with unfinished labels.
We require—in writing—the suggested retail bottle price and the number of cases produced (and, for wines produced outside the United States, the number of cases imported). This information is printed with each review.
Other information we request if pertinent:
The official release date for the wine if the sample submission is in advance of the wine’s release
The grape varieties used in a blend, if not listed on the label
The percentage of residual sugar, if any, in the wine
We cannot return bottles submitted as tasting samples.
Regarding your skepticism of trustworthiness:
Perhaps some tests could be run on random samples for some data points that are testable. I don’t buy that they don’t remember what they did. And if they don’t, that’s on them. They can still get their wine reviewed but there will be a score cap as I explained. Once a database of information is published, they can’t do what the Barolo producers are doing now and attempting to rewrite history precisely because there is no independent database available.
I dislike straw-man arguments. As to chaptalization, I said.
The critic can say there was good reason and that’s that. Consumers can choose whether they agree or not. What’s the argument for hiding the truth?
All the current models I’m aware of are for profit.
And yes, I’m OK with defining some level of IP/trade secret protection in some fashion. Though if we believe so much in terroir then there shouldn’t be that much to protect.
As Peter Drucker once said, there is nothing so useless as doing something efficiently that doesn’t need to be done at all. You’ve introduced randomized sampling of tech sheet wine review data to be done by to be yet hired staff despite having shown any reason why any of this information is actually useful to a typical consumer.
This is a solution in search of a problem, as I said.
Thanks for starting this thread Guillermo. We spend a lot of time hypothesizing about things here in Internet land and this seems like a reasonable subject to talk about.
I especially like the idea of a not-for profit Consumer Reports high ethics wine review site. To have the conflicts & choices explicitly reported would be a major source of confidence and clarity.
Though, come to think of it, don’t we already have some good non-profit sources? Nobody gets paid to review on this site, or Cellar Tracker. One could do pretty well understanding the wine market with just these two sources. Then there are the free publications that are by definition non-profit and non-revenue. Mosel Fine Wines is an exemplar of such material. Not only is it a free source of reviews, but it is IMHO far and away the best source in English about these wines. I have no idea how these folks support themselves but I applaud their efforts. It sure would be nice if there was more high quality free stuff out there.
But I also am fine with reviewers making a living on their work, and subscribe to a handful of paid sources myself. Threads like this might give them a few things to think about.
Let me give you a non-exhaustive list of examples:
With very limited exceptions, I don’t buy any wine above 14.5% ABV. And to make those exceptions I want to know I’m making them. (Yes TTB allows deviation to the listed number but the allowed deviation in some wines I buy, Italian wine, is much less than TTB allows.)
The amount of RS in wine is a number I will use in every decision to purchase I make of Pradikat Riesling. – For diabetics this is potentially life-saving information.
Many people here, including me, want Barolo made exclusively in botti.
The other day someone was posting about a cheesy note on a Champagne and it turned out it had to do with the toast of the barrel. I bet that person would like to have that information moving forward.
I strongly prefer non-oxidative Champagnes.
Many people prefer no-malo Champagne (which may or may not overlap with the prior).
I definitely prefer no-malo Chardonnays.
I’ve noticed that the TA number for the vintage is strongly correlated to how much I like a Zin (one wine where I’ll make the ABV exception to a point).
Following the above there might come a time that I (and others) will find similar correlations with their palates and thus guide their purchases accordingly.
I’m sure there are many I’m forgetting. And I’m sure others have their own.
On this particular point I agree with Greg (and love the Drucker quote). The tech data is even barely interesting to me, and I’m a pretty big wine nerd.
To me, the tech data requirement is a ‘perfect is the enemy of the good’ requirement, to use another business phrase. This data is far and away the least important part of a wine’s description. The testing, independent verification, and management of same is real work and not cheap. I would prefer that our hypothetical publication spend its very small amount of income on paying the low paid writers a bit more, or paying for travel to visit more regions. These are kinds of trade-offs that any business (for profit or otherwise) always has to make.
This portion of the post exemplifies why I find this entire exercise quixotic. We had the discussion about chaptalizaton, where a number of people (including a winemaker) explained to you that chaptalization is a normal and unremarkable part of the winemaking. In some years some producers will chaptalize, in some years they will not. It is a decision like racking, picking times, etc.
The judge for the consumer is the quality of the wine they are drinking. Neither the producer nor the critic is hiding the truth by not disclosing chaptalization any more than they are any other aspect of winemaking. Your obvious implication is that there are “bad reasons” for chaptalizing, which doesn’t make sense. How is a consumer going to tell if they agree or not with the decision to chaptalize?
If you want to understand how a winemaker makes wine, visit the winemaker and talk to them. If you want to know if a wine is good, read the critic’s review. If you want to analyze the wine, send it to a lab.
The technical data you are requesting does not correspond to this list. You have not thought this through.
Besides, which, of course, I can tell you most of this information (ABV is stated on bottles, for example) without forcing Jon GIlman to become a lab rat.
I wholeheartedly agree about MFW. Jean and David do more for German wine than probably anyone else not actually making wine. More than any other source, their reviews guide my purchases. I have emailed them to thank them for their work several times. As far as I know, it’s a labor of love.
Jeff Leve and Panos Kakaviatos publish great, technically and subjectively informative, and free reviews of Bordeaux and a few other regions. (With the benefit of very different palates.)
Rusty Gaffney’s Prince of Pinot is outstanding and also free (but less comprehensive because he was dialing back in the last few years). Tom Lee does great Zin reviews. Also free. But those last two examples, while great, are not nearly as comprehensive as the first three.
Regardless, they are the exception. And sure, consumers could take a deep dive into WB and find stuff, but that takes a level of commitment that, while we give it, may be too much to expect from most. And, they could stop at any time. And when they do, their sites might go offline and the information lost. I’d like something a little more institutionalized.
Listen. Let’s say I don’t want to buy chaptalized wine. It may be a perfectly fine practice. But if I don’t want to buy it then I don’t want to buy it.
Why do some people only buy organic wines? Some wines that aren’t organic have better quality than some that are, but the consumer can make that decision.
Why do some want only dry-farmed, no irrigation? Because they do.
Stop trying to dictate what I and others should care about in making purchase decisions with our own money.
ABV is stated on bottles but as you know, or should know, many times we don’t have the bottle when making the decision to purchase.
And what do you mean the technical data does not correspond to that list? Wood regimen, residual sugar, ABV, total acidity, malo, etc are all on the data I’m requesting (there are three links there you should follow).
If the random testing and independent verification are too expensive we could scale it back. Producers who are proud of their wines are probably glad to tell the truth. And those who aren’t proud would probably not get their wines highly rated anyway so they would face no consequences from withholding the data. I think a lot of the data entry can be done by the producer (just like anyone can add a wine in CT) so the cost may not be as high. But it’s certainly something to discuss.
Now you’re thinking like a product manager (trade offs, ideas on how to solve problems and alleviate costs). Keep going down this path and next thing you know you’ll be saying ‘heck, I’m going to do this.’ I look forward to being a subscriber!
My two cents is that, first, the reviewer needs to actually be able to make money at it (aside from the Prince of Pinot hobby), and two, it can’t be forgotten that the reality is that the most useful review/score to the vast majority of the wine industry taken as a whole is one that can be put on a shelf talker to move product.
Fascinating and not much time to add too much here but a few things:
Chaptalization is not legal in CA at all. Does it happen? Perhaps, but not legal and therefore no winery will state they do this (though of course adding grape concentrate is legal).
In Europe, you can chaptalize but cannot chaptalize and add acid to the same juice . . . but if you do add sugar to the must, you CAN add acid . . . to the wine.
Oak regiments are a rabbit hole - so many wineries state that they use little new oak - but may use a majority of 1 or 2 year old barrels, which still pack quite a bit of ‘oak punch’. Defining ‘neutral’ will be challenging.
More later but interesting . . … and I didn’t think people on this board cared about scores?
Definitely. Knowing exactly how much residual sugar is in that Spätlese is definitely going to be the difference between DKA and a blood sugar of 112.
I get that you are passionate about having access to all the technical aspects of any wine you are thinking about purchasing, and you’re doing your level best to convince the rest of us how grand it would be to have critics reporting all of this, but the diabetic argument…
love the blockchain analogy, although I wouldn’t go quite so far as to say what we’re discussing above is a solution looking for a problem but that’s own my opinion.
