The Definitive Guide to Scoring Wine

WINE TALK
#1

A carryover from this thread with a few tweaks. I’m interested as to what you guys think:

  1. Wine should be scored out of 100 points. This is (in America, at least), the most intuitive rating system for the wine community to understand.

  2. Scores are meaningless without a tasting note, and vice versa. Joe G at the WLTV forums has expounded at great length on this subject.

  3. Scores are meaningless without context: Wine style, region, grape, vintage, producer, and most importantly: the identity of the scorer. I have a 90 point wine and an 85 point wine. Which do you want? You can’t tell without context!

  4. Composite scores (i.e. 5 points for color, 5 points for finish, etc.) have no meaning. Composite scores are a simplistic linear transformation from a multi-dimensional abstract vector space into a single dimension. This matrix is non-invertible, and so is totally useless to the reader. The individual component scores might be useful, but the sum is useless. To put this another way, when you add all the components together, you collapse a multi-dimensional object into a single dimension, which loses a lot of information. Also the particular method of that collapse (how many points are allocated to each component) may differ from taster to taster.

  5. Scores are ordinal, but not quantitative. Averages have no meaning. Error-bars have no meaning. Scores are not additive, multiplicative, or easily decomposable into causal factors. You cannot do an ANOVA or an MSA. Scores are a qualitative communication tool that give nuanced information within a specific context. Any statistical treatment of the numbers is misguided because they are not measuring anything. Orley Ashenfelter’s work is a prime example of totally misguided statistical analysis. Just because you analyze the sh!t out of the numbers and p<.05 doesn’t mean you have discovered anything meaningful, because the question was a bad one in the first place.

  6. Precision is nice to calculate, but meaningless. You can measure informal repeatability, but you cannot quantitatively get a sense of precision - 89+/-1 point is not the same precision as 84+/-1 point. Repeatability is important, but is just one factor in interpreting scores.

  7. Interpreting scores is difficult and complex. The tasting note, wine type, context, scorer identity, and your own palate all matter enormously. The question of whether you will like a wine and whether the wine is of high quality are very different.

#2

The original post in this thread is the best argument I have ever seen for abolishing the practice of scoring altogether. While I think ratings have some utility and often convey some meaning that the written word doesn’t, they obviously lend themselves to obsessiveness and pseudo-science and don’t deserve to have nearly as much thought put into them as is done here.

Separately, the phrasing “a 90-point wine” is a particular pet peeve of mine. There is no such thing as “a 90-point wine” – as if the points were an actual property of the wine the same way you could refer to “a 10-pound weight” or “a 212-degree pot of water.” The wine is the wine and just because a member of the priestly wine-critic class decided to rate it 90 points doesn’t mean it somehow acquires a materially different essence. I find this whole aspect of the points religion fascinating. A couple dudes on the board or CT can rate a wine and they’re just spouting their opinion, but as soon as Robert Parker or the Wine Spectator decides to speak ex cathedra, suddenly the wine transmogrifies into “a 90-point wine” – a certification it will carry for eternity even as it develops, closes down, reopens, declines, and dies.

#3

I disagree at least to some extent with the above points.

Scores without Tasting Notes are not meaningless. They just mean less. Tasting Notes without scores are often as valuable or more so to me than when coupled with a score. A score can be distracting and misleading.

While the results of advanced (or even basic) statistical analysis is possibly meaningless but definitely pointless, I think average scores with a good number of data points do eventually tell you something about the wine.

#4

No need to get more complex than “50-100, here is the range for good, here is the range for very good, here is the range for outstanding, etc.”

#5

I know that scores and prices are 2 very different things. And both have to be interpreted and valued by the end consumer according to his/her own desires/preferences/budget. But I’ve always felt that there’s a missing link somewhere. Assuming scores are correct and comparable, then what is the basis to judge a $100 wine that scored 90 points vs a $10 wine that scored 90 points? Or how similar is that same $100 wine vs a $10 wine that only scored 88 points (but obviously only costs 10% as much). I know this is the mythical “QPR” that we all talk about. But some nerdy part of me wants to be calculate a numerical and comparable value using both cost and points. But that makes my head hurt, so then the less nerdy part of me justs wants a drink.

#6

I disagree with some of the above, particularly point five. Distributions of scores are meaningful. The fact that there is no common “scorer” or system makes the distribution all the more insightful. Agree that scores without context or notes are less than helpful. Composite scores where the discrete components remain visible are the most helpful, (still requiring context).

#7

I disagree with a few points, especially #5 but need time to formulate a rational response.

#8

P.S. And there are, for the record, valid statistical tests for Ordinal data. Think tests for non-parametrics, often on ranked data. One must be knowledgeable enough to use and interpret the appropriate test.

#9

The ONLY system that matters;

http://www.the-stupids.com/aboutus_3stooges.html" onclick="window.open(this.href);return false;

#10

Since taste is entirely subjective and thereby conveyed inaccurately at best from one person to another, we use a binary scoring system: “good/bad” with prose and metaphor to illustrate our perceptions.

Seems only fair to the wines since they didn’t ask to be quantified by a bunch of wine geeks.

[berserker.gif]

#11

The original post in this thread is the best argument I have ever seen for abolishing the practice of scoring altogether.

[thumbs-up.gif] [welldone.gif] [rofl.gif]

A score is a shorthand way of saying you liked it or you didn’t, and roughly quantifying how much you liked it.

Tasting notes may or may be of utility. If someone says a wine is reductive and actually has no idea what that means, what good is the note? Or if someone says it’s got VA and has no idea what that means, what good is the note? And don’t assume that those things don’t happen. I’ve sat with many people and listened to them opine in exactly that way.

On the other hand, you do know whether they liked the wine or not if they give it a 70 or a 90. So I’d argue that for the most part, the score is much MORE meaningful than the notes from most people.

#12

When I started my blog I decided not to use the 100 point system.
I’d just write notes, and tag the ones I thought deserved them with “Value” and “Recommended”.
I discovered that’s very similar to the way that Rusty scores things in the Pinot File.

The few people that bother to read my blog convinced me to start scoring the wines,
since a tasting note alone doesn’t necessarily convey how much you liked the wine.
(Is ‘concentrated’ good or bad? How about ‘elegant’?)

Since I focus on a very specific area, there aren’t many other people who post notes on the wines I taste.
The two that post the most are Richard Jennings and Wes Barton. We each have our personal preferences,
so there’s some variation, but overall I find that our scores agree more often than not. However to read our
notes you might think we were talking about different wines; we each pick up different characteristics.

There are no ‘90 point wines’, there are only ‘90 point bottles’, or more accurately ‘90 point experiences’.
How a wine shows can vary remarkably depending on outside factors.

You can draw conclusions based on other peoples scores and the variation, especially if you know those
people and the styles of wines that they like. But in the end every bottle, every tasting is different.
Every one can surprise and/or disappoint.

#13

Why can’t it? I find it hard to believe you can’t put into words a sentence describing how much you liked a wine, if that’s what you set out to do.

#14

I think wines should be scored using oridinals from the Fibonacci sequence

#15

I prefer the Zanotti binary system: 0 not worth drinking, 1 worth drinking.
alan

#16

I state it as follows:

I really liked this wine.

I only liked this wine a little bit.

I did not like this wine.

I hated this wine.

But here’s the thing - if someone is going to spend time with a bottle and talk about it and make that bottle the subject of an essay, that’s one thing. If you read that, you get an idea of what was going on. It’s nice writing but it takes a lot on the part of the reader. I tend to write that way and as a result, I rarely post TNs.

If you do a sip and spit and score tasting, like many of the trade tastings I go to, the tasting note is pointless. Just use a number and move on. A minute per glass? What are you really going to say about that? Numbers, letters, stars, any kind of symbol becomes more useful in these cases.

#17

The opening post makes me think of guys who find the joys of baseball in grinding out intricate stats from an entire seasons and arguing about whether Park A added two home runs a season to visiting left-hand hitters while Park B added three, but don’t actually go to see any games.

I’m there for the smell of the grass, the roar of the crowd and the joys and mysteries of the game.

I don’t score wines.

#18

Scores have their uses but far too much emphasis is placed on them.

I think the 100 point system is extremely misleading, even if it attractive at first sight. The score is actually out of 20 and 80 points are added to make it look like a percentage. Similarly the European 20 point scale is really only a 10 point scale (although I personally rate faulty wines under 10).

The misleading part is the precision implied by 100 points. I assume this was first brought in to try to rate one top Bordeaux against another. The problem is that lots of different critics, with different palates and preferences, are using the same scale to score red Bordeaux, Napa Cabernets, Riojas, Barolos, Volnays, Vouvrays and Chilean Sauvignon blancs - which is crazy. It’s like saying “Darkness on the Edge of Town” is a 96 point album and the Cleveland Quartet’s recording of Beethoven’s late quartets is 95 points.

With wine you have the added problem of comparing vintages at different stages in their development.

I think a numerical rating for a wine is useful but I think a single number is, as point 4 says, trying to collapse several dimensions into one. I would like to see critics develop a scoring system that gives a score to attributes of the wine - not the compenents of colour, aroma, palate, finish - but the real attributes which wine lovers look for - correctness, complexity and potential, for example.

The tasting note often tells you this but I want to be able to differentiate quickly between a wine that scores highly becuase it is soft, round and concentrated and one which is tightly structured and complex.

I don;t agree that you can’t average scores. A lot of people value CellarTracker becuase that is essentially what you can do.

#19

It can. I said it doesn’t necessarily, and quoted ‘concentrated’ and ‘elegant’ as descriptors that have very polarising views.
I bet if you took random tasting notes by any major critic you’d have a hard time guessing the scores from the note alone.

#20

I can’t be completely sure, but I think the original post might have been written by the same author as the chapter that was ripped from the book in The Dead Poets Society.

deadhorse