Why Amateur Wine Scores Are Every Bit As Good As Professionals'

Interesting piece in Vox today – a couple of social scientists compared Cellartracker reviews to the professional ones… and found a reasonable amount of correlation. Since folks on this board post here and on CellarTracker, I figured it could make for some interesting discussion.

(Disclaimer: The editor of this story (not the authors) is a former colleague of mine, which is how I learned about this piece.)

I didn’t read closely but it seems to have a major methodological flaw. The observed correlation doesn’t mean much if you don’t account for the professional scores influencing the Cellartracker scores. If you were going to do the experiment in a lab, you would make sure that the amateur raters weren’t aware of the professional rating when they did their own evaluation. If you’re using Cellartracker data, you would probably want to limit it to ratings that predate the professional ratings. Otherwise, it may be that all you have shown is that people are sheep who are highly susceptible to the influence of self-professed experts.

Some brilliant mind just made this same point

ha! just saw the other thread (looks like we posted within minutes of each other).

Smart DC area lawyers think alike? Who knew?

Keith, you featured prominently in the piece:

The '99 Testarossa Chardonnay outlier is your doing

I find there is similar overlap between amateur and professional astrologers and phrenologists.

Now that just made my day!

And there’s another flaw in the whole exercise. My note was made when the wine was 10 years old. I don’t know when the WA’s note was from, but presumably the time of release. So we’re evaluating two totally different wines, unless you buy into the fallacy that points are some objective and constant attribute baked into a wine like DNA, such that an 88-point wine is an 88-point wine from the moment it’s scored until the end of time.

I’ve always thought that CT notes are often a regurgitation of professional scores and notes, aided by a thesaurus to change the adjectives.