Should critics taste blind?

[WvanGorp]

@Daniel Rogov

First of all, I am much in favor of blind tasting and that, as is said, whenever possible and under certain circumstances. Whenever possible means largely when wines are sampled in one’s own tasting room. True, some 50% of all wines that many critics taste are tasted at trade exhibitions (where it sometimes possible to taste blind and sometimes not), when visiting wineries (when it is almost impossible to set up blind tastings), and when visiting a wine consortium (again sometimes possible and sometimes not to taste blind).

…

Some claim that not only should we critics always taste blind but that we should not allow ourselves to receive samples from producers. Now being perfectly honest, there is not an individual critic on this planet who has the budget to buy all of the wines that need to be tasted. That would mean that instead of tasting 5,000 or more wines a year, most critics would be limited to tasting 250-500 wines. Not exactly what we expect of a professional critic. That also means that we should not be tasting at wineries or at exhibitions. And that in turn means no barrel tastings and no advance tasting notes.

…

Questions, comments and objections will be most welcome.

Best
Rogov

Excellent points and thanks for chiming in. Although your point about the expense of tasting blind being prohibitive in cost to a critic makes perfect sense and is logical, Bob Parker states that he buys about 70% of the wines he tastes outside wineries: http://dat.erobertparker.com/bboard/showpost.php?p=1227677&postcount=12" onclick="window.open(this.href);return false; so apparently one critic feels the cost isn’t prohibitive.

[Daniel_Posner]

@Max Marinucci

Great post Daniel and it all makes perfect sense.

You are welcome.

Wilfred,

Thank you for finding that quote from Parker. Joan Passman, his assistant of 25 years, spoke otherwise in the New York Times. Parker said she did not know what she was talking about, and blamed the New York Times for bad journalism. Shocking that he would do that.

[Bruce_G]

@Daniel Posner

Bruce,

What % would you put on the term, “whenever possible?” Because I am sure the courts in the US have decided somewhere that “whenever possible” has a % attached to it.

Would any number that I put forth have any meaning here, Daniel?
I doubt it.

And I think any attempt to quantify would miss the (to my mind) greater point: Mr. Parker’s statement is at heart an admission that his methodology diverges from what is best, and invites bias and inconsistency.

[Bruce_G]

@GregT

3. Peer should mean peer. So tasting red Bordeaux with red Bordeaux makes sense, tasting nebbiolo with nebbiolo makes sense, tasting sauvignon blanc from one vintage right next to garnacha from a different vintage makes absolutely no sense at all. You should be able to go back and forth and how do you do that with whites and reds mixed up?

IMO this is a fallacy that enjoys a fairly wide distribution.
Sensory testing has shown conclusively that our perceptions of a certain wine are affected by the samples we’ve tasted just previously. Which is why we tend to taste like with like (peer groups). But even within peer groups the effect is important. Everyone knows that tasting an acidic wine after a sweet wine will cause the acidic wine to taste more acidic. This could hold true within a peer group as well, though.
Peer groups are also impossible to define in any consistent way (in part because of what is noted above).
The only way around these confounding points is to taste a number wines multiple times in a randomized order (not worrying about peer groups), noting the rating each time. Then the data can be subjected to statistical analysis and outliers can be discarded. This will get you close to an “absolute” rating of quality for the wine.
Of course, such a methodology would severely limit the number of samples that could be effectively tested, which is why nobody uses it.

[mauss]

If we had here the possibility to post polls (maybe ?), the 3 questions will be :

a : I prefer blind
b : I prefer no blind
c : I do not care since I trust the critic (s) who has influence on my choices

BTW : even those who may taste blind, how to be sure that, in their office, before printing, they do not erase (or add) some points according the fame of the wine ?

And, again and again : a wine is done for a meal and so, circumstances may play a far larger role than we may think of.

[Daniel_Rogov]

@mauss

…BTW : even those who may taste blind, how to be sure that, in their office, before printing, they do not erase (or add) some points according the fame of the wine ?

M. Mauss or, if I may, Francois…

I have followed and respected your connections with the world of wine for quite some time and it is good to finally feel “in contact” with you.

Indeed you make a good point… I am certain that there are many who do just that. Forgive a bit of vulgarity, but I have a name for those people: I call them “whores”

Best wishes
Rogov

[Peter_Cargasacchi]

@Daniel Rogov

@Peter Cargasacchi

The notion of the Ideomotor effect is that we are subject to psychological reactions that are detached from conscious decision. It’s not just this behavioral concept, but the idea that there are also many reactions on a more primitive level of the brain that derive from perception of the immediate environment associated with survival and well being, that guide our decision making and outlook.

Peter, Hi…

Respectfully disagree and that is why I referred to “professionalism” earlier. As the psychologist/psychiatrist learns to deal with transference and counter-transference, so can the critic deal with many of the more primitive and indeed subconscious messages from the more primitive part of the brain.

That all bias can never be done away with is apparent. That, in the end, is one of the reasons we select this or that critic to best follow…we learn their biases and adjust to them. (e.g. my own rather obvious biases in favor of Brunello di Montalcino and Chateauneuf-du-Pape, indeed even when tasted double blind)

Best
Rogov

Dude, you can’t hide behind professionalism!!! SEE: PROFESSIONALISM IS AN ILLUSION!

I think your answer is reflecting a little bit of male testosterone and typical human belief in our own uniqueness. One of the aspects of behavior that we all share, is that to some extent, more or less, and depending on experiences and training, humans all feel themselves in some ways to be “exceptions” from the general rule.

We all have some sense of an elitist perspective when considering our own individual capabilities. It is in some sense what motivates us, the belief that we are individually exceptional.

This becomes a pitfall, especially when you start to believe it without second thoughts. The gist is, and I was trying to find some other articles and research on this, but the exact opposite of what you stated above and believe, was found to be true.

The professional critics were found to have the least ability to detach themselves from their subconscious biases and expectations, while the amateurs were the best able.

I am trying to recall, but I think it had to do with the fact that we tend to follow patterns, and that critics had the most difficulty detaching themselves from the pattern because of ingrained expectations. Whereas amateurs did not have the ingrained expectations and were subsequently better able to detach themselves and show less bias when offered visual indicators that were unrelated to the physical properties and characteristics of the subject matter being evaluated.

The closer you are to something the harder it is to really see it. Tastings need to be blind to be valid even for and especially for professionals!

[Peter_Cargasacchi]

I got the link above and this one,
Consumers warned over consistency of wine reviews, from Steve Heimoff’s blog.

[Bill_Tex_Landreth]

@mauss

If we had here the possibility to post polls (maybe ?), the 3 questions will be :

a : I prefer blind
b : I prefer no blind
c : I do not care since I trust the critic (s) who has influence on my choices

BTW : even those who may taste blind, how to be sure that, in their office, before printing, they do not erase (or add) some points according the fame of the wine ?

And, again and again : a wine is done for a meal and so, circumstances may play a far larger role than we may think of.

Polls are easy enough.

[Bruce_G]

@Daniel Rogov

Indeed you make a g ood point… I am certain that there are many who do just that. Forgive a bit of vulgarity, but I have a name for those people: I call them “whores”

I call them “critics”, but what’s in a name, right?

The way around Francois’ problem is to have all tasting sheets collected and recorded before the identities of the wines are revealed. This would have to be done by an independent party, someone who would later certify that all published tasting comments are faithful representations of the original notes.
Again, this represents another layer of compleixty and an additional cost added to the process, so I doubt anyone would do it.

[Dale_Williams]

@Peter Cargasacchi

We all have some sense of an elitist perspective when considering our own individual capabilities. It is in some sense what motivates us, the belief that we are individually exceptional.

There’s a front page article in NYT today about how people think that THEY are able to drive just as safely while talking on cell phones, texting, etc (often while accepting that others are less safe). We’re all above average drivers (as previous studies have shown), that’s as likely to me as professionals being able to eliminate (unconscious) bias.

[mauss]

Bruce :

This is the way we do it at GJE for 13 years now !

[CFu]

@Daniel Posner

Never is a strong word, I suppose. He will taste blind at EWS in September. When has he ever stated recently that any of his reviews were done in a blind fashion?

“The wine was tasted at the usual venue where RMP and I taste with importers, The Oregon Grill in Hunt Vally, MD. Present was Mark Clinard and his partner in the Well-Oiled Wine Company, and Mark Noah, the sommelier who pours wine for Bob and myself. The wine was tasted from bottle; it was not presented as a barrel sample.”-Jay Miller

Clearly, he is not tasting blind at the Oregon Grille…

“can i please have these wines bagged 1-100”

[Mo_K_a_n_g]

@Peter Cargasacchi

I got the link above and this one,
Consumers warned over consistency of wine reviews, from Steve Heimoff’s blog.

This is why I crack up when people dismiss CellarTracker reviews and claim that only “expert” opinions are valuable. I am sure the experts know more ABOUT wine than the unwashed masses, but I don’t think they know better which wines the unwashed masses will like-- and frankly, the number one job of a critic is to tell the unwashed masses which wines we are likely to enjoy, and which ones to stay away from.

[Bruce_G]

@mauss

Bruce :

This is the way we do it at GJE for 13 years now !

Very impressed, Francois.
I didn’t think anybody out there would go to those lengths.
Now, for extra credit… do you taste the wines multiple times, each time in a different, randomly generated order?

[gene_keenan]

@Daniel Posner

Even at his home, while watching the All Star game, Jay Miller still has no idea what he tastes…blind or not blind…

http://dat.erobertparker.com/bboard/showthread.php?t=206168" onclick="window.open(this.href);return false;

Does anyone believe that he actually opened any bottle of wine. Robert Kenney sent him the btl 10 months ago. He claims to have tasted it on Tuesday night…only he posts NO TASTING NOTE (shocking) and gets the vintage wrong.

Robert Kenney, desperate to cover up the cover up, tells Jay Miller what Jay Miller tasted…classic!

I am not for blind tastings, per se, I am for fair reporting. Clearly this was not fair reporting. He scores a wine 96 points, and now just panned it with no TN, calling it undrinkable?

either that thread was deleted or the link is wrong… or does this happen when a thread is merged?

[Daniel_Greathouse]

Nobody arranges a tasting better than François.

Funny thing is that blind tasting was the rule when we began in various pursuits, but “pragmatism” got in the way. Like vowing to return to church, it’s hard to get back in the habit (so to speak) but tremendously gratifying when we do. As critics are asked to clarify non-blind or degrees of blind tasting, now is an excellent time for all of us to fit a few more blind events into our schedule.

It’s also fair to say a good, tutored non-blind tasting can be instructive too!


Peace Out,

–dang (itb)

[Roy_Hersh]

In the discussion point re: professionalism and bias, I have to side with Peter and not Daniel R. in this case.

There is a nuanced difference between “label bias” and preconceived notions based on past performance, no matter how professional the critic is.

Looking at a label has to influence, no matter how ethical and professional a critic is.

This says nothing about methodology or ethics or ability, whatsoever … it is just human nature and nothing more and nothing less.

[Daniel_Posner]

@gene keenan

either that thread was deleted or the link is wrong… or does this happen when a thread is merged?

Merged

[mauss]

Bruce :

Yes, for the very best vintages, we try to taste minimum 3 times over 6/10 years . We did it for the 90 and, in fact, the best remain inside the best, the worst inside the worst and only in the middle you may find some sensible movements.

What we do also : I put the same wine tasted in the morning , in a different order in the afternoon session. To check the variation of points. Surprisingly, at GJE, it is usually ± 1 point for my best tasters.

I do also - always blind - the same wine twice in the same session : so to check the influence of the glas tasted before.

Since, as you know, I compute the scores of at least 12 top GJE Members, it “smooths” the results in some way.

In 1998, we did taste top Bordeaux at Dutournier restaurant. Then, we did flight to the Raffles in Singapore (7 tasters) with the same wines, bought at the same place, bit, of course, served in a different blind order : trust me or not : they came exactly, for the first ten, in the same ranking !!

So, I do trust collective scores since, as I say usually : “12 top tasters cannot make the same mistake on the same ine at the same moment” while, even the very best like Parker, due ti the rythm of their tastings, they do, once in a while, a mistake, they “miss” a wine, no doubt. BTW : this may be an official reason to make some “corrections” in the office later. We are just human, as Wilfred tells me everytime !

DANIEL : so, you do officialize my wedding with Wilma ?

Wait I see you at Villa d’Este ! Mama mia ! Make a diet before ! Daily pasta will be a must !

Best,