CellarTracker Scoring Methodology

Tasting notes, varietals, grapes - anything related to wine
Post Reply
Message
Author
User avatar
Sh@n A
GCC Member
GCC Member
Posts: 1184
Joined: July 9th, 2018, 8:21 am

CellarTracker Scoring Methodology

#1 Post by Sh@n A » January 8th, 2019, 12:39 pm

I want to update my score methodology for more consistency and am curious for folks input/how they approach scoring levels. I feel at times my scores don't have enough dispersion and I would like better personal guidelines for my scores.

As I think about a framework, I am gravitating to the following guidepoints:
- Generally speaking I feel the total score range is 85-100, at least at the price point of wines I generally drink (for the most part $35+, although the occassional $15-25 bottle).
- I generally want these scores to pattern match existing CT scores meaning, a 91-93 score from me would correlate to a 92 on CT (not 100% of the time, but generally)
- CT has its own recommended scoring guide, but this doesn't seem applicable to how people score (CT recommends 80 = good, but I imagine folks would rate good a bit higher)
- I think there are 4 major scoring categories: best ever/epiphany good, above average/pleasurable, house/cusp of drinkable, and harsh/burning/acidic. And then one can decide what guideposts are within this range.
- Overall this would have: Lower scores are characterized by rough alcohol/tannin/acidity, more so than issues with fruit. 93 being a point where a wine feels balanced and pleasurable. Higher scores are more subjective (e.g., a qualitative appreciation for the many quantitative scoring of aroma, balance, finish, etc.), and for me, having not had as many 95+ scoring wines as I have 90-93, a category I will just have to get better at.


So step 1, I was thinking guide post ratings are:
85 = Harsh/burning/acidic - don't want to swallow
89 = House wine/cusp of drinkability
93 = Above Average/Pleasurable
99 = Epiphany good wine

Leading to then fill out (at 2 point intervals)

85 = Harsh/burning/acidic - Don't swallow
87 = Harsh/burning/acidic - Swallow if you want
89 = Slightly harsh/burning/acidic, but not overly so
91 = Well made, but perhaps lacking a component or not particularly pleasurable
93 = Above Average/Pleasurable
95 = Extremely good wine
97 = Makes time stop wine
99+ = Epiphany good wine
/ @ g r @ \

User avatar
Mike Francisco
Posts: 223
Joined: March 30th, 2017, 1:59 pm
Location: Ann Arbor, MI

Re: CellarTracker Scoring Methodology

#2 Post by Mike Francisco » January 8th, 2019, 2:39 pm

[winner.gif] Tasting Notes and Ratings
Tasting notes are one of the core components of CellarTracker; in fact, we currently have over 5 million user-generated reviews in our database! They offer a way for users to share their thoughts on how a particular wine is tasting at any given point in time.

Tasting notes are public and are seen by all users. These should not contain copyrighted material, scores from critics, or promotional material from retailers. In a nutshell, a tasting note should reflect your impressions on how a wine tastes. It is important to keep in mind that tens of thousands of people see these tasting notes every day. Any remarks that are defamatory, offensive, or strongly political may be edited or deleted. Play nicely!

With regard to copyrights, please note the following which appears on the personal tasting note entry screen:

Note: Material you enter here is publicly visible to all viewers of the site and should not include copyrighted material. The use of tasting notes is governed by Section 6 (Contributions) of the Terms and Conditions. Please use Pro Reviews to privately store copyrighted material and Articles to store winemaker notes.
CellarTracker also supports a variety of other public and private note types; for more information, see the Public and Private Note Types help topic.

Wine Rating Scale
CellarTracker uses a 50-100 point scale for rating wines.

Score Grade Meaning
98-100 A+ Extraordinary
94-97 A Outstanding
90-93 A- Excellent
86-89 B+ Very Good
80-85 B Good
70-79 C Below/Average
50-69 D Avoid

User avatar
Sh@n A
GCC Member
GCC Member
Posts: 1184
Joined: July 9th, 2018, 8:21 am

Re: CellarTracker Scoring Methodology

#3 Post by Sh@n A » January 8th, 2019, 2:48 pm

I don’t have empirical proof handy, but I doubt the average user awards 80 points to good wine. Or 86 to a very good wine. Versus something much higher.
/ @ g r @ \

User avatar
Marcus Dean
GCC Member
GCC Member
Posts: 1341
Joined: May 31st, 2017, 4:45 pm
Location: Auckland Noo ZEEEEland

Re: CellarTracker Scoring Methodology

#4 Post by Marcus Dean » January 8th, 2019, 3:41 pm

Sh@n A wrote:
January 8th, 2019, 2:48 pm
I don’t have empirical proof handy, but I doubt the average user awards 80 points to good wine. Or 86 to a very good wine. Versus something much higher.
I dont know, I found a note the other day and the comment was "best wine I have ever had" and then the score was 83.

User avatar
David Glasser
GCC Member
GCC Member
Posts: 6974
Joined: August 16th, 2009, 6:03 pm
Location: Maryland

Re: CellarTracker Scoring Methodology

#5 Post by David Glasser » January 8th, 2019, 6:00 pm

So your goal is for your scores to conform to those of the average CT poster? That isn’t a goal I would personally pursue. Scores and TNs are primarily for me. That may be a bit selfish, but I’ll usually add some context to give a reader an idea of where my palate is. That’s more important than a score IMO.

Don’t want to swallow is an 85? Don’t want to swallow is a 75 tops, maybe a 70 in my book. I’d expect an 85 to be a good wine. This is one of the reasons CT average scores are useless to me. At least if there’s a TN I can ignore the score.

Mike Evans
GCC Member
GCC Member
Posts: 3078
Joined: January 30th, 2012, 9:22 am
Location: Atlanta, GA

Re: CellarTracker Scoring Methodology

#6 Post by Mike Evans » January 8th, 2019, 6:22 pm

One of my resolutions this year is to use more of the scoring range, not less. The range of my scores has become so compressed as to be all but meaningless, which annoys me. In any case I’m going to use scores in the mid to high 80s more frequently for wines that I find to be good but not great. A reasonably priced wine should not feel any shame for getting 87-89 points.

I’m also seriously considering stealing A. So’s approach of using increments to address to some degree the false precision of the 100 point scale (see https://www.cellartracker.com/m/users/103791 for his scale).

User avatar
Sh@n A
GCC Member
GCC Member
Posts: 1184
Joined: July 9th, 2018, 8:21 am

Re: CellarTracker Scoring Methodology

#7 Post by Sh@n A » January 8th, 2019, 6:47 pm

What I am finding is:
1) I think my historical scoring paradigm/what I posted in OP correlates pretty well with current CT socres
2) But as I drink smarter/better, I need to create scoring room for these better wines (said another way, a 95 to me today is a much better wine than a wine I would have said was a 95 five years ago).

Example:
I just had a 2009 Drouhin Clos Sorbe. Historically and per my qualifications in the OP it would be a 90. As it so happens, all four prior tasting notes gave it a 90, so success! However, I feel in my gut that was not a 90 wine. That over the recent years I have had more enjoyable 90 scoring wines. So this should be an 89, or something lower.... I need need more dispersion as I get more educated.

ASo:
Generally speaking, I think I suspect my scores are ~1-3 points above his. e.g., what I score a 95 he would give a 92-94, and what I give a a 90 would be a 88-90 in his book (I think). It is mentally difficult to emulate his scores, as I don't drink enough 95+ wines in his book. Further I haven't drunk enough of what he considers 93 vs. 95 vs. 97 wines to really appreciate the difference between his 100 and a 93... although I feel scores 94+ in CT are a bit of a random walk so this is OK.

Conclusion so far:
Both the 2009 Drouhin and ASo example suggest I should use a slightly lower points scale, which will give me more dispersion to play with on average wines. However, I have to accept this will likely cause me to be a more conservative CT scorer.

Secondary conclusion is CT has a missed opportunity of doing a better job of harmonizing scores. Perhaps they will take this flag up in the future, which has added bonus of increasing user engagement on the site.
/ @ g r @ \

User avatar
Sebastian C.
Posts: 699
Joined: June 22nd, 2012, 3:00 pm
Location: CT

Re: CellarTracker Scoring Methodolog

#8 Post by Sebastian C. » January 8th, 2019, 7:15 pm

I would prefer if CT had he option of standardized scores. That is, I choose the adjectives and CT provides the score. Therefore, if i choose the adjective “good” it would start at 82 and build from there. Thus if i want a wine to rate at least 90, i would have to choose “excellent”, so on and so forth. I just dont understand people who use private scales for PUBLIC notes. Why not just use private notes.?
C0rr3@

User avatar
P@u1_M3nk3s
GCC Member
GCC Member
Posts: 1566
Joined: June 7th, 2009, 2:14 pm
Location: El Cerrito CA

Re: CellarTracker Scoring Methodology

#9 Post by P@u1_M3nk3s » January 8th, 2019, 9:20 pm

I use the CellarTracker system. Typically, I put the adjective “Good”, “Outstanding” etc. at the end of the note, along with the corresponding numerical score.
Cheers,
Paul

Maxwell A.
Posts: 105
Joined: October 12th, 2018, 9:48 pm
Location: Chicago, Illinois

Re: CellarTracker Scoring Methodology

#10 Post by Maxwell A. » January 8th, 2019, 10:35 pm

Marcus Dean wrote:
January 8th, 2019, 3:41 pm

I dont know, I found a note the other day and the comment was "best wine I have ever had" and then the score was 83.
83 points... [rofl.gif] I love the honesty.
Max Andrle

Instagram - musignymax

User avatar
Rory K.
Posts: 372
Joined: January 9th, 2017, 7:19 pm
Location: Los Angeles, California

Re: CellarTracker Scoring Methodology

#11 Post by Rory K. » January 8th, 2019, 11:02 pm

The rating system is so all over the place with how different folks use it that I just have given up. No scores, just written notes that emphasize helping another know if they would enjoy it/if it is ready to drink. Some people think 93pts is just 'good' and for others that's 85, still another might try a wine that is obviously corked and give it a 75 or whatever. I have rarely found any use for the numerical scores on CT. Then there's another guy who just writes what Wine Spectator scored it and gives the same number...
K i n n e a r

ITB

User avatar
David Glasser
GCC Member
GCC Member
Posts: 6974
Joined: August 16th, 2009, 6:03 pm
Location: Maryland

Re: CellarTracker Scoring Methodolog

#12 Post by David Glasser » January 8th, 2019, 11:11 pm

Sebastian C. wrote:
January 8th, 2019, 7:15 pm
I just dont understand people who use private scales for PUBLIC notes. Why not just use private notes.?
Easier to search or create a rank order list on the score field?
The score field seems like the obvious place to record a score?
They’re not thinking about the community average?

I asked Eric if they could list wines in order of the average score of people you were a fan of, since CT calculates that. He gave me an explanation of why they couldn’t that was above my head technically. Think he said it was because it was a dynamic score, but I didn’t understand how that made it different from the average of all scores.

User avatar
Mike Francisco
Posts: 223
Joined: March 30th, 2017, 1:59 pm
Location: Ann Arbor, MI

Re: CellarTracker Scoring Methodology

#13 Post by Mike Francisco » January 9th, 2019, 5:33 am

David Glasser wrote:
January 8th, 2019, 6:00 pm
So your goal is for your scores to conform to those of the average CT poster? That isn’t a goal I would personally pursue. Scores and TNs are primarily for me. That may be a bit selfish, but I’ll usually add some context to give a reader an idea of where my palate is. That’s more important than a score IMO.

Don’t want to swallow is an 85? Don’t want to swallow is a 75 tops, maybe a 70 in my book. I’d expect an 85 to be a good wine. This is one of the reasons CT average scores are useless to me. At least if there’s a TN I can ignore the score.
[winner.gif] We have another winner!

The whole point of Cellar Tracker notes and scores is to add what you think, not guessing what others think.

Hitting the average isn't like getting an answer right on a test.

User avatar
Sh@n A
GCC Member
GCC Member
Posts: 1184
Joined: July 9th, 2018, 8:21 am

Re: CellarTracker Scoring Methodology

#14 Post by Sh@n A » January 9th, 2019, 5:37 am

If you have a totally different impression of the wine, then indeed you should not have the same score. But if you think similar things as another taster's note, the better outcome is for two similar point scores versus one at 90 and one at 80.
/ @ g r @ \

User avatar
Brian Tuite
GCC Member
GCC Member
Posts: 19587
Joined: July 3rd, 2010, 8:53 am
Location: Podunk CA

Re: CellarTracker Scoring Methodology

#15 Post by Brian Tuite » January 9th, 2019, 6:04 am

You could just use the YELP scale that so many these days are tied to.

1986 Chateau d’Yquem - Popped this bad boy at Red Lobster for the gf’s birthday. This didn’t pair well with anything on the Fisherman’s Platter. I asked the guy at the wine shop for something real special for her birthday and he came up with this. For $300 I would have expected at least a full bottle. Way too sweet. We switched to beer and gave this to some unsuspecting guy at the next table. Love the biscuits! *****
Bob Wood - 1949-2013 Berserker for eternity! RIP

"On self-reflection, I think a big part of it was me just being a PITA customer..." ~ Anonymous Berserker

"Something so subtle only I can detect it." ~ Randy Bowman

2019 WOTY...

User avatar
David Glasser
GCC Member
GCC Member
Posts: 6974
Joined: August 16th, 2009, 6:03 pm
Location: Maryland

Re: CellarTracker Scoring Methodology

#16 Post by David Glasser » January 9th, 2019, 6:28 am

Sh@n A wrote:
January 9th, 2019, 5:37 am
If you have a totally different impression of the wine, then indeed you should not have the same score. But if you think similar things as another taster's note, the better outcome is for two similar point scores versus one at 90 and one at 80.
But that’s impossible when there are hundreds of different scoring paradigms in play, and you don’t have a representative sample of them participating in the scoring of the particular wine you are evaluating.

Whatever scoring system you employ, it will align with only some of those that others use. If your goal is to use one that fits somewhere near the middle of the road, your scores will be close to the mean of other CTers. That’s fine if you value the collective, but it means you will have to adopt what I perceive to be a distorted and compressed scale.

I understand that the bulk of people posting scores on CT are wine-savvy to the point where they are drinking more high quality wines than plonk, and that that contributes to the narrow range of scores. But it doesn’t explain all of it. Some of it is avoidance of cognitive dissonance: not wanting to give a wine you bought a low score. Some of it is only posting scores on the good stuff. But if that leads to a wine so unappealing you don’t want to swallow it getting a score of 85, it’s contributing to a distorted, inflated, compressed scoring system.

My personal approach to this is to avoid point scores altogether. I use 6 broad categories for non-flawed wine: poor, average, good, very good, excellent, outstanding. The main reason I do this: I’m not a good enough or frequent enough or consistent enough taster to be able to repeatedly score a wine within 1 or 2 points on a 50 point (or 30 or 25 point) scale anyway. I only use points for tastings where the organizer requests it. Even with broad categories, most of the wines I drink I rate as very good or excellent. But that’s because I’ve learned what I like over the years.

User avatar
Josh Grossman
GCC Member
GCC Member
Posts: 2125
Joined: August 30th, 2017, 11:26 am

Re: CellarTracker Scoring Methodology

#17 Post by Josh Grossman » January 9th, 2019, 7:02 am

I subjectively give bad wines lower scores and good wines higher scores neener

User avatar
Mike Francisco
Posts: 223
Joined: March 30th, 2017, 1:59 pm
Location: Ann Arbor, MI

Re: CellarTracker Scoring Methodology

#18 Post by Mike Francisco » January 9th, 2019, 7:17 am

Sh@n A wrote:
January 9th, 2019, 5:37 am
If you have a totally different impression of the wine, then indeed you should not have the same score. But if you think similar things as another taster's note, the better outcome is for two similar point scores versus one at 90 and one at 80.
There are so many reasons why that is just not reality, and why frankly it's OK that it's not. Here are just a few.

You are using different scales see posts 1 & 2
You come at said wine from very different perspectives i.e. he loves The Prisoner & you drink nothing but Chablis (the real stuff, not the stuff in the box)
The other taster may have an ax to grind, be winery fan boy or just nuts
If you like looking at scores use the CT average as that weeds out the the highs & lows (to me still useless)
Or if you use CT like I do, other tasters scores are irrelevant as, my scores are just a reference point for myself.
I hate to get existential here, but none of this means anything, your enjoyment or not, should not change because someone else liked a wine 2 points or 20 points differently then you did.

Cellar Tracker, like Democracy, is flawed but be glad that it is still better then everything else we have come up with so far.

User avatar
Matt Snow
Posts: 269
Joined: May 13th, 2009, 5:19 pm
Location: Boston, MA

Re: CellarTracker Scoring Methodology

#19 Post by Matt Snow » January 15th, 2019, 5:10 pm

Shon --

Unlike others who have commented, I think trying to normalize your scores is laudable (albeit probably difficult). If you are scoring wines at all, you are either doing it because it is helpful to you or because you think it may be helpful to others. In order for your scores to be helpful to others, they need to understand what the scores mean. Trying to discern a community consensus as to the scoring scale is the most natural way to post scores that are understandable to others.

Now, since individuals’ own scoring scales do not necessarily correspond to a community average/consensus scale, putting your scores on the latter scale will not necessarily make them correspond to a particular reader’s own scale. It should, however, make them intelligible in the context of other community scores, and that should be helpful. I have previously analogized this to grading in academia: I can complain all I want that today’s grading scale is inflated compared to what it was when I studied, but if I want prospective employers and other academic institutions to be able to understand the grades that I give my students, I had better not give a B+ for work that is excellent, bordering on superb. Doing so will only confuse the consumers of the grades I give. Since I am not arrogant enough to believe that consumers will try to figure out how my scale differs from the norm, I try to conform to the norm.

I also think that the scale you set out is probably somewhat close to the CT norm (and, indeed, the norm among critics, if you consider them as a group), with the following caveats:

1) People probably allow a little more space than you do between “barely drinkable” and “above average” – an average wine might be somewhere in the 87-90 range, depending on the scorer, but most scorers would not give a score in their subjective “average” range for a wine with decidedly unpleasant characteristics. (Note, also, that some of your unpleasant descriptors – harsh, burning, acidic – would not necessarily show as unpleasant in the context of what is expected from a particular wine, e.g., a Bourgeuil, a young Port, a Riesling.)

In my post linked above (which is from 9 years ago, during which period scales have compressed upward a little), I suggested dividing the range below “above average” into a 3-point range for “unobjectionable but not particularly special,” with scores below that indicating that the taster “did not really like it very much.”

2) For usability, it should not be so much your goal to have your scores uniformly fall near a median CT score for the corresponding wine as to have the distribution of your scores match the median distribution of a CT user. Statistically, the median scores for wines on CT (at least wines with a reasonable number of reviews) will tend much closer to the overall community average than do individual scores for a given wine – because the median scores are themselves averages that eliminate a degree of variability.

a) Just because a wine is fantastic (or terrible) in some quasi-objective sense does not mean it will show that way to every consumer on every night, so that variability will lead to some variation in scores.
b) Add to that the variability of different users’ score ranges.
c) Add to that the fact that the most fantastic wines are (because of cost, accessibility, or simply the fact that those who like repurchase) more likely to be consumed again by those who have most enjoyed them in the past, who are more likely to be jaded, inured to excellence, or disappointed by the contrast of today’s experience to an even better previous one. And conversely, those who truly hate a given wine likely will not drink it again, giving it a better chance of being scored by someone who might find a redeeming characteristic.

As a result, if you were always hitting the CT average, you would be scoring on a compressed scale by comparison to other users.

-- Matt

User avatar
Brian G r a f s t r o m
GCC Member
GCC Member
Posts: 18329
Joined: February 3rd, 2009, 12:54 am
Location: westside

Re: CellarTracker Scoring Methodology

#20 Post by Brian G r a f s t r o m » January 15th, 2019, 5:18 pm

David Glasser wrote:
January 8th, 2019, 6:00 pm
So your goal is for your scores to conform to those of the average CT poster? That isn’t a goal I would personally pursue....
*Exactly* what I was about to post.
Los Angeles Workers' Compensation and Personal Injury

“All these characters spend their time explaining themselves, and happily recognizing that they hold the same opinions … how important they consider it to think the same things all together.” --- A.R.

CT handle: grafstrb

User avatar
Matt Snow
Posts: 269
Joined: May 13th, 2009, 5:19 pm
Location: Boston, MA

Re: CellarTracker Scoring Methodology

#21 Post by Matt Snow » January 15th, 2019, 5:29 pm

Brian G r a f s t r o m wrote:
January 15th, 2019, 5:18 pm
David Glasser wrote:
January 8th, 2019, 6:00 pm
So your goal is for your scores to conform to those of the average CT poster? That isn’t a goal I would personally pursue....
*Exactly* what I was about to post.
I am not a big believer in the importance of scores (vs. a narrative description of impressions of the wine), but if you are going to score, would your goal be to have a wildly different scoring range than other posters? And if so, why?

-- Matt

User avatar
Brian G r a f s t r o m
GCC Member
GCC Member
Posts: 18329
Joined: February 3rd, 2009, 12:54 am
Location: westside

Re: CellarTracker Scoring Methodology

#22 Post by Brian G r a f s t r o m » January 15th, 2019, 5:32 pm

I do score wines (when I have a mind to, which is about 10 - 20% of the time). My goal(s) in scoring wines have nothing to do with scores given by others. Others' scores for a wine are completely not relevant to the score I give the same wine.
Los Angeles Workers' Compensation and Personal Injury

“All these characters spend their time explaining themselves, and happily recognizing that they hold the same opinions … how important they consider it to think the same things all together.” --- A.R.

CT handle: grafstrb

User avatar
Matt Snow
Posts: 269
Joined: May 13th, 2009, 5:19 pm
Location: Boston, MA

Re: CellarTracker Scoring Methodology

#23 Post by Matt Snow » January 15th, 2019, 5:44 pm

Brian G r a f s t r o m wrote:
January 15th, 2019, 5:32 pm
I do score wines (when I have a mind to, which is about 10 - 20% of the time). My goal(s) in scoring wines have nothing to do with scores given by others. Others' scores for a wine are completely not relevant to the score I give the same wine.
Of course not, but the OP's point was not (I am assuming) to try to make each individual score match community scores for a given wine, but rather to try to make his range of scores match. That seems to me a good goal if you care whether your scores are meaningful to others.

-- Matt

User avatar
Brian G r a f s t r o m
GCC Member
GCC Member
Posts: 18329
Joined: February 3rd, 2009, 12:54 am
Location: westside

Re: CellarTracker Scoring Methodology

#24 Post by Brian G r a f s t r o m » January 15th, 2019, 5:55 pm

Matt Snow wrote:
January 15th, 2019, 5:44 pm
Brian G r a f s t r o m wrote:
January 15th, 2019, 5:32 pm
I do score wines (when I have a mind to, which is about 10 - 20% of the time). My goal(s) in scoring wines have nothing to do with scores given by others. Others' scores for a wine are completely not relevant to the score I give the same wine.
Of course not, but the OP's point was not (I am assuming) to try to make each individual score match community scores for a given wine, but rather to try to make his range of scores match. That seems to me a good goal if you care whether your scores are meaningful to others.

-- Matt
Well, that begs the question: Are "community scores" helpful/meaningful to others? There are many threads on this site, alone, wherein many folks emphatically answer that question with a "No."

At what point does a collection of CT scores become a "community score" with which one should endeavor to match? Regardless of whether one's TN is the first, tenth, one hundredth, or one thousandth TN for a wine, it still has just as much standing/merit as all other notes. If anything, the more recent notes should be valued more. Unless one would have the same worries about "matching" scores for a given wine with zero notes as they do for a gifen wine that has more than zero notes, it seems the whole notion of matching is logically flawed for the sole reason I put in bold, above.

A "community score" is all scores, averaged together. If, after some arbitrary number of scores, an user is merely looking to match their score to the community score then there is no point in scoring that wine, other than to further solidify/entrench the status quo.

It will **never** be possible to get everyone on the same page w/r/t scoring wines, or just about damn near anything, for that matter. Hell, look at Olympics, boxing, MMA, dog shows, cat shows, cooking competitions, writing competitions, Oscars, Grammys, Emmys, {anything where the scoring is, at least partially, subjective} and you will see that Opinions Will Differ. And differing opinions are to be celebrated and discussed, rather than denigrated and suppressed.

In my opinion, the best way to be helpful/meaningful to others with one's TNs is to be *consistent* with oneself. The best way to be consistent is to have a strict procedure for assessment. I have a strict procedure for assessment; I feel I'm far more consistent/reliable since I've implemented this system than I was before using my system. Experience has also played a part in my consistency/reliability, but I do believe having a strict system of assessment plays a big role, too. As alluded-to earlier, however, I usually have not the interest or energy to go through such a strict process, which is why I don't score most wines. ... one kind of has to ask oneself when setting upon this Community Score Matching Path, "How much time and attention did *others* put into their scoring?" Is the tail wagging the dog?
Los Angeles Workers' Compensation and Personal Injury

“All these characters spend their time explaining themselves, and happily recognizing that they hold the same opinions … how important they consider it to think the same things all together.” --- A.R.

CT handle: grafstrb

User avatar
Sh@n A
GCC Member
GCC Member
Posts: 1184
Joined: July 9th, 2018, 8:21 am

Re: CellarTracker Scoring Methodology

#25 Post by Sh@n A » January 15th, 2019, 7:42 pm

Brian, it’s pretty easy to disagree. Someone who rates wines on a 1-3 point scale because they feel privileged to be different, should not have there scores averaged into a 0-100 point system. I am not saying those notes are not helpful on their own, but then they should, as you note, refrain from scoring or be open that their scoring is not based on any scale understandable by the other commoners using the site.
/ @ g r @ \

User avatar
Sh@n A
GCC Member
GCC Member
Posts: 1184
Joined: July 9th, 2018, 8:21 am

Re: CellarTracker Scoring Methodology

#26 Post by Sh@n A » January 15th, 2019, 7:53 pm

Matt Snow wrote:
January 15th, 2019, 5:44 pm
Brian G r a f s t r o m wrote:
January 15th, 2019, 5:32 pm
I do score wines (when I have a mind to, which is about 10 - 20% of the time). My goal(s) in scoring wines have nothing to do with scores given by others. Others' scores for a wine are completely not relevant to the score I give the same wine.
Of course not, but the OP's point was not (I am assuming) to try to make each individual score match community scores for a given wine, but rather to try to make his range of scores match. That seems to me a good goal if you care whether your scores are meaningful to others.

-- Matt
Yes that was the point, Matt. My goal isn’t to try and conform my own tasting notes to match the CT notes. It was asking whether my scoring system in the OP could be be refined, as I plan on refining it myself for greater consistency. I personally reject the idea that there is so little consistency in scores such that the average CT score is a random walk. My personal experience is wine that consistently gets notes of 94-95 is better in my own personal tasting than CT wines that score 85. One can debate where the line is, and it seems clear based on responses here that the refining I discussed in the OP is outside the bounds of error for CT (and I will take my score less seriously as a result). But that does not mean there should be no rules at all.

One thing I definitely took away from this discussion, is I am scoring average wines too high. I plan to rework my system per the recommendations, to lower my DPIM scores and be more willing to score wines a 88-89 (lowering my average wine to 88-90 instead of 90). I freely admit that I will score wines lower in the future than in the past, but (I) in the margin of error this is reasonable for the CT universe and (II) will allow me to be more consistent going forward. Barring a more refined set of guidelines from CT, this is a reasonable outcome.
/ @ g r @ \

Philip G
GCC Member
GCC Member
Posts: 269
Joined: January 4th, 2019, 3:04 pm
Location: Denver

Re: CellarTracker Scoring Methodology

#27 Post by Philip G » January 15th, 2019, 9:31 pm

My scoring is close to the CT scoring system but drops off quicker, though not as quickly as the OP's "85 = Harsh/burning/acidic - don't want to swallow". From my experience, wines that WA or WS rate mid to high 80s are what I would call "Good", not "Very Good" as in the CT scoring system. These are the wines that I might buy for parties or as cellar defenders depending on the price. Below 85 I would call below average.

Curious if there there is a wine rated low 80s on WA, WS and CT that would get a consensus as a "Good" wine.
Philip "Chip" G

User avatar
Joe B
GCC Member
GCC Member
Posts: 9288
Joined: October 28th, 2012, 2:40 pm
Location: North Burbs Chicago Illinois

Re: CellarTracker Scoring Methodology

#28 Post by Joe B » January 16th, 2019, 5:41 am

We need to start a group who will adhere to the same guidelines when rating a wine. 85 is not a score that says do not swallow. That is a 70-75. But people don't want to attach these low scores to a wine and then their user identity.
J W@ll @ce

User avatar
Jason T
GCC Member
GCC Member
Posts: 2122
Joined: June 8th, 2014, 7:45 am
Location: London

Re: CellarTracker Scoring Methodology

#29 Post by Jason T » January 16th, 2019, 6:18 am

Joe B wrote:
January 16th, 2019, 5:41 am
We need to start a group who will adhere to the same guidelines when rating a wine. 85 is not a score that says do not swallow. That is a 70-75. But people don't want to attach these low scores to a wine and then their user identity.
Joe, it would be nice but I don’t see how it’s possible. I do agree with what you’re saying though in terms of scoring. I’m thinking of a note I saw recently where the wine was ultimately described as “not good” and yet was still given 91 pts.

But that also hits on why I don’t score wines. I did in the beginning, as more of an academic excercise. But I I discovered quickly that few followed the methodology laid out by Parker or CT (I’d argue towards the end even Parker didn’t...). My scoring wasn’t comparable to others, and therefore couldn’t easily be translated. I also found that my scoring didn’t even mean anything to me....

In the end, it’s the notes that matter for me. My own notes as well as the notes of others whose palates seem to align with mine.
J@son Tr@ughber

User avatar
Matt Snow
Posts: 269
Joined: May 13th, 2009, 5:19 pm
Location: Boston, MA

Re: CellarTracker Scoring Methodology

#30 Post by Matt Snow » January 16th, 2019, 8:03 am

Brian G r a f s t r o m wrote:
January 15th, 2019, 5:55 pm
Matt Snow wrote:
January 15th, 2019, 5:44 pm
Brian G r a f s t r o m wrote:
January 15th, 2019, 5:32 pm
I do score wines (when I have a mind to, which is about 10 - 20% of the time). My goal(s) in scoring wines have nothing to do with scores given by others. Others' scores for a wine are completely not relevant to the score I give the same wine.
Of course not, but the OP's point was not (I am assuming) to try to make each individual score match community scores for a given wine, but rather to try to make his range of scores match. That seems to me a good goal if you care whether your scores are meaningful to others.

-- Matt
Well, that begs the question: Are "community scores" helpful/meaningful to others? There are many threads on this site, alone, wherein many folks emphatically answer that question with a "No."

At what point does a collection of CT scores become a "community score" with which one should endeavor to match? Regardless of whether one's TN is the first, tenth, one hundredth, or one thousandth TN for a wine, it still has just as much standing/merit as all other notes. If anything, the more recent notes should be valued more. Unless one would have the same worries about "matching" scores for a given wine with zero notes as they do for a gifen wine that has more than zero notes, it seems the whole notion of matching is logically flawed for the sole reason I put in bold, above.

A "community score" is all scores, averaged together. If, after some arbitrary number of scores, an user is merely looking to match their score to the community score then there is no point in scoring that wine, other than to further solidify/entrench the status quo.
I agree with you, but I think we still are reading the point of this thread differently. To my understanding, everyone posting here agrees that individual scores on a given wine can and should vary from (distribute around) the community score, i.e., the average of the individual scores.

Personally, I am much more interested in the language of particular notes on a wine, at particular points in time, than I am in any individual score or in the average score.

That said, I and many others do read the individual scores, and if people are using them, I think that it helps for the scorers to be generally in line with a community norm as to what a given score means. Your notes, for instance, are very helpful (easily top 5% of the notes I read), because you give contextual information such as serving temperature and decant, and you are very specific and descriptive about aspects of the aroma, palate impression, etc. I don't find your scores particularly helpful because (as far as I've followed) you compose them as an aggregate of a variety of subscores and I don't know what those subscores represent, so in reading a number of your notes in passing I have never successfully calibrated what a particular number in a particular subscore might mean to me.

That's all ok: I'm sure the notes are helpful to you and to people who understand your system, and you don't have any obligation to make them helpful to me or anyone else in particular. But by using a non-standard system, you definitely make what you are saying more obscure, even if the intention is to be more specific.
Brian G r a f s t r o m wrote:
January 15th, 2019, 5:55 pm
It will **never** be possible to get everyone on the same page w/r/t scoring wines, or just about damn near anything, for that matter. Hell, look at Olympics, boxing, MMA, dog shows, cat shows, cooking competitions, writing competitions, Oscars, Grammys, Emmys, {anything where the scoring is, at least partially, subjective} and you will see that Opinions Will Differ. And differing opinions are to be celebrated and discussed, rather than denigrated and suppressed.

In my opinion, the best way to be helpful/meaningful to others with one's TNs is to be *consistent* with oneself. The best way to be consistent is to have a strict procedure for assessment. I have a strict procedure for assessment; I feel I'm far more consistent/reliable since I've implemented this system than I was before using my system. Experience has also played a part in my consistency/reliability, but I do believe having a strict system of assessment plays a big role, too. As alluded-to earlier, however, I usually have not the interest or energy to go through such a strict process, which is why I don't score most wines. ... one kind of has to ask oneself when setting upon this Community Score Matching Path, "How much time and attention did *others* put into their scoring?" Is the tail wagging the dog?
Agreed, and agreed.

-- Matt

Alex N
Posts: 201
Joined: December 3rd, 2018, 1:57 pm

Re: CellarTracker Scoring Methodology

#31 Post by Alex N » January 16th, 2019, 10:18 am

I use a point system similar to some of the critics
Something like:
45 - minimum points just for being a wine
10 - color
15 - nose
20 - taste
5 - style for the varietal and location (example: a 100% steel tank 2013 Oakville cab would probably get a lower score)
5 - QPR
@13x N1ch•!z

User avatar
Mike Francisco
Posts: 223
Joined: March 30th, 2017, 1:59 pm
Location: Ann Arbor, MI

Re: CellarTracker Scoring Methodology

#32 Post by Mike Francisco » January 16th, 2019, 10:38 am

Joe B wrote:
January 16th, 2019, 5:41 am
We need to start a group who will adhere to the same guidelines when rating a wine. 85 is not a score that says do not swallow. That is a 70-75. But people don't want to attach these low scores to a wine and then their user identity.
Don't forget about ego, so many tasters on CT don't want to admit they spent money on a wine they shouldn't give 90 points to. [stirthepothal.gif]

User avatar
Brian G r a f s t r o m
GCC Member
GCC Member
Posts: 18329
Joined: February 3rd, 2009, 12:54 am
Location: westside

Re: CellarTracker Scoring Methodology

#33 Post by Brian G r a f s t r o m » January 16th, 2019, 3:07 pm

Matt Snow wrote:
January 16th, 2019, 8:03 am

I agree with you, but I think we still are reading the point of this thread differently. To my understanding, everyone posting here agrees that individual scores on a given wine can and should vary from (distribute around) the community score, i.e., the average of the individual scores.

Personally, I am much more interested in the language of particular notes on a wine, at particular points in time, than I am in any individual score or in the average score.

That said, I and many others do read the individual scores, and if people are using them, I think that it helps for the scorers to be generally in line with a community norm as to what a given score means.
I agree with this. However, I believe said conformity, or community norm, should be established up-front, rather than back-doored.
Your notes, for instance, are very helpful (easily top 5% of the notes I read), because you give contextual information such as serving temperature and decant, and you are very specific and descriptive about aspects of the aroma, palate impression, etc. I don't find your scores particularly helpful because (as far as I've followed) you compose them as an aggregate of a variety of subscores and I don't know what those subscores represent, so in reading a number of your notes in passing I have never successfully calibrated what a particular number in a particular subscore might mean to me.

That's all ok: I'm sure the notes are helpful to you and to people who understand your system, and you don't have any obligation to make them helpful to me or anyone else in particular. But by using a non-standard system, you definitely make what you are saying more obscure, even if the intention is to be more specific.
All very reasonable points, Matt. And thank you for the kind words. FYI, and for the benefit of others who also are confused by my system, I'll happily share how I arrive at my scores. I use the Robert Parker scoring methodology. I believe CT promotes the same scale, if not the methodology in particular. As you've noted, my scores are broken-down into sub-scores --- I do this for two main reasons: (1) discipline, and (2) information. As it was taught to me, the Parker scoring system works like this:

50 points just for showing up to the party as a liquid that is actually wine.
5 points for Color/Appearance/Body
15 points for Nose
20 Points for Taste/Structure/Palate
10 Points for Overall/Ability to Improve With Age

So, my TN scores are indicated as: Showing up, Color, Nose, Taste, Overall = Total Score
Breaking-down the score like this informs others of how I arrived at my final score; obviously, this won't be informative if folks don't know what categories I'm using, or what the total available points in each category are. I've found that analyzing a wine in such a manner requires discipline and focus, and leads to more thoughtful assessments. I've also found this system yields greater consistency, and that, in my opinion, is one of the most important attributes of a helpful taster. YMMV.

Yes, it would be great if we all analyzed, and scored, wines using the same methodology. That's a big, and unreasonable, ask. But I do see the value in that hypothetical. I think we're more in agreement than I previously thought. Thank you for your thoughtful posts on this topic, Matt. [cheers.gif]
Last edited by Brian G r a f s t r o m on January 21st, 2019, 9:03 am, edited 1 time in total.
Los Angeles Workers' Compensation and Personal Injury

“All these characters spend their time explaining themselves, and happily recognizing that they hold the same opinions … how important they consider it to think the same things all together.” --- A.R.

CT handle: grafstrb

William Jones
GCC Member
GCC Member
Posts: 50
Joined: December 8th, 2010, 10:46 pm

Re: CellarTracker Scoring Methodology

#34 Post by William Jones » January 16th, 2019, 5:36 pm

I am not as trusting as others here about the equality of tasters experience or palates on CT. With thousands of tasters, given the usual Bell curve the scores should be weighted toward the less experienced I would think.
Another problem with taking CT scores at face value is a problem I have encountered more that once--2005 Freixenet Cava Brut Nature Vintage $ 8.64. Zero reviews but ranked as a score of 97 when searching <cava>
That might skew the " community average" little!

User avatar
Jeremy Holmes
Posts: 6753
Joined: April 28th, 2010, 3:50 pm

Re: CellarTracker Scoring Methodology

#35 Post by Jeremy Holmes » January 16th, 2019, 7:56 pm

Is it my imagination, or do an awful lot of people simply point their wines on Cellartracker within 1 point of their favourite critic?
ITB

User avatar
M.Kaplan
GCC Member
GCC Member
Posts: 4986
Joined: April 18th, 2009, 9:10 am
Location: Los Angeles

Re: CellarTracker Scoring Methodology

#36 Post by M.Kaplan » January 16th, 2019, 8:49 pm

Jeremy Holmes wrote:
January 16th, 2019, 7:56 pm
Is it my imagination, or do an awful lot of people simply point their wines on Cellartracker within 1 point of their favourite critic?
[winner.gif] And they use the same word salad to describe the wines.
---Mark

Post Reply

Return to “Wine Talk”