Blind and sighted tastings 10 May 2006

From Vivian Watts:
I cannot see how Platter ratings can be taken all that seriously, given the fact that (as far as I know) the wines are not tasted blind. What about the little concept called prejudice? A famous icon wine will therefore never score two stars (as often happens when such wines are tasted blind, for example by Wine magazine. Your comments?

Tim James responds:

Vivian is largely correct that Platter wines are generally tasted with the judge knowing exactly what is being tasted. Only the final line-up of five-star candidates is subjected to a blind tasting.

The main argument in favour of blind tasting is clear (it avoids prejudice), so I’ll argue the other side, which I’m very happy to do. Perhaps we should first point out that there are not many areas of criticism where the judges are expected to not know what they’re judging, and where they’re not expected to overcome their prejudices as far as possible. How many films, books, etc are rated ‘blind’? Can you imagine, for example, a film critic expected to look at a 10-minute cut from a movie, not knowing whether its intended to be a two-hour mass-appeal blockbuster or an art-house short? That is not a ludicrous analogy of what happens when people taste wines blind. In fact, when there’s a line-up of wines, it would be more like a film critic viewing a few dozen two-minute clips from movies and being expected to make judgements on that basis....

As to Vivian’s specific question about the fate of famous wines: in fact there have been occasions when such wines have been given notably reduced scores in Platter in certain vintages: I remember that Veenwouden Classic went down from 4+ stars to 3+ stars for the 2001 vintage. Plaisir de Merle once had a bigger drop.

Admittedly, it doesn’t often happen – but that is partly because most good wines tend to be fairly consistent in quality from year to year. It’s interesting in this light to look at what Vivian contrasts with Platter: Wine mag’s blind tastings. Let’s take the example of Boekenhoutskloof Syrah – locally and internationally one of the most respected wines of the last decade. Wine mag’s panel have rated different vintages of this wine everything from five stars down to two stars. Within this spread, the 2002 vintage was tasted on two occasions, two months apart. On the first occasion it scored three stars, on the second occasion it scored five. Is that really the sort of scoring that inspires Vivian to say that blind tastings should be taken seriously and sighted tastings should not? Would Vivian really feel more confident going by those results than by Platter, which has scored the wine consistently well?

Honestly, Vivian, if you know the wines, which do you think is the more plausible rating for Rustenberg's John X Merriman: the 2+ stars awarded it in the latest Wine mag or the 4+ that Platter gives it? And the 3+ stars given to Sadie Family Palladius by Wine mag or the five stars given it by Platter? Will you really stick with the blind tasting result?

For me the bottom line is that both sighted and blind tastings are going to come up with what are, in fact, errors of judgement. In my opinion, sighted tastings produce fewer ludicrousnesses, and generally give more credible results, if they are done responsibly and carefully. When you know the background to what you’re tasting (the price point its aiming at, whether it’s an old-fashioned wine designed to improve over many years or something designed to give immediate, short-term satisfaction, etc), you can bring in more relevant criteria for judging. And don’t assume that only expensive wines get high ratings in Platter and vice versa. That is certainly not the case. Or that Platter judges are always overawed by reputations.

If I may bring in my own Platter experience. When I taste a wine that I know is a serious product (take Cordoba Crescendo as an example), not only do I have an idea of what the wine’s intentions are (as it were!), but I will normally sample it at least three times, over at least two days before making my final decision. When I tasted the 2002 Crescendo, I remember that I was very disappointed at first and was going to lower its rating quite seriously; but I gave the wine time; after three days it had opened up and was showing what a few years in bottle would do to it (in my opinion), and I gave it a good score.

With all wines, fancy or otherwise, I tend to give them a second chance one they have been open for a while: everyone knows that wine changes when exposed to air – that is another reason why blind quick line-ups can be very misleading. It is not only that Platter wines are judged sighted: they are also (a lot of them, though not all) judged in a more leisurely fashion.

I have given some very expensive bottles low scores, and some cheap wines higher scores. Last year I tasted for Platter the maiden releases wine of an excellent new winery, Tulbagh Mountain Vineyards; their ‘top’, very expensive wine I gave 3+ stars, a much less expensive wine I gave 4+  stars. I can also point out that I have on a few occasions told the editor that I should not be given X’s wines to taste as I dislike the style so much that it would be hard for me to be fair to them.

Platter judges are not fools, nor do they simply go by reputation or glitter. Most of us try hard to be responsible, to not let their prejudices get the better of us. We don't simply look at the label and write down a score. We taste the wines, and I think most of us do it better for know what the wines are trying to be.

COMMENT

From Andras Salamon:
Blind tastings seem no worse than sighted – in my experience what causes the inconsistent and sometimes clearly anomalous results found in blind tastings is the panel system. Very seldom will the same panel be used to taste the same wine on different occasions. This is why for me knowing who did the review is as important as what they wrote, since I can then adjust the comments and score if their palate is clearly different to my own.

From Keith Prothero:
There is no question that everyone no matter how talented a taster they are,is strongly influenced by the label. In view of this, I too wish Platter wines were rated blind although I assume the sure numbers involved would make this impractical. I would like to pose a question to Tim. How did the judges possibly rate the Glen Carlou Syrah as Wine of the Year? It is not even a three-star wine in my opinion, and was the worst wine we tasted in an international syrah tasting held at Dornier a few months ago.

Response from Tim James:
The Glen Carlou was tasted by a panel of 13 (I think) Platter tasters, and emerged as the best performer out of the range of some 70 five-star candidates tasted totally blind. Your point rebounds on itself a bit, Keith if you're trying to argue for blind panel tastings! I would say that the Glen Carlou is a wine that is likely to perform well in most large lineups of wine (not the case at the Dornier tasting).

To which Keith comes back:
I accept your point Tim that "big" wines are more likely to show well whether the tasting is blind or open. However, the fact that this wine earned the WOTY tag devalues IMHO the whole Platter system.
I spend a good deal of my time in the UK promoting Cape wine for no fee, and often the rating of this wine is mentioned by people who remain unconvinced by the merits of the wine produced here, or indeed the Platter rating system.