'Tasting with one's eyes': some Platter discrepancies? 15 July 2006

We have recently received two emails about wines in the Platter Guide getting rated substantially differently although the wines were identical – but differently labelled.

1. From Andrew Milne of Southern Sky Wines:
'People taste with their eyes'. I will not mention names but I am aware of a time when the exact same wine (with different labels) received vastly different Platter ratings (4* vs 2*) when tasted by the same Platter judge. The one label was an Estate and the other label was unknown. Judges are human but this discrepancy is rather drastic and supports my opening line.
 

Response from Tim James:
This is, of course, quite a serious allegation. No names of tasters are mentioned, which means that all the Platter tasters, myself included, are potentially smeared. Not nice. Nor are the wines mentioned. All this lack of specificity means that the accusations remain at the level of damaging gossip, a smear. If someone knows that this happened, why not give the details so that the taster involved gets a chance to answer, or be properly convicted of incompetence or whatever? Perhaps what is claimed here is even true, perhaps the situation is not so simple – but how is anyone to know one way or another? Anyone can make any vague accusation about whatever they like. (I wonder how Andrew would feel if we published a letter from someone saying something pretty rude about some aspect of his wine business, without being specific enough for him to deny them properly?).  

Tasting with our eyes? I accept fully that any sighted tasting (like those for Platter) has to contend with the problem of prejudice. It is a problem. I have no doubt that I myself make mistakes and misjudgements in the guide – but not through simply applying my prejudices. Most professional tasters really try to be as honest and objective as possible – just like film reviewers, book reviewers, etc. It is often remarked that ratings in, for example, Platter tend to go with established reputations of producer, winemaker, region, etc – this is hardly surprising, really, given that reputations generally get established for good reason! But it's less often pointed out that, for example, some co-op wines from 'the other side of the mountain' have been put forward, and received, five stars, while some very expensive wines get pretty poor ratings on occasion in Platter.

There are plenty of forums around which taste wines blind, and I genuinely believe that it is a useful thing to also have an important set of tastings done sighted, taking track records, intentions of winemakers, etc, into account. Consumers can, and do, use both.
 

Response from Tim James:
Here it's the accuser who is anonymous (as well as rather snide!) .... But at least we have a specific case to refer to (not the same one that Andrew Milne was referring to, he tells me).

The story alluded to by the brave and well-informed correspondent is this: in the 2006 edition of Platter, two wines from Mooiplaas were given higher ratings than supposedly identical wines labelled for the Checkers supermarket. The wines were tasted by two different judges. Checkers complained about this, suggesting that their wines were under-rated because they bore a supermarket label rather than an estate one.

It is not quite as simple a case as it seems, however. I should say at the outset that I am directly involved, as I was the Platter taster of the Mooiplaas wines.

The complaint was, I know, taken very seriously by the Platter publisher and editor, and there was a good deal of discussion between Checkers and Platter. At one stage, the Platter team got together as many of the wines involved as they could (one of the Checkers versions was no longer available), and all the Platter people involved sat down to try them. It was very clear to us that, with regard to the Cabernet Sauvignon in question, the Mooiplaas sample was definitely superior. Generally, it seemed to be in much better condition; if taste is subjective, it could be pointed out that the Mooiplaas cork was fresh and clean, for example, while that of the Checkers bottle was quite seriously soaked.

We did agree, however, that probably the Checkers example was worth a little bit more than Platter had given it, while the Mooiplaas one was worth a little bit less than it had been given. Thus far, it would seem that the complaint had some validity, and that the estate wine had been given, shall we say, the benefit of any doubt, while the supermarket one had been slightly under-rated. I cannot speak on behalf of the Platter editor and publisher, but personally I think this ought to be admitted.

However, two things must be stressed: firstly, while the wines originally submitted for Platter presumably did came from exactly the same tanks (I don't know of any attempt to confirm this), they certainly went on different paths thereafter. I don't know whether the supermarket wine was bottled by the estate at the same time as the estate wines (a crucial quality stage), or where it was stored once sold. The Mooiplaas samples for Platter came straight from their cellar. The Checkers wines had, presumably, been shunted around and subjected to whatever pressures wines are subjected to when moved and warehoused for a supermarket (note, I am making no accusations here, but the sample we had of the Checkers version was, to repeat, not in the best of condition, for whatever reason).

The second thing that must be stressed is that the Platter tastings were done a year apart! That is, the Checkers versions were tasted when a year older than the Mooiplaas ones. (And where had they been for that year?).

I have been told that Checkers had at some stage given the pairings to an independent professional, who found no difference between the Mooiplaas and Checkers bottlings. There clearly has been some degree of bottle variation within the Checkers versions. It is possible that the sample tasted by the Platter taster was a particularly poor one. Who knows? I do not know if sufficient investigation has been done – or, if it has, whether the results have been made as public as the rumours and insinuations have been.

As for my own part in this business – perhaps slightly over-rating the Mooiplaas version: the Platter Guide is a conservative institution on the whole, because it is an annual, ongoing review rather than, like the competitions and shows, starting off on a clean slate each year. There are 'track record' ratings given for the better wines as well as individual ones for the latest vintage, and I certainly would tend to keep to the track-record rating unless there is a good reason to change – which there often is: I had previously downrated one vintage of a Mooiplaas wine, for example. But sometimes, even if I might prefer to give the wine a quarter star less, I would leave it where it is rather than giving it a half-star less (Platter only moves in half-star increments). This is not a scientific thing, after all, and I do not have huge faith in the absolute precision of my own rating skills!

Granted, again, the problems inherent in sighted tastings, in this particular case, it seems to me probable that the discrepancy (though perhaps too large) was primarily caused by a genuine difference between the samples. Also, there were two different tasters involved with possibly differing scoring standards operating (a problem for the Platter team, admittedly, and one they try to overcome in various ways). It happens. Not too often, I think, and certainly not often enough to be able to demolish the value of a methodology or the usefulness of the Platter Guide. By the way, I think people underestimate the difficulty of tasting sighted in a responsible manner: it can sometimes lead to a lot more worry than blind tastings!