Grape open space

Assisting the judges to get the right result? 19 November 2007

But organisers deny manipulating the Diners Club results

From Anonymous [we know the identity of this correspondent]:

It seems that the South African wine industry finds it difficult to stay out of controversy regarding issues that should be straightforward such as basic arithmetic. If it isn’t a scandal involving the counting of votes at the election of KWV directors [click for the story], it is about the counting of scores at wine competitions (refer also the Wine Magazine Chenin Blanc Challenge [click here for the discussion about that]). The latest rumour concerns the Diner’s Club Winemaker of the Year judging process and the gossip is so disturbing and seemingly impossible that I thought it would actually be best to make sure that there is no truth in this vicious rumour.

The story is about the latest Diners Club Winemaker of the Year judging goes like this. According to panel chair Dave Hughes:

162 different wines were entered .... To allow the judging panel a better chance of determining the best wine a "screening" panel was assembled the day before, to taste through the 162 entries and eliminate those that were not going to be in the running for top honours. The judging panel was thus able to devote itself to a lineup of 71 wines in coming up with a shortlist of 13 for closer scrutiny.

Now the above would be all fair and well and according to the general principles of wine competitions (however flawed they may be). But the alternative view (i.e. the “rumour”) goes like this:

162 different wines were entered.... Entries were weeded down to 71 then to 5. An extra wine was added as a seeded player to make a final 6. The added wine wins - turns out to be Boekenhoutskloof Syrah 2005.

Now if this alternative model is true (where a wine gets added at the final stage due to it being a “seeded” player) this could be seen as a serious indictment as to the ability of competitions (or, to keep it statistically correct for the moment, the panel in this specific competition) to find the “best” wine through the normal competition process.

My personal views:

Wine competitions where huge numbers of wines have to be tasted in order to find a “winner” are inherently flawed. Boekenhoutskloof and Marc Kent deserved to be the Diner’s Club winemaker of the year but the normal rules involved in wine competitions did not favour his chances.

In their wisdom, the organisers of the competition felt it ethically acceptable to include Boekenhoutskloof into the final wines to be evaluated due to its ‘history’. If this really did happen, it would be very unfair to the other entrants who had not lobbied to have their previous successes taken into consideration (if this were the case, I’m sure other producers who have won major awards in the past would also have motivated their wines to be included at the last moment as being “seeded” (for example Raka Biography or Cloof Crucible that did not even make it into the Top 13).

I honestly hope that the above is not true (i.e. that a sixth wine wasn’t added to the final group of 5 wines as determined by the judging panel to determine the ultimate winner) and that the process was honest and without intervention (I’m sure the auditors should be able to say whether the process as established at the outset was followed or whether rules were changed during the process.)

I am not a huge believer in wine competitions as having the ability to consistently determine the 'best' wines (however you would define that). Maybe that is why the Platter team as well as other luminaries such as Robert Parker do not deign to the use of the blind tasting method of evaluation? Blind tasting is a great learning tool but fraught with inherent flaws in determining, for example, the longevity and maturation potential of a wine.

From the editor:
This version of events has also been stated, we believe, by Neil Pendock in a blog. Before printing this contribution from our reader we thought it responsible to ask tasting panel chair Dave Hughes and Mike Froud, Managing Editor of Wine mag which convened the tasting, for their responses to the suggestion of adding in a 'seeded' wine. While they somewhat strangely disagree about how many wines were included in the final taste-off, both deny anything untoward in their accounts of the process:
[Mike Froud subsequently pointed out that he was working from the audited results in giving his response, while Dave Hughes was working from memory, and apparently made a mistake.]

From Mike Froud's response:
...In the second round of judging on day two, 13 wines emerged as top-scorers, and a third/final round of judging involving five of the 13 top-scoring wines was then held to determine the winning wine. At the end of the process, using a 20-point rating system, the panel rated the top wine 17.7 versus 16.83 in respect of the first of the four runners-up, and all of the judges led by panel chair Dave Hughes expressed their satisfaction with the competition results as audted by PriceWaterhouse-Coopers.

From Dave Hughes:

[T] here were 162 entries. These were whittled down to 71. Then I suggested that out of the second flight we should take the top five and re-look at those wines. The auditor then pointed out that there were in fact six wines with near as dammit same score. So we (the whole panel) accepted we should look at six instead of five. Out of that six the Boekenhoutskloof was selected as the top wine. As you have written an "extra wine was added" is not how it happened. It was there all the time. As far as I am concerned everything was totally above board.

From winewriter Neil Pendock:
Two of the Diners judges independently told me a sixth wine was added to the final round of five wines by Mike Froud which generated some discussion among the judges at the time. I mentioned this in my blog . I did NOT say that the sixth wine was the Boekenhoutskloof but rather that the identity of the latecomer “will exercise anoraks for weeks” as indeed it looks like doing.

From Angela Lloyd:
Let's spare a thought for Diners Club in this debacle. They have sponsored this high profile and eagerly sought award since it was inaugurated in 1980. When the late Peter Devereux and Tony Mossop ran the judging process, the competition had the prestige it deserved; no one questioned the results. Others in between let things slip a bit, but there was no suggestion of any mishandling the judging process. Now it looks as though Diners Club efforts and financial backing are being compromised and it is even unclear whose competition it is. This year's international judge was announced not by a Diners Club Press Release, as one might expect, but by Wine magazine's Deputy Editor, Christian Eedes in his Gulp! newsletter. Further surprises came at the awards lunch, when the latest edition of Wine magazine was handed out (at a Diner's Club event?) and found to include interviews with both Marc Kent and Ruth Penfold (Young Winemaker of the Year); they obviously had advance knowledge of the winners, again making it look as though Wine magazine is hijacking the event. It doesn't help the Diners Club Winemaker of the Year/Young Winemaker of the Year Awards that the running of Wine magazine's own competitions have been called into question this year. Both the Chenin Blanc Challenge and Peter Schultz Port Award have caused calculators to work overtime and raised eyebrows to a startling level at the success of the number of wines from a winemaker member of the panel. Diners Club need to take command of their own event if they want it to maintain its past reputation.

From Anonymous again:
It does seem that there may be a problem determining whether there were five or six wines in the final line-up. That is the type of basic mathemical problems we seem to have in the South African wine industry and it causes confusion and a certain scepticism with regard to competition results. Apart from that the reassurance by Dave Hughes should surely now lay this rumour to rest. Or will it? Anybody out there who could maybe give a first-hand account of whether it was five or six wines in the final line-up?

From André van Rensburg (Vergelegen winemaker):
This debate is as irrelevant as the ‘Whites only’ cover of Wine Mag. No one gives a shit. The wine industry is always involved in some or other underhand business - according to wine writers! Congratulations to Boekenhoutskloof.

Response to André from Tim James:
Well, André, in my experience if there's a group of people that does a lot of moaning about underhand business it is surely winemakers more than winewriters – perhaps the difference is that (some) winewriters try to do something about it rather than just complaining to each other and saying to winewriters "Please don't quote me!'

Note from the editor:
We're happy to say that it would appear that there is no basis for continued speculation about any possible improper manipulation of results. The rumour appears to have originated with Neil Pendock reporting on remarks to him from two unnamed judges of the Diners Club tasting, and Neil stands by his original claim The judges have not come forward to repeat the claims – and we did not think it correct to approach any of the judges; we are content to accept the words of the tasting organisers. Their words are no longer necessarily conflicting. Dave Hughes, whose original statement to Grape perhaps gave ground for the rumour having plausibility by saying that six wines were tasted, now says 'Maybe five it was. I really don't recall exactly.'

Mike Froud of Wine mag quotes in an email a statement from the auditor stating that

• The top 13 average scores were identified (i.e. the top 13 wines were identified based on the average scores)
• 5 wines were then selected by the tasting panel - from the top 13 wines identified above - for the final blind tasting
• The Boekenhoutskloof Syrah 2005 was the winner (i.e. it was the wine with the highest average score following the final blind tasting)

It's all a strange set of unfortunate coincidences whereby Dave Hughes' initial recall was that there were six wines in the taste-off, which was part of the basic point of the rumour.

Amidst a necessarily persistent debate about methods of judging wine, what is clear and has never been challenged here, is that Boekenhoutskloof Syrah is an extraordinarily fine wine whose producers thoroughly deserve the accolade they have received.

From Tiny:
Do we really believe this? Hughes has a reputation for honesty.... does he honestly claim that he can't remember something that he seemed so sure about? Do you really mis-remember about putting in a sixth wine and having discussion about it? And in fact I see that the so-called auditor doesn't say there were five wines in the final round - he just says that the panel selected five. Something doesn't smell so good about this – not so much sweaty saddles as sweaty organisers. Is there brett at Wine magazine?

From Martin Paulson:
Would the Diners auditors please explain exactly how many tastings took place?
- one to select 71 from 162 entries
- another to select 13 from 71
- another to select 5 from 13
- yet another to find the top wine from the 5?
So four rounds in all with fresh bottles at each stage?
I really don't know where the 13-wine round comes from. It sounds like a replay of the Chenin Challenge when the wines kept being tasted until the 'correct' result was achieved.

Note from the editor:
We did ask Mr Froud to get the auditors to clear up any ambiguities, including making a clear statement that there were five wines in the final round and also whether the wines in the final round were the top scorers – as opposed to just coming 'from the top 13', but Mr Froud replied that

Diners Club does not wish to be drawn into any further debate about or explanation of the judging. Correspondence from our side and that of Diners Club in this matter is now closed.'

So there'd be no response to Martin's question. It does seem that there were four tastings, yes. Probably the last three stages took place in the space of a morning, and were probably conducted using one bottle, unless a fresh pour was thought useful for the final round and they had enough bottles.

The fact that it was felt necessary to have so many rounds of whittling down the numbers does suggest, perhaps, some lack of faith in the accuracy or reliability of this method of judging wines - otherwise they could simply have declared the winner to be the highest scorer in the first round. It would be interesting to hear the rationale for not doing that, of course. (And very interesting to know what that first top-scoring wine was!) Why, indeed, the need for so many rounds?

From Anonymous

Re: Dr Fraud and Mr Hoax – I can quite understand why Mr Froud would not like to be further drawn into this debate as it is becoming quite clear that even his own recollection of what actually transpired is starting to become as hazy as that of Mr Hughes. If you read closely you will see that in his initial response Froud indicates:

In the second round of judging on day two, 13 wines emerged as top-scorers, and a third/final round of judging involving five of the 13 top-scoring wines was then held to determine the winning wine.

It is very definitely indicated that there were a total of 3 rounds, i.e. the first round on day one, the second round on day two and the third/final round (also on day two). Therefore no fourth round as wrongly assumed by both the editor of Grape and Martin Paulson took place, unless they were looking at the auditors report as sent to the editor by Froud which indicates:

The top 13 average scores were identified (i.e. the top 13 wines were identified based on the average scores)
5 wines were then selected by the tasting panel - from the top 13 wines identified above - for the final blind tasting

How could the tasting panel have selected 5 wines from the 13 without another round of tasting? Maybe the tasting panel just looked at the scores and said, O.K. let’s taste the 5 wines with the highest scores. Or was it 6 as there was that one that scored “near as dammit” (to quote Hughes) to the fifth wine on the list? So my assumption would be that this elimination round was not a tasting but a paper exercise.

So what was the take of Hughes on the process? He says in his response to Grape:

There were 162 entries. These were whittled down to 71. Then I suggested that out of the second flight we should take the top five and re-look at those wines.

So round 1 was the tasting of 162 wines that got trimmed down to 71. Round 2 was 71 wines scored by the panel of judges. Hughes then suggested that they look at the top 5 (i.e. not “selected by the tasting panel” as indicated by the auditors but “suggested” by Hughes and probably accepted by the judges).

If I’m not mistaken, this is yet another interpretation of what transpired (and please don’t accuse me of using semantics to obfuscate the matter – these are very clear statements that have been made by people who should know that the information will be open for discussion and debate).

Mr Froud (and oh it’s becoming so much more difficult to spell your surname correctly), I think your cop-out by using Diners Club as a smoke-screen for what may or may not have transpired is in direct contrast with the so-called honesty and transparency that Wine magazine supposedly champions. If I were Diners Club I would like to see the truth emerge and not be associated with something that is starting to smell somewhat pongy.

So what has happened to the judges in this competition and why have they not come to the fore to side with either Froud or Hughes as to whether there were 5 or 6 wines in the final line-up? OK, I can understand that ”cinq” and “six” can sound close to one another to the untrained ear (so you are pardoned monsieur Graillot) but the difference between five and six should be quite obvious for our South African judges.

Wine competitions (especially those audited by fancy bean-counting firms) should leave a very definite paper-trail. How about allowing someone without an axe to grind (for example the venerable editor of this publication) to have a look at the score sheets and the process followed? Thereby all of this conjecture could easily be answered in a straightforward manner. If Froud is not willing to allow an outside party to try and figure out what actually happened (seeing that there are now 3 different versions on the table), I would think that this could be seen as an indication by Froud that there definitely is something to hide as originally indicated – or am I being a bit too harsh?

From Mark:
I think some clarity is called for so yes, Wine should publish the auditors results, or at the least make them available for scrutiny on the web. Anther point, and perhaps it draws reference to an article Alex Dale wrote in Grape some years back. How can we have the same panel judging Shiraz and Dry white wine?

From John:
An excellent point was made previously, but then was ignored...PISH to Mr. Frouds 'closing' of comments from either Wine or Diners Club...we ought to hear from PWC as the auditors. Why can we not contact PWC direct and ask for their findings? It would be a brave accountant in that well reputed and international firm to conspire with elements in the wine industry...as far as I know the paper-shredding was all across the road at Andersons!

I personally never believe any accusations of fixing and skull-duggery in the wine industry.

CLICK HERE TO SEND US YOUR COMMENT