Tom Fazio did a good job with the very limited piece of land at the Wynn resort in Las Vegas, especially considering that it is in the middle of the city with a distracting monorail going around. But the second best golf resort in the Western US? Are you f*cking kidding me? I will admit it is the second best golf resort in Las Vegas. Number one US course Bandon Dunes? Did not make the list. Red Sky Ranch? Nope. Shadow Creek, perennial top ten? Nope. Sun River, Oregon? The list goes on and on and on.
What follows is no mere rant. Rather, it is a continuation of annoyance after years of following the golf and travel industry’s top polls, ratings and rankings.
Five years ago, I was put in charge of launching a series of travel polls for Cigar Aficionado, a high-end luxury lifestyle magazine. We wanted to do something different – and better – than the alternatives on the market. After looking at the major polls in other magazines, I saw significant flaws, and developed a methodology to avoid these pitfalls. I continue to serve this roll as the contributing travel editor, and have produced several polls on the best hotels, resorts, and yes, best golf travel. In the intervening years, I have watched the majority of polls and rankings, especially the Golf and Golf Digest Top 100 lists, improve. There are still questionable calls, and maybe too much emphasis on history, but all in all the differences have become more subjective than objective.
But not at Condé Nast Traveler.
I have nothing against the magazine. It is well produced, with pretty pictures, and occasionally even includes a story of interest. My parents have a subscription and as I suspect many readers do, keep the copies in the bathroom to read during important business. I saw a poll once saying that less than half the readers own passports, and based on the reader poll responses I can believe this. The problem with CNT’s polls isn’t just that they are poorly executed, edited and unscientific, which they are. But the real problem is that they are important. People in the travel industry take them very seriously, hotel GMs are promoted or scolded based on rankings over which they and the quality of their operation have little or no control. Likewise, industry anecdotes suggest that these polls are among the most acted on – award a property and people will go. If the polls were accurate, or even mediocre, this would be okay, but they are so horribly flawed that they are in affect sending readers off, in many cases, to waste their time and money.
Not every hotel, cruise line, spa or golf resort they recommend is bad. It’s just that they portray them as the best of the best, the top choice on earth or in a geographic region. Once in a blue moon they are, but more often they are merely good or even very good properties elevated way above their proper status. At the poor end of the spectrum are choices that simply do not belong on any “best” list for any reason but still show up.
Ireland's Doonbeg is not the nation's best, but it is a very good golf course on an incredible piece of land, and virtually every golf travel expert agrees that the Lodge at Doonbeg is among the handful of finest golf hotels on earth, making it a standout in Ireland, where many top courses have no lodging. But did it make CNT's top foreign resorts list? Nope, it got beat out of the World's Top 10 by three "resorts," and I use the term loosely as CNT does, in Canada, and another three in Mexico. Scotland and Ireland combined landed just three spots. It's worth noting that CNT's Top Ten Foreign Golf Resorts include one that has only a par-3 course, and one, unbelievably, that has no golf course at all. Mexico has lots of great golf resorts but even they don't make this list - Doonbeg got beat out by the Hilton Los Cabos, which is not even top ten in Los Cabos!
Here’s the nitty gritty:
I have three big problems with the poll, reasonableness, methodology and accuracy.
Over the years I have looked into the polls’ methodologies and they simply suck. A single respondent can make the difference between a property being ranked among the best in the world or being left off the list (in the small print of the new golf poll, they say that a minimum number of unique respondents were required, but don’t say what that was. Based on past experience, my guess is one). If you follow the rankings from year to year, you’ll notice that in the resort and hotel poll, the same property pretty much never wins more than once. In the real world, the best hotel on earth would likely hold that title for more than 12 months, or at least drop from first to second or third. But in CNT-land it is routine for a hotel to win and then disappear from the charts. One year the Four Seasons Nevis, a wonderful golf resort in the Caribbean, won for “Best Resort North America.” Several months later I visited for a story and found this reasonable. It may or may not have been the best, but it was good enough to make the argument. The following issue it was not even on the list. Since I knew that the amount of time between my visit and the production of the next issue could not have been more than a few months, I questioned this, directly from an editor I knew at CNT and got a simple explanation. They use an average of reader rankings, and the year before, just one respondent had included the hotel, given it a perfect score (10 or 100, I can’t remember the scale at the time) and it got a 10 (or 100). The following year no one who took the poll (it’s a notoriously small sample) stayed at the hotel so it was unrated. That’s it. The best resort on the continent the magazine is published on went from first to worst thanks to one reader.
To make matters worse, they have little control over who the one reader is. It could have been the GM’s wife. The poll has been mailed (now online) to a selected (and small) subgroup of subscribers. I’ve gotten it, and I’ve not gotten it. They ask you to limit your responses to places you’ve visited in the past 12 months, but they have no way of knowing when and if you ever visited them. Surely someone who honeymooned at a favorite resort 30 years ago is still going to be inclined to rate it favorably. Likewise, what do they think hotel employees, management, friends of management, investors, employees of suppliers, marketing companies associated with the hotel – or their competitors – are going to do in terms of honesty? There are no apparent safeguards against this, and in fact, I have heard numerous stories within the hotel industries of concerted campaigns to get ballot recipients to vote for (or against) particular properties. Likewise I have similar stories from reliable sources, which could be mere rumors, of editorial changes to the poll results to reflect advertiser complaints or purchases.
One other shocker is that the poll, at least the last time I got a ballot, is a fill in the blank questionnaire, basically a self fulfilling prophecy. Given the size of the world and hotel (or cruise for or golf or spa) industry, there is a rather limited number of pre-selected properties to choose from (maybe because they advertise?). CNT has already selected the potential winners before a vote is cast.
The poll still uses this highly unscientific averaging formula to generate its numbers. So if one reader loves the golf course at say, the World Golf Village in St. Augustine, Florida and give sit a perfect score, it gets a golf course quality ranking of 100. Likewise if 1000 readers vote for the Old Course at St. Andrews and 999 give it a perfect score and one gives it a 99, it will finish second with a 99.99 to the WGV’s 100.
Now from my standpoint, this is where the subjective point comes in. Some people, even the most expert and informed critics, are going to have varying opinions on golf course quality, but you have to draw the line somewhere. If CNT were to tell you that the Marriott Indianapolis Airport (I’m making that up, don’t know if there is one) is the best hotel in the country, better than the Four Seasons Beverley Hills or Peninsula New York, you have to draw a line in the sand and say “that’s ridiculous.’ Likewise giving the 36 slightly-better-than-average-for-Florida holes at the World Golf Village the highest possible ranking while giving the Old Course or the vaunted Ailsa Course at Scotland’s Turnberry, which many critics, including myself, feel is the planet’s very best golf course, a 99.99, is ridiculous.
Of course, I am exaggerating, and they did not do this. I don’t know how many voted for each, What really happened was the World Golf Village got a perfect 100 score for its 36-combined holes, neither of which is particularly memorable (though they both would be in the top ten for the great St. Augustine region), while the Old Course got a golf course rating not of 99.99, but of 96.3 edged out by Turnberry’s 36 which got 96.4. Bear in mind that in every mainstream golf poll produced, both Old and Ailsa are routinely ranked in the world’s top ten, while I have never seen either course at the World Golf Village crack any list, domestic, public, or global in scope.
This idiocy demonstrates two key faults in the poll: lack of subjective reasonableness and the unscientific nature.
In part two, I will tackle the purely unacceptable inaccuracies and show you why the actual golf results totally suck.
Stay tuned…
