By Mark Schatzker and Richard Bazinet | Updated May 25, 2018
Article summary
Few consumer products offer as staggering a range of choice as wine. You can buy a bottle of Dark Horse Big Red Blend for $8. Or for around $500, you can get a 2012 bottle of Sloan Proprietary Red. Yet for each bottle, the same question applies: Is it any good?
The rise of the wine-rating crowd
In 2004, Eric LeVine — then a group program manager at Microsoft — launched CellarTracker, a site where amateur wine enthusiasts can rate wines. Today, CellarTracker is the web’s most popular “community” or “crowdsourced” wine review website, containing 6.3 million reviews from 113,000-plus users for more than 2.2 million different wines.
Amateur and professional wine scores correlate very tightly
How similar? We ran a statistical tool called a Spearman correlation and got a figure of 0.576. A perfect correlation is 1. An utter non-correlation is 0. A score of 0.576 may not sound impressive at first, but it can actually get worse than 0 — a negative correlation, which is what you would see if you compared, say, shortness with the likelihood of playing professional basketball.
Amateurs appear more expert than the experts
It looks very much like the enthusiasts actually do a better job of agreeing with the experts than the experts do with each other. That might sound odd, but out of thousands of wines we analyzed, only a handful contradicted this pattern. Simply put, if you want to know what the experts think, the best place to look appears to be, of all places, CellarTracker.
The better the wine, the more experts agree with the amateurs
How do wine enthusiasts compare with experts like Robert Parker and Jancis Robinson? Very well.
There is also a tendency for scores to converge as wines improve in quality. This is evident in the arrow shape of the clusters in figures comparing CellarTracker with Wine Advocate and CellarTracker with International Wine Cellar. Average scores, furthermore, are high. On Wine Advocate, the average score was 89, on International Wine Cellar it was 91, and it was 17 out of 20 for Jancis Robinson. On CellarTracker, it was 89. This tells us that experts and enthusiasts alike don’t seem to be spending a great deal of time scoring mediocre wines.
Read the full article