Why most nutritional research is useless

Nutrition research is big money these days.  Our national obsession with weight loss is at a fever pitch, and any new or interesting research is sure to make headlines.

Like, oh say this recent study Maxed Out Mama referred me to on how people who eat more chocolate tend to be thinner.  Only a partial abstract is available free, but here it is.

Here's some basic guidelines on what to look for in nutritional research (any study, not just this one):

1. Was the data self reported? Even CNN brought this up in their article.  People, especially those embarrassed about their weight, don't accurately assess what they eat.  My mother, skinny little thing that she is, could eat one peppermint patty and tell you she'd had a serving of chocolate.  I don't think I'd even count it until I had 3 or so.
2. How much was "more"? I actually can't find this for this study.  Is it the difference between 1 and 2 servings per week?  Or the difference between 1 and 5?  Both would produce statistically significant correlations, but the practical outcome would be different.  In 2005, researchers made news by saying that eating more fruits and veggies did not, in fact, prevent cancer.  The cancer treating establishment (which I work in, btw) promptly responded by pointing out that they compared people who ate half a fruit per day to those who at 1-2 fruits per day.  It was all reported in grams too, so the data look extra impressive "Those eating less than 114 grams showed no difference from those eating 367 grams".  The link gives more examples, but 250 grams is one medium apple.  Watch out for this.
3. Who classified people as "normal weight" or "overweight"?  If this was also self reported (and in this study, it looks like there were clinic visits), then look out.  There's a great study I can't find right now that shows that women tend to lie about weight, and men tend to lie about height.  Both lies will screw up the BMI calculation (the most common metric for assessing "normal").
4. Were the overweight people actively (or even somewhat) trying to modify their diets to lose weight?  A few years ago, I heard about the study that suggested diet soda was linked to obesity.  I remember my first reaction was "are we sure they're all not just on diets?".  This seemed like a classic correlation/causation issue.  All the analysis seemed to presume they were overweight because they drank diet soda.  I wondered why they never seemed to look at the idea that they could be drinking diet soda because they were overweight.  That's one of the first swaps most people I know make when they try to lose weight.  
5. Don't even get me started if it's a population study.  That's a big topic for another time, but lets just say they're really really tricky.

If you ever want a fabulous crash course in how nutrition research can be skewed, pick up two diet books that contradict each other, and read through their parts on research.  Take something like Atkins (high protein, low carb) and Joel Fuhrman (nearly vegan), and watch them rip to shreds the research the other one builds their whole case on.  

He may have his own controversy, but this is why I like Michael Pollan.  The book I linked to has a great crash course in why most nutritional research just sees what it wants to.  He refused to take a strict nutritional stance and instead condensed it down to a few "rules" that he gleaned from quizzing nutritionists on "what they could say for sure".  The answer? Eat real food, not too much, mostly plants.