Showing posts with label Research Pet Peeves. Show all posts
Showing posts with label Research Pet Peeves. Show all posts

Sunday, August 19, 2012

Does egg = cigarette?

Oh CNN, your headlines make me sad sometimes.

Is eating egg yolks as bad as smoking?

No.  No it is not.  The study you're reporting on does in fact claim that eating egg yolks accelerate heart disease about 2/3rds as much as smoking does, but acceleration of heart disease is not actually the health problem smoking is most known for.  But you know that.  Sigh.

Not that I'm buying the study anyway.  They asked people, who already had heart disease, to report their egg yolk consumption over the course of their lives.  How accurately can you recall your average egg yolk consumption over the course of your life?  Additionally, people who have heart disease have most likely been told to cut down on consumption of saturated fat and cholesterol.  Those still eating more eggs have likely heard this advice, and disregarded it.  What are the chances that they're disregarding other advice as well?  Lastly, it does not appear the study asked about the consumption of any other food, meaning egg consumption could actually just be co-occuring with the consumption of something else that was even worse.  Surveys that ask only about very specific foods tend to see what they want to see.

So basically, another correlation/causation issue here, combined with those terrible consumption recollection surveys, with a sprinkle obnoxious headline writing.   Yeehaw.


Wednesday, May 16, 2012

Correlation and Causation: the Teen Pregnancy Edition

One of the first posts I ever did was on correlation and causation.  In it, I spelled out the three rules to consider whenever two variables (x and y) are linked:


  1. X is causing Y
  2. Y is causing X
  3. Something else is causing both X and Y
While most people jump to the conclusion that it's number 1, Matthew Yglesias wrote a piece for Slate.com this week where he rather awkwardly jumps to conclusion number 2.  

He starts off well with the second paragraph, but then goes to very strange place in the third: 

Delivering the commencement address last weekend at the evangelical Liberty University, Mitt Romney naturally stuck primarily to “family values” and religious themes. He did, however, make one economic observation that intersects with some fascinating new research. “For those who graduate from high school, get a full-time job, and marry before they have their first child,” he said, “the probability that they will be poor is 2 percent. But if [all] those things are absent, 76 percent will be poor.”
These are striking numbers, but they raise the age-old question of correlation and causation. Does this mean that the representative high-school dropout would be doing much better had he stuck it out in school for a few more years? Or is it instead the case that the population of high-school dropouts is disproportionately composed of people who have attributes that lead to low earnings?
When it comes to early pregnancy, surprising new evidence indicates that Romney and most everyone else have it backward: Having a baby early does not hamper a young woman’s economic prospects, as Romney implies. Rather, young women choose to become mothers because their economic outlook is so objectively bleak.
Say what?

As a former teenage girl myself, this is a strange conclusion....I certainly never met a teen mom who would have put it that way.  But surely there was some wonderful evidence to support this scathing conclusion?

Well, not really.  Here's the original paper....and  here's how the authors conveyed their thoughts:

We describe some recent analysis indicating that the combination of being poor and living in a more unequal (and less mobile) location, like the United States, leads young women to choose early, non-marital childbearing at elevated rates, potentially because of their lower expectations of future economic success. ...These findings lead us to conclude that the high rate of teen childbearing in the United States matters mostly because it is a marker of larger, underlying social problems.
The emphasis was mine....but notice how much more careful they are in their language.  If you take my list above, you see that they are challenging possibility number 1, seeing if #2 is a feasible conclusion, but ultimately pointing the finger at #3....i.e. "larger, underlying social problems".

For example, the cite low maternal education as a risk factor for teen pregnancy...which one could presume could be either the result of or the cause of low income.

Teen pregnancy is complicated, and honestly I would be very surprised if you could ever figure out a way to pin it on just one factor.  Additionally, so much information is unavailable that it can be hard to parse through what you have left.  A key factor in all of this would be to determine if higher income girls weren't having babies because they weren't getting pregnant or because they were having abortions....data which could lead to very different conclusions.

I fully support this study, by the way, questioning the prevailing wisdom is always a good thing. What I resent is when people think just by flipping the order of a normal conclusion that they're being clever.

X could cause Y, Y could cause X, something else could be causing both.

Then again, it could also just be a coincidence.  

Friday, April 20, 2012

Food Deserts and Big City Living

The Assistant Village Idiot did a good post on a new report on the prevalence of "food deserts" and if this was the crisis it's been reported to be.

While I will point out that the study refuting the idea of food deserts uses self reported data for height, weight and eating habits (check out my previous post on this issue), I was glad to see someone take this issue on.  Food deserts reporting has always fascinated me, mostly because I lived in the middle of the Boston area (albeit in different locations) for about 9 years.   The food desert idea always sort of baffled me, and when I took a look at the USDAs food desert locator, I notice that the only part of Boston proper or the close suburbs that qualifies as a food desert is.....Logan Airport.

I currently live in a suburb that is near 2 food deserts, so checking those out was interesting as well.  One is actually a small peninsula, and I happen to know you have to drive by a grocery store to get on the main route out there.  The other is next to the docks.

For cities, this data gets complicated by the fact that many very small grocers sell all sorts of produce in small spaces that wouldn't make the list.  For rural areas, personal gardens are not counted.  I also liked that the article pointed out that some people researching this have done grocery stores/1000 people, a metric which make cities look bleak.  That's a classic case of needing to review why you actually want the data.  A busy grocery store is not a lack of a grocery store.  Additionally, I have never seen one of these surveys that added in farmer's markets or grocery store delivery services.  While not always the cheapest option, delivery services allowed me (when I was a broke college student) to buy in bulk and save money other ways.  They run about $7 ($5 when I was in college), when a train ride to and from the store was $4 round trip, and a taxi would have been at least $10 (not counting ghost taxis that exist almost exclusively in front of city grocery stores and help you with your groceries for around $5).

Overall, I'm sure access is an issue for some people, I just balk when people who don't live in the middle of cities on a limited budget like I did try to tell me what it's like.  I DO think that before we flip out about an issue, doing research as to how much access really affects obesity is key.  The number of regulations and reforms that get pushed without any data proving their relevance staggers me, and I'm glad to see someone questioning the wisdom in this case.

Saturday, March 31, 2012

Say What?

I can't figure out if this is the worst statistic I've read this week, or just the most poorly phrased good statistic.  I'm leaning towards that first one:

"....more than half of students earning bachelor’s degrees at public colleges – 56 percent – are graduating with $22,000 of debt, on average."  -Nancy Zimpher on CNN.com

If I'm reading this correctly, they tossed out everyone at private college, then anyone who didn't graduate, then (most disturbingly) 44% of who was left?  Why did they get the boot?  I mean, if we're just tossing out arbitrary numbers of students, can't we get any average we want?

Nancy Zimpher, what are you up to??? 

Tuesday, March 27, 2012

THE KIND OF DATA I ABSOLUTELY HATE

Hate's a strong word.  I get that.  I also get that data and survey types are not always the sort of thing that inspires people to strong hatred, but here we are.

In this post I mentioned my annoyance at perception/prediction polls.  The one I referenced was based on women who didn't change their last names and their level of marital commitment.  Commenter Assistant Village Idiot mentioned another example, which I also liked ""Do you think earthquakes are more likely now because of climate change?" What we think has nothing to do with anything. The earthquakes will happen according to their own rules."  

In writing that post however, I forgot to mention that same study included an even worse piece of data.  As a rebuttal to the "Midwestern college kids don't think non-name changing women are committed" they included a remark that women who didn't plan on changing their names didn't feel less committed. 


I HATE STATEMENTS LIKE THAT.

I would really love it if someone could tell me if there's a proper name for this sort of thing, but I always think of it as "the embarrassing question debacle".  Basically, researchers ask people questions with a potentially embarrassing answer, and then report it as meaningful when people do not answer embarrassingly.

There are only two types of people I have ever heard who will admit they went in to their marriages less than completely committed:

  1. Those who have been married successfully for quite some time who are now comfortable in admitting they were totally naive when they walked down the aisle.
  2. Those who are already divorced and reflecting on what went wrong.
Level of commitment is best assessed in retrospect, and I look with great skepticism at anyone who says they can gauge it before the fact.  

Getting at the reasons people do things can be brutal.  Your only source for your data also has the biggest motivation to conceal it from you.  Some people are actually doing things for good reasons, some just want to look like they are, and some are lying to themselves.  Unless a study at least attempts to account for all 3 scenarios, I would hold all answers suspect.