In this post I mentioned my annoyance at perception/prediction polls. The one I referenced was based on women who didn't change their last names and their level of marital commitment. Commenter Assistant Village Idiot mentioned another example, which I also liked ""Do you think earthquakes are more likely now because of climate change?" What we think has nothing to do with anything. The earthquakes will happen according to their own rules."
In writing that post however, I forgot to mention that same study included an even worse piece of data. As a rebuttal to the "Midwestern college kids don't think non-name changing women are committed" they included a remark that women who didn't plan on changing their names didn't feel less committed.
I HATE STATEMENTS LIKE THAT.
I would really love it if someone could tell me if there's a proper name for this sort of thing, but I always think of it as "the embarrassing question debacle". Basically, researchers ask people questions with a potentially embarrassing answer, and then report it as meaningful when people do not answer embarrassingly.
There are only two types of people I have ever heard who will admit they went in to their marriages less than completely committed:
- Those who have been married successfully for quite some time who are now comfortable in admitting they were totally naive when they walked down the aisle.
- Those who are already divorced and reflecting on what went wrong.
Level of commitment is best assessed in retrospect, and I look with great skepticism at anyone who says they can gauge it before the fact.
Getting at the reasons people do things can be brutal. Your only source for your data also has the biggest motivation to conceal it from you. Some people are actually doing things for good reasons, some just want to look like they are, and some are lying to themselves. Unless a study at least attempts to account for all 3 scenarios, I would hold all answers suspect.
Amen and amen.
ReplyDeleteWhen it's not a subject I care about much, I can look at such errors analytically, as examples of birds-chirping or elephant-trumpeting. People making noises to just bond and say "I'm human... I'm here...I'm this sort of firefly." It's not meant to have content, it's meant to allow tribe members to touch each other's hands.
When I care about the subject, though, my nostrils flare, because people are going to be influenced by this. There is a tribal feedback loop of opinion that allows people to keep taking the pulse of those around them and titrate where they are going to stand. Even subtle and unmeant suggestions that women who change/don't change are better/worse bends the curve a bit.
I did think of another correlation nuance, which you mention indirectly here. There could be a subgroup of women who don't change their names who are measurably less committed by some standard who skew the results for the whole group. 10% who were pathologically undercommitted make the whole group look slightly less committed.
And also, what about the hyphenateds and who they drew from each group over the years...and also, the college students in 1990 were born in 1970, and knew only younger women who had not changed their name versus 2006...and, and, and. I can find possible theories and explanations all day - it's what I'm good at. But they remain only speculations unless we have some data that might at least point us to a theory.
About that correlational nuance...it's funny, I had actually thought of one in the opposite direction. I was thinking that perhaps it's that the MOST committed (or at least signaling) women had been taken out of that group. You rarely hear a woman say "my goal in life is to be a stay at home mom and raise babies" follow that up with "oh and I'm keeping my name".
DeleteOne small related conjecture: in 1990 the school could have had a professor or two who hadn't changed her name who the students liked. In engineering I had very few female professors, but not one of them had changed their name (though interestingly at least one or two were married to other faculty members). Alternatively, in 2006 there could have been a unliked or divorced professor who kept her name.
"I would really love it if someone could tell me if there's a proper name for this sort of thing, but I always think of it as "the embarrassing question debacle"."
ReplyDeleteIt sounds like the question might most resemble a Complex Question fallacy, where the person is asked a 2 part question in disguise as a single question, so he/she is guaranteed to fail or embarrass himself/herself in any answer given. The classic example is "Have you stopped beating your wife?" because both yes and no answers would imply that he beat his wife at one point in time. The question should be broken into "Do you beat your wife?" and "If so, have you stopped?".
The situation also resembles Faulty Dilemma, where a scenario is set up as either-or when that is not the case. In this instance, women are portrayed as either committed or retaining their maiden names, when clearly the issue is much more complex.
I teach this stuff to my AP students...I love trying to get them to understand how to break apart political rhetoric and other arguments around them. I figure even if we disagree wildly in politics or social issues, at least I'll have an intelligent opponent to argue with someday.
Commenter Assistant Village Idiot mentioned another example, which I also liked ""Do you think earthquakes are more likely now because of climate change?" What we think has nothing to do with anything. The earthquakes will happen according to their own rules."
ReplyDeleteActually, to me, this is an interesting topic - what do people think of when presented with some bit of scientific research? How did they arrive at their conclusions? What influences them more - the scientists or the media? Or their Cousin Larry? I'm working on a post related to peoples' opinions/positions on climate change.