My Operaattractive jerks and selection bias and proxy variables! Oh my!
Monday, January 5, 2009 4:34:51 PM
A poster at ladyblog offered Selection Bias as an answer to the perennial question of why girls like jerks, and nice guys finish last. That isn’t something I have to worry about anymore, but my criminal justice students might be more interested in selection bias in this context than the political polling context I usually use, so I figured I’d take a look.
She says:
I'll let that crack about my nerdy fashion sense go this time. Moving on...to test her claim I used data on roughly 1800 dating decisions in a Columbia University speed dating study (available at http://www.stat.columbia.edu/~gelman/arm/examples/speed.dating/). I had to massage the data set into a format where each observation is a single decision (by a woman) to date or not to date, with information about what she thought of the guy, and what the guy thinks about himself.
Here’s what I got. If the selection bias hypothesis is correct, we should see the following (I think):
To test this, I had to create two new measurements, Jerkiness and Hotness. For Jerkiness I created a scale out of several variables, essentially measuring the size of a guy’s ego and whether they are as a fun as they think they are.
The women in this data set rate attractiveness, sincerity, and intelligence about equally, and all much more important than being fun, ambitious, or having shared interests. So my hotness scale simply adds up a guy’s score on those three variables (in the girl’s eyes). I arbitrarily said that above average guys are “Hot Guys” and below average guys are not.
That done, I imported the whole thing into STATA to see what turned up. Short story--the story doesn’t hold up. Here are the technical results, with ordinary English results afterwards.
What does all this mean in English?
First, there doesn’t seem to be much sorting of jerkiness by hotness, or vice versa. In other words, hot guys are just as likely to be jerks as average guys, which suggests that selection is not taking place--hot guys don’t seem to let out their inner jerks any more than average guys. If anything, the kind of guy that girls in the study found attractive was less of a jerk than non-attractive guys.
Second, being a jerk does not appear to extract a price from a guy’s chance at getting a date. In fact, for hot guys, being a jerk appears to increase his chances--even when we control for hotness. In other words, for two equally attractive/smart/sincere guys, the jerk is more likely to get a date.
In other words, the selection bias idea is neat (or I wouldn’t have spent an hour doing this), but unfortunately, it appears to be wrong.
The tricky part, though, is how we measure “Jerkiness.” The researchers never asked participants “Is this guy a jerk,” so I had to use an arbitrarily created scale as a proxy measure instead. If that scale doesn’t accurately indicate who girls think are jerks, then we can throw out all my results. Is it accurate? I really haven’t a clue. Still, mabye I can get my students to think about it.
She says:
Jerks get girls. This deeply regrettable morsel of conventional wisdom is true. It is likely, however, that the lesson that men often draw from it – that it pays to be a jerk in the dating market – is false. How can these apparently contradictory claims be simultaneously correct? The answer, dear reader, lies in a deep, dark secret harbored by nerdy statisticians who hope to offset their lack of fashion sense with superior analytical ability. This answer is: selection bias.
I'll let that crack about my nerdy fashion sense go this time. Moving on...to test her claim I used data on roughly 1800 dating decisions in a Columbia University speed dating study (available at http://www.stat.columbia.edu/~gelman/arm/examples/speed.dating/). I had to massage the data set into a format where each observation is a single decision (by a woman) to date or not to date, with information about what she thought of the guy, and what the guy thinks about himself.
Here’s what I got. If the selection bias hypothesis is correct, we should see the following (I think):
- H1: Among attractive men, the average jerkiness should be higher (this is the selection bias in action--hot guys can let out their inner jerks, average guys have to keep it in check)
- H2: Jerkiness should reduce the chance of a girl saying “yes,” but,
- H3: The jerkiness effect in H2 should be lower for hot guys than for average guys.
To test this, I had to create two new measurements, Jerkiness and Hotness. For Jerkiness I created a scale out of several variables, essentially measuring the size of a guy’s ego and whether they are as a fun as they think they are.
- His expectation of how happy they’d be with their partners (lower=more jerky),
- His expectations of how many girls would want to date him (higher=more jerky),
- His self-rated “funness” minus the girl’s opinion of how fun his is (higher=more jerky).
The women in this data set rate attractiveness, sincerity, and intelligence about equally, and all much more important than being fun, ambitious, or having shared interests. So my hotness scale simply adds up a guy’s score on those three variables (in the girl’s eyes). I arbitrarily said that above average guys are “Hot Guys” and below average guys are not.
That done, I imported the whole thing into STATA to see what turned up. Short story--the story doesn’t hold up. Here are the technical results, with ordinary English results afterwards.
- H1: Comparing the mean jerkiness of hot guys and average guys, I found that Hot guys are less likely to be Jerks. The difference isn’t huge (average guy mean jerkiness=2.4, hot guy mean jerkiness=.08, on a variable that ranges from -12 to 27), but big enough that we can be confident of our result (p<.001).
- H2: A logistic regression to find the effect of a guy’s jerkiness on the girl’s willingness to say yes to a date, I found no appreciable effect (p.=841).
- H3: To test this, I did separate regressions (like the one from H2) for observations categorized as hot guys and observations that weren’t. Among average guys, jerkiness has no effect (p=.851). Among hot guys, though, jerkiness actually increases the probability of a girl saying yes (p<.052 when holding hotness constant among that subgroup)
What does all this mean in English?
First, there doesn’t seem to be much sorting of jerkiness by hotness, or vice versa. In other words, hot guys are just as likely to be jerks as average guys, which suggests that selection is not taking place--hot guys don’t seem to let out their inner jerks any more than average guys. If anything, the kind of guy that girls in the study found attractive was less of a jerk than non-attractive guys.
Second, being a jerk does not appear to extract a price from a guy’s chance at getting a date. In fact, for hot guys, being a jerk appears to increase his chances--even when we control for hotness. In other words, for two equally attractive/smart/sincere guys, the jerk is more likely to get a date.
In other words, the selection bias idea is neat (or I wouldn’t have spent an hour doing this), but unfortunately, it appears to be wrong.
The tricky part, though, is how we measure “Jerkiness.” The researchers never asked participants “Is this guy a jerk,” so I had to use an arbitrarily created scale as a proxy measure instead. If that scale doesn’t accurately indicate who girls think are jerks, then we can throw out all my results. Is it accurate? I really haven’t a clue. Still, mabye I can get my students to think about it.
How to use Quote function: