An in-depth look at the Regnerus study

I’ve promised you this post for a while now.  Because this seems to be a misuse of science, it’s a frustrating article to write, and the Regnereus study is a hard read… But enough with excuses!

Mark Regnerus did a study with University of Texas called the New Family Structures Study.  He recently published a paper in Social Science Research (SSR) entitled “How different are the adult children of parents who have same-sex relationships? Findings from the New Family Structures Study.

This has been controversial for a number of reasons.

  1. It was rushed through the review process.
  2. It contradicts what every other well-designed comparative study has found
  3. It appears that the way the data was classified for sampling was manipulated in such a way as to produce bad outcomes.
  4. The funding for the study was from conservative foundations with anti-gay leanings.

Ultimately this has resulted in a letter from 205 academics, including 5 Uni of Texas faculty members, writing an open letter to Social Science Research which essentially suggests that they may now consider it a second-grade journal.

UT is also conducting an investigation into scientific misconduct by Regnerus.  This is an independent panel of UT professors who are looking into the claims that Regnerus essentially faked poor outcomes for same-sex-parented kids.

So I’m going to take these points in order… (any page references are to the page number in SSR)

1. It was rushed through the review process

The timeline is

Received on Feb 1
Revised Feb 29
Accepted March 12

The peer review policy of SSR states that their process is as follows

  1. Editor evaluates it to make sure it meets minimum criteria
  2. 2 experts are secured for peer review.  The review process typically takes 2-3 months.  “But substantially longer review times are not uncommon”.
  3. Revised manuscripts are usually returned to the initial referees upon receipt.
  4. The editor then has the final decision to include the paper.

So in this instance, it went through not one, but two iterations of the review process in about half the time it “typically” takes for a paper to be reviewed once.

This on its own is not necessarily a problem.  There may have been two appropriate reviewers available to read and review it straight away…  It’s rare, but it happens.

2. It contradicts what every other well-designed study has found

This is not specifically a problem either.  Science finds out that it was wrong in all sorts of fascinating and interesting ways, with surprising frequency…

In the introduction to Regnerus’s paper, he notes that “Since [Stacey and Biblarz’s Study in 2001] the conventional wisdom emerging from comparative studies of same-sex parenting is that there are very few differences of note in the child outcomes of gay and lesbian parents. … Moreover, a variety of possible advantages of having a lesbian couple as parents have emerged in recent studies.” (p753)

There is nothing wrong with a scientist contradicting the status quo, whatever your local quack might suggest.  The thing is, when you are overturning established theories, you need damn good evidence to suggest that you are right!  And you’d better be prepared for your studies to be picked apart – wrong presumptions in some circumstances mean that whole fields need to restart using the new theories.

Curiously, given the apparent manipulation of the data in this study, he later talks about sampling concerns in previous studies – I’d classify that as bold…

3. Misclassified data, other sample problems

The real scientific problems with this paper all stem from the sampling.

The study surveyed young adults, 18-39 (p755). For the purposes of Regnerus’s study, they were divided into a number of groups:

IBF – Intact biological family – lived with parents form 0-18 and parents are still married.

LM – Lesbian Mother – Respondent’s mother had a same sex relationship with a woman

GF – Gay Father – Respondent’s father had a same-sex relationship with a man

Adopted: Adopted by two or more strangers between birth and age 2

Divorced later: lived with parents to age 18, parents are not married at present

Stepfamily: Biological parents where Respondent’s primary custodial parent was married to a step-parent before Respondent turned 18

Single Parent: Biological parents were either never married or else divorced, and the primary custodial parent did not marry/remarry before Respondent turned 18

Others: Any other family situation, including deceased parent.

There are some definition problems here that I will address shortly, but the issue noted by the UTexas letter relates to classifications…

“Respondents might fit more than one group.  I have, however, forced their mutual exclusivity here for analytic purposes.  For example a respondent whose mother had a same-sex relationship might also qualify in [Divorced] or [Single Parent], but in this case my analytical interest is in maximising [LMs] and [GFs] so the respondent would be placed in [LMs].”  GF is the trump card here, “since [GFs] is the smallest and most difficult to locate randomly in the population”.

There were 12 respondents in the study to have both an LM and a GF – they were all represented as GF for the study.

Lets think about what this classification decision means…

The only groups not eligible for consideration as LM or GF were IBF or Adopted (since the parents in same-sex relationships were biological for LM or GF).

What this means is that the study, when looking at LMs and GFs include Divorced, Stepfamily, Single parents and Other family configurations.  We do not know if any respondents were raised exclusively by one pair of gay parents using surrogacy or a sperm donor .

Given that IBF is the control group here, what we’re doing (I would suggest almost exclusively) is comparing married people with situations where children were raised in multi-homed or single-parent situations.

This study does not compare similar relationships

And then there’s the definition problem: “had a same-sex relationship” – as far as we know there’s no requirement that this happened while the child was growing up.  it just has to have happened.  If my mother were to call up tomorrow or even 10 years from now and announce she was in a relationship with a woman, I would then be classified as a LM although my entire upbringing was in an IBF.  Or my parents might have divorced and my father taken a male lover for a fortnight – if I as a child perceive that as a relationship, he’s a GF for sure.

In short, the study’s sampling methods promote a misinterpretation of the data by comparing non-comparable relationships and drawing conclusions on the quality of gay parenting based on those comparisons. It also does not seem to draw any requirement of meaning or permanence in a same-sex relationship before qualifying participants as Gay Father or Lesbian Mother.

4. Funding sources

The Witherspoon Institute and the Bradley Foundation “are commonly known for their support of conservative causes” (p755)  Places fund studies all the time – the wages have to come from somewhere… grad students don’t just grow on trees y’know!  The funding only becomes a problem when the sample or other methodologies seem to be constructed to be biased in one direction or another – like the data in the NFSS…

Conclusions

The study is flawed.  This is pretty clear, and plenty of people actually qualified to pick this study apart agree with me (I’m a computer scientist, not a social scientist).  This along with the other  problems like the amazingly short review period, the funding sources, and the fact that it goes against what every other researcher has found suggest some rather odd dealings.  It seems strange that SSR reviewers who were not already sympathetic to the article would miss the errors that I and many others noticed.

Ultimately, I think Regnerus has done his reputation and that of SSR some serious damage.  In the process, they’ve produced a study the religious right will use to advocate against same-sex parenting, but which has essentially no relevance, validity, or accepted authority on the matter.

If you have any questions, feel free to leave a comment!   I look forward to hearing from you!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s