Here’s Why Surveys Are Still Lowballing the Percentage of Non-Religious Americans

If you’ve paid any attention to surveys that ask what religion people are, you know the responses depend a lot on how the questions are worded. Consistency matters. Not being prodded to say “Christian” by default matters. Who’s doing the asking matters.

So when you’re trying to find out what percentage of Americans are non-religious, there are a lot of factors you need to consider.

No one has dug as deeply into this as the ever-prolific Ryan P. Burge of Eastern Illinois University, who just posted about two major data sets he’s been analyzing for a while: the General Social Survey (GSS), which has gone out at least every two years since 1972, and the Cooperative Congressional Election Study (CCES), which has been around since 2006.

The surveys are great because they involve a lot of participants… and yet Burge notes they give very different answers regarding how many non-religious people are in the U.S.

In 2008, the GSS indicates that 16.7% of the population are nones, while the CCES has the estimate higher at 22.3% — a gap of 5.6%. The Census bureau reports that there are 250 million adults in the United States, therefore that difference amounts to 15 million adult Americans. And the discrepancy widens from there. In 2014, it jumped to 7.1%. By 2018, it had expanded to 8.2%. These differences can’t be chalked up to random sampling error. There has to be something systematically happening that produces such a consistent and growing discrepancy.

While it’s impossible can’t pinpoint the exact problem, a few theories make a lot of sense.

For example, how are the questions asked? On the GSS, the question looks like this:

What is your religious preference? Is it Protestant, Catholic, Jewish, some other religion, or no religion?

The CCES asks:

What is your present religion, if any?

It then offers 11 options — including atheist, agnostic, and nothing in particular (which you can combine to find a total “no religion” number).

Not surprisingly, the latter question gets a higher response from the Nones. I’m not surprised. It’s far more open-ended. It treats “atheist” and “agnostic” like every other option. Even “nothing in particular” may be less grating to some than “no religion,” which could sound harsh or negative.

(In fact, when Burge compared response rates for individual religions, the CCES found significantly lower rates for Protestants and Catholics… which suggests there are many Americans who might say they’re Christian in one survey while saying they’re not religious in the other.)

It’s because of this that atheists in Ireland and Australia have pushed for changes in how their nations’ censuses ask about religion.

Burge also notes another important difference between the two surveys, which I believe also accounts quite a bit for the discrepancies:

… the GSS is still conducted by a survey administrator asking questions of respondents, while the CCES is computer-based. People are less likely to lie to a computer than they are to a person.

It’s a lot easier for some people to tell a computer they’re not religious than saying it to a human. People are a lot more honest when they don’t risk any judgment.

Burge has even more analysis of the differences on his site. Check it out. And then remember to be skeptical whenever you see these surveys. Even when the numbers show a rise of the Nones, it’s possible we’re still seeing an underestimate because of how the question is worded and asked.