My last post offered general guidelines for evaluating researchers and organizations who create data about American religion. Using these guidelines, here are some of the good, and not so good, sources of religion data.
Sources of Really Good Data
We are fortunate to have lots of high quality data about American religion. These sources include:
Pew Center on Religion and Public Life. I like Pew a lot. They hire good people and collect lots of valuable data. Their reports describe rather than interpret religion, which minimizes ideological bias. They focus on the way things are, not the way somebody thinks that they should be.
Gallup. Gallup is the survey expert, and they ask lots of questions about religion. Because Gallup has been collecting data since before World War II, their data can be used to track long-term trends in American religion.
538. The data-journalism website fivethirtyeight.com doesn’t write about religion often, unfortunately, but when they do it’s really good. A while ago they covered one of my studies, and they did a great job—accurately summarizing it and correctly identifying its primary methodological limitation.
General Social Survey (GSS). The GSS is the gold standard of American social surveys. It’s been collected every year or two since 1972 with strong survey methodology. Every wave of the GSS contains a base set of religion questions plus occasional new ones. Descriptions of American religion that use GSS data are usually trustworthy.
Sources of Data that Could Be Good or Bad
Some sources of religion data are a mixed bag. They can be informative, but they require scrutiny.
Academic research. Academic researchers are (usually) well-trained, and we submit our research to a peer-review process. This (usually) catches obvious errors and mistakes. But it misses ideological bias, and we professors certainly have that when it comes to religion. Far fewer professors believe in God relative to the general population, and this shows up in our work. When I read academic research on religion, I watch for this bias.
Journalists. Journalists find new and interesting research, but they usually don’t have the time or expertise to carefully evaluate it. Furthermore, journalists have strong financial incentive to report what is interesting, unexpected, or provocative. This gets clicks and sells copies. It also skews our understanding of American religion. When I read about a religion study in the media, I’ll sometimes go find and read the original study. Often the journalist has misrepresented an important aspect of the study. There are a few reporters who focus exclusively on religion—that is their beat—and they do good work. Unfortunately, their number has declined over the years as media companies have pared back their budgets.
Christian Marketers. There’s a cottage industry of people and organizations who collect data about the Christianity. They use these data to sell articles, books, and consulting services. Some of these groups refer to themselves as Christian marketers. Their expertise varies as does the quality of their data. While they have good intentions, they have financial incentive to create data that sells. This can lead to negative or sensationalistic portrayals of Christianity.
Sources that are Always Bad
Activists. Activists work to make the world a better place, and that is commendable. But they are a terrible source for data. They want data that supports their cause. Period. Accuracy might be an added benefit, but it’s not required. When activists present accurate data, it’s only because those data happen to support their work. Many times activists have argued against better data because they disagree with its implications.
Rogue statistics. Some religion statistics are passed around, widely accepted, but nobody knows who created them. They are little more than urban legends dressed up as an empirical fact. Let’s call them rogue statistics. For a variety of reasons, rogue statistics are invariably wrong. They also hide well, being difficult to identify. Suppose that Writer A includes a rogue statistic in his/her book with no citation. Reading it, you could figure out that the statistic is rogue. But then Writer B comes along and includes this statistic in his/her book, and Writer B cites Writer A’s book. If you only read Writer B’s book, you might think that the statistic has a known and trustworthy origin. It doesn’t. Practicing safe statistics means always avoiding statistics that do not have known, trustworthy origins.
These are some of the best known sources of religion data, and while there are certainly other ones as well, this guide will get you started in finding good data.