Meaningless Surveys: The Faulty Mathematics of the Nones

CE Huffpo headerCulture on the Edge’s Monica Miller and Steven Ramey co-authored the following post,
published originally at the Huffington Post on November 7, 2013.

People unaffiliated with a religion, commonly grouped as the ‘Nones’, are all the rage right now and have beckoned responses from faith leaders to philosophers and scholars of religion. Common among such responses is an unwavering and uncritical belief in the statistical reality of this group; very few, in our opinion, have questioned how this group came to exist in the laboratory of statistical analysis and myopic survey questions. Most recently, a series on the New York Times Room for Debate page featured references to the Nones and the similar Pew report on the status of Judaism in America. However, the methodological basis for all of this excitement is actually quite thin.

Might the Nones, in fact, be a social construction rather than an actual reality? If so, what’s the hype, and why aren’t people questioning the method behind the (socially constructed) madness? The excitement about the “rise” of the Nones seemingly draws on one question in surveys, such as “What is your religious affiliation?” in the Pew Center poll.

As argued previously on the Huffington Post, the excitement actually hinges on a non-answer to that question: no affiliation. Then analysts assume those who answer in such a way form a group with particular characteristics. That is, once this constructed group is re-constructed and legitimated, people begin to imbue this group with a material face, social interests and political persuasions, as if this group, always there but now with a name, is available for their commentary and speculation.

Are the survey responses actually capturing something “significant” about changing patterns in society? The Economic Values Survey report includes the interesting tidbit that 18 % of the “nonreligious” reported being affiliated with a religious identification, whether Catholic, Mainline Protestant, non-Christian religion or other Christian communities. Similarly, those who answered unaffiliated included a signify cant number who were identified by the researchers as religiously liberal, moderate or conservative, along with many of the nonreligious. These statistics reflect the findings in the Pew Forum poll that many of the unaffiliated participate in practices commonly described as religious and/or believe in some higher power. Thus, the public responses to such claims may reflect a variety of points: a defense of the faith, an embrace of non-institutional religion, or a celebration of the growing “fuzziness” of religion and spirituality.

What do such contradictory responses tell us? Perhaps, it says something about just how malleable these categories and terms are. Survey responses are also susceptible to inflated reports of presumably socially approved answers, especially with fluid identifications. That is, each person surveyed understands religious/non-religious or affiliated/unaffiliated in different sorts of ways.

So, do their responses necessarily reflect assent to certain practices or beliefs? Might assuming a correlation between affiliation/identity, belief/religion and spiritual/not religious be misleading at best?  If so, then the statistics are virtually meaningless, and participant’s responses are not comparable.

So, what are we actually responding to in all the excitement, debate and dithering over these apparently meaningless reports? They correlate closely to some common narratives about the religious nature of the United States and the shift towards a more secular, less religious society (whether understood as a good or bad development). Both those who celebrate such changes and those who fear them find statistical confirmation of their preconceived narratives. That is, the responses to such surveys tells us little about the group the surveys have claimed to statistically discover and more about the ways in which people manipulate such data to fit their own ideas and platforms surrounding religion, belief, and spirituality.  Interestingly, similar problems with polls and analysis of them are present in discussions of contemporary China.

These narratives, particularly among those fearing secularization, draw on nostalgia for a simpler time when everyone met at the local church on Sunday. Those who celebrate such changes maintain a similar narrative of prior decades but value the narrative and the presumed dominance of particular churches very differently. And that perception of church dominance is an important part of the interest in these reports, as the debates participate in contests over societal power that involves various institutions that are waxing and waning. The assumption that people’s identifications have a definite, stable meaning combines this interest in competitions of power with the desire to construct communities that can be managed through their enumeration into definite groups.

We are not denying that society is changing but want to emphasize that society is always changing. Measuring change is always in the eye of the beholder. Each generation tends to fixate on the changes that they are experiencing, and the confrontation of such change is always met with a crisis and moral panic that society can rally around, re-creating new narratives and re-presenting new ideas that are anything but new. Such assumptions depend on this simplistic nostalgic notion of what came before, which was always a matter of contest, change, and uncertainty.

So, when scholars and the New York Times create a series concerning how institutions can attract people in the current environment, it serves to promote the nostalgic view of a simplified past in a way that seems to serve the interests of institutions that are losing the influence that they, and perhaps the media and many scholars, think that they once had and should continue to have.

Thus, we ask, what’s really in a number?

Discover more from Culture on the Edge

Subscribe now to keep reading and get access to the full archive.

Continue reading