Polling What People Believe
Problems with Polls
by Bob Burridge ©2018
Polls and statistical studies are often used to determine many things. They are used in politics, religious articles, debates, and in determining less profound matters such as the popularity of restaurants. The results only show the views of those answering what they believe the survey is asking about. They do not prove that what most believe is actually true, or morally right. Surveys and polls can be helpful in understanding the opinions and beliefs of a population, but if they are not conducted carefully, they can produce completely misleading conclusions.
Unfortunately what we think most people want or believe often becomes the standard that shapes the way some churches conduct their worship, design their outreach ministries, counsel their members, and set up their leadership structure.
When trying to communicate with people it’s helpful to understand how they perceive things and what they already believe. However, the final source of what the church is, what it teaches as truth, and how it conducts its work must be the written word of God, not what appeals to any particular target group of people. Within the boundaries the Bible sets there’s room for variations as we reach out to different cultures and groups. But this liberty within those boundaries is to help us communicate better. It’s not a liberty to adjust or change the truths we teach, or the principles we apply which are given to us in Scripture.
Polls or surveys can be helpful tools in determining the most effective way to explain God’s word to those who are not yet redeemed believers, or to those who are in search of a sound church where they can worship, fellowship, and grow. But the surveys need to be done properly. There are three main factors which need to be considered when a study of people’s views or perceived needs is conducted.
1. Defining the Population: The first thing we have to do is to accurately define the group we’re studying.
For example, there was a poll reported by the Pew Research Center in 2014 which attempted to determine how many adults in the United States believe in “hell” and “heaven”. The results showed that 58% believe in hell, and 72% believe in heaven. Their survey report is appended at the end of this article.
Included in that study were various groups they identify as “Christian“. If we are trying to understand what most Christians understand and believe about something, we first need to define who is being defined as “Christian”. Those taking the survey had to identify themselves with just one of five groups in the “Christian” heading. The choices were: “Protestant”, “Roman Catholic”, “Orthodox Christian”, “Mormon”, or “Jehovah Witness”. Most in those groups would not consider the others to be legitimate “Christians”.
Within the category of “Protestant” there were just three choices. You could identify yourself as either: “Evangelical”, “Mainline”, or “Historically black”. These are strange divisions which do not reflect clearly defined demarcations. Within each group there are some who accept sources of “divine revelation” other than the Bible, and some don’t accept the Bible’s definition of heaven and hell.
With such poorly defined categories those responding had to decide which group they belonged in based upon how they understood the choices listed. The results showed a wide range of responses. Over-all the survey indicated that among those identifying as “Protestants”, 75% believed in hell, and 86% believed in heaven.
When a population is not clearly and accurately defined, the results of a poll are unreliable and can be misleading.
2. Sampling the Population: It’s rare that all members of a population can be studied, so a sampling of the population must be made.
This sampling must be done with careful accuracy to be sure it represents the range and distribution of views held by the population as a whole. Some taking part may believe they belong to the larger population or a sub-group when they don’t really fit the category they choose.
Social or economic differences in the sample should reflect the percentage of each group in the larger population. The means of polling such as phone calls, postal mail, e-mail, tables in pubic places, or door-to-door surveys may not reach an honest sampling. The results may also be skewed because some might choose not to participate because of convictions unique to a legitimate segment of the population.
Admittedly the poll reported by Pew Research couldn’t ask everybody in the whole population of adults in the United States. It had to select a sample of those who claimed to belong to each sub-group. If the sub-groups are not accurately representative of the larger population the results will be skewed.
When the population being polled is not accurately sampled, the results will not reflect the actual position of the larger group.
3. Considering the Variables: All things that might effect the results of a survey or poll need to be considered.
One crucial variable that must be considered is the definition of the position being asked about. If questions in a poll are worded differently, people will understand the choices differently and will not answer the same way. The terms used should be understood to mean the same thing to all those in the whole population defined by the survey. If samples are taken in rural as opposed to urban or suburban settings there may be some ways of wording questions that reflect lifestyle differences and understandings. The percentage represented by each sub-group in the sample should be the same as in the larger population being studied.
In some cases it’s been shown that if choices were presented in a different order the results of the survey were not the same. Current events and issues cause the same questions to be answered differently than before those topics rose to greater interest or concern.
In that Pew poll the sampling results would be effected because those responding may have had a different understanding of the sub-categories they had to identify with. Also, there were no definitions reported to clarify what was meant by “hell” and “heaven”. Those responding had to rely upon their own understanding of those terms which mean very different things in the different sub-groups.
What “Christians” believe will differ depending upon the religious background of those being polled. Within each sub-group some will be active members of the body they identify with, and some will merely identify with it because of family history. The percentages of all variations in the sample need to be the same as in the primary population.
When all the variables are not considered and controlled, the results will be unreliable.
A clear example of polls gone wrong are the ones conducted before the 2016 Presidential Election for the President of the United States. Not all but most polls predicted a clear win by Hillary Clinton. They were wrong. The error was in the difficulty faced in considering the three main factors needed to make the poll accurate. It’s difficult in our world with so many means of communication to get meaningful responses from a truly representative sample of the whole population of actual voters in the United States. At times the poll questions were worded in ways that favored certain responses.
When it comes to surveys that guide us in the work and mission of Bible believing churches we should be very careful not to rely upon polls independently of the direct statements of Scripture when determining how to structure Christian worship, conduct evangelism, organize church leadership, explain the Gospel, define marriage, and determine moral boundaries.
The polls we may use to help us communicate God’s truth in a particular culture or environment must be conducted properly by carefully defining the population being surveyed, considering the method and accuracy of sampling that population, and diligently considering all the variables involved in how those surveyed respond.
We should not structure our doctrines or styles of worship based upon what the majority of people think is right or what they want. The goal is not to please some population of people. It should be to be faithful to what God tells us in his word about what is right and good.
This is a published copy from the 2014 Religious Landscape Study poll reported by the Pew Research Center:
