I cannot claim that I have recent information but, about 30 years ago, when I was studying Japanese culture -- trying to decide to make the leap and start learning Japanese (one of my favorite Aunts was born in Japan and that has always attracted me to the country and its people) -- I came across an interesting tidbit. It seems that, at least at that time, minutely detailed polls were conducted with people. Things like "do you brush your upper teeth first or your lower teeth first?" or "Do you sleep on the left side of the bed or the right side of the bed?" Many readers would go to the poll results of previous polls before checking out the rest of the paper. It is not unlike enjoying the horoscopes first, or the crosswords, or the comic page of a newspaper (back when printed media held a primary position). The difference was the importance placed. It mattered to the readers as to what the answer was. Were they doing it "right"? In Japanese culture, the group comes first.
Within the US, and many countries, polls are associated with political matters. Popularity, or unpopularity, of politicians. Popularity, or unpopularity of issues and possible approaches to those issues. It is not unusual for the polls to "say" one thing in an election and, following the election, have the results vary substantially from what the polls forecast.
For political candidates, the questions on the poll are fairly direct and simple. Do you, or do you not, favor Politician X. In the case of issues, however, it is much like a general survey. How well are the questions phrased? Are there assumptions that lie behind the question. "Do you think that we are doing the correct response to situation A?" This question assumes that situation A is as stated. If situation A is different from that which is stated then the entire question becomes invalid. Given Situation Z do you prefer option G, H, J, or K? What happens if Situation Z is unlikely (or impossible)? What if you don't like G, H, J, OR K? Once again, any results from the poll question becomes unusable.
Let's assume that the question is either simple and straight-forward OR it is formulated in such a manner such that the situation is true and the possible solutions cover the possibilities. What is the next problem in getting good results from a poll (or survey)? Audience. A survey, or poll, is supposed to be getting samples of answers from a small audience that can be applied to a much larger population.
So, is the sample representative of the larger population? It doesn't have to be representative for ALL people. If the poll/survey concerns something related to college students, then it has to represent all college students. If it concerns only homemanagers, then it only has to represent all who do the multiple-job position of a homemanager. But the sample must be an accurate representation of the population. If you multiply the sample by a hundred, thousand, or a million then the sample should expand out into a close copy of the full population. What are possible problems:
- Getting in touch with a representative population. Once upon a time, this could be done via door-to-door polling or telephone polling. Not anymore. The population connects with the rest of the world via many different potential avenues. And, by leaving out all of the people who communicate only via SMS, you may no longer be representative. Or by leaving out all of the people who communicate only via a particular social app, or game chat conduit. Not only does this make it much harder to communicate with your representative group but it makes the poll/survey much more complicated, and expensive, to do correctly.
- Getting people to respond. Our landline rings 10 times a day. We only answer if we recognize the caller (and occasionally that is spoofed and the caller is someone unknown). Some days, we never answer the phone. A message can be left -- but usually they do not leave a message. The same situation exists in other types of connections. Even if you locate the appropriate people, they have to be willing to respond.
- Will they give a truthful answer? For some topics and questions, the respondent won't want to tell you what they really feel. They feel that it is the "incorrect" answer and may be bashful, or ashamed, to tell you. But, when they have anonymity, they will answer as they really feel. Closed, anonymous, survey answers can avoid this -- if they are trusted to remain anonymous -- but that makes the survey/poll more complicated, and expensive, once again.
More than ever, polling is harder to do accurately. So, how much faith should be placed in a poll? Given the lack of an alternative, they may be better than nothing but don't expect them to be fully accurate. In addition, people like to be "part of the winning team" and, if they think an answer is more popular and acceptable, they may choose answers based on what they think is the "winner". Whether they will continue to support that opinion is unknown.
Polls are meant to be an indication of the thoughts of a larger population. But improper use of polling methods can actually be used to influence the thoughts of the larger population. In a parallel sense with the group mindset of the polling of the Japanese population as mentioned at the beginning of this blog, the poll results can be used as indicators to the larger population as to what they "should" think. Thus, polling can work both directions. The sample can reflect the opinions of the larger population and the larger population may allow the smaller sample to indicate the opinions that they should have.
Be careful with the interpretations of poll results and carefully examine the questions and answers.
