When conducting research, significant attention is paid to data collection, so once respondents' answers are collected, they are assumed to be correct by default, and the report based on such answers is deemed objective. However, situations often arise where a more detailed review of individual responses reveals clear misunderstandings by respondents of the survey phrasing or instructions.
1. Misunderstanding of professional terms or certain words. When designing a survey, it is important to consider the target groups of respondents: the age and status of the participants, whether they live in large cities or remote villages, etc. Specialized terms and various slang should be used cautiously, as they may not be understood by all respondents or interpreted uniformly. Nevertheless, this misunderstanding often does not prompt respondents to abandon the survey (which would, of course, be undesirable), and they may answer randomly (which is even more undesirable due to data distortion).
2. Misunderstanding of the question. Many researchers believe that every respondent has a clear and unambiguous opinion on every question. This is not the case. Sometimes, survey participants find it difficult to answer a question because they have never considered the subject as a whole or from that perspective before. This difficulty can lead to respondents abandoning the survey, or answering in a completely uninformative manner. Assist survey participants by clearly articulating the question and providing a variety of response options.
Source: news.sportbox.ru
3. Misunderstanding of survey instructions or specific questions. Like the entire survey text, the wording of the instructions must be tailored for all groups of intended respondents. Try to avoid a large number of questions where a specific number of responses must be marked (e.g., 'Select the three most important...'), or in all similar questions, specify the same number of responses that need to be marked. It is also worth simplifying complex types of questions (matrices, ranking, etc.) by replacing them with simpler ones. If you believe that respondents may answer the survey from a mobile phone, further simplify the survey structure.
4. Misunderstanding of the rating scale. When using a rating scale in a survey, explain the meanings to respondents even if they seem obvious to you. For example, the familiar scale from 1 to 5 is often understood in analogy with the school grading system; however, sometimes respondents mark '1', attributing it to the value of first place. In verbal scales, it's better to avoid subjective criteria. For example, the scale 'never – rarely – sometimes – often' is very subjective. Instead, it is advisable to offer concrete values ('once a month', etc.).
5. Generalized-positive and averaged assessments. The tendency of respondents to give generalized-positive evaluations often interferes with, for example, surveys of software users and other similar studies. If a user is generally satisfied with your program, it's difficult for them to break it down into parts and evaluate the personal cabinet, new functional solution, etc., separately. Most likely, they will give high marks across the board. Yes, the report on the survey results will look very positive, but the results will not allow for a real assessment of the situation.
Averaged assessments often hinder, for example, during a 360-degree employee evaluation. Employees tend to assign an average score across all competencies: if their relationship with a colleague is positive, you will see inflated scores throughout the survey; if relations with a colleague are tense, even their clearly strong leadership qualities will be rated low.
In both cases, it's wise to thoroughly work through the response options, replacing familiar scales with detailed verbal answers for each individual question.
6. Manipulating opinions. This item differs from the previous ones in that researchers consciously guide respondents toward advantageous answers for a 'successful' report. Common methods of manipulation include the illusion of choice and focusing on positive characteristics. Usually, managers studying the positive results of a survey do not consider the correctness of data interpretation. However, it's worth taking an objective look at the survey itself: what is its logic, is there a certain bias in the questionnaire, and are positive and negative response options evenly distributed? Another common trick for 'stretching' data is the substitution of concepts. For example, if most employees rated the new incentive program as 'satisfactory', the report might state that 'the majority of employees are satisfied with the new incentive program.'
Source: habr.com
