Lenzner, T., Kaczmirek, L., & Lenzner, A. (2010). Cognitive burden of survey questions and response times: A psycholinguistic experiment. Applied Cognitive Psychology, 24(7), 1003-1020.

Lenzner, T.; Kaczmirek, L.; Lenzner, A.

2009

Lenzner, T., Kaczmirek, L., & Lenzner, A. (2010). Cognitive burden of survey questions and response times: A psycholinguistic experiment. Applied Cognitive Psychology, 24(7), 1003-1020.

An important objective in survey question design is to write clear questions that respondents find easy to understand and to answer. This contribution identifies the factors that influence question clarity. Theoretical and empirical evidence from psycholinguistics suggest that specific text features (e.g., low-frequency words (LFRWs), left-embedded syntax) cause comprehension difficulties and impose a high cognitive burden on respondents. To examine the effect of seven different text features on question clarity, an online experiment was conducted in which well-formulated questions were compared to suboptimal counterparts. The cognitive burden of the questions was assessed with response times. Data quality was compared in terms of drop-out rates and survey satisficing behavior. The results show that at least six of the text features are relevant for the clarity of a question. WWe provide a detailed explanation of these text features and advise survey designers to void them when crafting questions.

The present findings show a strong support for the relevance of text features on survey satisficing behavior to examine the effects of cognitive burden on data quality. First, the overall effect of text features on total response times was highly significant. Secondly, six text features differed significantly between conditions: respondents answering the suboptimal questions had longer response times. The highest impact was shown for complex syntax and working memory overload.

985

28