Questionnaire Pretesting Methods: A Comparison of Cognitive Interviewing and Respondent Debriefing Vis-à-vis the Study of the Adoption of Decision Support Systems by Knowledge Workers

Emmanuel Matthew Ikart

《International Journal of Business and Information》 13卷2期 (2018/06) Pp. 119-154

https://doi.org/10.6702/ijbi.201806_13(2).0001

摘要

As a vehicle for data collection, the questionnaire is a vital component in achieving higher quality in research studies. A frequent difficulty with questionnaire design is that respondents often misinterpret the questions. This difficulty has been recognized in the literature. In recent years, there has been increased emphasis on building quality into questionnaire design through pretesting. A pretest aims to ensure quality by subjecting the questionnaire to some types of evaluations to assess its ability to collect the desired data. Whether constructing a new scale or revising an existing scale, through pretesting, researchers may be able to confirm whether the scale uses clear and appropriate language, has no obvious errors or omissions, and has adequate psychometric properties. Questionnaire pretesting, therefore, is a crucial element of a good study design. Prior to the use of pretests, questionnaire problems were diagnosed mainly by the questionnaire designers (through their expertise in the subject) and the interviewers (through their experience in administering the questionnaire). Thus, detailed reports of appropriate methods to undertake pretesting are currently underrepresented. As the first of its kind, this paper compares the results of two pretesting methods - namely, cognitive interviewing and respondent debriefing - used to evaluate a series of questions from the questionnaire developed for the study of executive information systems (EIS) adoption factors by knowledge workers. We believe the protocols contribute to reducing shortfalls in pretesting guidance for researchers and practitioners. This paper also aims to inform others about the research and to encourage discussions on the topic.

關鍵字

Cognitive interviewing ； respondent debriefing ； EIS ； CBIS ； pretesting

參考文獻

Willis, G.B. (2005). Cognitive Interviewing. A Tool for Improving Questionnaire Design, London: Sage Publishing. DOI: 10.1007/s11136-005-5263-8

Beatty, P.C., and Willis, G.B. (2007). Research synthesis: The practice of cognitive interviewing, Public Opinion Quarterly 71, 287-311. DOI: https://doi.org/10.1093/poq/nfm006

Beatty, P. (2004). The dynamic of cognitive interviewing, In S. Presser, J. M. Rothgeb, M.P. Couper, and J.L. Lesser, Elizabeth Martin, Jean Martin and Eleanor Singer (eds.), Methods for Testing & Evaluating Survey Questionnaires (pp. 109-130). New York: Wiley & Sons Inc. DOI: https://doi.org/10.1002/0471654728.ch3

Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quarterly of Life Research 12, 229-238. DOI: 10.1023/A:1023254226592

Drennan, J. (2003). Cognitive interviewing: Verbal data in the design and pretesting of questionnaires, Journal of Advanced Nursing 42, 57-63. DOI: https://doi.org/10.1046/j.1365-2648.2003.02579.x