Questionnaires are useful instruments for gathering responses to specific factual questions. However, the problems of questionnaire responses impede the effective use of the questionnaire. Some of these problems including non-responses, non-completion, issues of judgement, and social desirability such as information that the respondent is not willing to disclose need to be dealt with. The situation is more compounded in a scenario where an information security practice study in a typical hospital consists of broad categories of respondents and the survey findings are to be relied on to actually address information security compliance issues. This study, therefore, shares "pitfalls" to watch when preparing a questionnaire to measure the information security practice level in a hospital which is characterized by different respondents with varying domain knowledge such as knowledge in information security, information communication technology, and that of the domain knowledge of healthcare. A synergy of a conventional pretesting method and behaviour coding were therefore used to pretest the questionnaire. Questionnaire problems including a lack of understanding of the healthcare information systems’ structure of all hospitals, unclear questions, the insignificant difference between questions, problematic questions, inadequate questions, and complex terms were among the identified pitfalls to watch. Out of a total of 118 questionnaire items that were used in the pretesting, a total of 50 questionnaire items (representing 42%) were identified to have problems after the pretesting was conducted with a total of 36 respondents in behavior coding and 21 respondents in conventional pretesting.