Research Questionnaire

A research questionnaire is the structured written instrument itself: a fixed, standardized set of questions (items) with defined response formats, administered identically to every respondent so their answers can be compared, coded, and analyzed. It is the tool. Survey research is the broader methodology built around that tool — defining a sampling frame, choosing an administration mode, fielding the questionnaire, and reporting a response rate. A study can also use a questionnaire outside a full probability-sample survey (for example, fielding it to a convenience sample in a pilot study), so the two terms are related but not interchangeable.

Questionnaire vs. survey: what the difference actually is

The distinction matters most when writing a manuscript’s Methods section, where the two terms get conflated:

• The questionnaire is the document or digital form: the specific items, their wording, their order, and their response formats. Two studies can use the identical questionnaire while running completely different surveys around it — different sampling frames, different administration modes, different response rates.
• The survey is the overall data-collection process: defining the target population and sampling frame, selecting an administration mode (online panel, mail, telephone, in-person), fielding the questionnaire, tracking the response rate, and handling nonresponse. See Survey Research Methods for the sampling and administration side of this distinction, including how manuscripts are expected to report response rate and representativeness.

A questionnaire can also be administered outside a survey design entirely — for example, as a pre/post outcome measure in an experimental study, where sampling and response-rate reporting work differently than in a cross-sectional survey. See Experimental Design.

Question types

Most questionnaires mix several item formats, chosen for what each construct requires:

• Likert-type (rating-scale) items — an ordered agreement, frequency, or satisfaction scale, typically 5 or 7 points (e.g., Strongly Disagree to Strongly Agree). Strictly, a “Likert scale” is the summed score across a set of related Likert-type items measuring one construct; a single item is more precisely a Likert-type item, though the terms are used loosely in practice.
• Dichotomous items — a binary choice (yes/no, true/false, present/absent).
• Multiple-choice / categorical items — a fixed list of mutually exclusive response options, sometimes with a write-in “other” option.
• Ranking items — respondents order a set of options by preference or priority, producing ordinal rather than interval data.
• Semantic differential items — a scale anchored by two opposite adjectives (e.g., Weak…Strong), used to measure attitudes toward a concept.
• Open-ended items — free-text response, with no predefined options.

Open-ended vs. closed-ended questions

Closed-ended items (Likert-type, dichotomous, multiple-choice, ranking, semantic differential) constrain responses to a predefined set, which makes them fast to complete and straightforward to code and analyze quantitatively, at the cost of forcing respondents into categories that may not fit their actual answer. Open-ended items let respondents answer in their own words, which can surface unanticipated themes and nuance a closed list would miss, but responses require qualitative coding before they can be analyzed systematically, and they typically have higher item non-response than closed items. Most questionnaires used in primary research combine both: closed-ended items for the variables the study is designed to measure, and a smaller number of open-ended items to capture context, explanation, or anything the closed items didn’t anticipate.

Validity and reliability

Because a questionnaire is the measurement instrument, a manuscript reporting questionnaire data is expected to address whether the instrument actually measures what it claims to (validity) and does so consistently (reliability):

• Content validity — whether the item set adequately covers the construct, typically established through expert review of the draft items.
• Face validity — whether the instrument appears, on its face, to measure what it’s intended to measure, usually judged informally during drafting and pilot testing.
• Construct validity — whether the measure relates to other variables the way theory predicts it should.
• Internal-consistency reliability — how consistently a multi-item scale’s items measure the same construct, commonly reported as Cronbach’s alpha.
• Test-retest reliability — whether the same respondents produce consistent scores when the instrument is re-administered after an interval.

See Survey Research Methods for how these properties are typically reported alongside sampling and response-rate information in a manuscript’s Methods section, and Operationalizing Variables for how a questionnaire item translates an abstract construct into a measurable variable.

Pilot testing

Before a questionnaire is fielded to the full study sample, it is standard practice to pilot test it with a small group drawn from, or similar to, the target population — commonly cited guidance for questionnaire development in educational and health research (e.g., AMEE Guide No. 87 on developing questionnaires) recommends this as a distinct step after expert content-validity review and before full fielding. A pilot test is used to check:

• Item clarity — whether respondents interpret the wording as intended, often assessed through cognitive interviewing or think-aloud protocols, where a respondent verbalizes their reasoning while answering.
• Completion time and burden — whether the instrument is a reasonable length for its administration mode.
• Missing-data and floor/ceiling patterns — items that respondents skip, or items where nearly everyone selects the same extreme response, which signals a wording or scale problem.
• Preliminary reliability — an early internal-consistency check on multi-item scales, with the expectation of revision if it comes in low.

Pilot testing is iterative: items that perform poorly are revised or dropped, and a further small round of testing may follow before the instrument is finalized and fielded.

Illustrative example, not a real study or institution: a researcher drafting a 20-item questionnaire on faculty data-management practices might combine 15 closed-ended Likert-type items with 5 open-ended items on specific barriers, circulate the draft for content-validity review by two subject-matter colleagues, then pilot it with 10 people outside the target sample to check item clarity and completion time before revising wording and fielding it to the full department.

Common design pitfalls

• Double-barreled questions — a single item that actually asks two things at once (e.g., “Is the training clear and useful?”), which makes it impossible to know which part a response is answering.
• Leading or loaded wording — phrasing that suggests a desired answer rather than presenting the question neutrally.
• Acquiescence bias — the tendency of some respondents to agree with statements regardless of content, mitigated by mixing positively- and negatively-worded items within a scale.
• Question-order effects — earlier items shaping how respondents interpret or answer later ones.
• Jargon or ambiguous terms — wording that means different things to different respondents, which pilot testing and cognitive interviewing are specifically meant to catch.

What isn’t a questionnaire

A structured interview guide administered by a trained interviewer who can rephrase questions, ask follow-up probes, and adapt wording to the respondent is not a questionnaire in the strict sense — the interviewer’s real-time adaptation breaks the uniform-administration requirement that defines the instrument. A self-administered instrument with genuinely fixed wording remains a questionnaire regardless of delivery channel — paper, online form, or read aloud verbatim over the phone — as long as the wording and order stay the same for every respondent.