A survey instrument is built from a small catalogue of question types, and picking the wrong one for a given piece of information is one of the most common, and most consequential, design mistakes in questionnaire design. This guide is a practical reference: every question type a researcher is likely to need, a copy-ready worked example with its response options, the design rules that keep it analyzable, and the specific pitfall that shows up most often in real data. For the underlying mechanics of closed-ended item construction and wording, see Closed-Ended Questions in Research and Open-Ended vs. Closed-Ended Questions; this page is the type-by-type catalogue those two build on, with an emphasis on the mechanics — multi-select coding, skip logic, matrix usability, demographic wording — that neither covers in depth.
Multiple Choice: Single-Answer vs. Multiple-Answer
The single most consequential fork in question design is whether a respondent may select one option or more than one. These produce fundamentally different data and are not interchangeable.
Single-answer multiple choice
Example: “Which of the following best describes your current employment status?” — Full-time / Part-time / Self-employed / Student / Retired / Not currently employed / Prefer not to say
Exactly one option is selected, so the response is a single categorical variable. The category list must be exhaustive (every respondent has a truthful option, usually via an “other” or catch-all) and mutually exclusive (no respondent’s true status fits two options).
Multiple answer / multi-select / checkbox questions
Example: “Which of the following data repositories has your lab used in the past year? (Select all that apply)” — Dryad / Zenodo / figshare / an institutional repository / a domain-specific repository / None of the above
A multi-select question lets the respondent choose as many options as apply, and it needs its own set of design rules that a single-answer item doesn’t:
- Say “select all that apply” explicitly in the stem. Respondents default to reading a list of options as single-select unless told otherwise; an ambiguous instruction produces inconsistent behavior across respondents in the same dataset.
- Include an explicit “None of the above.” Without it, a respondent for whom no option applies simply skips the question, and that skip is indistinguishable in the data from a respondent who didn’t see the question or declined to answer — a real zero and a missing value end up looking identical.
- Keep the list exhaustive. Multi-select items still need every plausible true answer represented, or respondents under-report by omission rather than picking a wrong option (the failure mode is different from single-answer, but the fix — an “other, please specify” option — is the same).
- Randomize option order across respondents where the list is long. Options presented earlier in a list receive disproportionately more selections purely from position — a primacy effect that is well documented in the survey-methodology literature, particularly in self-administered (visual) surveys. Randomizing (or at minimum rotating) option order spreads this bias out instead of systematically favoring whichever option happens to be listed first.
- Mutual exclusivity is usually not required — but check. Unlike single-answer items, overlapping options are often fine (a respondent legitimately uses both Dryad and an institutional repository). Only enforce exclusivity where the options genuinely can’t co-occur.
The critical analysis rule: multi-select data cannot be analyzed like single-answer data. A multi-select question does not produce one categorical variable — it produces one binary (selected / not selected) variable per option. “Which repository has your lab used” with six options becomes six yes/no variables, each summarized as its own frequency and percentage of respondents who selected it. Percentages across a multi-select question’s options will not sum to 100%, because respondents can contribute to more than one option’s count — a table that reports multi-select percentages as if they were single-answer proportions (implying they sum to 100%) is reporting the data wrong. Cross-tabulating a multi-select item against another variable requires treating each option as its own binary outcome, not collapsing the question into a single categorical column.
Dichotomous Questions: When a Forced Binary Distorts the Answer
Example: “Did your study receive IRB or ethics committee approval before participant recruitment began?” — Yes / No
Dichotomous (yes/no, true/false) items are the fastest item type to answer and the easiest to code, and they are the right choice when the underlying reality genuinely is binary. The failure mode is applying a yes/no format to a question whose true answer is conditional, partial, or a matter of degree. “Are you satisfied with the training program?” forced into Yes/No loses everyone whose real answer is “mostly, except for one part” or “it depends which session.” The tell that a dichotomous item is the wrong choice: if you can imagine a reasonable respondent answering “it depends,” the true variable is not binary, and a category, rating, or a dichotomous item plus a conditional follow-up (see filter questions below) is the better fit. See Closed-Ended Questions in Research for dichotomous coding and analysis.
Category Questions
Example: “Which of the following best describes your primary research discipline?” — Life sciences / Physical sciences / Social sciences / Humanities / Engineering / Other (please specify)
A category question is a single-answer multiple-choice item whose options are unordered, non-numeric groupings rather than points on a scale — discipline, department, role, region, method type. Because the categories have no inherent order, they can only be summarized with frequencies, proportions, and chi-square-family tests of association, never with a mean. The main design risk is the same exhaustive/mutually-exclusive pair covered above, with one addition specific to categories: keep the list at a length a respondent can actually scan (roughly eight to ten visible options before a dropdown or search field becomes easier to use than a long radio-button list), and order the list either alphabetically or by expected frequency rather than arbitrarily, since option order itself is not neutral (see the randomization point above).
Filter / Screening Questions and Skip Logic
Example: “Have you submitted a grant application in the past 12 months?” — Yes → continue to funding-source questions / No → skip to Section 4
A filter (or screening) question routes different respondents to different subsequent questions based on their answer — the mechanism behind “skip logic” in survey software. Filter questions do two jobs at once: they keep irrelevant questions from being asked of respondents they don’t apply to, and they define subgroups within the sample.
The design and analysis implications are specific to this type:
- The filter question changes the denominator for everything downstream of it. If 400 of 1,000 respondents answer “Yes” to the grant-application filter question above, every question that follows it in that branch has a base of 400, not 1,000. Reporting percentages for those follow-up questions against the full sample of 1,000 misrepresents them — the correct denominator is the number of respondents who were actually routed to and answered that question.
- State the base explicitly in any report or table (“n = 400, respondents who submitted a grant application in the past 12 months”) rather than leaving it implied, since a reader cannot otherwise tell whether a percentage is out of the full sample or a filtered subgroup.
- Keep the filter question itself simple and unambiguous — a poorly worded filter routes respondents incorrectly, and unlike a normal item, that error doesn’t just corrupt one answer, it corrupts every downstream answer in the wrong branch.
- Test skip logic before fielding. Broken branching (a respondent routed to the wrong section, or a dead end with no path forward) is one of the most common technical failures caught in pretesting and cognitive interviewing — see Questionnaire Design.
Likert, Rating, and Ranking Questions
These three ordered-response formats are covered in construction detail — number of scale points, midpoint inclusion, endpoint vs. full labeling, and respondent-burden limits — in Closed-Ended Questions in Research and Questionnaire Design. Two points worth restating as design rules here, since they determine whether the resulting scale is usable at all:
- Likert scales: five and seven points are the most common conventions; whether to include a neutral midpoint depends on whether the topic is low-salience (include it) or subject to social-desirability pressure that a “neutral” option would let respondents hide behind (consider omitting it). Fully labeling every point, not just the endpoints, improves comparability across respondents when the scale will be summed into a composite score — see Cronbach’s alpha for checking that composite’s internal consistency once it is.
- Ranking questions (“Rank the following in order of priority”) become unreliable past roughly seven to eight items — respondents can meaningfully order a short list but not a long one, and a ranking item forced past that length degrades into effectively-random ordering of the lower-priority items. If more than about eight items need to be compared, consider rating each independently (a series of Likert or rating items) instead of forcing a single strict rank order.
Matrix / Grid Questions
Example: A single grid with the stem “For each of the following, how satisfied are you?” and rows — core facility access, IT support, administrative processing, training opportunities — each rated on the same Strongly dissatisfied … Strongly satisfied scale in the grid’s columns.
A matrix (or grid) question presents several items sharing the same response scale in a compact table, letting a respondent answer many related questions without re-reading the scale for each one. It saves space and respondent time on paper and desktop, but it carries two well-documented problems that have gotten worse, not better, as survey-taking has shifted to mobile:
- Mobile usability. A grid with several rows and a five- or seven-point scale that renders cleanly on a desktop screen frequently has to be either shrunk to illegible size or converted to a scrolling, one-row-at-a-time format on a phone — either degrades the response experience and can measurably raise item nonresponse and abandonment on mobile-heavy samples.
- Straightlining. A matrix invites a specific low-effort response pattern — selecting the same column for every row regardless of the row’s actual content — because the format makes it visually easy to do. This is a genuine data-quality threat, not a rare edge case, and it is hard to distinguish from a respondent who legitimately feels the same way about every row without additional checks (response-time flags, an occasional reverse-worded row, or an attention check, below).
Where the item count is small (three or four rows) and the sample is desktop-heavy, a matrix is a reasonable, efficient choice. Where the sample is expected to be predominantly mobile or the row count is large, breaking the matrix into individual single items, or using a mobile-adapted grid pattern the survey platform explicitly supports, is usually the safer choice.
Open-Ended Questions: When the Qualitative Cost Is Worth It
Example: “What, if anything, would have made the grant application process easier for you?” — free-text response field
An open-ended question is worth its higher respondent burden and analysis cost specifically when a fixed list would risk missing the real range of respondent experience — exploratory work, a topic without an established set of categories, or a question about reasoning and meaning rather than magnitude. It is a poor substitute for a closed item when the researcher already knows the plausible answer set; in that case an open item just adds coding work to recover a categorical variable that could have been asked directly.
Open-ended responses do not analyze themselves. Before any pattern from open-text data can be reported as a finding, it needs to go through a coding process: developing a codebook (a documented set of categories, usually built from an initial read-through of a subset of responses), applying it systematically across all responses, and, for anything beyond a small illustrative sample, checking inter-rater agreement if more than one coder is involved. Reporting a percentage from open-ended responses (“40% mentioned time constraints”) without a documented coding process behind it overstates the rigor of the analysis. See Open-Ended vs. Closed-Ended Questions for the full comparison and the common workflow of piloting open-ended, then converting recurring answers into a closed response set for the full-scale instrument.
Demographic Questions Done Well
Demographic items look simple and are one of the most common sources of both bad data and participant frustration when they are not. A few items deserve specific, deliberate wording decisions rather than a default template copied from another survey.
Age
Example (ranges): “What is your age?” — 18–24 / 25–34 / 35–44 / 45–54 / 55–64 / 65+
Example (exact): “What is your current age in years?” — numeric entry
Example (year of birth): “In what year were you born?” — numeric entry
All three formats are legitimate; the right one depends on what the analysis needs and how the survey will be used over time. Ranges are faster to answer and feel less exposing to respondents sensitive about disclosing an exact age, but they discard information (two respondents 18 and 24 years old are coded identically) and force an arbitrary choice of band boundaries that can misalign with the boundaries an analysis actually needs later. Exact age captures full precision but is the most identifying of the three, which matters for de-identification and small-cell disclosure risk in reporting. Year of birth is generally the better choice for any survey that will be repeated on the same population over time (a longitudinal panel, an annual staff survey, a cohort study), because year of birth is a fixed fact that doesn’t need updating at each wave, while an “age in years” or “age range” answer collected at wave one silently goes stale by wave two unless it is deliberately recalculated — year of birth ages correctly on its own with each subsequent administration.
Gender
Example: “What is your gender?” — Woman / Man / Non-binary / A gender not listed here (please specify) / Prefer not to say
An inclusive gender item offers more than a binary Man/Woman choice, includes a genuine option for respondents whose gender isn’t captured by the listed categories, and always includes “prefer not to say.” Where the research question specifically requires biological sex rather than gender identity (e.g., certain clinical or biomedical measures), ask for sex and gender as two separate, explicitly labeled items rather than one item conflating the two — they are different constructs and increasingly treated as such in survey-methodology and public-health guidance.
Race and ethnicity
Race and ethnicity categories are jurisdiction-dependent and there is no single universal category list: the categories used in a U.S. federal survey (following OMB standards), a UK census (following ONS categories), and a Canadian survey (following Statistics Canada categories) differ from each other in real, non-interchangeable ways. Use the standard category set relevant to the jurisdiction and reporting requirement the study is actually subject to, rather than a generic list assembled from memory, and always include an “other” option and “prefer not to say.”
Income and education
Example (income): “What is your total annual household income, before taxes?” — presented as ranges (e.g., under $25,000 / $25,000–$49,999 / $50,000–$74,999 / … ) rather than an exact figure, both for respondent comfort and because ranges are the standard format for this sensitive item.
Example (education): “What is the highest level of education you have completed?” — ordered categories from no formal education through a doctoral or professional degree, matched to the jurisdiction’s standard education-level categories.
Both are sensitive items with real nonresponse risk; both should always include “prefer not to say,” and both benefit from the sensitive-question placement guidance (later in the questionnaire, after rapport is established) covered in Questionnaire Design.
The ethics and GDPR angle
Race/ethnicity, health status, and (per GDPR Article 9) genetic data, biometric data used for identification, and data concerning sex life or sexual orientation are all “special category” personal data under GDPR Article 9(1), which prohibits their processing by default unless a specific Article 9(2) condition applies — explicit consent, or, commonly for research, the scientific-research derogation in Article 9(2)(j) read together with Article 89(1) safeguards. In practice this means: collect special-category demographic data only when the research question genuinely requires it (not by default “in case it’s useful”), document the legal basis and safeguards in the study’s data-protection documentation, and always give respondents a genuine, honored “prefer not to say” option. This sits alongside, not instead of, the separate informed consent requirement to disclose what demographic data is collected and why before the respondent answers.
Attention Checks and Instructional Manipulation Checks
Example: “This question is an attention check — please select ‘Somewhat agree’ regardless of your actual opinion.” — embedded within a block of ordinary Likert items, using the identical response scale.
An attention check (also called an instructional manipulation check, or IMC) is an item that verifies a respondent is reading questions rather than clicking through mechanically. Rather than testing knowledge, it gives an explicit instruction unrelated to the survey’s actual content and checks whether the respondent followed it. Placement matters: an IMC embedded inside a matrix or a long Likert block (where straightlining is most likely) is more diagnostic than one placed in isolation, but placing more than one or two across a survey risks annoying attentive respondents and can itself depress data quality if overused.
Whether to exclude respondents who fail an attention check is a genuine, unsettled methodological debate, not a default. The case for excluding: a failed IMC is reasonable evidence the respondent wasn’t engaging with the content, so their other answers are less trustworthy. The case against blanket exclusion: some respondents fail an IMC for reasons unrelated to inattention (misreading the specific instruction once, a translation or accessibility issue), and excluding based on a single check can introduce its own bias if failure correlates with a demographic or trait the study cares about. The generally recommended practice is to decide and pre-register the exclusion rule before data collection (rather than deciding post hoc after seeing which respondents look inconvenient), and to report both the full-sample and IMC-passed-only results when the two differ, rather than silently reporting only one.
Quick Reference: Question Type, Data, Analysis, and Common Pitfall
| Question type | Data produced | Analysis it enables | Common pitfall |
|---|---|---|---|
| Single-answer multiple choice | Categorical (nominal) | Frequencies, chi-square tests of association | Options not exhaustive or not mutually exclusive |
| Multiple answer / multi-select | One binary variable per option | Per-option frequencies; cannot be summed to 100% | Analyzed as if it were single-answer categorical data |
| Dichotomous (yes/no) | Binary (nominal) | Frequencies, proportions, chi-square | Forces a binary answer onto a conditional/partial true answer |
| Category question | Categorical (nominal, unordered) | Frequencies, chi-square; never a mean | Arbitrary or overlong option ordering biases selection |
| Filter / screening question | Routes respondents into branches | Subgroup analysis on the routed base | Reporting downstream percentages against the full sample instead of the filtered base |
| Likert scale item | Ordinal (or approx. interval when summed into a scale) | Medians/non-parametric tests per item; means for a validated multi-item scale | Unlabeled or unbalanced scale points bias responses |
| Rating scale | Ordinal or interval, depending on scale length and treatment | Means/SD if treated as interval; otherwise ordinal tests | Treating an ordinal scale as interval without justifying it |
| Ranking | Ordinal, relative | Rank-order comparisons; not directly comparable in magnitude across respondents | Asking respondents to rank more than ~7–8 items |
| Matrix / grid | Same as the underlying item type (usually ordinal), per row | Same as the underlying scale, analyzed row by row | Straightlining and mobile-rendering data-quality loss |
| Open-ended | Unstructured qualitative text | Coding / thematic analysis, not direct statistics | Reporting a percentage from uncoded open text |
| Demographic (age, gender, race/ethnicity, income, education) | Categorical, ordinal, or numeric depending on item | Cross-tabulation and subgroup comparison | No “prefer not to say”; collecting special-category data without a documented basis |
| Attention check / IMC | Pass/fail flag, not a substantive variable | Data-quality screening decision (pre-registered, ideally) | Deciding the exclusion rule after seeing the data |
Frequently Asked Questions
What is a multiple answer question?
A multiple answer (multi-select, or “select all that apply”) question lets a respondent choose more than one option from a list, unlike a standard multiple-choice item that allows exactly one. Each option is coded as its own binary yes/no variable, and the resulting percentages do not sum to 100% because respondents can select more than one option.
What is a category question, with an example?
A category question is a single-answer item whose response options are unordered groupings rather than points on a scale — for example, “Which of the following best describes your primary research discipline? Life sciences / Physical sciences / Social sciences / Humanities / Engineering / Other.” It produces nominal data suited to frequencies and chi-square tests, never a mean.
What is a filter question, with an example?
A filter (screening) question routes different respondents to different follow-up questions based on their answer — for example, “Have you submitted a grant application in the past 12 months? Yes / No,” where only respondents answering “Yes” continue to a block of funding-source questions. Filter questions change the correct denominator for every question in the branch that follows them.
How should a survey ask about age?
Any of exact age, age ranges, or year of birth is legitimate depending on precision needs and respondent comfort, but year of birth is usually the better choice for a survey that will be repeated on the same population over time, since it stays accurate at every future wave without recalculation, unlike an age or age-range answer captured at a single point in time.
What is an open-ended survey question, with an example?
An open-ended question supplies no fixed response options and lets the respondent answer freely in their own words — for example, “What, if anything, would have made the grant application process easier for you?” It captures richer, unanticipated detail than a closed item but requires a documented coding process before any pattern in the responses can be reported as a finding.
What is an instructional manipulation check?
An instructional manipulation check (IMC), or attention check, is a survey item that gives an explicit instruction unrelated to the survey’s actual content (e.g., “select ‘Somewhat agree’ for this item”) to verify the respondent is reading carefully rather than clicking through. Whether to exclude respondents who fail an IMC is debated in the survey-methodology literature; the recommended practice is to pre-register the exclusion rule before data collection rather than deciding after seeing the results.
Why can’t multi-select data be analyzed the same way as single-answer data?
A multi-select question produces one binary variable per option rather than one categorical variable for the whole question, because respondents can select more than one option. Treating it as a single categorical variable, or reporting its option percentages as if they summed to 100%, misrepresents the data.
Related Guides
- Closed-Ended Questions in Research — the six core closed-ended item types, construction rules, and coding by data level.
- Open-Ended vs. Closed-Ended Questions — direct comparison, sequencing (funnel/inverted-funnel), and converting open pilot data into closed items.
- Questionnaire Design: Writing Survey Questions That Work — wording failure patterns, Likert scale construction, question order, sensitive-question handling, and pretesting.
- Cronbach’s Alpha — checking the internal consistency of a multi-item Likert scale.
- Types of Variables — the nominal/ordinal/interval/ratio distinctions that determine which analysis a given question type supports.
- Sampling Methods — how the sample a survey is fielded to relates to the population it aims to represent.
- Informed Consent in Research — disclosure obligations when collecting demographic or otherwise sensitive survey data.
- GDPR — the special-category data rules relevant to race/ethnicity, health, and other sensitive demographic items.
- Research Methods & Statistics — the full cluster hub.
