Written and maintained by CASRAI Editorial Board
Published
On this page
A matrix question organizes related survey items sharing an identical rating scale into a compact grid of rows and columns. This guide provides matrix question examples across survey formats, reviews platform settings in Qualtrics and SurveyMonkey, and walks through the quantitative mechanics: composite scoring formulas, reverse coding, and within-respondent standard deviation for straightline detection.
Ask CASRAI · free to tryQuestion about this guide? Ask CASRAI answers with linked sources. 2 free questions a day, no account.
What is a matrix question?
A matrix question (also called a grid or battery question) is a closed-ended survey format that lists several statements along rows and evaluates them against a shared set of response options across columns. Qualtrics defines a matrix table as an efficient format to condense multiple items sharing the same scale points into one table. SurveyMonkey defines a matrix question as a closed-ended format where respondents evaluate multiple row items using consistent column choices. Because each statement shares the same scale anchors — such as a 5-point agreement scale — respondents evaluate related criteria without re-reading scale definitions for each item. Matrix questions belong to the family of closed-ended questions and are widely deployed alongside standard survey question types and rank order scale examples.
Matrix question examples and platform layouts
The visual structure of a matrix question depends on whether items require single-choice evaluation, multi-attribute selection, or bipolar semantic ratings.
1. Single-Answer Likert Agreement Matrix
In a standard Likert agreement grid, each row presents an independent statement and columns represent an ordered scale from Strongly Disagree to Strongly Agree. The respondent selects one radio button per row. For deeper background on scale design, see Likert scale construction.
| Statement (Row) | Strongly Disagree (1) | Disagree (2) | Neutral (3) | Agree (4) | Strongly Agree (5) |
|---|---|---|---|---|---|
| The user portal is intuitive to navigate. | ○ | ○ | ○ | ○ | ○ |
| Technical support resolves issues promptly. | ○ | ○ | ○ | ○ | ○ |
2. Bipolar and Checkbox Formats
In a bipolar matrix, opposing ends of each row are anchored with contrasting adjectives (e.g., Inefficient vs. Efficient) for semantic differentials. In a checkbox matrix, respondents select multiple non-exclusive attributes per row (e.g., software tools evaluated across API and export features).
Platform Capabilities: Qualtrics vs. SurveyMonkey
Key capabilities differ across major platforms:
| Capability | Qualtrics | SurveyMonkey |
|---|---|---|
| Matrix Types | Likert, Bipolar, Profile, Rank Order, Constant Sum, Text Entry | Matrix / Rating Scale, Dropdown Matrix |
| Scoring & Weighting | Custom recode values assigned to scale points in settings | Assign numerical weights; calculates automatic weighted average per row |
| Forced Ranking | Rank Order matrix variant | Forced Ranking setting prevents picking same column across multiple rows |
| Mobile Optimization | Matrix Carousel (slides statements one at a time) or Accordion view | Responsive horizontal wrapping; single-row rating scale conversion |
| Platform Sizing | Supports up to 100 statements | Recommends under 50 total choices across question to prevent fatigue |
For a step-by-step setup in Qualtrics, consult the guide to the Qualtrics Likert matrix table.
Formula for matrix scoring, reverse coding and straightline detection
Analyzing matrix questions requires converting selections into numbers, handling reverse-worded statements, and screening for straightlining.
Item Mean: X_bar_j = sum(X_ij, i=1..n) / n
Composite Scale Mean: S_bar_i = sum(X_ij, j=1..k) / k
Reverse-Coding: X_rev = (m + 1) - X_raw
Within-Row SD (Check): s_row_i = sqrt( sum((X_ij - X_bar_row_i)^2, j=1..k) / (k - 1) )
Cronbach's Alpha: alpha = (k / (k - 1)) * (1 - sum(s_j^2, j=1..k) / s_total^2)
- X_ij: Numeric rating for respondent i on item j.
- n, k, m: Screened sample size (n), row item count (k), and scale points (m).
- X_rev: Inverted rating for negatively keyed items, where (m + 1) anchors reversal.
- s_row_i: Within-respondent sample SD on raw items. An s_row of 0.0 indicates complete straightlining.
- alpha: Cronbach’s alpha internal consistency; s_j^2, s_total^2: Item variance and total composite variance.
Worked example: scoring a 5-item matrix and screening straightliners
This worked example evaluates 8 researchers assessing a facility across 5 matrix items on a 5-point Likert scale (1 = Strongly Disagree to 5 = Strongly Agree): Item 1 (Instrumentation), Item 2 (Scheduling), Item 3 (Support), Item 4 (Training confusing, negative), and Item 5 (Timeliness).
Data Matrix, Straightline Screening and Recoding
Row standard deviations (s_row) are calculated on raw ratings. Item 4 is recoded as X_rev = 6 − X_raw.
| Respondent | Item 1 | Item 2 | Item 3 | Item 4 (Raw) | Item 5 | Row SD | Composite Sum | Status |
|---|---|---|---|---|---|---|---|---|
| 1 | 4 | 5 | 4 | 2 | 5 | 1.225 | 22 | Valid |
| 2 | 3 | 4 | 3 | 2 | 4 | 0.837 | 18 | Valid |
| 3 | 5 | 5 | 5 | 1 | 5 | 1.789 | 25 | Valid |
| 4 | 3 | 3 | 3 | 3 | 3 | 0.000 | 15 | Straightliner (Excluded) |
| 5 | 4 | 4 | 3 | 2 | 4 | 0.894 | 19 | Valid |
| 6 | 2 | 3 | 2 | 4 | 2 | 0.894 | 11 | Valid |
| 7 | 5 | 4 | 4 | 2 | 4 | 1.095 | 21 | Valid |
| 8 | 3 | 3 | 4 | 3 | 3 | 0.447 | 16 | Valid |
Respondent 4 selected 3 across every row, producing s_row = 0.000. Under standard data hygiene protocols, Respondent 4 is flagged as a straightliner and excluded from scale evaluation ($N = 7$).
Item Statistics and Scale Reliability ($N = 7$)
Sample means and variances ($s^2$, $df = 6$) across valid cases are: Item 1 ($M = 3.714, s^2 = 1.238$), Item 2 ($M = 4.000, s^2 = 0.667$), Item 3 ($M = 3.571, s^2 = 0.952$), Item 4 Recoded ($M = 3.714, s^2 = 0.905$), and Item 5 ($M = 3.857, s^2 = 1.143$), with sum $sum s_j^2 = 4.905$. Composite sums (22, 18, 25, 19, 11, 21, 16) yield variance $s_{ ext{total}}^2 = 20.476$. Cronbach’s alpha is:
alpha = (5 / (5 - 1)) * (1 - (4.905 / 20.476)) = 1.25 * 0.7605 = 0.951
The scale demonstrates high internal consistency reliability ($lpha = .95$).
Interpretation: reading the results
Evaluating matrix questions requires reviewing item ratings alongside data quality indicators:
| Metric | Observed Value | Benchmark Threshold | Interpretation & Action |
|---|---|---|---|
| Item Mean (X_bar_j) | 3.57 – 4.00 | 1.0 – 5.0 scale | Item 2 (4.00) rated highest; Item 3 (3.57) indicates priority for service improvement. |
| Composite Scale Mean | 3.77 (SD = 0.90) | > 3.0 indicates net agreement | Overall sentiment toward facility operations is positive across the user cohort. |
| Straightline Flag (s_row) | s_row = 0.0 (Resp 4) | s_row = 0.0 | Zero variance across items indicates satisficing; exclude from composite psychometrics. |
| Scale Reliability (alpha) | 0.951 | alpha ≥ 0.70 acceptable, ≥ 0.80 good | High internal consistency confirms all 5 items measure a cohesive construct. |
Assumptions and common mistakes
While matrix tables offer visual economy on desktop screens, survey methodologists document recurring vulnerabilities:
- Straightlining and Satisficing. Grids invite respondents to check the same column across rows. AAPOR standards recommend keeping batteries short (5–7 items) and auditing row-level variance during data cleaning.
- Mobile Viewport Failures. Pew Research Center studies show that wide matrix tables force smartphone users to scroll horizontally and vertically, elevating break-offs. Pew recommends avoiding grids on mobile surveys in favor of vertically stacked items.
- Grammatical and Scale Mismatch. Forcing disparate questions into a single matrix distorts measurement. Every statement in rows must make direct grammatical sense with column response options.
- Neglecting Reverse Scoring. Mixing reverse-keyed statements disrupts automatic clicking, but analysts often fail to recode negative items prior to calculating scale means and alpha, producing artificially degraded reliability figures.
- Excessive Battery Depth. Matrices exceeding 10–15 rows induce cognitive fatigue. SurveyMonkey advises keeping total response options below 50 across the question to preserve data quality.
How to report it (APA example)
Report scale properties, screening criteria, item metrics, and reliability in APA 7th format:
“Core facility satisfaction was assessed using a 5-item matrix battery rated on a 5-point Likert scale (1 = Strongly Disagree to 5 = Strongly Agree), with one negatively keyed item reverse-coded prior to composite scoring. Data screening identified one respondent displaying complete non-differentiation (straightlining with within-respondent SD = 0.00 across all rows), who was excluded from psychometric analyses ($N = 7$). The composite scale showed high internal consistency reliability, Cronbach’s $lpha = .95$. Overall satisfaction was positive ($M = 3.77, SD = 0.90$). At the item level, scheduling ease received the highest approval ($M = 4.00, SD = 0.82$), followed by timeline adherence ($M = 3.86, SD = 1.07$), equipment calibration ($M = 3.71, SD = 1.11$), training documentation ($M = 3.71, SD = 0.95$), and troubleshooting assistance ($M = 3.57, SD = 0.98$).”
How to calculate it in Excel, R or Python
The code snippets below demonstrate how to screen straightliners, reverse-score items, and calculate composite means and Cronbach’s alpha.
Excel
Row SD (Cell G2): =STDEV.S(B2:F2)
Straightline flag (Cell H2): =IF(G2=0, "FLAG", "OK")
Reverse-code Item 4 (Cell I2): =6 - E2
Composite mean (Cell J2): =AVERAGE(B2, C2, D2, I2, F2)
Cronbach's alpha: =(5/(5-1))*(1 - SUM(B10:F10)/G11)
R
raw <- matrix(c(4,5,4,2,5, 3,4,3,2,4, 5,5,5,1,5, 3,3,3,3,3,
4,4,3,2,4, 2,3,2,4,2, 5,4,4,2,4, 3,3,4,3,3), nrow=8, byrow=TRUE)
valid <- raw[apply(raw, 1, sd) > 0, ]
valid[, 4] <- 6 - valid[, 4]
colMeans(valid)
scale_scores <- rowMeans(valid)
k <- ncol(valid)
alpha <- (k / (k - 1)) * (1 - sum(apply(valid, 2, var)) / var(rowSums(valid))) # 0.951
Python
import statistics
raw_data = [[4,5,4,2,5],[3,4,3,2,4],[5,5,5,1,5],[3,3,3,3,3],[4,4,3,2,4],[2,3,2,4,2],[5,4,4,2,4],[3,3,4,3,3]]
screened = [r[:] for r in raw_data if statistics.stdev(r) > 0]
for r in screened: r[3] = 6 - r[3]
k = len(screened[0])
item_means = [statistics.mean([r[j] for r in screened]) for j in range(k)]
alpha = (k / (k - 1)) * (1 - sum(statistics.variance([r[j] for r in screened]) for j in range(k)) / statistics.variance([sum(r) for r in screened]))
print("Item means:", [round(m, 3) for m in item_means], "Alpha:", round(alpha, 3))
Frequently asked questions
What is a matrix question in a survey?
A matrix question is a closed-ended survey grid grouping related items in rows evaluated against shared column response options. It conserves space and lets respondents rate multiple items on a single scale.
How do you score a matrix question?
Score matrix items individually using row means, or combine rows into a composite construct score by averaging numerical ratings across items. Reverse-code negative statements before computing scale totals.
What is straightlining in a matrix question?
Straightlining is a satisficing shortcut where a participant selects the same column across all rows without reading item text. It is detected by computing within-respondent row standard deviation (SD = 0 indicates non-differentiation).
Are matrix questions mobile-friendly?
Traditional grids perform poorly on mobile devices due to horizontal scrolling and tiny tap targets. Pew Research Center recommends avoiding grid layouts on mobile surveys and using vertically stacked items instead.
How many rows should a matrix question have?
AAPOR guidelines recommend limiting matrix batteries to 5–7 rows. Batteries exceeding 10 rows significantly increase cognitive fatigue, straightlining, and abandonment.
Sources
- Qualtrics Support, “Matrix Table Question”, https://www.qualtrics.com/support/survey-platform/survey-module/editing-questions/question-types-guide/standard-content/matrix-table/ (checked 2026-11-09).
- SurveyMonkey Help, “Matrix/Rating Scale Question”, https://help.surveymonkey.com/en/surveymonkey/create/matrix-rating-scale/ (checked 2026-11-09).
- Pew Research Center, “Tips for Creating Web Surveys for Completion on a Mobile Device”, https://www.pewresearch.org/methods/2015/06/11/tips-for-creating-web-surveys-for-completion-on-a-mobile-device/ (checked 2026-11-09).
- American Association for Public Opinion Research (AAPOR), “Best Practices for Survey Research”, https://aapor.org/standards-and-ethics/best-practices/ (checked 2026-11-09).
The worked example uses illustrative data; all arithmetic and test statistics were computed and verified on the page.







