Skip to main content
v2026.11,986 entries · CC-BY 4.0

Pivot Long in R: Reshape with pivot_longer

Reshape wide research data into tidy format in R using tidyr::pivot_longer(). Step-by-step syntax, multi-column splitting, troubleshooting, and alternatives.

Written and maintained by CASRAI Editorial Board

Published · Last updated

On this page

In R, pivoting data from wide to long format reshapes a data frame so that multiple measurement columns are collapsed into key-value pairs, producing a tidy structure where each row represents a single observation and each column represents a single variable. The modern standard function for this operation is pivot_longer() from the tidyr package (part of the core tidyverse). It supersedes older reshaping commands like gather() by providing a more expressive interface for handling complex column names, multi-variable prefixes, and in-flight data type conversions.

What it is used for in research

In scientific research, clinical trials, and epidemiological studies, raw data is frequently recorded or exported in wide format. For instance, an electronic data capture system or laboratory assay instrument often outputs a separate column for each measurement timepoint (such as baseline, week 4, and week 12 scores). While wide tables are easy for human data entry, most downstream statistical analysis tools and graphics packages in R require tidy (long) data.

Researchers routinely pivot data long for three primary purposes:

  • Statistical Modeling: Repeated-measures analysis of variance (ANOVA), generalized estimating equations (GEE), and linear mixed-effects models (such as those fitted with lme4::lmer or nlme::lme) require one row per observation, with an explicit variable indexing the repeated time or condition factor.
  • Data Visualization: The ggplot2 graphics library relies on tidy aesthetics. Plotting longitudinal trajectories or faceted boxplots across experimental conditions requires grouping variables to be stored in dedicated columns rather than dispersed across column headers.
  • Harmonized Data Pipelines: Integrating data across multiple collection sites as part of a broader data wrangling workflow requires converting varied schemas into a canonical structure that clearly delineates distinct types of variables.

Unlike simply transposing matrices and data frames in R, which inverts the entire matrix orientation without regard to variable semantics, pivot_longer() selectively lengthens target measurement columns while preserving subject identifiers and experimental covariates.

How to install or access it

The pivot_longer() function is maintained in the open-source tidyr package, published on the Comprehensive R Archive Network (CRAN) under the MIT license by Posit Software, PBC. It is compatible with R versions 4.1.0 and higher.

To install tidyr from official CRAN mirrors, run the standard package installer in the R console:

install.packages("tidyr")

Alternatively, if you use the complete tidyverse suite for research analysis, you can install the complete meta-package:

install.packages("tidyverse")

Once installed, load the library into your active R script or interactive session:

library(tidyr)

To confirm that your installed version supports all modern pivoting features, inspect the package version:

packageVersion("tidyr")

Step-by-step instructions

Follow these numbered steps to reshape a wide research dataset into tidy long format using tidyr::pivot_longer().

  1. Inspect the wide dataset and identify invariant key columns.

    Determine which columns should remain intact as observation identifiers (such as subject IDs, trial arms, or demographic factors) and which columns contain measurement values spread across headers. Consider this representative clinical trial dataset:

    library(tidyr)
    
    # Sample wide research dataset: 3 participants with repeated biomarker measurements
    trial_wide <- data.frame(
      patient_id = c("PT-101", "PT-102", "PT-103"),
      arm = c("Control", "Treatment", "Treatment"),
      biomarker_m0 = c(14.2, 16.5, 12.8),
      biomarker_m3 = c(14.0, 14.1, 11.2),
      biomarker_m6 = c(14.5, 12.3, 10.1)
    )
    
    print(trial_wide)
  2. Execute a basic pivot using cols, names_to, and values_to.

    Specify the columns to pivot using tidyselect helpers, name the new key column that will hold the old header strings, and name the new column that will hold the measurement values:

    trial_long <- pivot_longer(
      data = trial_wide,
      cols = starts_with("biomarker_"),
      names_to = "timepoint",
      values_to = "biomarker_value"
    )
    
    print(trial_long)

    You can also use negative column selection (e.g., cols = -c(patient_id, arm)) to pivot all columns except the designated identifier fields.

  3. Strip header prefixes with names_prefix.

    To eliminate redundant text from the resulting key column, supply a regular expression to names_prefix. For instance, to remove the leading "biomarker_" text and isolate the time labels:

    trial_clean_time <- pivot_longer(
      data = trial_wide,
      cols = starts_with("biomarker_"),
      names_to = "month",
      names_prefix = "biomarker_m",
      values_to = "biomarker_value"
    )
    
    print(trial_clean_time)
  4. Decompose multi-variable column headers using names_sep or names_pattern.

    When wide column names encode multiple experimental factors (for example, assay name and timepoint, such as crp_wk0 and crp_wk4), split them into distinct tidy columns in one call:

    # Multi-variable wide dataset
    complex_wide <- data.frame(
      patient_id = c("PT-101", "PT-102"),
      crp_wk0 = c(1.2, 3.4),
      crp_wk4 = c(1.1, 2.1),
      il6_wk0 = c(5.4, 7.8),
      il6_wk4 = c(5.2, 5.9)
    )
    
    complex_long <- pivot_longer(
      data = complex_wide,
      cols = -patient_id,
      names_to = c("biomarker", "visit"),
      names_sep = "_",
      values_to = "concentration"
    )
    
    print(complex_long)
  5. Enforce strict type conversions using names_transform and values_transform.

    Column headers are initially parsed as character strings. To convert numerical indices (like numeric months or dose amounts) directly into integers or factors during pivoting, supply a list of transform functions:

    trial_typed <- pivot_longer(
      data = trial_wide,
      cols = starts_with("biomarker_"),
      names_to = "month_num",
      names_prefix = "biomarker_m",
      names_transform = list(month_num = as.integer),
      values_to = "biomarker_value"
    )
    
    # Confirms month_num is integer rather than character
    str(trial_typed$month_num)
  6. Manage missing observations with values_drop_na.

    If participants missed specific visits, their wide rows contain NA cells. Set values_drop_na = TRUE to drop missing rows from the long output, or retain them (values_drop_na = FALSE, default) to maintain explicit record completeness compliant with FAIR data principles:

    # Drops rows where biomarker_value is NA
    trial_complete_cases <- pivot_longer(
      data = trial_wide,
      cols = starts_with("biomarker_"),
      names_to = "timepoint",
      values_to = "biomarker_value",
      values_drop_na = TRUE
    )

Output and how to read it

The output of pivot_longer() is a restructured data frame (or tibble) with fewer columns and more rows than the input.

To verify the transformation, compare the dimensions and column classes before and after pivoting:

# Input dimensions: 3 rows by 5 columns
dim(trial_wide)

# Output dimensions: 9 rows by 4 columns (3 patients * 3 timepoints)
dim(trial_long)

When inspecting the printed output of trial_long, look for the following characteristics:

  • Repeating Keys: Each patient_id and arm now appears 3 times, once for each corresponding measurement row.
  • Key Column (timepoint): Contains the names of the original columns (e.g., "biomarker_m0", "biomarker_m3", "biomarker_m6").
  • Value Column (biomarker_value): Contains the numerical readings that were previously dispersed across horizontal headers.
  • Row Ordering: By default, cols_vary = "fastest" arranges the output rows so that all timepoints for a single patient appear together consecutively, which matches the required format for longitudinal modeling.

Troubleshooting

When reshaping complex research datasets, you may encounter specific errors or unexpected outputs. Here is how to diagnose and resolve them:

  • Error: Can’t combine <character> and <double>:

    Cause: The columns passed to cols contain conflicting data types. For example, if one timepoint column contains numeric values while another contains text strings (such as "<0.01" or "missing"), R cannot merge them into a single atomic vector.

    Resolution: Inspect column classes using sapply(df, class). Harmonize data types prior to pivoting (e.g., replacing text flags with NA using dplyr::na_if()), or force coercion during pivoting by adding values_transform = list(values_to = as.character).

  • Error: ‘names_to’ contains 2 elements, but ‘names_sep’ / ‘names_pattern’ was not supplied:

    Cause: Supplying multiple output column names in names_to = c("var1", "var2") without telling the function how to split the incoming header strings.

    Resolution: Add names_sep = "_" for simple character delimiters, or provide regular expression capture groups in names_pattern = "(.*)_(.*)".

  • Unintended Row Multiplication or Missing Identifiers:

    Cause: The columns not selected in cols do not uniquely identify each observational unit, causing R to duplicate rows or group multiple measurements together unexpectedly.

    Resolution: Ensure the non-pivoted columns form a composite primary key. If your dataset lacks a unique subject or run identifier, generate one prior to pivoting using trial_wide$row_id <- seq_len(nrow(trial_wide)).

  • Columns Pivoting Unexpectedly:

    Cause: Loose pattern matching in tidyselect helpers (e.g., starts_with("m") matching both month_1 and medical_record_number).

    Resolution: Use strict character vectors cols = c("month_1", "month_2") or precise negative selections cols = -c(medical_record_number, site_id).

Free alternatives

While tidyr::pivot_longer() is the standard in the tidyverse ecosystem, several other free and open-source alternatives exist for reshaping data in R and related research computing environments.

Tool / Function Environment / Package Syntax Approach Memory & Performance Best Research Application
tidyr::pivot_longer() R (tidyr / tidyverse) Declarative tidyselect syntax with integrated splitting (names_sep) and casting Fast C++ execution via vctrs; optimized for standard in-memory analysis General scientific analysis, tidy data workflows, and direct input preparation for ggplot2.
stats::reshape() Base R (Built-in stats) Argument-driven (varying, timevar, idvar, direction = "long") Minimal overhead; zero external dependencies Base R scripts, lightweight server automation, and package development minimizing external imports.
data.table::melt() R (data.table) Vector arguments (id.vars, measure.vars, variable.name) Extremely fast in-place memory management; handles tens of millions of rows High-throughput genomics, electronic health record registries, and memory-constrained compute clusters.
pandas.melt() Python (pandas) DataFrame method (id_vars, value_vars, var_name, value_name) C/Cython vectorized execution within the Python scientific stack Cross-language research projects and workflows manipulating columns in pandas.

FAQ

What is the difference between pivot_longer() and the older gather() function?

pivot_longer() was introduced in tidyr version 1.0.0 to supersede gather(). While both convert wide data to long format, pivot_longer() provides a cleaner and more consistent syntax that aligns with pivot_wider(), supports modern tidyselect helpers directly, and allows multi-variable column headers to be parsed into separate columns using names_sep and names_pattern without requiring external string manipulation.

How do I keep specific identifier columns from being pivoted?

To exclude identifier columns from pivoting, use the negative selection operator - in the cols argument. For example, cols = -c(patient_id, treatment_group) instructs R to retain patient_id and treatment_group as intact identifying columns while pivoting all remaining columns into key-value pairs.

Can pivot_longer() split column names into multiple new variables?

Yes. By specifying multiple names in the names_to argument (such as names_to = c("measurement", "time")) and providing either names_sep = "_" or a regex pattern in names_pattern, pivot_longer() splits compound column headers and places the resulting parts into distinct tidy variables simultaneously.

Does pivoting long in R change the underlying data values?

No. Pivoting is strictly a structural reshaping operation that rearranges rows and columns. It does not alter, recompute, or aggregate your recorded data values unless you explicitly request a data type conversion via values_transform or omit missing values using values_drop_na = TRUE.

When should researchers use long format instead of wide format?

Researchers should use long format whenever they need to fit repeated-measures regression models (such as linear mixed models or GEEs), visualize grouped data in ggplot2, or conduct group-level summaries across repeated experimental intervals. Wide format is primarily useful for initial manual data entry and tabular report presentation.

Sources

Follow CASRAI

Research-administration guidance, standards updates and independent tool reviews.

Ask CASRAI · free to try

Ask about Pivot Long in R: Reshape with pivot_longer

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →