pandas concat: Syntax, axis, join, keys and ignore_index

pandas.concat() stacks or places side by side pandas Series and DataFrames. According to the pandas 3.1.1 documentation, it “concatenates pandas objects along a particular axis, with optional set logic along the other axes” and “can also add a hierarchical index on the concatenation axis.” This guide is for researchers and analysts who need to combine exports (one file per site, wave or instrument) into a single table and understand what the result looks like. Documentation read 2026-11-07.

What pandas concat is used for in research data

Research data frequently arrives in pieces: one CSV per site, one file per data-collection wave, or one sheet per instrument run. pd.concat joins those pieces by position along an axis, without matching on a key column. The pandas user guide describes concat() as combining objects along a shared index or column axis, and merge() as the SQL-style tool for joining on key columns (see the comparison table below).

• Stack rows (axis=0, the default): append records from several files that share the same columns.
• Combine columns (axis=1): place Series or DataFrames side by side, aligned on their index.

Concatenating is usually a step after tidying each piece; see tidy data rules for structuring research data and data cleaning steps.

Install or access

concat is part of pandas itself, so there is nothing separate to install. The official pandas installation page lists these routes: pip install pandas (PyPI) and conda install -c conda-forge pandas (conda-forge), plus source from GitHub. It recommends working inside a virtual environment (venv for pip users). Other distributions are not managed by the pandas team. Then import it:

import pandas as pd

Syntax and arguments

The documented signature in pandas 3.1.1 is:

pandas.concat(objs, *, axis=0, join='outer', ignore_index=False, keys=None, levels=None, names=None, verify_integrity=False, sort=<no_default>, copy=<no_default>, union_categories=False)

Argument	What the documentation says
objs	An iterable or mapping of Series and DataFrame objects. If a mapping is passed, its keys are used as keys. None objects are dropped silently; if all are None, a ValueError is raised.
axis	0/'index' (default) or 1/'columns': the axis to concatenate along.
join	'outer' (default) takes the union of the other axis; 'inner' takes the intersection.
ignore_index	If True, the index along the concatenation axis is discarded and labelled 0 to n-1. The other axis’s index is still respected in the join.
keys	Builds a hierarchical index using the keys as the outermost level.
levels, names	levels sets the levels for the hierarchical index; names names the index levels.
verify_integrity	If True, checks the new concatenated axis for duplicates. The documentation warns this “can be very expensive relative to the data concatenation.”
sort	Default False. Exception: a DatetimeIndex on the non-concatenation axis with join='outer' that is not aligned is always sorted.
copy	Deprecated since pandas 3.0.0 and ignored; scheduled for removal in 4.0. Since 3.0, concat always returns a new object using lazy copy (Copy-on-Write).
union_categories	Added in 3.1.0. If True and all values for a column are categorical, the result keeps a categorical dtype with the union of the categories.

Return type: if every object is a Series and axis=0, you get a Series; if any object is a DataFrame, you get a DataFrame; with axis=1 the result is a DataFrame.

Step-by-step: stack per-site exports into one DataFrame

The following example uses small, invented, illustrative data (it is not from a real study) to show the pattern the pandas user guide recommends: collect the pieces in a list, then call concat once.

1. Load or create each piece. Each has the same columns.

   import pandas as pd
   
   site_a = pd.DataFrame({"participant_id": ["A01", "A02"], "score": [12, 15]})
   site_b = pd.DataFrame({"participant_id": ["B01", "B02"], "score": [14, 11]})

   For building frames from scratch, see how to create a DataFrame in Python.

2. Put the pieces in a list and concatenate once.

   combined = pd.concat([site_a, site_b], ignore_index=True)

   With ignore_index=True the row labels become 0, 1, 2, 3 instead of repeating 0, 1, 0, 1.

3. Label where each row came from with keys:

   labelled = pd.concat([site_a, site_b], keys=["site_a", "site_b"], names=["site", "row"])

   The result has a two-level row index: the outer level holds “site_a” or “site_b”, the inner level holds each piece’s original row labels. labelled.loc["site_b"] returns just the second piece.

4. Catch duplicate row labels early when labels must be unique:

   pd.concat([site_a, site_a], verify_integrity=True)

   This raises ValueError: Indexes have overlapping values because both inputs carry the labels 0 and 1.

5. When you have many files, build the list first. The pandas user guide’s own pattern is:

   frames = [process_your_file(f) for f in files]
   result = pd.concat(frames)

The pandas documentation states that “it is not recommended to build DataFrames by adding single rows in a for loop. Build a list of rows and make a DataFrame in a single concat.” It adds that concat() makes a full copy of the data, so calling it repeatedly creates unnecessary copies.

Combine columns: pandas concat axis=1

Use axis=1 to place objects side by side, aligned on the index. For example, pd.concat([df1, df4], axis=1) from the pandas documentation puts the columns of df4 next to df1. Rows whose index label appears in only one object get missing values (NaN) in the other object’s columns under the default join='outer'. To keep only the rows of the first object, the user guide gives pd.concat([df1, df4], axis=1).reindex(df1.index); join='inner' keeps only labels present in every object.

Concat two DataFrames with different columns

In the documentation’s example, df1 has columns letter and number, and df3 has an extra column animal. pd.concat([df1, df3], sort=False) keeps all three columns and fills NaN for the df1 rows in animal. Adding join="inner" keeps only letter and number.

Append a single row

The documented pattern is pd.concat([df7, new_row.to_frame().T], ignore_index=True), where new_row is a Series.

Output and how to read the index

• Repeated index labels: by default, row labels from each input are kept as they were, so stacking two frames that each start at 0 gives labels 0, 1, 0, 1. Use ignore_index=True for a fresh 0 to n-1, or keys to keep the source visible.
• NaN cells: these appear where one input lacks a column (or index label) that another has, under join='outer'. They mean “not present in that input”, not “measured as missing”; decide deliberately whether to keep 'outer' or switch to 'inner'.
• Result type: Series in, Series out only when all objects are Series and axis=0; otherwise a DataFrame.
• Column order: sort defaults to False, so non-concatenation-axis labels are not sorted (except for the DatetimeIndex case above).

Troubleshooting

Symptom	Cause per the documentation	Fix
ValueError: Indexes have overlapping values	verify_integrity=True found duplicate labels on the concatenated axis.	Use ignore_index=True, add keys, or reset the index of each input before concatenating.
Unexpected NaN columns or cells	Inputs have different columns or index labels and join='outer' (default) keeps the union.	Use join='inner', or align column names first.
ValueError when every object is None	Only None objects were passed; None objects are otherwise dropped silently.	Check that your list of inputs was actually populated.
Duplicate row labels in the result	Default behaviour keeps each input’s own labels.	ignore_index=True or keys.
Slow loop that grows a DataFrame row by row	Each concat call copies all the data.	Collect pieces in a list, call concat once.
Warning or error about copy=	copy is deprecated since 3.0.0, ignored, and set for removal in 4.0.	Remove the argument.

If the pieces you are combining come from tab-separated exports, see what is a TSV file for reading them in first.

pandas concat vs merge vs join

Direct answer: use concat to stack or align whole objects along an axis; use merge when you need to match rows on key columns, as in a database join.

Tool	What the pandas documentation says it does	Typical research use
pd.concat	Concatenates along an axis with optional set logic (join) on the other axis; can add a hierarchical index.	Stacking same-shape exports from several sites or waves.
DataFrame.merge	“Database-style join.” Default how='inner'; on defaults to the intersection of column names; when joining columns on columns, indexes are ignored. validate accepts "one_to_one", "one_to_many", "many_to_one" or "many_to_many" to check key uniqueness.	Attaching participant demographics to outcomes using a participant ID column.
DataFrame.join	Listed under “See also” on the concat reference page alongside merge.	Index-based combining (see the pandas docs for details).

A concat with axis=1 aligns on index labels only; it does not match on a key column. If a participant ID is a column rather than the index, use merge or set it as the index first. For a related data-reshaping topic in another tool, see the Stata reshape command.

Free alternatives

Within pandas itself, the documented alternatives for combining data are DataFrame.merge and DataFrame.join (above). pandas is open-source software distributed through PyPI and conda-forge.

FAQ

What does concat pandas do?

It combines Series or DataFrames along one axis, row-wise by default (axis=0), with join controlling how the other axis is handled.

How do I concat columns in pandas?

Pass axis=1: pd.concat([df_a, df_b], axis=1). Objects are aligned on the index.

How do I reset the index after concatenation?

Pass ignore_index=True, which labels the concatenation axis 0 to n-1.

How do I keep track of which DataFrame each row came from?

Pass keys (a list of labels, or pass a dict as objs) to build a hierarchical index whose outermost level identifies the source.

Is it a problem to call concat in a loop?

The pandas documentation advises against it: build a list of objects and call concat once, because each call copies the data.

Does pandas have an append method for DataFrames?

The pandas documentation’s own pattern for adding a row is pd.concat with the row converted to a one-row frame, as shown above.

Sources

• pandas documentation, pandas.concat (version 3.1.1), read 2026-11-07.
• pandas user guide, Merge, join, concatenate and compare (3.1.1), read 2026-11-07.
• pandas documentation, pandas.DataFrame.merge, read 2026-11-07.
• pandas documentation, Installation, read 2026-11-07.

The data in the worked example is invented for illustration and does not come from any real study.