Skip to main content
v2026.11,986 entries · CC-BY 4.0

pandas query: DataFrame.query() Syntax and Examples

How pandas DataFrame.query() filters rows: syntax, @ variables, backticks, arguments, a worked example, common errors and plain-pandas alternatives.

Written and maintained by CASRAI Editorial Board

Published · Last updated

On this page

DataFrame.query() is the pandas method that filters the rows of a DataFrame with a string expression, so df.query("age >= 40") returns the rows where the age column is at least 40. It returns a new DataFrame of matching rows, takes column names directly inside the string, and refers to Python variables with an @ prefix. This page covers the syntax and arguments as documented for pandas 3.1.1, a worked example on a small research-style table, how to read the output, the errors people hit, and the in-pandas alternatives.

What DataFrame.query() does in research analysis

Analysts use query() to define a subset of observations, such as participants in one study arm, records above an age threshold, or samples from one site, in a form that reads like the inclusion criterion itself. The pandas user guide describes it this way: “DataFrame objects have a query() method that allows selection using an expression.” It is a row filter: it does not select columns; selecting columns is a separate task and it does not reshape data.

Because the criterion is a single readable string, it is easy to store next to the analysis, log, and review. The user guide also points out a second use: when several DataFrames share column names, one expression string can be applied to each, for example expr = '0.0 <= a <= c <= 0.5' followed by map(lambda frame: frame.query(expr), [df, df2]). For research teams that care about computational reproducibility, a named filter string is easier to report than a chain of nested boolean masks.

Query pandas: DataFrame.query() versus pd.eval()

Searches for “query pandas” usually mean this method. DataFrame.query() evaluates an expression against the frame’s own index and columns and returns the matching rows. pandas.eval() is a separate top-level function for evaluating expression strings; use query() when the goal is “give me the rows where…”.

Install or access

query() ships with pandas; there is nothing separate to install. The official installation page gives these commands:

  • pip install pandas
  • pip install "pandas[performance]", which also installs the optional performance dependencies numexpr, bottleneck and numba
  • conda install -c conda-forge pandas, or, in a new environment, conda create -c conda-forge -n name_of_my_env python pandas followed by conda activate name_of_my_env

The installation page recommends working in a virtual environment. numexpr is listed under “Performance dependencies (recommended)” with a minimum version of 2.11.0. Without it, query() still works, falling back to the Python engine (see the engine argument below). The examples on this page were run on pandas 3.1.1, the version of the documentation read for this guide.

Step-by-step: filtering rows with query()

The example uses a small illustrative table invented for this page, not real study data. Six participants, two groups, a pre and post score, and a column whose name contains a space:

import pandas as pd

df = pd.DataFrame({
    "participant": ["P01","P02","P03","P04","P05","P06"],
    "group": ["control","treatment","treatment","control","treatment","control"],
    "age": [34, 41, 29, 52, 47, 38],
    "score_pre": [61.0, 58.5, 70.2, 49.8, 64.1, 55.0],
    "score_post": [63.2, 66.0, 77.5, 50.1, 71.4, 54.2],
    "site code": ["A","A","B","B","A","B"],
})
df["gain"] = df["score_post"] - df["score_pre"]

The new gain column is created as in adding a column to a pandas DataFrame. Gains for P01 to P06 are 2.2, 7.5, 7.3, 0.3, 7.3 and -0.8.

  1. Filter on one column. df.query("age >= 40") returns P02, P04 and P05 (rows 1, 3 and 4).
  2. Combine conditions. df.query("group == 'treatment' and age < 45") returns P02 and P03. Use a different quote type inside the string than the one that wraps it.
  3. Use a Python variable with @. After min_gain = 7, df.query("gain > @min_gain") returns P02, P03 and P05. The documentation’s own example is df.query("A <= @local_var").
  4. Quote awkward column names with backticks. df.query("`site code` == 'A'") returns P01, P02 and P05. The documentation uses df.query("B == `C&C`") for a column containing an operator character, and notes that a literal backtick is escaped by doubling it.
  5. Test membership with in and not in. df.query("group in ['control']") returns P01, P04 and P06; df.query("group not in ['control']") returns the three treatment rows. This is the query form of isin; see pandas isin for the mask version.
  6. Chain comparisons. df.query("score_pre < score_post < 75") returns P01, P02, P04 and P05, because P03’s post score of 77.5 is over 75 and P06’s post score is lower than its pre score.
  7. Store the result. subset = df.query("age >= 40"). The method returns a new DataFrame; assign it to a name.

Signature and arguments

The reference gives the signature as DataFrame.query(expr, *, parser='pandas', engine=None, local_dict=None, global_dict=None, resolvers=None, level=0, inplace=<no_default>). Only expr is positional; every other argument is keyword-only.

  • expr (str): the query string. Python keywords cannot be used as identifiers in it.
  • parser (‘pandas’ or ‘python’, default ‘pandas’): with the pandas parser, & and | get the precedence of and and or; the python parser uses strict Python semantics.
  • engine (‘python’ or ‘numexpr’, default None): None tries numexpr and falls back to python. The python engine is described as “generally not recommended” because it is inefficient compared with numexpr.
  • local_dict, global_dict: the namespaces for @ variables, defaulting to locals() and globals().
  • resolvers: extra namespaces to search. The method already injects the index and columns.
  • level (int, default 0): how many frames up to look for local variables.
  • inplace: deprecated since pandas 3.1.0 and scheduled for removal in pandas 4.0 (PDEP-8). Assign the returned DataFrame instead. Running query(..., inplace=False) on pandas 3.1.1 emits a deprecation warning.

The documentation carries an explicit warning: “This method can run arbitrary code, which can make you vulnerable to code injection if you pass user input to this function.” Never build an expression string from text a user typed into a form or app. Hard-code the expression and pass user-chosen values through @variables where the logic allows it.

Output and how to read it

query() returns a DataFrame containing only the matching rows, with their original index labels preserved (the P02, P04, P05 example above keeps labels 1, 3 and 4). Two checks belong in every analysis script:

  • Row count. len(subset) or subset.shape shows how many rows met the criterion. If nothing matches, you get an empty DataFrame with the same columns, not an error: df.query("age > 100").shape is (0, 7) on the example frame. Downstream means and models on an empty frame yield NaN or errors, so check the count first.
  • Index labels. Original labels carry through, so use .reset_index(drop=True) if later code assumes positions 0 to n-1.

When it is unclear whether a threshold was applied as intended, profiling the filtered frame helps; see what is data profiling.

Querying the index and MultiIndex levels

The index is part of the query namespace. According to the user guide, a named index is used if no column has that name; otherwise the column wins. The name index always refers to the index, so df.query('index < b < c') works, and if a column is itself called index, use ilevel_0 for the index. On the example frame, df.set_index("participant").query("index == 'P03'") returns P03’s row. For a MultiIndex, level names behave like columns (df.query('color == "red"')), and unnamed levels are ilevel_0, ilevel_1 and so on.

Performance

The user guide states that DataFrame.query() using numexpr is slightly faster than Python for large frames, and that the benefit appears “only if your frame has more than approximately 100,000 rows.” For typical study-sized tables, choose between query and a boolean mask on readability, not speed. Also, in and not in are evaluated in Python because numexpr has no equivalent; only the in expression itself is evaluated that way, while something like a in b + c + d first computes b + c + d with numexpr.

Troubleshooting

Symptom Cause Fix
UndefinedVariableError: name 'x' is not defined A bare name that is not a column or index level is treated as a variable lookup that fails. Reproduced with df.query("age > nothing"). Check the column spelling; for a Python variable, write @x.
SyntaxError: invalid syntax A column name with a space, such as site code, written without backticks. Reproduced with df.query("site code == 'A'"). Wrap it in backticks: `site code`. Python keywords cannot be identifiers.
Filter silently picks the wrong value String values written without quotes are read as names, not text. Quote string literals: group == 'treatment'.
ValueError about a non-unique column label The docs state this is raised if the expression refers to a non-unique label that evaluates to a DataFrame. Reduce it, as in the docs’ example df.query("a.max(axis=1) > 4"), or rename duplicate columns.
A variable inside a function is not found Local variable lookup depends on local_dict and level. Pass the value explicitly with local_dict={'x': value}.
Deprecation warning mentioning inplace inplace is deprecated since 3.1.0 and removed in 4.0. Drop the argument and assign the result.
Unexpected grouping of conditions Under the default parser & and | take the precedence of and and or. Under parser='python' they follow strict Python rules. Use parentheses, or the words and and or.

One further detail from the reference: in backticked names, characters such as operators, spaces, ?, ! and $ are replaced when the name is resolved, so backticks are the supported route for such column names.

Free alternatives in pandas

There is no paid version of query(); the alternatives are other ways to get the same rows inside pandas. The first two rows are documented as equivalents in the pandas user guide’s indexing page.

Approach Example When it fits
Boolean mask df[(df['age'] >= 40) & (df['group'] == 'treatment')] Each condition must be a separate, reusable object, or the column names are computed in code. The user guide shows it is equivalent to the query string.
Series.isin() df[df['group'].isin(['control'])] Membership tests; the guide lists it as the equivalent of in in a query. See pandas isin.
Resampling by time df.resample(...) Time-indexed data aggregated by period rather than filtered by value; see pandas resample.

The data table below lists the syntax elements covered by the user guide next to their boolean-mask equivalents, with sources.

FAQ

What does pandas query() return?

A new DataFrame containing the rows for which the expression is true, with the original index labels. It returns None only when the deprecated inplace=True is used.

How do I use a variable inside df.query()?

Prefix the variable name with @, for example df.query("gain > @min_gain"). The variable is looked up in the local and global namespaces.

How do I query a column name with spaces?

Wrap it in backticks: df.query("`site code` == 'A'").

Is query() faster than boolean indexing?

The documentation says only that the numexpr engine is slightly faster for large frames, with a benefit visible above roughly 100,000 rows. For smaller frames, pick the more readable form.

Is DataFrame.query() safe to use with user input?

No. The official reference warns that it can run arbitrary code, which creates a code-injection risk if user input is passed in.

Does df.query(inplace=True) still work?

It is deprecated since pandas 3.1.0 and the documentation says it will be removed in pandas 4.0. Assign the returned DataFrame instead.

Sources

Example outputs were produced by running the code shown on pandas 3.1.1.

Data: DataFrame.query() expression forms and their boolean-mask equivalents in pandas 3.1.1

Compares query-string syntax elements with the equivalent boolean-mask or isin code, compiled from the pandas user guide and API reference for DataFrame.query (version 3.1.1), with the engine behaviour the documentation states for each.

Period covered: pandas 3.1.1.

DataFrame.query() expression forms and their boolean-mask equivalents in pandas 3.1.1
Query elementquery() expressionEquivalent in plain pandasEvaluation note in the documentation
Chained boolean conditiondf.query('(a < b) & (b < c)')df[(df['a'] < df['b']) & (df['b'] < df['c'])]The docs show 'a < b & b < c' and 'a < b and b < c' as equivalent under the default parser
Chained comparisondf.query('a < b < c')df[(df['a'] < df['b']) & (df['b'] < df['c'])]Listed in the docs as equivalent to the explicit form
Membershipdf.query('a in b')df[df['a'].isin(df['b'])]Evaluated in Python; numexpr has no equivalent
Negated membershipdf.query('a not in b')df[~df['a'].isin(df['b'])]Evaluated in Python; numexpr has no equivalent
Equality with a listdf.query('b == ["a","b","c"]')df[df['b'].isin(['a','b','c'])]The docs state == with a list behaves like in
Boolean column negationdf.query('~bools')df[~df['bools']]'not bools' is documented as an alternative spelling
Local Python variabledf.query('A > @local_var')df[df['A'] > local_var]@ refers to a variable in the local or global namespace
Column name with special charactersdf.query('B == `C&C`')df[df['B'] == df['C&C']]Backticks are required for names with spaces or operator characters

Sources

Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.

Follow CASRAI

Research-administration guidance, standards updates and independent tool reviews.

Ask CASRAI · free to try

Ask about pandas query: DataFrame.query() Syntax and Examples

Ask your first 2 questions free below. Subscribers get 150 a day for $29 a month.

An AI assistant specialized in research administration. It cites the sources behind every answer, labels web answers and says when it can't answer.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays free to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →