Written and maintained by CASRAI Editorial Board
Published · Last updated
On this page
DataFrame.query() is the pandas method that filters the rows of a DataFrame with a string expression, so df.query("age >= 40") returns the rows where the age column is at least 40. It returns a new DataFrame of matching rows, takes column names directly inside the string, and refers to Python variables with an @ prefix. This page covers the syntax and arguments as documented for pandas 3.1.1, a worked example on a small research-style table, how to read the output, the errors people hit, and the in-pandas alternatives.
What DataFrame.query() does in research analysis
Analysts use query() to define a subset of observations, such as participants in one study arm, records above an age threshold, or samples from one site, in a form that reads like the inclusion criterion itself. The pandas user guide describes it this way: “DataFrame objects have a query() method that allows selection using an expression.” It is a row filter: it does not select columns; selecting columns is a separate task and it does not reshape data.
Because the criterion is a single readable string, it is easy to store next to the analysis, log, and review. The user guide also points out a second use: when several DataFrames share column names, one expression string can be applied to each, for example expr = '0.0 <= a <= c <= 0.5' followed by map(lambda frame: frame.query(expr), [df, df2]). For research teams that care about computational reproducibility, a named filter string is easier to report than a chain of nested boolean masks.
Query pandas: DataFrame.query() versus pd.eval()
Searches for “query pandas” usually mean this method. DataFrame.query() evaluates an expression against the frame’s own index and columns and returns the matching rows. pandas.eval() is a separate top-level function for evaluating expression strings; use query() when the goal is “give me the rows where…”.
Install or access
query() ships with pandas; there is nothing separate to install. The official installation page gives these commands:
pip install pandaspip install "pandas[performance]", which also installs the optional performance dependencies numexpr, bottleneck and numbaconda install -c conda-forge pandas, or, in a new environment,conda create -c conda-forge -n name_of_my_env python pandasfollowed byconda activate name_of_my_env
The installation page recommends working in a virtual environment. numexpr is listed under “Performance dependencies (recommended)” with a minimum version of 2.11.0. Without it, query() still works, falling back to the Python engine (see the engine argument below). The examples on this page were run on pandas 3.1.1, the version of the documentation read for this guide.
Step-by-step: filtering rows with query()
The example uses a small illustrative table invented for this page, not real study data. Six participants, two groups, a pre and post score, and a column whose name contains a space:
import pandas as pd
df = pd.DataFrame({
"participant": ["P01","P02","P03","P04","P05","P06"],
"group": ["control","treatment","treatment","control","treatment","control"],
"age": [34, 41, 29, 52, 47, 38],
"score_pre": [61.0, 58.5, 70.2, 49.8, 64.1, 55.0],
"score_post": [63.2, 66.0, 77.5, 50.1, 71.4, 54.2],
"site code": ["A","A","B","B","A","B"],
})
df["gain"] = df["score_post"] - df["score_pre"]
The new gain column is created as in adding a column to a pandas DataFrame. Gains for P01 to P06 are 2.2, 7.5, 7.3, 0.3, 7.3 and -0.8.
- Filter on one column.
df.query("age >= 40")returns P02, P04 and P05 (rows 1, 3 and 4). - Combine conditions.
df.query("group == 'treatment' and age < 45")returns P02 and P03. Use a different quote type inside the string than the one that wraps it. - Use a Python variable with @. After
min_gain = 7,df.query("gain > @min_gain")returns P02, P03 and P05. The documentation’s own example isdf.query("A <= @local_var"). - Quote awkward column names with backticks.
df.query("`site code` == 'A'")returns P01, P02 and P05. The documentation usesdf.query("B == `C&C`")for a column containing an operator character, and notes that a literal backtick is escaped by doubling it. - Test membership with in and not in.
df.query("group in ['control']")returns P01, P04 and P06;df.query("group not in ['control']")returns the three treatment rows. This is the query form ofisin; see pandas isin for the mask version. - Chain comparisons.
df.query("score_pre < score_post < 75")returns P01, P02, P04 and P05, because P03’s post score of 77.5 is over 75 and P06’s post score is lower than its pre score. - Store the result.
subset = df.query("age >= 40"). The method returns a new DataFrame; assign it to a name.
Signature and arguments
The reference gives the signature as DataFrame.query(expr, *, parser='pandas', engine=None, local_dict=None, global_dict=None, resolvers=None, level=0, inplace=<no_default>). Only expr is positional; every other argument is keyword-only.
- expr (str): the query string. Python keywords cannot be used as identifiers in it.
- parser (‘pandas’ or ‘python’, default ‘pandas’): with the pandas parser,
&and|get the precedence ofandandor; the python parser uses strict Python semantics. - engine (‘python’ or ‘numexpr’, default None): None tries numexpr and falls back to python. The python engine is described as “generally not recommended” because it is inefficient compared with numexpr.
- local_dict, global_dict: the namespaces for
@variables, defaulting tolocals()andglobals(). - resolvers: extra namespaces to search. The method already injects the index and columns.
- level (int, default 0): how many frames up to look for local variables.
- inplace: deprecated since pandas 3.1.0 and scheduled for removal in pandas 4.0 (PDEP-8). Assign the returned DataFrame instead. Running
query(..., inplace=False)on pandas 3.1.1 emits a deprecation warning.
The documentation carries an explicit warning: “This method can run arbitrary code, which can make you vulnerable to code injection if you pass user input to this function.” Never build an expression string from text a user typed into a form or app. Hard-code the expression and pass user-chosen values through @variables where the logic allows it.
Output and how to read it
query() returns a DataFrame containing only the matching rows, with their original index labels preserved (the P02, P04, P05 example above keeps labels 1, 3 and 4). Two checks belong in every analysis script:
- Row count.
len(subset)orsubset.shapeshows how many rows met the criterion. If nothing matches, you get an empty DataFrame with the same columns, not an error:df.query("age > 100").shapeis(0, 7)on the example frame. Downstream means and models on an empty frame yield NaN or errors, so check the count first. - Index labels. Original labels carry through, so use
.reset_index(drop=True)if later code assumes positions 0 to n-1.
When it is unclear whether a threshold was applied as intended, profiling the filtered frame helps; see what is data profiling.
Querying the index and MultiIndex levels
The index is part of the query namespace. According to the user guide, a named index is used if no column has that name; otherwise the column wins. The name index always refers to the index, so df.query('index < b < c') works, and if a column is itself called index, use ilevel_0 for the index. On the example frame, df.set_index("participant").query("index == 'P03'") returns P03’s row. For a MultiIndex, level names behave like columns (df.query('color == "red"')), and unnamed levels are ilevel_0, ilevel_1 and so on.
Performance
The user guide states that DataFrame.query() using numexpr is slightly faster than Python for large frames, and that the benefit appears “only if your frame has more than approximately 100,000 rows.” For typical study-sized tables, choose between query and a boolean mask on readability, not speed. Also, in and not in are evaluated in Python because numexpr has no equivalent; only the in expression itself is evaluated that way, while something like a in b + c + d first computes b + c + d with numexpr.
Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
UndefinedVariableError: name 'x' is not defined |
A bare name that is not a column or index level is treated as a variable lookup that fails. Reproduced with df.query("age > nothing"). |
Check the column spelling; for a Python variable, write @x. |
SyntaxError: invalid syntax |
A column name with a space, such as site code, written without backticks. Reproduced with df.query("site code == 'A'"). |
Wrap it in backticks: `site code`. Python keywords cannot be identifiers. |
| Filter silently picks the wrong value | String values written without quotes are read as names, not text. | Quote string literals: group == 'treatment'. |
ValueError about a non-unique column label |
The docs state this is raised if the expression refers to a non-unique label that evaluates to a DataFrame. | Reduce it, as in the docs’ example df.query("a.max(axis=1) > 4"), or rename duplicate columns. |
| A variable inside a function is not found | Local variable lookup depends on local_dict and level. |
Pass the value explicitly with local_dict={'x': value}. |
Deprecation warning mentioning inplace |
inplace is deprecated since 3.1.0 and removed in 4.0. |
Drop the argument and assign the result. |
| Unexpected grouping of conditions | Under the default parser & and | take the precedence of and and or. Under parser='python' they follow strict Python rules. |
Use parentheses, or the words and and or. |
One further detail from the reference: in backticked names, characters such as operators, spaces, ?, ! and $ are replaced when the name is resolved, so backticks are the supported route for such column names.
Free alternatives in pandas
There is no paid version of query(); the alternatives are other ways to get the same rows inside pandas. The first two rows are documented as equivalents in the pandas user guide’s indexing page.
| Approach | Example | When it fits |
|---|---|---|
| Boolean mask | df[(df['age'] >= 40) & (df['group'] == 'treatment')] |
Each condition must be a separate, reusable object, or the column names are computed in code. The user guide shows it is equivalent to the query string. |
Series.isin() |
df[df['group'].isin(['control'])] |
Membership tests; the guide lists it as the equivalent of in in a query. See pandas isin. |
| Resampling by time | df.resample(...) |
Time-indexed data aggregated by period rather than filtered by value; see pandas resample. |
The data table below lists the syntax elements covered by the user guide next to their boolean-mask equivalents, with sources.
FAQ
What does pandas query() return?
A new DataFrame containing the rows for which the expression is true, with the original index labels. It returns None only when the deprecated inplace=True is used.
How do I use a variable inside df.query()?
Prefix the variable name with @, for example df.query("gain > @min_gain"). The variable is looked up in the local and global namespaces.
How do I query a column name with spaces?
Wrap it in backticks: df.query("`site code` == 'A'").
Is query() faster than boolean indexing?
The documentation says only that the numexpr engine is slightly faster for large frames, with a benefit visible above roughly 100,000 rows. For smaller frames, pick the more readable form.
Is DataFrame.query() safe to use with user input?
No. The official reference warns that it can run arbitrary code, which creates a code-injection risk if user input is passed in.
Does df.query(inplace=True) still work?
It is deprecated since pandas 3.1.0 and the documentation says it will be removed in pandas 4.0. Assign the returned DataFrame instead.
Sources
- pandas documentation, pandas.DataFrame.query (version 3.1.1), read 2026-11-13.
- pandas user guide, Indexing and selecting data, “The query() Method”, read 2026-11-13.
- pandas documentation, Installation, read 2026-11-13.
Example outputs were produced by running the code shown on pandas 3.1.1.
Data: DataFrame.query() expression forms and their boolean-mask equivalents in pandas 3.1.1
Compares query-string syntax elements with the equivalent boolean-mask or isin code, compiled from the pandas user guide and API reference for DataFrame.query (version 3.1.1), with the engine behaviour the documentation states for each.
Period covered: pandas 3.1.1.
| Query element | query() expression | Equivalent in plain pandas | Evaluation note in the documentation |
|---|---|---|---|
| Chained boolean condition | df.query('(a < b) & (b < c)') | df[(df['a'] < df['b']) & (df['b'] < df['c'])] | The docs show 'a < b & b < c' and 'a < b and b < c' as equivalent under the default parser |
| Chained comparison | df.query('a < b < c') | df[(df['a'] < df['b']) & (df['b'] < df['c'])] | Listed in the docs as equivalent to the explicit form |
| Membership | df.query('a in b') | df[df['a'].isin(df['b'])] | Evaluated in Python; numexpr has no equivalent |
| Negated membership | df.query('a not in b') | df[~df['a'].isin(df['b'])] | Evaluated in Python; numexpr has no equivalent |
| Equality with a list | df.query('b == ["a","b","c"]') | df[df['b'].isin(['a','b','c'])] | The docs state == with a list behaves like in |
| Boolean column negation | df.query('~bools') | df[~df['bools']] | 'not bools' is documented as an alternative spelling |
| Local Python variable | df.query('A > @local_var') | df[df['A'] > local_var] | @ refers to a variable in the local or global namespace |
| Column name with special characters | df.query('B == `C&C`') | df[df['B'] == df['C&C']] | Backticks are required for names with spaces or operator characters |
Sources
- pandas user guide: Indexing and selecting data, The query() Method (accessed 2026-11-13)
- pandas.DataFrame.query API reference (3.1.1) (accessed 2026-11-13)
Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.







