Skip to main content
v2026.11,986 entries · CC-BY 4.0

sklearn GridSearchCV: Syntax, Worked Example and Common Errors

How sklearn GridSearchCV works: a runnable pipeline example, fit-count arithmetic, cv_results_ and common errors.

Written and maintained by CASRAI Editorial Board

Published

On this page

sklearn GridSearchCV (scikit-learn’s sklearn.model_selection.GridSearchCV) is an exhaustive hyperparameter search: you give it an estimator and a grid of candidate settings, it fits and cross-validates every combination, and it exposes the best combination and the full score table afterwards. With the default refit=True it then refits the best setting on all the data you passed in, so the fitted search object behaves like a model.

Ask CASRAI · free to tryQuestion about this guide? Ask CASRAI answers with linked sources. 2 free questions a day, no account.

Ask your first question freeSubscribe

What GridSearchCV is used for in research

Many models have settings that are not learned from the data, such as the penalty C of a support vector machine or the depth of a tree. scikit-learn calls these hyperparameters: they are constructor arguments of the estimator, and estimator.get_params() lists them. GridSearchCV replaces trial-and-error tuning by hand with a documented, repeatable procedure that you can describe in a methods section: the grid, the number of folds, the scoring rule and the selection rule.

It is built on cross-validation, so every candidate is judged on held-out folds rather than on the data it was trained on. That guards against choosing the setting that merely memorises the training set (see what overfitting is), although, as the reporting section below explains, it does not make the winning score an unbiased estimate of future performance. Typical research uses are tuning a support vector machine, a random forest, a k-nearest-neighbours classifier or a regularised regression such as elastic net.

gridsearchcv sklearn and scikit learn gridsearchcv: same tool

Searches for “gridsearchcv sklearn” and “scikit learn gridsearchcv” lead to the same class. The package is installed as scikit-learn and imported as sklearn, so the import line is from sklearn.model_selection import GridSearchCV. Older tutorials sometimes show a different module path; the current documentation lists the class under sklearn.model_selection.

Install or access

GridSearchCV ships with scikit-learn; there is nothing separate to install. The installation page (read 2026-11-14, scikit-learn 1.9.1) gives pip3 install -U scikit-learn inside a virtual environment, or, with conda, conda create -n sklearn-env -c conda-forge scikit-learn. To check what you have, run python -c "import sklearn; sklearn.show_versions()" and record the version in your project notes, because default behaviour and available arguments can differ between releases.

Step-by-step: tune a model with GridSearchCV

The signature in the 1.9.1 documentation is GridSearchCV(estimator, param_grid, *, scoring=None, n_jobs=None, refit=True, cv=None, verbose=0, pre_dispatch='2*n_jobs', error_score=nan, return_train_score=False). The steps below use a scaler plus a support vector classifier on the breast cancer dataset that ships with scikit-learn, so you can reproduce every number on this page.

1. Hold out an evaluation set first

The user guide warns that tuning hyperparameters and then reporting the score on the same data gives an optimistic result. It recommends splitting the data into a development set, on which GridSearchCV runs, and an evaluation set that the search never sees. Use train_test_split for this (see train/test split in Python).

from sklearn.datasets import load_breast_cancerfrom sklearn.model_selection import GridSearchCV, train_test_splitX, y = load_breast_cancer(return_X_y=True)X_dev, X_eval, y_dev, y_eval = train_test_split(    X, y, test_size=0.25, stratify=y, random_state=0)

Here the development set has 426 rows and the evaluation set 143.

2. Wrap preprocessing and model in a Pipeline

If a scaler is fitted on all the data before the search, information from the validation folds leaks into training. Putting the scaler inside a Pipeline means it is refitted inside each training fold. The user guide’s naming rule for pipelines is <step>__<parameter>, with a double underscore, and nesting levels can be chained (its example is model__estimator__max_depth).

from sklearn.pipeline import Pipelinefrom sklearn.preprocessing import StandardScalerfrom sklearn.svm import SVCpipe = Pipeline([("scale", StandardScaler()), ("model", SVC())])

3. Define param_grid

param_grid is a dictionary or a list of dictionaries. With a list, each dictionary is its own grid and the grids are explored in turn, which is how you avoid invalid combinations such as a gamma value for a linear kernel. The user guide’s own example is a linear grid and an rbf grid, which it describes as two grids that contribute 4 and 8 combinations.

param_grid = [    {"model__kernel": ["linear"], "model__C": [0.1, 1, 10]},    {"model__kernel": ["rbf"], "model__C": [0.1, 1, 10],     "model__gamma": [0.001, 0.01]},]

This grid has 3 combinations in the first dictionary and 3 × 2 = 6 in the second, 9 in total.

4. Choose cv, scoring and refit, then fit

With cv=None the search uses 5-fold cross-validation; for a binary or multiclass classifier the folds are stratified, otherwise plain KFold without shuffling. The default score is the estimator’s own score method (accuracy for classifiers, R² for regressors), and the user guide suggests setting scoring explicitly for cases such as imbalanced classification. To track several metrics, pass a list or dictionary to scoring and name the metric that decides the winner in refit.

search = GridSearchCV(    pipe, param_grid, cv=5,    scoring={"accuracy": "accuracy", "f1": "f1"}, refit="f1")search.fit(X_dev, y_dev)print(search.best_params_, search.best_score_)

Run on scikit-learn 1.9.1 this reports {'model__C': 0.1, 'model__kernel': 'linear'} with a mean cross-validated F1 of 0.9906.

5. Score the winner once on the evaluation set

print(search.score(X_eval, y_eval))

Because refit="f1" was set, score delegates to the refitted best estimator and uses the F1 scorer. On the 143 held-out rows the F1 was 0.9674, lower than the 0.9906 from cross-validation: a small illustration of why the search score should not be quoted as the final result.

Grid size: how many fits will it run?

The number of model fits is the number of parameter combinations multiplied by the number of folds, plus one refit when refit is on. In the example, 9 combinations × 5 folds = 45 cross-validation fits, and the fitted object reports n_splits_ = 5 with 9 entries in cv_results_['params']. Every value you add to one parameter multiplies the total, so a grid of five parameters with four values each is 45 = 1,024 combinations, or 5,120 fits at five folds. Setting verbose=1 prints the number of fits before the search starts, which is a cheap way to check your arithmetic before committing compute. Setting verbose=2 adds timings and verbose=3 adds the fold scores.

n_jobs runs fits in parallel, but the documentation notes that data is copied for each grid point when n_jobs is above 1, and pre_dispatch (default '2*n_jobs') exists to limit the number of copies and avoid memory errors.

Output: how to read the results

After fitting, the attributes below are the ones to report or inspect. The documentation notes that some are absent depending on refit.

  • best_params_: the setting that scored best on the held-out folds.
  • best_score_: the mean cross-validated score of that setting (not available when refit is a callable).
  • best_estimator_: the model refitted on the whole development set (not available when refit=False).
  • best_index_: the row of cv_results_ that belongs to the best setting.
  • cv_results_: a dictionary of arrays that converts directly to a pandas DataFrame, with param_* columns, split0_test_score through split{k}_test_score, mean_test_score, std_test_score, rank_test_score, fit and score timings, and a params column.
  • n_splits_, scorer_ and refit_time_: the number of folds, the scorer used and the seconds spent on the final refit.

With multi-metric scoring the score columns carry the metric name instead, for example mean_test_f1 and rank_test_accuracy. The data table below lists all nine candidates of the worked example, ranked by mean F1. The ranking carries information that best_params_ alone hides: the top three settings are within about 0.008 F1 of each other, and the weakest candidate (a very small C with rbf and gamma 0.001) is far behind at 0.8123, which shows how much a poorly placed grid can cost. Compare mean_test_score with std_test_score before declaring a winner: when two settings differ by less than the fold-to-fold spread, the choice between them is not strongly supported by the data.

import pandas as pdresults = pd.DataFrame(search.cv_results_)cols = ["params", "mean_test_f1", "std_test_f1", "rank_test_f1"]print(results[cols].sort_values("rank_test_f1"))

Set return_train_score=True to add training-fold scores to cv_results_. The documentation notes it is off by default because it costs extra compute; it is useful for diagnosing overfitting, since a large gap between training and validation scores points that way.

Reporting discipline for papers and theses

Three practices follow from the user guide. First, report the grid (every parameter and value), cv, scoring and the selection rule, and the scikit-learn version. Second, treat best_score_ as a development-set number: the same folds both chose and scored the winner, which is why the guide separates the development set from the evaluation set and points to its “Nested versus non-nested cross-validation” example for a fuller treatment. Third, say which data the final number came from. In the example above, 0.9906 is the cross-validation figure and 0.9674 is the evaluation-set figure; only the second is a fair estimate of out-of-sample performance.

Troubleshooting

Some fits fail and scores come back as nan

error_score defaults to nan, so a setting whose fit raises an exception is scored as nan rather than stopping the search. Set error_score="raise" while debugging to see the real traceback, or pass a number to score failing settings with that value. If the final refit fails, the error is raised regardless.

Error about refit when using several metrics

With multi-metric scoring, refit must be the name of one scorer (or False, or a callable that takes cv_results_ and returns the best index). Leaving it as None raises an error, and best_estimator_ does not exist when refit=False. The halving searches do not support multi-metric scoring at all.

The estimator has no score method

The estimator must provide a score method, or you must pass scoring explicitly.

The search is too slow or runs out of memory

Reduce the grid (fewer values, or split it into a list of grids), lower cv, or switch to one of the cheaper searches in the next section. If you use n_jobs above 1 and memory is the problem, lower n_jobs or pre_dispatch because of the data copies described above.

Methods raise errors on the search object

predict, predict_proba, decision_function, transform and the other prediction methods are available on the search object only when refit=True; they delegate to best_estimator_.

Free alternatives

GridSearchCV is one of several selection tools in scikit-learn, all open-source. The user guide describes the following alternatives; the right one depends on how large your search is.

Tool How it differs from GridSearchCV Note from the user guide
RandomizedSearchCV Samples settings from distributions instead of trying every combination The budget is set by n_iter and does not depend on the number of parameters; the guide cites Bergstra and Bengio (2012)
HalvingGridSearchCV Starts all candidates on small resources and keeps only the best at each stage Experimental: needs from sklearn.experimental import enable_halving_search_cv; the elimination factor is usually 3
HalvingRandomSearchCV The same successive-halving idea applied to randomly sampled candidates Experimental, same enabling import
LassoCV, ElasticNetCV, RidgeCV, LogisticRegressionCV Estimators with built-in cross-validation of the regularisation setting Listed as alternatives for regularisation paths
Out-of-bag estimates Estimate performance without cross-validation for random forests and gradient boosting Listed as an alternative where the estimator supports it

Frequently asked questions

What does GridSearchCV do in scikit-learn?

It runs a cross-validated, exhaustive search over a grid of hyperparameter values you specify, records the scores in cv_results_, and (with refit=True) refits the best combination on the full data it was given.

How many folds does GridSearchCV use by default?

Five. With cv=None the 1.9.1 documentation uses 5-fold cross-validation, stratified for classifiers on binary or multiclass targets and plain KFold otherwise.

Is best_score_ the accuracy I should report?

No. It is the mean cross-validated score of the winning setting on the same data used to choose it, so it is optimistic. Report the score of the refitted model on a separate evaluation set, or use nested cross-validation.

How do I tune a Pipeline with GridSearchCV?

Name each parameter as the step name, two underscores, then the parameter, for example model__C. This keeps preprocessing inside each training fold.

What is the difference between GridSearchCV and RandomizedSearchCV?

GridSearchCV tries every combination in the grid. RandomizedSearchCV samples a fixed number of settings (n_iter) from lists or distributions, so its cost does not grow with the number of parameters.

Sources

Data: GridSearchCV results for an SVC pipeline on the scikit-learn breast cancer dataset (9 candidates, 5-fold CV)

Cross-validated results of every candidate in the worked example on this page: a StandardScaler plus SVC pipeline tuned with GridSearchCV(cv=5, scoring accuracy and F1, refit='f1') on a stratified 75% development split (random_state=0) of the breast cancer dataset bundled with scikit-learn 1.9.1. Values were produced by running the code on the page and read from cv_results_; the parameter names and attributes follow the scikit-learn documentation.

Period covered: 2026.

GridSearchCV results for an SVC pipeline on the scikit-learn breast cancer dataset (9 candidates, 5-fold CV)
Candidate (SVC settings)Mean CV F1Std of CV F1Rank by F1Mean CV accuracy
linear, C=0.10.99060.010410.9883
rbf, C=10, gamma=0.010.98510.004520.9812
linear, C=10.98330.010830.9789
rbf, C=10, gamma=0.0010.97980.010440.9742
rbf, C=1, gamma=0.010.97460.013650.9672
linear, C=100.9590.017160.9484
rbf, C=1, gamma=0.0010.95760.022970.9438
rbf, C=0.1, gamma=0.010.95570.021880.9414
rbf, C=0.1, gamma=0.0010.81230.026290.7091

Sources

Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.

Follow CASRAI

Research-administration guidance, standards updates and independent tool reviews.

Ask CASRAI · free to try

Ask about sklearn GridSearchCV: Syntax, Worked Example and Common Errors

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →