Written and maintained by CASRAI Editorial Board
Published
On this page
sklearn GridSearchCV (scikit-learn’s sklearn.model_selection.GridSearchCV) is an exhaustive hyperparameter search: you give it an estimator and a grid of candidate settings, it fits and cross-validates every combination, and it exposes the best combination and the full score table afterwards. With the default refit=True it then refits the best setting on all the data you passed in, so the fitted search object behaves like a model.
Ask CASRAI · free to tryQuestion about this guide? Ask CASRAI answers with linked sources. 2 free questions a day, no account.
What GridSearchCV is used for in research
Many models have settings that are not learned from the data, such as the penalty C of a support vector machine or the depth of a tree. scikit-learn calls these hyperparameters: they are constructor arguments of the estimator, and estimator.get_params() lists them. GridSearchCV replaces trial-and-error tuning by hand with a documented, repeatable procedure that you can describe in a methods section: the grid, the number of folds, the scoring rule and the selection rule.
It is built on cross-validation, so every candidate is judged on held-out folds rather than on the data it was trained on. That guards against choosing the setting that merely memorises the training set (see what overfitting is), although, as the reporting section below explains, it does not make the winning score an unbiased estimate of future performance. Typical research uses are tuning a support vector machine, a random forest, a k-nearest-neighbours classifier or a regularised regression such as elastic net.
gridsearchcv sklearn and scikit learn gridsearchcv: same tool
Searches for “gridsearchcv sklearn” and “scikit learn gridsearchcv” lead to the same class. The package is installed as scikit-learn and imported as sklearn, so the import line is from sklearn.model_selection import GridSearchCV. Older tutorials sometimes show a different module path; the current documentation lists the class under sklearn.model_selection.
Install or access
GridSearchCV ships with scikit-learn; there is nothing separate to install. The installation page (read 2026-11-14, scikit-learn 1.9.1) gives pip3 install -U scikit-learn inside a virtual environment, or, with conda, conda create -n sklearn-env -c conda-forge scikit-learn. To check what you have, run python -c "import sklearn; sklearn.show_versions()" and record the version in your project notes, because default behaviour and available arguments can differ between releases.
Step-by-step: tune a model with GridSearchCV
The signature in the 1.9.1 documentation is GridSearchCV(estimator, param_grid, *, scoring=None, n_jobs=None, refit=True, cv=None, verbose=0, pre_dispatch='2*n_jobs', error_score=nan, return_train_score=False). The steps below use a scaler plus a support vector classifier on the breast cancer dataset that ships with scikit-learn, so you can reproduce every number on this page.
1. Hold out an evaluation set first
The user guide warns that tuning hyperparameters and then reporting the score on the same data gives an optimistic result. It recommends splitting the data into a development set, on which GridSearchCV runs, and an evaluation set that the search never sees. Use train_test_split for this (see train/test split in Python).
from sklearn.datasets import load_breast_cancerfrom sklearn.model_selection import GridSearchCV, train_test_splitX, y = load_breast_cancer(return_X_y=True)X_dev, X_eval, y_dev, y_eval = train_test_split( X, y, test_size=0.25, stratify=y, random_state=0)
Here the development set has 426 rows and the evaluation set 143.
2. Wrap preprocessing and model in a Pipeline
If a scaler is fitted on all the data before the search, information from the validation folds leaks into training. Putting the scaler inside a Pipeline means it is refitted inside each training fold. The user guide’s naming rule for pipelines is <step>__<parameter>, with a double underscore, and nesting levels can be chained (its example is model__estimator__max_depth).
from sklearn.pipeline import Pipelinefrom sklearn.preprocessing import StandardScalerfrom sklearn.svm import SVCpipe = Pipeline([("scale", StandardScaler()), ("model", SVC())])
3. Define param_grid
param_grid is a dictionary or a list of dictionaries. With a list, each dictionary is its own grid and the grids are explored in turn, which is how you avoid invalid combinations such as a gamma value for a linear kernel. The user guide’s own example is a linear grid and an rbf grid, which it describes as two grids that contribute 4 and 8 combinations.
param_grid = [ {"model__kernel": ["linear"], "model__C": [0.1, 1, 10]}, {"model__kernel": ["rbf"], "model__C": [0.1, 1, 10], "model__gamma": [0.001, 0.01]},]
This grid has 3 combinations in the first dictionary and 3 × 2 = 6 in the second, 9 in total.
4. Choose cv, scoring and refit, then fit
With cv=None the search uses 5-fold cross-validation; for a binary or multiclass classifier the folds are stratified, otherwise plain KFold without shuffling. The default score is the estimator’s own score method (accuracy for classifiers, R² for regressors), and the user guide suggests setting scoring explicitly for cases such as imbalanced classification. To track several metrics, pass a list or dictionary to scoring and name the metric that decides the winner in refit.
search = GridSearchCV( pipe, param_grid, cv=5, scoring={"accuracy": "accuracy", "f1": "f1"}, refit="f1")search.fit(X_dev, y_dev)print(search.best_params_, search.best_score_)
Run on scikit-learn 1.9.1 this reports {'model__C': 0.1, 'model__kernel': 'linear'} with a mean cross-validated F1 of 0.9906.
5. Score the winner once on the evaluation set
print(search.score(X_eval, y_eval))
Because refit="f1" was set, score delegates to the refitted best estimator and uses the F1 scorer. On the 143 held-out rows the F1 was 0.9674, lower than the 0.9906 from cross-validation: a small illustration of why the search score should not be quoted as the final result.
Grid size: how many fits will it run?
The number of model fits is the number of parameter combinations multiplied by the number of folds, plus one refit when refit is on. In the example, 9 combinations × 5 folds = 45 cross-validation fits, and the fitted object reports n_splits_ = 5 with 9 entries in cv_results_['params']. Every value you add to one parameter multiplies the total, so a grid of five parameters with four values each is 45 = 1,024 combinations, or 5,120 fits at five folds. Setting verbose=1 prints the number of fits before the search starts, which is a cheap way to check your arithmetic before committing compute. Setting verbose=2 adds timings and verbose=3 adds the fold scores.
n_jobs runs fits in parallel, but the documentation notes that data is copied for each grid point when n_jobs is above 1, and pre_dispatch (default '2*n_jobs') exists to limit the number of copies and avoid memory errors.
Output: how to read the results
After fitting, the attributes below are the ones to report or inspect. The documentation notes that some are absent depending on refit.
best_params_: the setting that scored best on the held-out folds.best_score_: the mean cross-validated score of that setting (not available whenrefitis a callable).best_estimator_: the model refitted on the whole development set (not available whenrefit=False).best_index_: the row ofcv_results_that belongs to the best setting.cv_results_: a dictionary of arrays that converts directly to a pandas DataFrame, withparam_*columns,split0_test_scorethroughsplit{k}_test_score,mean_test_score,std_test_score,rank_test_score, fit and score timings, and aparamscolumn.n_splits_,scorer_andrefit_time_: the number of folds, the scorer used and the seconds spent on the final refit.
With multi-metric scoring the score columns carry the metric name instead, for example mean_test_f1 and rank_test_accuracy. The data table below lists all nine candidates of the worked example, ranked by mean F1. The ranking carries information that best_params_ alone hides: the top three settings are within about 0.008 F1 of each other, and the weakest candidate (a very small C with rbf and gamma 0.001) is far behind at 0.8123, which shows how much a poorly placed grid can cost. Compare mean_test_score with std_test_score before declaring a winner: when two settings differ by less than the fold-to-fold spread, the choice between them is not strongly supported by the data.
import pandas as pdresults = pd.DataFrame(search.cv_results_)cols = ["params", "mean_test_f1", "std_test_f1", "rank_test_f1"]print(results[cols].sort_values("rank_test_f1"))
Set return_train_score=True to add training-fold scores to cv_results_. The documentation notes it is off by default because it costs extra compute; it is useful for diagnosing overfitting, since a large gap between training and validation scores points that way.
Reporting discipline for papers and theses
Three practices follow from the user guide. First, report the grid (every parameter and value), cv, scoring and the selection rule, and the scikit-learn version. Second, treat best_score_ as a development-set number: the same folds both chose and scored the winner, which is why the guide separates the development set from the evaluation set and points to its “Nested versus non-nested cross-validation” example for a fuller treatment. Third, say which data the final number came from. In the example above, 0.9906 is the cross-validation figure and 0.9674 is the evaluation-set figure; only the second is a fair estimate of out-of-sample performance.
Troubleshooting
Some fits fail and scores come back as nan
error_score defaults to nan, so a setting whose fit raises an exception is scored as nan rather than stopping the search. Set error_score="raise" while debugging to see the real traceback, or pass a number to score failing settings with that value. If the final refit fails, the error is raised regardless.
Error about refit when using several metrics
With multi-metric scoring, refit must be the name of one scorer (or False, or a callable that takes cv_results_ and returns the best index). Leaving it as None raises an error, and best_estimator_ does not exist when refit=False. The halving searches do not support multi-metric scoring at all.
The estimator has no score method
The estimator must provide a score method, or you must pass scoring explicitly.
The search is too slow or runs out of memory
Reduce the grid (fewer values, or split it into a list of grids), lower cv, or switch to one of the cheaper searches in the next section. If you use n_jobs above 1 and memory is the problem, lower n_jobs or pre_dispatch because of the data copies described above.
Methods raise errors on the search object
predict, predict_proba, decision_function, transform and the other prediction methods are available on the search object only when refit=True; they delegate to best_estimator_.
Free alternatives
GridSearchCV is one of several selection tools in scikit-learn, all open-source. The user guide describes the following alternatives; the right one depends on how large your search is.
| Tool | How it differs from GridSearchCV | Note from the user guide |
|---|---|---|
RandomizedSearchCV |
Samples settings from distributions instead of trying every combination | The budget is set by n_iter and does not depend on the number of parameters; the guide cites Bergstra and Bengio (2012) |
HalvingGridSearchCV |
Starts all candidates on small resources and keeps only the best at each stage | Experimental: needs from sklearn.experimental import enable_halving_search_cv; the elimination factor is usually 3 |
HalvingRandomSearchCV |
The same successive-halving idea applied to randomly sampled candidates | Experimental, same enabling import |
LassoCV, ElasticNetCV, RidgeCV, LogisticRegressionCV |
Estimators with built-in cross-validation of the regularisation setting | Listed as alternatives for regularisation paths |
| Out-of-bag estimates | Estimate performance without cross-validation for random forests and gradient boosting | Listed as an alternative where the estimator supports it |
Frequently asked questions
What does GridSearchCV do in scikit-learn?
It runs a cross-validated, exhaustive search over a grid of hyperparameter values you specify, records the scores in cv_results_, and (with refit=True) refits the best combination on the full data it was given.
How many folds does GridSearchCV use by default?
Five. With cv=None the 1.9.1 documentation uses 5-fold cross-validation, stratified for classifiers on binary or multiclass targets and plain KFold otherwise.
Is best_score_ the accuracy I should report?
No. It is the mean cross-validated score of the winning setting on the same data used to choose it, so it is optimistic. Report the score of the refitted model on a separate evaluation set, or use nested cross-validation.
How do I tune a Pipeline with GridSearchCV?
Name each parameter as the step name, two underscores, then the parameter, for example model__C. This keeps preprocessing inside each training fold.
What is the difference between GridSearchCV and RandomizedSearchCV?
GridSearchCV tries every combination in the grid. RandomizedSearchCV samples a fixed number of settings (n_iter) from lists or distributions, so its cost does not grow with the number of parameters.
Sources
- scikit-learn 1.9.1 documentation, sklearn.model_selection.GridSearchCV, read 2026-11-14.
- scikit-learn 1.9.1 user guide, Tuning the hyper-parameters of an estimator, read 2026-11-14.
- scikit-learn 1.9.1 documentation, Installing scikit-learn, read 2026-11-14.
- Worked-example figures: produced by running the code shown on this page with scikit-learn 1.9.1, run 2026-11-14.
Data: GridSearchCV results for an SVC pipeline on the scikit-learn breast cancer dataset (9 candidates, 5-fold CV)
Cross-validated results of every candidate in the worked example on this page: a StandardScaler plus SVC pipeline tuned with GridSearchCV(cv=5, scoring accuracy and F1, refit='f1') on a stratified 75% development split (random_state=0) of the breast cancer dataset bundled with scikit-learn 1.9.1. Values were produced by running the code on the page and read from cv_results_; the parameter names and attributes follow the scikit-learn documentation.
Period covered: 2026.
| Candidate (SVC settings) | Mean CV F1 | Std of CV F1 | Rank by F1 | Mean CV accuracy |
|---|---|---|---|---|
| linear, C=0.1 | 0.9906 | 0.0104 | 1 | 0.9883 |
| rbf, C=10, gamma=0.01 | 0.9851 | 0.0045 | 2 | 0.9812 |
| linear, C=1 | 0.9833 | 0.0108 | 3 | 0.9789 |
| rbf, C=10, gamma=0.001 | 0.9798 | 0.0104 | 4 | 0.9742 |
| rbf, C=1, gamma=0.01 | 0.9746 | 0.0136 | 5 | 0.9672 |
| linear, C=10 | 0.959 | 0.0171 | 6 | 0.9484 |
| rbf, C=1, gamma=0.001 | 0.9576 | 0.0229 | 7 | 0.9438 |
| rbf, C=0.1, gamma=0.01 | 0.9557 | 0.0218 | 8 | 0.9414 |
| rbf, C=0.1, gamma=0.001 | 0.8123 | 0.0262 | 9 | 0.7091 |
Sources
- scikit-learn GridSearchCV documentation (cv_results_ keys, refit, scoring) (accessed 2026-11-14)
- scikit-learn user guide: tuning hyper-parameters (accessed 2026-11-14)
Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.







