Written and maintained by CASRAI Editorial Board
Published · Last updated
On this page
A neural network in scikit-learn is the multi-layer perceptron (MLP), available as MLPClassifier for class labels and MLPRegressor for continuous outcomes in the sklearn.neural_network module. You fit it like any other estimator (fit, predict, score), but it works best after you scale the features, and it is not built for GPUs or very large datasets. This guide covers neural networks in sklearn from install to a reproducible worked example, with the errors people hit most often.
What it is used for in research
The scikit-learn user guide describes the supervised MLP as a model with one or more non-linear hidden layers between the input and the output. The learned weight matrices are stored in coefs_ and the bias vectors in intercepts_. Because the hidden layers are non-linear, the model can learn relationships that a linear model cannot, which is why researchers reach for it when a logistic regression or linear regression underfits a tabular dataset.
The same documentation is explicit about the limits. The implementation “is not intended for large-scale applications”, and scikit-learn offers no GPU support. For deep learning or GPU training the docs point to related projects (see the alternatives table below). The documented drawbacks are a non-convex loss (so different random initialisations can give different validation accuracy), the need to tune the number of hidden neurons, layers and iterations, and sensitivity to feature scaling. The documented advantages are the ability to learn non-linear models and online learning through partial_fit.
Neural networks in sklearn: classifier or regressor?
Choose by the type of outcome. MLPClassifier trains with backpropagation and, per the user guide, supports only cross-entropy loss, which is what makes predict_proba available. For more than two classes it uses a softmax output; for multi-label problems it uses a logistic output and assigns 1 where the value is at least 0.5. MLPRegressor uses an identity output and squared error, returns continuous values, and supports multiple outputs. Its loss parameter also accepts 'poisson' (added in version 1.7), which requires non-negative targets.
How to install or access it
The neural network estimators ship inside scikit-learn; there is nothing extra to install. The official installation page gives two routes. With pip: pip install -U scikit-learn. With conda: conda create -n sklearn-env -c conda-forge scikit-learn, then conda activate sklearn-env. To confirm what you have, run python -m pip show scikit-learn or python -c "import sklearn; sklearn.show_versions()". The text of this guide follows the scikit-learn 1.9.1 documentation; record your own version in any analysis you report, because defaults and parameters change between releases.
If you want a small dataset to practise on, the bundled loaders are described in our guide to scikit-learn datasets, and the wider workflow is in the scikit-learn tutorial.
Step-by-step task: fit a scaled MLPClassifier
The documentation says it is “highly recommended to scale your data”, fitting the scaler on the training data only, and suggests StandardScaler inside a Pipeline. The steps below follow that advice with a train/test split and early stopping. This is an illustrative example, not a benchmark, and it prints results you should read from your own run.
- Load data and split it. Hold out a test set before any fitting so that scaling and tuning never see it.
- Build a pipeline. Put
StandardScalerfirst andMLPClassifiersecond, so the scaler is fitted on training data only. - Fix
random_state. Weight initialisation, the early-stopping split and mini-batch sampling are all controlled by it. - Fit, then score on the held-out set.
scorereturns mean accuracy for the classifier. - Inspect
loss_curve_andn_iter_. They show whether training converged.
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPClassifier
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.25, stratify=y, random_state=42
)
model = Pipeline([
("scale", StandardScaler()),
("mlp", MLPClassifier(
hidden_layer_sizes=(50,),
early_stopping=True,
max_iter=500,
random_state=42,
)),
])
model.fit(X_train, y_train)
print("test accuracy:", model.score(X_test, y_test))
mlp = model.named_steps["mlp"]
print("iterations run:", mlp.n_iter_)
print("final training loss:", mlp.loss_curve_[-1])
print(model.predict_proba(X_test[:3]))
The scikit-learn user guide also shows the smallest possible call: MLPClassifier(solver='lbfgs', alpha=1e-5, hidden_layer_sizes=(5, 2), random_state=1) fitted on [[0., 0.], [1., 1.]] with labels [0, 1], which then predicts array([1, 0]) for [[2., 2.], [-1., -2.]]. It demonstrates the API, not a realistic analysis.
Choosing a solver
The user guide describes three solvers: sgd, adam and lbfgs. Per the docs, adam is “very robust” on relatively large datasets, lbfgs “converges faster and with better solutions on small datasets”, and sgd with momentum can outperform both if the learning rate is tuned. The practical consequences are documented too: sgd and adam support online and mini-batch learning, lbfgs does not, and options such as early_stopping only take effect with sgd or adam. The default is adam.
Regularisation and tuning
alpha is the L2 penalty. The user guide suggests tuning it with GridSearchCV, usually over the range 10.0 ** -np.arange(1, 7). Our guide to scikit-learn hyperparameter tuning covers the search tools, and the logistic regression guide is a useful linear baseline to compare an MLP against before you accept its extra complexity.
Neural network sklearn for regression
The regressor follows the same pattern: scale the inputs, then fit MLPRegressor in the pipeline. Its score is the coefficient of determination R², where 1.0 is best, it can be negative, and a constant model that always predicts the mean scores 0.0. Two attribute differences matter: with early_stopping=True, best_loss_ is None, so use best_validation_score_ (an R² value) and validation_scores_ instead; both exist only for the sgd and adam solvers.
Output and how to read it
predict returns class labels (or continuous values for the regressor). predict_proba returns one probability per class, and predict_log_proba returns their logarithms. score on the classifier is mean accuracy, which can mislead on imbalanced classes; if your classes are skewed, consider a class-balance-aware metric such as balanced accuracy instead of reporting a single accuracy figure.
The attributes that tell you how training went:
loss_curve_: the loss at each iteration. A curve that is still falling at the last point means training stopped before convergence.n_iter_: how many iterations were run. Forsgdandadam,max_itercounts epochs, son_iter_equal tomax_itermeans the limit was reached.coefs_andintercepts_: the weight matrices and bias vectors for each layer.
Training stops at the iteration limit or when the improvement in loss falls below the tolerance (tol, default 0.0001). With early_stopping=True the estimator sets aside validation_fraction of the training data (default 0.1) as a validation set and stops when the validation score has not improved for n_iter_no_change epochs (default 10). With learning_rate='adaptive', the rate is divided by 5 when two consecutive epochs fail to improve.
To watch training yourself, the user guide describes a manual loop: set warm_start=True and call fit repeatedly with max_iter=1, recording whatever you want to monitor after each call. The data table below lists the documented defaults for the main MLPClassifier parameters.
Reproducibility and reporting
Because the loss is non-convex, two runs with different random initialisations can give different results, and the docs say as much. The random_state parameter controls the initialisation, the early-stopping split and the batch sampling; its default is None, so an unseeded run is not repeatable. In a methods section, report the scikit-learn version, the full set of non-default parameters, the seed, how the data were split, and which data the scaler was fitted on. If you repeat the fit across several seeds, report the spread, not only the best run.
Troubleshooting
| Symptom | Cause per the documentation | What to try |
|---|---|---|
| Warning that the stochastic optimizer reached maximum iterations | For sgd and adam, max_iter is the number of epochs and training stopped at the limit (default 200) before the tolerance was met |
Raise max_iter, scale the features, or use lbfgs on a small dataset |
| Poor or unstable results on raw features | The model is sensitive to feature scaling | Put StandardScaler in a Pipeline and fit it on training data only |
| Different scores on each run | Non-convex loss; random_state defaults to None |
Set random_state; compare several seeds |
early_stopping seems to do nothing |
It only takes effect with sgd or adam |
Switch solver, or choose lbfgs deliberately and drop the option |
best_loss_ is None on the regressor |
Early stopping is on | Read best_validation_score_ and validation_scores_ |
| Training is slow or the dataset is large | Not intended for large-scale applications; no GPU support | Use a library built for deep learning (see below) |
poisson loss raises an error on the regressor |
It requires non-negative targets and exists from version 1.7 | Check the target range and your installed version |
Free alternatives
The scikit-learn “related projects” page lists two projects under deep neural networks. Both are open-source libraries on GitHub.
| Project | Description on the scikit-learn page | Address |
|---|---|---|
| skorch | A scikit-learn compatible neural network library that wraps PyTorch | github.com/dnouri/skorch |
| SciKeras | Provides a wrapper around Keras to interface it with scikit-learn; successor of tf.keras.wrappers.scikit_learn |
github.com/adriangb/scikeras |
Within scikit-learn itself, tree ensembles are a common non-neural comparison on tabular data; see the guides to random forests and decision trees.
Frequently asked questions
Does scikit-learn have neural networks?
Yes. MLPClassifier and MLPRegressor in sklearn.neural_network implement multi-layer perceptrons. They were added in version 0.18.
Can sklearn neural networks use a GPU?
No. The scikit-learn user guide states that it offers no GPU support and that the implementation is not intended for large-scale applications.
Do I need to scale features before an MLP?
The documentation says it is highly recommended, because the model is sensitive to feature scaling. Fit the scaler on the training data only, ideally inside a pipeline.
What are the default settings of MLPClassifier?
One hidden layer of 100 neurons, relu activation, the adam solver, alpha of 0.0001, a maximum of 200 iterations and no early stopping. The data table below lists the rest.
How do I get reproducible results?
Set random_state, record the scikit-learn version, and report the seed with your results.
Sources
- scikit-learn documentation (1.9.1): Neural network models (supervised), https://scikit-learn.org/stable/modules/neural_networks_supervised.html, checked 2026-11-15.
- scikit-learn documentation: MLPClassifier, https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html, checked 2026-11-15.
- scikit-learn documentation: MLPRegressor, https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html, checked 2026-11-15.
- scikit-learn documentation: Installing scikit-learn, https://scikit-learn.org/stable/install.html, checked 2026-11-15.
- scikit-learn documentation: Related Projects, https://scikit-learn.org/stable/related_projects.html, checked 2026-11-15.
Data: MLPClassifier default parameters (scikit-learn 1.9.1)
Default values and roles of the main MLPClassifier parameters, compiled from the scikit-learn 1.9.1 MLPClassifier API reference and the supervised neural network user guide. Values are as documented on the pages checked on 2026-11-15; check your installed version because defaults can change between releases.
| Parameter | Default | Meaning or allowed values |
|---|---|---|
| hidden_layer_sizes | (100,) | Number of neurons in each hidden layer |
| activation | relu | One of identity, logistic, tanh, relu |
| solver | adam | One of sgd, adam, lbfgs |
| alpha | 0.0001 | L2 regularization strength |
| batch_size | auto | Mini-batch size for sgd and adam |
| learning_rate | constant | Schedule for sgd; adaptive divides the rate by 5 after two epochs without improvement |
| learning_rate_init | 0.001 | Initial learning rate for sgd and adam |
| max_iter | 200 | Maximum iterations; epochs for sgd and adam |
| tol | 0.0001 | Loss-improvement tolerance for stopping |
| early_stopping | False | If True, holds out validation_fraction of training data; sgd and adam only |
| validation_fraction | 0.1 | Share of training data held out when early_stopping is True |
| n_iter_no_change | 10 | Epochs without improvement before stopping |
| random_state | None | Controls initialization, early-stopping split and batch sampling |
Sources
- scikit-learn: MLPClassifier (accessed 2026-11-15)
- scikit-learn: Neural network models (supervised) (accessed 2026-11-15)
Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.







