What Is a Support Vector Machine (SVM)?

Q: Which kernel should I start with?

scikit-learn’s SVC defaults to rbf . For a linear problem with a very large sample, LinearSVC is the faster implementation.

A support vector machine (SVM) is a supervised learning method for classification, regression and outlier detection. For classification, it finds the hyperplane that best separates the data points of one class from those of another, and it keeps only the training points closest to that boundary, the support vectors, to define it. This guide explains the idea and then runs one in scikit-learn, with the outputs the official documentation shows and the current-version details that older tutorials miss.

What it is used for in research

SVMs are a standard choice when a study needs to assign samples to classes from many measured features. The scikit-learn documentation lists these strengths: SVMs are effective in high-dimensional spaces, remain effective when the number of dimensions is greater than the number of samples, are memory efficient because they use a subset of the training points (the support vectors), and are versatile because different kernel functions can be specified. MathWorks lists signal processing, natural language processing, and speech and image recognition among typical applications.

Three terms carry most of the idea:

Hyperplane. The boundary that separates one class from another.
Support vectors. According to MathWorks, a subset of the training observations that identify the location of the separating hyperplane.
Margin. The maximal width of the slab parallel to the hyperplane that has no interior data points. A soft margin allows the hyperplane to separate many, but not all, data points.

When classes cannot be separated by a straight boundary, nonlinear SVMs use kernel functions to transform the features. scikit-learn offers the linear, polynomial (poly), radial basis function (rbf) and sigmoid kernels through the kernel parameter.

The same caution applies as for any flexible classifier. The scikit-learn documentation warns that if the number of features is much greater than the number of samples, avoiding over-fitting through the choice of kernel function and the regularization term is crucial. See what overfitting is and working with high-dimensional data before trusting a score from a small sample. For where SVMs sit among other methods, see what machine learning is and supervised versus unsupervised learning (an SVM is a supervised method).

Install or access

The scikit-learn installation page (checked 2026-10-29) gives the pip command:

pip install -U scikit-learn

The same page lists the install-time dependencies and their minimum versions as numpy 1.24.1, scipy 1.10.0, joblib 1.4.0, narwhals 2.0.1 and threadpoolctl 3.5.0. The SVM classes live in the sklearn.svm module. If you work in MATLAB instead, the equivalent functions are fitcsvm for classification and fitrsvm for regression, with Classification Learner and Regression Learner apps for point-and-click use.

Step-by-step: fit and predict with SVC

This is the minimal example from the scikit-learn user guide.

Import the module and define a tiny training set of two samples with two features each, and one class label per sample:
```
from sklearn import svm
X = [[0, 0], [1, 1]]
y = [0, 1]
```
Create the classifier and fit it:
```
clf = svm.SVC()
clf.fit(X, y)
```
Predict the class of a new sample:
```
clf.predict([[2., 2.]])
```

For real data, scale the features and fit both steps as one pipeline. The documentation states that SVM algorithms are not scale invariant, so scaling is highly recommended, and the same scaling must be applied to the test vector. The SVC reference page shows:

import numpy as np
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
X = np.array([[-1, -1], [-2, -1], [1, 1], [2, 1]])
y = np.array([1, 1, 2, 2])
clf = make_pipeline(StandardScaler(), SVC(gamma='auto'))
clf.fit(X, y)
print(clf.predict([[-0.8, -1]]))

Holding out data to test the fitted model is covered in train/test splitting in Python, and feature scaling in what data normalization is. A longer scikit-learn walk-through is in the scikit-learn tutorial.

Output and how to read it

For the first example, the documentation shows clf.fit(X, y) returning SVC() and clf.predict([[2., 2.]]) returning array([1]): the new point is assigned to class 1. In the pipeline example the printed result is [1].

The fitted model exposes the support vectors directly:

clf.support_vectors_ returns array([[0., 0.], [1., 1.]]) in the first example. Both training points are support vectors, as expected with only two samples. The attribute has shape (number of support vectors, number of features).
clf.support_ gives the positions of those points in the training data ([0, 1] here).
clf.n_support_ gives the number of support vectors per class ([1, 1] here).

MathWorks recommends evaluating a trained SVM with accuracy, precision, recall and F1 on held-out data, and tuning hyperparameters (for example with Bayesian optimization) afterwards.

The two parameters you will tune: C and gamma

The scikit-learn user guide says C is 1 by default and is a reasonable default choice; with many noisy observations, decrease it, since decreasing C corresponds to more regularization. The SVC reference adds that the strength of the regularization is inversely proportional to C and that C must be strictly positive. For the rbf kernel, gamma defines how much influence a single training example has: the larger gamma is, the closer other examples must be to be affected. The default is gamma='scale', equal to 1 / (n_features * X.var()); 'auto' uses 1 / n_features. The default changed from 'auto' to 'scale' in version 0.22, so tutorials written before then may behave differently. The documentation suggests searching C and gamma with GridSearchCV, spaced exponentially.

Troubleshooting

Poor results on unscaled data

SVMs are not scale invariant. Put StandardScaler in a pipeline ahead of SVC, as in step 4, so the same scaling is applied to training and test data.

Training is slow on a large dataset

The documentation states that the libsvm-based solver behind SVC scales between O(n_features x n_samples²) and O(n_features x n_samples³). For a linear kernel, LinearSVC is a faster implementation and, per the documentation, can scale almost linearly to millions of samples and/or features.

You need probabilities, not just classes

SVMs do not directly provide probability estimates. Older tutorials use SVC(probability=True), which the user guide describes as relying on an expensive five-fold cross-validation. In the scikit-learn 1.9.1 reference, the probability parameter is deprecated and scheduled for removal in 1.11, and the documentation recommends CalibratedClassifierCV(SVC(), ensemble=False) instead. If you only need confidence scores, use decision_function.

Imbalanced classes

Set class_weight='balanced', which weights classes by n_samples / (n_classes * np.bincount(y)). The default is None.

Overfitting with many features and few samples

Choose the kernel and regularization carefully and check performance on held-out data, as the documentation advises (see the overfitting guide linked above).

Free alternatives

Option	What it does	Source note
`SVC` (scikit-learn)	Binary and multi-class classification; kernels linear, poly, rbf, sigmoid or precomputed; rbf by default	scikit-learn SVC reference, 1.9.1
`NuSVC` (scikit-learn)	Binary and multi-class classification, like SVC	scikit-learn user guide, SVM page
`LinearSVC` (scikit-learn)	Faster implementation for the linear-kernel case; scales almost linearly to millions of samples and/or features	scikit-learn user guide, SVM page
`fitcsvm` / `fitrsvm` (MATLAB)	SVM classification and regression, plus the Classification Learner and Regression Learner apps	MathWorks SVM page

FAQ

What is a support vector machine in simple terms?

A supervised learning method that draws the boundary with the widest possible margin between classes, defined only by the training points nearest that boundary, the support vectors.