Skip to main content
v2026.11,986 entries · CC-BY 4.0

PolynomialFeatures in scikit-learn: Research Implementation Guide

A guide to sklearn PolynomialFeatures: generating polynomial terms and interactions in Python pipelines with stable numerical scaling and regularized models.

Written and maintained by CASRAI Editorial Board

Published

On this page

The scikit-learn PolynomialFeatures transformer in sklearn.preprocessing generates explicit polynomial combinations and interaction features from numeric input variables, enabling linear regression and classification models to capture complex non-linear relationships. By projecting an input matrix into a higher-dimensional feature space composed of monomial powers and cross-products, researchers can model polynomial response curves and multi-variable interactions while retaining the mathematical guarantees, rapid convergence, and convex optimization solvers of linear estimators. The data table below compares basis expansion algorithms across scikit-learn and scientific Python.

What PolynomialFeatures is used for in research

In scientific computing, quantitative social science, econometrics, and biomedical research, empirical phenomena rarely follow strictly linear trajectories. Physical systems frequently exhibit accelerating returns, saturation thresholds, quadratic drag, or synergistic interaction effects where the impact of an exposure variable depends upon the concentration of a secondary factor. When working with observational or experimental datasets, researchers often evaluate non-linear hypotheses without abandoning the interpretability and inferential structure of linear regression models. The sklearn.preprocessing.PolynomialFeatures class provides an automated, standardized mechanism to perform basis function expansion on feature matrices.

Rather than manually calculating and maintaining squared, cubic, or multiplicative columns in data tables, PolynomialFeatures systematically computes all polynomial combinations up to a specified maximum degree. For a feature vector with two continuous predictors [x₁, x₂] and a designated degree of 2 with bias included, the transformer produces the expanded coordinate vector [1, x₁, x₂, x₁², x₁x₂, x₂²]. When supplied to a standard linear estimator, the resulting model fits the regression equation y = β₀ + β₁x₁ + β₂x₂ + β₃x₁² + β₄x₁x₂ + β₅x₂². Crucially, although the relationship between the target and the original raw features is non-linear, the equation remains strictly linear with respect to the regression parameters β. This linear-in-parameters formulation allows researchers to utilize standard convex optimization methods, such as ordinary least squares (OLS) or penalized regression, which converge reliably to a unique global optimum without the local minima or initialization sensitivity typical of deep neural architectures.

Research applications of polynomial feature expansion span several core experimental domains:

  • Experimental response surface methodology: Modeling physical or chemical response surfaces where researchers investigate optimal temperature, pressure, or reactant concentration combinations to maximize experimental yield.
  • Moderator and interaction testing: Detecting synergistic or antagonistic interactions across treatments in clinical and epidemiological studies, identifying whether treatment efficacy varies systematically across baseline subgroups. In observational designs, modeling interactions is essential to adjust for a confounding variable whose moderating influence distorts direct causal paths.
  • Econometric production functions: Estimating translog and quadratic cost functions where input substitution elasticities vary across firm scale or capital intensity levels.
  • Kinetic and biomechanical modeling: Capturing acceleration and velocity power laws in biomechanics, physiological signal processing, and physical laboratory instrumentation.

Despite its mathematical flexibility, polynomial expansion introduces significant computational and statistical hazards that researchers must manage. The primary hazard is combinatorial feature explosion. For n input features expanded to degree d with bias included, the total number of output features P is governed by the binomial coefficient:

P = (n + d)! / (n! × d!) = C(n + d, d)

When the bias column is excluded (include_bias=False), the count is C(n + d, d) − 1. If only distinct interaction terms are requested (interaction_only=True), the feature count collapses to the sum of binomial combinations ∑ C(n, k) for k from 1 to d. As the input dimensionality n or the polynomial degree d increases, P grows exponentially. For example, an input matrix with 10 features expanded to degree 2 yields 66 features; expanded to degree 3, it produces 286 features; and expanded to degree 5, it generates 3,003 features. Expanding 50 features to degree 4 generates 316,251 features, requiring gigabytes of memory for a single dense matrix and drastically exceeding the sample size in most scientific experiments. When feature count exceeds observation count (where P is greater than N), ordinary least squares becomes unidentifiable, leading to severe what is overfitting where the model fits random sample noise perfectly but fails completely on holdout validation data.

Install or access scikit-learn

The PolynomialFeatures transformer is part of the standard scikit-learn machine learning library. Researchers can install and access scikit-learn through official package managers across Linux, macOS, and Windows operating systems. The package requires a functioning Python environment (Python 3.9 or higher) along with its foundational scientific computing dependencies, NumPy and SciPy.

To install scikit-learn using the standard Python package installer (pip), execute the following command in your terminal or virtual environment:

pip install -U scikit-learn

For scientific research workflows managed through Anaconda or Miniconda environments, install scikit-learn from the community-maintained conda-forge channel to ensure binary compatibility with optimized BLAS/LAPACK linear algebra libraries:

conda install -c conda-forge scikit-learn

To verify that the installation is complete and confirm the installed version from your command line, run:

python -c "import sklearn; print('scikit-learn version:', sklearn.__version__)"

In cloud-hosted scientific environments such as Google Colab, Kaggle Kernels, Amazon SageMaker, and JupyterHub institutional clusters, scikit-learn is pre-installed in the default runtime. Researchers can import PolynomialFeatures directly from the sklearn.preprocessing module without extra configuration.

Step-by-step task: generating polynomial and interaction features

Executing polynomial feature expansion within a reproducible scientific pipeline requires disciplined data preparation, strict leakage prevention, and model regularization. Follow this five-step walkthrough to configure, fit, and evaluate polynomial features in Python.

Step 1: Partition data and apply prerequisite feature scaling

Never apply PolynomialFeatures to raw, unscaled predictors. Raising unstandardized features with large magnitudes to powers of 2, 3, or 4 leads to severe numerical scaling disparities. For instance, if an input variable ranges from 10 to 1,000, its fourth power ranges from 10,000 to 1,000,000,000,000, causing floating-point rounding errors and destabilizing gradient descent or matrix inversion. Furthermore, feature scaling and polynomial transformations must be fitted exclusively on the training partition and applied downstream to test partitions to avoid data snooping.

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# Simulated research dataset: 200 observations, 3 biological covariates
np.random.seed(42)
X = np.random.uniform(1.0, 10.0, size=(200, 3))
y = 2.5 * X[:, 0] + 1.8 * (X[:, 1]**2) - 0.7 * (X[:, 0] * X[:, 2]) + np.random.normal(0, 1, size=200)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

Step 2: Configure PolynomialFeatures parameters

Initialize PolynomialFeatures by setting four primary hyperparameters depending on research objectives:

  • degree: An integer (e.g., 2) specifying maximal degree, or a tuple of two integers (min_degree, max_degree) available in scikit-learn 1.0+. Passing degree=(2, 2) isolates pure second-order terms while omitting original linear features.
  • interaction_only: Set to True if research theory posits that individual factors act linearly in isolation but exhibit joint synergistic effects (e.g., drug-drug interactions), thereby excluding squared terms like x₁².
  • include_bias: Set to False when pairing with linear estimators that already include an intercept parameter (fit_intercept=True). If both include bias, the resulting design matrix contains duplicate constant columns, creating rank deficiency.
  • order: Memory layout order ('C' for row-major dense C-order, or 'F' for column-major Fortran order). Fortran order can accelerate feature computation during fitting.
from sklearn.preprocessing import PolynomialFeatures

# Configure second-degree expansion with interaction terms, omitting duplicate bias
poly = PolynomialFeatures(degree=2, interaction_only=False, include_bias=False)
X_train_poly = poly.fit_transform(X_train_scaled)
X_test_poly = poly.transform(X_test_scaled)

print(f'Original features: {X_train.shape[1]}')
print(f'Expanded features: {X_train_poly.shape[1]}')

Step 3: Assemble an end-to-end Pipeline with regularized regression

Because polynomial expansion drastically inflates multicollinearity and variance, ordinary least squares regression often yields wildly fluctuating, uninterpretable coefficients. Researchers should integrate preprocessing and estimation into a unified Pipeline object paired with regularized shrinkage. Examining ridge vs lasso regression clarifies this choice: Ridge (L2 penalty) smoothly shrinks correlated polynomial coefficients toward zero, whereas Lasso (L1 penalty) enforces sparsity by setting non-informative interaction terms exactly to zero, performing automated variable selection.

from sklearn.pipeline import Pipeline
from sklearn.linear_model import RidgeCV

# Construct atomic pipeline to eliminate data leakage during cross-validation
polynomial_ridge_pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('poly', PolynomialFeatures(degree=2, include_bias=False)),
    ('regressor', RidgeCV(alphas=np.logspace(-3, 3, 20), cv=5))
])

polynomial_ridge_pipe.fit(X_train, y_train)
train_score = polynomial_ridge_pipe.score(X_train, y_train)
test_score = polynomial_ridge_pipe.score(X_test, y_test)
print(f'Train R²: {train_score:.4f}, Test R²: {test_score:.4f}')

Step 4: Extract feature names for research reporting

To communicate empirical findings in scientific publications, researchers must identify which specific polynomial terms and cross-product interactions correspond to the estimated regression weights. Call get_feature_names_out() on the fitted transformer:

feature_labels = ['Temperature', 'Pressure', 'Catalyst']
poly_transformer = polynomial_ridge_pipe.named_steps['poly']
expanded_names = poly_transformer.get_feature_names_out(feature_labels)

best_ridge = polynomial_ridge_pipe.named_steps['regressor']
for name, coef in zip(expanded_names, best_ridge.coef_):
    print(f'{name:25s}: {coef:10.4f}')

Step 5: Inspect residual diagnostics and non-linear curvature

Evaluate goodness-of-fit by computing residual plots against predicted values. If residual plots exhibit U-shaped or inverted patterns after fitting a linear model, the original specification suffered from functional form misspecification. The inclusion of degree-2 polynomial features should eliminate systematic curvature in the residuals, resulting in a homoscedastic, random scatter around zero.

Output and how to read it

The primary output of PolynomialFeatures.fit_transform() is a 2D NumPy ndarray (or SciPy sparse matrix when configured) of shape (n_samples, n_output_features_). Interpreting this output correctly requires inspecting both the numerical values and the internal structural attributes exposed by the transformer after fitting.

The internal state of a fitted PolynomialFeatures instance includes four essential attributes:

  • n_features_in_: An integer indicating the count of input features observed during the fit call.
  • feature_names_in_: A NumPy array containing input column names if the training data was passed as a Pandas DataFrame with string column headers.
  • n_output_features_: An integer specifying the total count of generated polynomial and interaction features.
  • powers_: An integer array of shape (n_output_features_, n_features_in_) detailing the exact exponent of each input variable for every generated output column.

Understanding the powers_ matrix is critical for automated feature auditing. Each row in powers_ represents an output column, and each column corresponds to an original input predictor. For an input dataset with three variables [x₀, x₁, x₂], an output feature representing the interaction x₀ × x₂² will have the row vector [1, 0, 2] in powers_. A bias row contains all zeros [0, 0, 0], corresponding to x₀⁰ × x₁⁰ × x₂⁰ = 1.

When mapping polynomial regression outputs back to analytical scientific formulations, researchers must distinguish between explicit basis expansion and implicit kernel methods. In what is a support vector machine, polynomial kernels compute inner products in a high-dimensional Hilbert space implicitly via the kernel function K(x, z) = (γ⟨x, z⟩ + r)ᵈ without ever generating or storing the intermediate monomial matrix in system memory. In contrast, PolynomialFeatures constructs the physical matrix explicitly. This explicit representation provides direct access to individual regression coefficients, enabling hypothesis testing on interaction terms, but imposes substantial memory overhead on large feature spaces.

Troubleshooting common errors and numerical instability

Researchers implementing polynomial expansions frequently encounter computational bottlenecks, linear algebra failures, and pipeline discrepancies. Review the diagnostic resolutions below to address common issues:

1. MemoryError and Kernel Crashes from Combinatorial Explosion

Symptom: Python terminates abruptly, Jupyter crashes with a memory allocation error, or system memory spikes to 100%.

Cause: Specifying a high degree (such as degree=4 or 5) on a dataset with more than 15 features causes the count of output columns to exceed available RAM. A matrix of 100,000 samples and 50 features expanded to degree 3 produces 23,426 columns, requiring approximately 18.7 GB of memory for a single float64 array.

Resolution: Restrict the expansion to degree=2, set interaction_only=True to eliminate pure higher powers, or apply dimensionality reduction (such as feature selection or principal component analysis) prior to polynomial expansion.

2. Multicollinearity and Singular Matrix Errors in OLS

Symptom: Scipy or NumPy emits LinAlgError: Singular matrix, or coefficient estimates explode to values like 10¹⁴ with immense standard errors.

Cause: Polynomial powers and cross-products of correlated variables are inherently collinear. As powers increase, column correlations approach 1.0, driving the condition number of the design matrix XᵀX toward infinity and rendering matrix inversion numerically unstable. Variance Inflation Factors (VIF) frequently exceed thousands.

Resolution: Replace unregularized LinearRegression with penalized estimators like RidgeCV or ElasticNetCV. The L2 penalty adds a positive constant α to the diagonal of XᵀX, ensuring the matrix is strictly invertible and stabilizing coefficient estimates.

3. Redundant Bias Column with Linear Estimators

Symptom: Regression output contains two intercept terms, or a warning indicates rank deficiency in the intercept column.

Cause: Both PolynomialFeatures(include_bias=True) and LinearRegression(fit_intercept=True) generate an intercept column. The design matrix contains an explicit column of ones while the estimator internally adds a second constant vector.

Resolution: Always set include_bias=False in PolynomialFeatures when using standard scikit-learn estimators that fit their own intercept by default.

4. Feature Name Mismatches in Cross-Validation and Deployment

Symptom: Calling predict() on new test data raises ValueError: The feature names should match those that were passed during fit.

Cause: The transformer was fitted on a Pandas DataFrame with specific column names, but downstream inference received an array or DataFrame with modified or reordered column labels.

Resolution: Encapsulate preprocessing in a Pipeline, maintain strict column ordering, or convert DataFrames to NumPy arrays consistently across training and scoring environments.

5. Boundary Divergence (Runge’s Phenomenon) in Extrapolation

Symptom: Predictions for data points slightly outside the training variable ranges diverge wildly toward positive or negative infinity.

Cause: High-degree polynomials oscillate severely near the boundaries of the training domain (Runge’s phenomenon). Monomial terms like x⁴ or x⁵ grow unbounded outside the convex hull of the training data.

Resolution: For non-linear relationships with bounded domain requirements, avoid global polynomials and switch to local piecewise approximations like SplineTransformer.

When researchers require formal p-values, standard errors, and confidence intervals for polynomial coefficients rather than purely predictive models, contrasting scikit-learn with specialized econometric packages is instructive. For instance, logistic regression in statsmodels provides comprehensive statistical summary tables displaying asymptotic covariance matrices, Wald tests, and log-likelihood metrics.

Free alternatives for non-linear feature engineering

While PolynomialFeatures is the standard baseline for generating non-linear representations in scikit-learn, several open-source alternative methods offer distinct advantages in computational efficiency, boundary stability, and high-dimensional scaling. The data table below compares these alternatives across key technical dimensions.

The table below summarizes five primary non-linear feature engineering methods available in scikit-learn and the scientific Python ecosystem:

Method Python Implementation Mathematical Formulation Dimensionality Growth Primary Advantage Primary Limitation
Polynomial Expansion sklearn.preprocessing.PolynomialFeatures Global monomial powers and cross-products Combinatorial O(nᵈ) Direct algebraic interpretability of interaction terms Boundary oscillation (Runge’s phenomenon) and rapid memory explosion
B-Spline Basis sklearn.preprocessing.SplineTransformer Piecewise polynomial B-splines with localized knots Linear O(n × n_knots) Local support prevents boundary divergence; stable numerical conditioning Does not generate multi-variable cross-product interactions by default
Random Fourier Features sklearn.kernel_approximation.RBFSampler Monte Carlo approximation of Radial Basis Function kernel User-defined O(n_components) Scales to large feature counts without combinatorial explosion Transformed features are random cosine projections lacking direct physical meaning
Nystroem Approximation sklearn.kernel_approximation.Nystroem Low-rank eigendecomposition of subsampled kernel matrix User-defined O(n_components) Captures arbitrary non-linear kernel metrics with fixed memory bounds Approximation quality depends heavily on representative landmark sampling
Manual Interaction Modeling Pure Python / NumPy / Pandas custom functions Theory-driven domain cross-products (e.g., x₁ × x₂) Linear O(k) for k selected terms Minimal multicollinearity; preserves degrees of freedom and statistical power Requires prior domain knowledge; fails to discover unexpected interactions

Frequently asked questions

Does PolynomialFeatures scale features automatically?

No. PolynomialFeatures performs algebraic exponentiation and multiplication on input values without normalization. Researchers must always precede polynomial feature generation with a feature scaler such as StandardScaler or RobustScaler within an atomic pipeline. Failing to scale features causes severe numerical disparities across terms and destabilizes regression solvers.

What is the difference between interaction_only and full polynomial expansion?

Full polynomial expansion generates both powers of individual features (e.g., x₁², x₁³) and cross-product terms between distinct variables (e.g., x₁x₂). Setting interaction_only=True restricts generation exclusively to products of distinct features, omitting all pure squared and cubic powers. This reduces dimensionality and isolates synergistic interactions without assuming non-linear self-effects.

When should include_bias be set to False?

You should set include_bias=False whenever you feed the transformed data into a downstream estimator that includes its own intercept parameter, such as LinearRegression(fit_intercept=True) or Ridge(). Setting include_bias=True creates an unnecessary column of ones, resulting in duplicate intercepts and potential linear algebra collinearity warnings.

How do you get feature names from PolynomialFeatures in scikit-learn?

In modern scikit-learn (version 1.0 and later), call get_feature_names_out(input_features) on the fitted transformer. Pass a list of your original variable names to receive an array of formatted strings indicating the exact powers and interactions (e.g., ‘x0^2’, ‘x0 x1’). The older get_feature_names() method was deprecated in version 1.0 and removed in version 1.2.

Why does polynomial regression cause extreme multicollinearity?

Polynomial terms are mathematical transformations of the original predictors, meaning that x and x² inherently share strong statistical correlation. As the degree increases, correlations between columns approach 1.0, causing the design matrix condition number to spike and inflating the Variance Inflation Factor (VIF). To mitigate multicollinearity, center your variables before expansion and fit regularized models like Ridge or Lasso.

How do you prevent data leakage when using PolynomialFeatures in cross-validation?

To prevent data leakage, encapsulate PolynomialFeatures and any prerequisite scalers inside a sklearn.pipeline.Pipeline. When evaluating models with cross-validation (such as cross_val_score), the pipeline ensures that transformers compute scaling parameters and expansion rules strictly on the training folds and apply them to validation folds without information snooping.

Sources

  • scikit-learn Development Team. “sklearn.preprocessing.PolynomialFeatures.” scikit-learn stable API documentation. https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html (accessed 2026-11-14).
  • scikit-learn Development Team. “Polynomial Regression: Extending Linear Models with Basis Functions.” scikit-learn User Guide §1.1.16. https://scikit-learn.org/stable/modules/linear_model.html#polynomial-regression-extending-linear-models-with-basis-functions (accessed 2026-11-14).
  • scikit-learn Development Team. “Preprocessing Data: Generating Polynomial Features.” scikit-learn User Guide §6.3.4. https://scikit-learn.org/stable/modules/preprocessing.html#generating-polynomial-features (accessed 2026-11-14).
  • scikit-learn Development Team. “sklearn.pipeline.Pipeline.” scikit-learn stable API documentation. https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html (accessed 2026-11-14).
  • scikit-learn Development Team. “sklearn.preprocessing.SplineTransformer.” scikit-learn stable API documentation. https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.SplineTransformer.html (accessed 2026-11-14).
  • scikit-learn Development Team. “Installing scikit-learn.” scikit-learn stable installation guide. https://scikit-learn.org/stable/install.html (accessed 2026-11-14).
  • SciPy Community. “scipy.interpolate.BSpline.” SciPy Reference Guide. https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.BSpline.html (accessed 2026-11-14).

Data: Non-Linear Feature Expansion and Transformation Methods in scikit-learn and Scientific Python

Comparative specifications of non-linear basis expansion, spline transformations, and kernel approximation algorithms in scikit-learn and SciPy, detailing mathematical formulation, feature growth rates, support bounds, sparse matrix support, and boundary extrapolation behavior.

Period covered: 2024/2026. Coverage: Global.

Non-Linear Feature Expansion and Transformation Methods in scikit-learn and Scientific Python
MethodScikit-Learn ClassMathematical BasisFeature Growth RateSupport TypeSparse Matrix InputBoundary Hazard
Polynomial Expansion (Full)sklearn.preprocessing.PolynomialFeaturesGlobal monomial powers and cross-productsCombinatorial O(n^d)GlobalYes (Fortran order)Runge's phenomenon / rapid boundary divergence
Polynomial Expansion (Interaction Only)sklearn.preprocessing.PolynomialFeaturesSquare-free feature cross-productsSum of binomial coefficientsGlobalYes (CSR and CSC)Unbounded multiplicative extrapolation
Piecewise B-Spline Basissklearn.preprocessing.SplineTransformerPiecewise polynomial B-splines with knotsLinear O(n * n_knots)Local (compact knot intervals)NoControlled (constant extrapolation or periodic)
Random Fourier Featuressklearn.kernel_approximation.RBFSamplerMonte Carlo cosine projection of RBF kernelFixed O(n_components)Global decayingNoDecays toward zero outside sample density
Nystroem Kernel Approximationsklearn.kernel_approximation.NystroemSubsampled eigendecomposition of kernel matrixFixed O(n_components)Data-dependent landmark supportYes (kernel dependent)Distortions outside sample convex hull

Sources

Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.

Follow CASRAI

Research-administration guidance, standards updates and independent tool reviews.

Ask CASRAI · free to try

Ask about PolynomialFeatures in scikit-learn: Research Implementation Guide

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →