Written and maintained by CASRAI Editorial Board
Published
On this page
Lasso logistic regression is ordinary logistic regression for a yes/no outcome with an L1 penalty added to the fitting criterion: the model minimises the average negative log-likelihood plus λ times the sum of the absolute values of the coefficients. Because that penalty can set coefficients exactly to zero, the method fits the model and selects predictors in one step, which is why it is used when there are many candidate predictors relative to the number of events.
What is lasso in logistic regression?
Lasso for logistic regression keeps the logistic link (the model is still for the log-odds of the outcome) but changes how the coefficients are estimated. Instead of maximising the likelihood alone, it maximises the likelihood minus a penalty on the size of the slopes. The scikit-learn documentation states that a logistic regression with an L1 penalty yields sparse models and can therefore be used to perform feature selection. The glmnet package, which implements the method in R, describes the tuning parameter λ as controlling the overall strength of the penalty.
The same method appears under several names: logistic regression lasso, lasso in logistic regression, lasso for logistic regression and L1-penalised (or L1-regularised) logistic regression all describe one model. It is a special case of the elastic net, with the mixing parameter set so that only the L1 part is active; see the elastic net guide for the general family and ridge versus lasso for the L2 alternative.
Formula for logistic regression with a lasso penalty
For a binary outcome, the glmnet vignette gives the objective for the binomial family as follows, with the lasso being the case α = 1:
minimise over (β₀, β): −[ (1/N) Σᵢ ( yᵢ(β₀ + xᵢ′β) − log(1 + e^(β₀ + xᵢ′β)) ) ] + λ [ (1 − α) ‖β‖₂² / 2 + α ‖β‖₁ ]
- N: the number of observations; i indexes them.
- yᵢ: the outcome for observation i, coded 0 or 1.
- xᵢ: the vector of predictor values for observation i; xᵢ′β is its inner product with the coefficient vector β.
- β₀: the intercept. It sits outside the penalty term, so it is not shrunk.
- β: the slope coefficients, on the log-odds scale.
- λ (lambda): the penalty strength. Larger λ means stronger shrinkage and more coefficients at exactly zero.
- α (alpha): the mixing weight between L1 and L2. α = 1 is the lasso (the glmnet default) and α = 0 is ridge.
- ‖β‖₁: the sum of the absolute values of the slopes. ‖β‖₂²: the sum of their squares.
With α = 1 the penalty reduces to λ‖β‖₁, so the lasso logistic objective is simply (mean negative log-likelihood) + λ × (sum of |βj|). The first part is the familiar logistic loss; the second is what forces some coefficients to zero.
scikit-learn writes the same idea with a different scale. Its objective minimises the mean log-loss plus r(w)/(S·C), where setting l1_ratio = 1 makes r(w) the L1 norm of the coefficients. C is an inverse regularisation strength: a smaller C means stronger regularisation, whereas in glmnet a larger λ means stronger regularisation. Do not copy a λ from R into the C argument; the two parameters move in opposite directions and are defined on different scales in the two documentation pages.
Worked example with the arithmetic shown
This is an illustrative composite built to be checkable by hand, not data from a study. Take four observations with one standardised predictor x = (−1.5, −0.5, 0.5, 1.5) and outcomes y = (0, 1, 0, 1). To keep the arithmetic short, hold the intercept at β₀ = 0 (the outcome is half ones, so zero is the fitted intercept when the slope is zero) and use λ = 0.1.
For a candidate slope b, each observation contributes yᵢ·z − log(1 + eᶻ) where z = b·xᵢ, and the loss is minus the average of those terms. Take b = 0.4, so z = (−0.6, −0.2, 0.2, 0.6):
- Observation 1 (y = 0, z = −0.6): −log(1 + e^−0.6) = −log(1.5488) = −0.4375
- Observation 2 (y = 1, z = −0.2): −0.2 − log(1 + e^−0.2) = −0.2 − log(1.8187) = −0.7981
- Observation 3 (y = 0, z = 0.2): −log(1 + e^0.2) = −log(2.2214) = −0.7981
- Observation 4 (y = 1, z = 0.6): 0.6 − log(1 + e^0.6) = 0.6 − log(2.8221) = −0.4373
The terms sum to −2.4710, so the loss is 2.4710 / 4 = 0.6178. The lasso penalty adds λ|b| = 0.1 × 0.4 = 0.04, giving a penalised objective of 0.6578. Repeating the calculation for other slopes gives the table below.
| Slope b | Loss (mean negative log-likelihood) | Penalty λ|b| (λ = 0.1) | Penalised objective |
|---|---|---|---|
| 0 | 0.6931 | 0 | 0.6931 |
| 0.2 | 0.6494 | 0.02 | 0.6694 |
| 0.4 | 0.6178 | 0.04 | 0.6578 |
| 0.6 | 0.5978 | 0.06 | 0.6578 |
The unpenalised loss keeps falling as the slope grows, but the penalty claws part of that gain back, so the penalised optimum sits at a smaller slope than the unpenalised one. How much smaller depends on λ. There is also a threshold: at b = 0 the slope of the loss is −(1/N) Σ xᵢ(yᵢ − 0.5) = −0.25, so for any λ ≥ 0.25 the penalty outweighs the loss gain and the fitted slope is exactly zero. This is why a lasso fit is usually run over a sequence of λ values and the final one chosen by cross-validation.
Interpretation: reading the output
Penalised coefficients are still log-odds coefficients, so exponentiating one gives an odds ratio for a one-unit increase in that predictor, holding the others fixed (see how to interpret an odds ratio). For example, a retained coefficient of 0.80 corresponds to exp(0.80) = 2.23, one of −0.40 to exp(−0.40) = 0.67, and a coefficient of 0 to an odds ratio of 1.00. The difference from ordinary logistic regression is that the estimates are deliberately shrunk toward zero, so they are biased low in magnitude in exchange for lower variance.
| What you see | How to read it |
|---|---|
| A coefficient is exactly 0 at the chosen λ | The predictor was dropped from the model at that penalty strength; in glmnet, predict with type = “nonzero” lists the retained variables. |
| A non-zero coefficient | Log-odds change per unit of the predictor; exponentiate for the odds ratio. It is shrunk, not an unpenalised maximum-likelihood estimate. |
| Larger λ (glmnet) or smaller C (scikit-learn) | Stronger penalty, sparser model. |
| lambda.min versus lambda.1se | lambda.min gives the minimum cross-validated error; lambda.1se is the largest λ whose error is within one standard error of that minimum, so it yields a sparser model. |
| predict(…, type = “link”) versus “response” | “link” returns the linear predictor (log-odds); “response” returns the fitted probability; “class” returns the predicted class. |
Choosing λ is done by cross-validation. The glmnet guide shows the full cv.glmnet workflow, and cross-validation in statistics explains k-fold design and how to report it.
Assumptions and common mistakes
- The outcome model is still logistic. The penalty changes estimation, not the link: the log-odds are assumed linear in the predictors you supply, and observations are assumed independent. A lasso does not repair a misspecified functional form or clustered data.
- Predictors should be on comparable scales. The L1 penalty treats all slopes alike, so a predictor measured in small units needs a large coefficient and is penalised more heavily. glmnet defaults to standardize = TRUE and always returns coefficients on the original scale, so you rarely standardise by hand there; check the documentation of any other tool for what it does.
- The intercept is not penalised. In the glmnet objective β₀ is outside the penalty term. Do not drop it from the model.
- Solver compatibility in scikit-learn. The documentation lists liblinear and saga as the solvers that support an L1 penalty; lbfgs, newton-cg, newton-cholesky and sag do not. The default solver, lbfgs, therefore cannot fit a lasso logistic model. saga is the documented solver of choice for sparse multinomial logistic regression.
- An all-zero model. If C is too small, every coefficient is zero. scikit-learn provides l1_min_c to give a lower bound for C so that the model is not all zeros.
- AUC as a CV metric is two-class only. In cv.glmnet, type.measure options for a binomial outcome are deviance, mse, mae, class and auc, and the vignette notes auc is for two-class logistic regression only. The default is deviance.
- Which level is modelled. glmnet returns results for the class of the second level of the factor response. Check your factor levels before reading signs.
- Correlated predictors. The lasso may keep one of a group of correlated predictors and zero the rest; if that matters for your question, consider the elastic net.
How to report a lasso logistic regression (APA style)
Report what a reader would need to reproduce the model: the penalty, how λ was chosen, the software and version, the retained predictors with their coefficients or odds ratios, and an out-of-sample performance measure. A template, with bracketed placeholders to be replaced by your own results:
We fitted an L1-penalised (lasso) logistic regression to predict [outcome] from [number] candidate predictors using glmnet in R ([version]). Predictors were standardised internally. The penalty parameter λ was chosen by [10]-fold cross-validation using the [lambda.1se / lambda.min] rule with deviance as the loss, giving λ = [value]. [Number] predictors had non-zero coefficients. Retained predictors and their odds ratios were: [predictor], OR = [value]. Cross-validated [AUC / deviance] was [value].
State that coefficients are penalised estimates. If you also report confidence intervals or p-values, say how they were obtained, because the standard maximum-likelihood formulas are not the ones that produced these estimates;.
How to calculate it in Excel, R or Python
Excel
Excel has no built-in function that fits a lasso logistic model, but you can evaluate the objective from the worked example and search over the slope yourself. With x in A2:A5, y in B2:B5, the slope in D1, the intercept in D2 and λ in D3, put this in C2 and fill down: =-(B2*($D$2+A2*$D$1)-LN(1+EXP($D$2+A2*$D$1))). Then the penalised objective is =AVERAGE(C2:C5)+$D$3*ABS($D$1). Solver can minimise it over D1; this reproduces the table above (for example 0.6578 at slope 0.4).
R (glmnet)
x is a numeric predictor matrix and y a two-level factor. coef() returns the penalised coefficients on the log-odds scale.
library(glmnet)
cvfit = cv.glmnet(x, y, family = "binomial", alpha = 1, type.measure = "deviance")
coef(cvfit, s = "lambda.1se") coefficients (log-odds scale)
predict(cvfit, newx = x_new, s = "lambda.1se", type = "response")
Python (scikit-learn)
l1_ratio=1 is the lasso, saga supports the L1 penalty, and a smaller C means a stronger penalty. Zeros in model.coef_ mark dropped predictors.
from sklearn.linear_model import LogisticRegression
model = LogisticRegression(l1_ratio=1, C=0.5, solver="saga", max_iter=5000)
model.fit(X_train, y_train)
print(model.coef_)
For full scikit-learn parameter coverage see the scikit-learn logistic regression guide.
Sources
- Hastie, Qian and Tay: An Introduction to glmnet (glmnet vignette), checked 2026-11-18
- glmnet reference: cv.glmnet, checked 2026-11-18
- Friedman, Hastie and Tibshirani (2010), Regularization Paths for Generalized Linear Models via Coordinate Descent, Journal of Statistical Software 33(1), checked 2026-11-18
- scikit-learn: Linear Models (logistic regression section), checked 2026-11-18
- scikit-learn: LogisticRegression API reference, checked 2026-11-18
Data: Lasso logistic regression settings in glmnet (R) and scikit-learn (Python)
Side-by-side comparison of how the two packages express a lasso logistic regression: the argument that selects the pure L1 penalty, the penalty-strength parameter and its direction, and the algorithm or solvers. Compiled by CASRAI from the glmnet vignette, the cv.glmnet reference, the glmnet JSS paper and the scikit-learn documentation.
| Aspect | glmnet (R) | scikit-learn (Python) |
|---|---|---|
| Argument that gives the pure lasso | alpha = 1 (the default) | l1_ratio = 1 |
| Penalty-strength parameter | lambda: larger means stronger penalty | C: inverse strength, smaller means stronger penalty |
| Algorithm or solvers for the L1 penalty | Coordinate descent (Friedman, Hastie and Tibshirani 2010) | liblinear or saga (lbfgs, newton-cg, newton-cholesky and sag do not support L1) |
Sources
- glmnet vignette (accessed 2026-11-18)
- glmnet reference: cv.glmnet (accessed 2026-11-18)
- Regularization Paths for Generalized Linear Models via Coordinate Descent (accessed 2026-11-18)
- scikit-learn Linear Models (accessed 2026-11-18)
Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.







