Written and maintained by CASRAI Editorial Board
Last updated
XGBoost is an open-source, supervised machine-learning library that builds gradient boosted trees: an ensemble of decision trees (CARTs) added one at a time, each correcting the previous ones. Its official tutorial describes it as the same family of tree ensembles as a random forest, differing in how the trees are trained: “random forests and boosted trees are really the same models; the difference arises from how we train them.” This guide covers what it does for a research project, how to install it from the official documentation, a runnable train-predict-save sequence, how to read the objective that the documented parameters control, and how to report it. Details were checked against the official documentation on 2026-10-29.
What is XGBoost used for in research?
XGBoost fits supervised models: you give it a table of input features and a label, and it learns a function that predicts the label. A researcher reaches for it when a regression or classification model on tabular data is needed and a single linear model is not enough. For the broader picture of how such models fit into a study, see what machine learning is and where it goes wrong in research.
The library was introduced by Tianqi Chen and Carlos Guestrin in “XGBoost: A Scalable Tree Boosting System” (arXiv:1603.02754, published at KDD ’16). The abstract names a sparsity-aware algorithm, a weighted quantile sketch, cache-access patterns, data compression and sharding as the system features behind its scalability.
How do you install XGBoost from the official source?
These commands come from the install page of the official XGBoost documentation (xgboost.readthedocs.io, stable, checked 2026-10-29). Use only one route per environment.
| Environment | Command or artifact from the docs |
|---|---|
| Python (pip) | pip install xgboost |
| Python, smaller CPU-only build | pip install xgboost-cpu |
| Python (conda) | conda install -c conda-forge py-xgboost (the docs list py-xgboost=*=cpu* and py-xgboost=*=cuda* variants) |
| R | install.packages('xgboost', repos = c('https://dmlc.r-universe.dev', 'https://cloud.r-project.org')) (R-universe is the recommended source; the docs mark the CRAN release as outdated) |
| JVM / Spark | Artifact xgboost4j-spark_2.12, group ml.dmlc |
Platform notes stated in the same documentation: on Windows, XGBoost needs DLLs from the Visual C++ Redistributable (not needed if Visual Studio is installed); GPU training (device=cuda:0) needs an NVIDIA GPU; macOS wheels have no GPU support; multi-GPU training is Linux only; on macOS, the R package needs brew install libomp, because without OpenMP XGBoost uses a single CPU core.
If you manage the project in a conda environment, sharing a reproducible conda environment shows how to record the exact versions.
Step by step: train, predict and save a model in Python
This is the sequence from the official Python introduction (checked 2026-10-29).
- Import the library:
import xgboost as xgb. - Wrap your NumPy array or pandas DataFrame in XGBoost’s data container:
dtrain = xgb.DMatrix(data, label=label). Builddtestthe same way from held-out rows. - Set the parameters in a dictionary (
param). The next section lists the documented defaults you are choosing between. - Train:
bst = xgb.train(param, dtrain, num_round, evallist), wherenum_roundis the number of boosting rounds andevallistnames the datasets whose metric is printed each round. - Predict:
ypred = bst.predict(dtest). - Save:
bst.save_model('model.ubj').
XGBoost also has a scikit-learn-style interface. The documentation example is reg = xgb.XGBRegressor(tree_method="hist", device="cuda"), followed by reg.fit(X, y) and reg.save_model("regressor.json"); leave out device to train on the CPU. If your analysis already runs through scikit-learn pipelines, the scikit-learn tutorial covers fitting, evaluating and citing in that ecosystem.
Output and how to read it
The prediction from bst.predict is the model’s output for each row of the test matrix, and the per-round lines from evallist are the evaluation metric. The documentation states that the default eval_metric depends on the objective: RMSE for regression and logloss for classification (other objectives, such as ranking, have their own defaults); the default objective is reg:squarederror.
What the objective and parameters mean
The official model tutorial writes the objective as obj(θ) = L(θ) + Ω(θ): a training loss plus a regularization term. It adds that “the regularization term is what people usually forget to add.” Training is additive: each round adds one tree, ft(x), to the previous prediction, using a second-order (gradient and Hessian) approximation so one solver serves many loss functions. The regularization is Ω(f) = γT + ½λΣwj², where T is the number of leaves and wj the leaf scores. A split is not added if its gain is below γ, which is how the tree is pruned.
| Parameter | Documented default | What the docs say it does |
|---|---|---|
eta (learning_rate) |
0.3 | Step size shrinkage used in update to prevent overfitting |
max_depth |
6 | Maximum depth of a tree |
lambda |
1 | L2 regularization term on weights |
alpha |
0 | L1 regularization term on weights |
gamma |
0 | Minimum loss reduction required to make a further partition on a leaf node of the tree |
subsample |
1 | Subsample ratio of the training instances |
objective |
reg:squarederror |
Regression with squared loss |
Source: XGBoost Parameters page, checked 2026-10-29.
How should you report it in a paper?
Cite the method as Chen, T. and Guestrin, C. (2016), “XGBoost: A Scalable Tree Boosting System,” KDD ’16 (DOI 10.1145/2939672.2939785; arXiv DOI 10.48550/arXiv.1603.02754). State the library version you installed and every parameter you set; for any parameter you did not set, the documented default applies, and listing those values (for example eta 0.3 and max_depth 6) makes the run reproducible. Keep tuning decisions separate from the rows you evaluate on, since information from the test set leaking into training is the failure described under data leakage, and see computational reproducibility for what a rerun needs. For citing the software itself, how to cite software, code and R packages gives the patterns.
Troubleshooting installation and runs
- Import fails on Windows: install the Visual C++ Redistributable, which provides DLLs XGBoost requires (not needed if Visual Studio is installed).
- GPU not used on a Mac: macOS wheels have no GPU support;
device=cuda:0needs an NVIDIA GPU. - Training uses one core in R on macOS: run
brew install libomp; without OpenMP XGBoost uses a single CPU core. - Old R version from CRAN: the docs mark the CRAN package as outdated; install from R-universe with the command above.
- Spark on Windows: XGBoost4J-Spark does not support the Windows platform.
- Multi-GPU training: supported only on Linux.
Free alternatives to XGBoost
The scikit-learn user guide (version 1.9.1, checked 2026-10-29) describes two gradient-boosted tree families that need no separate library.
| Option | What the scikit-learn documentation says |
|---|---|
GradientBoostingClassifier / GradientBoostingRegressor |
Gradient Tree Boosting (GBDT) generalizes boosting to arbitrary differentiable loss functions; “an excellent model for both regression and classification, in particular for tabular data.” The page says these may be preferred for small sample sizes, since binning can make split points too approximate. |
HistGradientBoostingClassifier / HistGradientBoostingRegressor |
Introduced in scikit-learn 0.21, inspired by LightGBM; can be orders of magnitude faster than the classes above when there are more than tens of thousands of samples; missing values are supported natively, with no imputer needed. |
FAQ
Is XGBoost the same as a random forest?
No, but the models are closely related. The official tutorial says random forests and boosted trees are really the same kind of model; the difference arises from how they are trained. XGBoost trains its trees additively, each new tree added to the running prediction.
What does the regularization term do?
It penalizes model complexity: γ per leaf, and an L2 penalty λ on leaf scores. A split whose gain is below γ is not added. The docs’ default gamma is 0 and lambda is 1.
Can XGBoost run on a GPU?
Yes, with device=cuda:0 on a machine with an NVIDIA GPU. macOS wheels have no GPU support, and multi-GPU training is Linux only.
How do I install XGBoost for R?
The documentation recommends R-universe: install.packages('xgboost', repos = c('https://dmlc.r-universe.dev', 'https://cloud.r-project.org')).
Sources
- XGBoost documentation, Installation Guide: https://xgboost.readthedocs.io/en/stable/install.html (checked 2026-10-29)
- XGBoost documentation, Python Package Introduction: https://xgboost.readthedocs.io/en/stable/python/python_intro.html (checked 2026-10-29)
- XGBoost documentation, Introduction to Boosted Trees: https://xgboost.readthedocs.io/en/stable/tutorials/model.html (checked 2026-10-29)
- XGBoost documentation, XGBoost Parameters: https://xgboost.readthedocs.io/en/stable/parameter.html (checked 2026-10-29)
- Chen, T. and Guestrin, C., XGBoost: A Scalable Tree Boosting System, arXiv:1603.02754, https://arxiv.org/abs/1603.02754 (checked 2026-10-29)
- scikit-learn user guide, 1.11 Ensembles: https://scikit-learn.org/stable/modules/ensemble.html (version 1.9.1, checked 2026-10-29)
