What is a random forest?

An ensemble of trees

A random forest is an ensemble: it builds many decision trees and aggregates their outputs rather than relying on one. For classification it takes a majority vote across the trees; for regression it averages their predictions. The guiding idea is that many diverse, individually imperfect models, combined, are more accurate and more stable than any single one — provided the models make different kinds of error, so that their mistakes tend to cancel out rather than reinforce.

Bagging and feature randomness

Two sources of randomness make the trees diverse. The first is bagging (bootstrap aggregating): each tree is trained on a random sample of the data drawn with replacement, so every tree sees a slightly different dataset.

The second is feature randomness: at each split, a tree considers only a random subset of features rather than all of them. Together these decorrelate the trees, which is what allows averaging to cut variance and reduce overfitting.

Strengths and trade-offs

Random forests are accurate, robust, and require relatively little tuning, which makes them a strong default for many tabular-data problems. They resist overfitting better than single trees and can rank the importance of features. The trade-offs are reduced interpretability — a forest of hundreds of trees cannot be read like one tree — and greater computational cost. Random forests, introduced by Leo Breiman in 2001, remain one of the most widely used and dependable supervised learning methods.

Random forests in research

In research, random forests are a popular, dependable baseline for prediction on structured data, often performing well with little tuning. Their feature-importance estimates can suggest which variables matter, though these must be interpreted cautiously, as they can be biased by correlated or high-cardinality features. As with any model, performance should be assessed on held-out data via cross-validation, and the random seed and settings reported for reproducibility.