8.4 Random Forests

A random forest is an ensemble of many decision trees that work together. Each tree is trained on a random sample of the data and a random subset of features, then their predictions are combined by majority vote (classification) or averaging (regression). This reduces the overfitting that single trees suffer from.

Think of asking many doctors instead of one before a diagnosis. Each may make a small mistake, but the consensus is usually more reliable than any single opinion. A random forest applies that same 'wisdom of the crowd' idea to trees.