8.5 K-Nearest Neighbors

K-Nearest Neighbors (KNN) predicts the label of a new data point by looking at the K examples closest to it in the feature space. For classification it takes the majority label among those neighbors; for regression it averages their values. 'Closeness' is measured by a distance, often Euclidean distance.

Picture a new shop in a neighborhood. To guess what it sells, you glance at the nearest few shops, if most are cafes, it is probably a cafe too. KNN reasons in exactly this 'judge by your neighbors' way.