AI glossary · Basic · Data & evaluation
KNN
KNN (k-Nearest Neighbors) is a method that, when new data comes in, finds the k most similar items among the data it already knows and makes a decision based on those neighbors' answers. It is one of the simplest machine learning methods: it has no separate complex training process and relies on the idea that 'similar things have similar answers'.
About this term
- Level
- Basic · Technical terms you often meet in AI news
- Category
- Foundations & applications
- Fields
- Data & evaluation
- Added
- 2026-10-08
How it works
In the training phase, KNN simply stores the data as is and does not build a separate model. When new data comes in, it calculates the distance (how different they are) to every stored data point. It then picks the k closest, and decides the answer by majority vote among the neighbors for classification problems, or by averaging the neighbors' values for numerical prediction problems. Distance is usually measured by converting data into lists of numbers and taking the difference between them.
Why it matters
Because its principle is intuitive, KNN is a good way to learn the concepts of 'similarity' and 'distance' when starting out in machine learning. The idea of 'finding what is close' carries directly into today's embedding-based search: converting documents or images into embeddings and finding the ones closest to a query is exactly a nearest neighbor search. The same idea underlies RAG and recommendation systems.
Things to know
If k is too small, results are swayed by noise; if it is too large, different groups get mixed together and the decision becomes blurry. Because it has to compute the distance to all data points every time, it becomes very slow as the data grows. That is why large-scale search uses ANN search, which gives up a little accuracy for a large gain in speed. Also, if numbers are on different scales, distances become distorted, so preprocessing that aligns the ranges of values is important.
Example
Consider a simple recommendation feature in an online store. A customer's purchase history is represented as numbers, the k customers with the most similar histories are found, and the products they bought most are recommended. The top-k search used in a vector DB to 'find the k documents most similar to the query' is also a term that comes from the same KNN concept.