Machine Learning ยท medium
How do you choose the number of clusters in k-means?
Asked in Data Scientist interviews, in the Machine Learning round.
Short answer
Elbow method, silhouette score, gap statistic, or domain knowledge.
How to answer it
Definition
There is no purely statistical answer, because k-means has no likelihood to optimize over k. The metrics narrow the range; the decision is made with domain knowledge and by what the clusters will be used for.
The methods
Elbow plot inertia (within-cluster sum of squares) against k
look for the bend where extra clusters stop paying
inertia always falls, so there is no minimum to find
Silhouette per point: how much closer to its own cluster than to the
nearest other one, from -1 to 1
has a genuine maximum, so it is less subjective than the elbow
Gap statistic compares inertia against a random reference distribution
more principled and more expensive
Stability cluster on resamples; a k whose clusters keep changing is
not a real structureExample
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
for k in range(2, 11):
labels = KMeans(n_clusters=k, n_init=10, random_state=0).fit_predict(X)
print(k, round(silhouette_score(X, labels), 3))Output
2 0.412
3 0.455 <- highest
4 0.398
5 0.361The part that matters more
For a customer segmentation, the honest answer is often that k is chosen by the business: how many segments can marketing actually run distinct campaigns for? Four actionable segments beat nine statistically cleaner ones that nobody can operationalise. Saying this is usually what the interviewer is listening for.
Key points
- The elbow has no maximum and is read by eye. Silhouette at least has an optimum.
- Scale the features first. K-means uses Euclidean distance, so an unscaled column dominates.
- K-means assumes roughly spherical, similar-sized clusters. If the shape is wrong, no k is right; use DBSCAN or a Gaussian mixture.
- Check stability across seeds and resamples. Unstable clusters are noise.
- The final call includes what the clusters are for.
FAQ
- Why does inertia always fall? More centroids always fit tighter. That is why the elbow is a bend rather than a minimum.
- What if the elbow and silhouette disagree? Look at the clusters themselves. Interpretability breaks the tie.
- Is k-means even right here? If clusters are elongated or of very different densities, no. Say so rather than tuning k forever.
The 30-second answer
I would run k over a range and look at two things: the elbow on inertia, where extra clusters stop paying for themselves, and the silhouette score, which I prefer because it has an actual maximum rather than a bend you read by eye. If I have time, the gap statistic is more principled, and I would check stability across seeds, because clusters that move every run are not real structure.
Before any of that I would scale the features, since k-means uses Euclidean distance and an unscaled column will dominate.
But for something like a customer segmentation the real constraint is usually the business: how many segments marketing can actually run different campaigns for. Four segments people can act on beat nine that score slightly better and nobody can use.
In the interview
- What they are checking: that you name a statistic and then override it with the use case.
- Common mistake: reading a k off the elbow plot as if it were a measurement.