Learn / Unsupervised
Clustering
Grouping points that belong together, without labels. Centroids, densities, mixtures and hierarchies.
On this page
Intuition
Clustering sorts points into groups so that members of a group are more alike than members of different groups. The algorithms differ in what “a group” means: a blob around a centre (k-means), a dense region (DBSCAN), a bell-shaped cloud (Gaussian mixture), or a branch of a tree (agglomerative).
What the data looks like
Each row is one thing you want to group. There is no answer column; the algorithm adds one.
| rowatm | inputwithdrawals_day | inputpeak_hour | inputweekend_share | inputcash_outs_month | outputcluster |
|---|---|---|---|---|---|
| ATM-17 | 310 | 18 | 0.22 | 4 | A |
| ATM-23 | 95 | 12 | 0.48 | 0 | B |
| ATM-31 | 288 | 17 | 0.25 | 3 | A |
| ATM-40 | 120 | 13 | 0.51 | 1 | B |
The cluster column is not in the data you start with. The algorithm writes it, and you name the groups afterwards ("busy evening ATMs", "quiet weekend ATMs").
Example rows are invented for illustration.
How the data is prepared
- Standardise every feature; otherwise the one measured in the largest units decides the clusters.
- Choose the distance: Euclidean for numeric features, cosine for embeddings.
- Handle outliers: k-means will happily spend a centroid on three extreme customers.
- Reduce dimensions first when there are many correlated features; distances lose meaning in high dimensions.
Algorithms
k-means
Alternate between assigning points to the nearest centroid and moving centroids to the mean.
- Cost
- O(iters·n·k)
- Strength
- Fast, simple, scales well.
- Weakness
- You must pick k; assumes round, equal-sized clusters.
- Used for
- customer segmentation, ATM grouping
E-means
Start with too many centroids and let evolution merge, delete and nudge them until the description length stops improving.
- Cost
- O(gens·n·k)
- Strength
- Finds the number of clusters on its own.
- Weakness
- Still assumes round clusters; slower than k-means.
- Used for
- segments when k is unknown, ATM grouping
Mini-batch k-means
k-means on small random batches.
- Cost
- O(iters·batch·k)
- Strength
- Handles very large or streaming data.
- Weakness
- Slightly worse clusters than full k-means.
- Used for
- segmenting millions of customers
DBSCAN
Grow clusters from dense regions; sparse points are noise.
- Cost
- O(n log n) with an index
- Strength
- Finds arbitrary shapes and outliers; no k.
- Weakness
- Struggles when densities differ; ε is sensitive.
- Used for
- anomaly pockets, geo clustering
Gaussian mixture (EM)
Model the data as a blend of Gaussians and fit them with expectation-maximisation.
- Cost
- O(iters·n·k)
- Strength
- Soft assignments; elliptical clusters.
- Weakness
- Can converge to poor local optima.
- Used for
- overlapping segments, density estimation
Agglomerative
Start with every point alone and repeatedly merge the two closest clusters.
- Cost
- O(n²) memory
- Strength
- Gives a full hierarchy (dendrogram); no k up front.
- Weakness
- Does not scale to large n.
- Used for
- document topics, taxonomy building
The maths
k-means minimises the within-cluster sum of squares (inertia):
by alternating two steps that can only lower : assign each point to its nearest centroid, then move each centroid to the mean of its points. The result depends on where the centroids start, which is why k-means++ spreads the initial centroids out.
E-means removes the need to choose . It treats the set of centroids as an organism that evolves: each generation tries merging the closest pair, deleting the smallest cluster and nudging a centroid, and keeps whichever change lowers a description-length fitness,
where is the likelihood of the data under one spherical Gaussian per cluster and the number of parameters. More clusters always fit better, but each one costs per parameter, so the search settles where an extra cluster stops paying for itself.
A Gaussian mixture softens this. Each point gets a responsibility for every component (E-step), and each component’s mean and covariance are re-estimated from the points weighted by (M-step).
The silhouette of a point compares its mean distance to its own cluster, , with the mean distance to the nearest other cluster, :
Real-world examples
- Customer segmentation: groups by product usage and transaction behaviour, for targeted offers.
- ATM grouping: clustering machines by usage pattern to plan cash loading and maintenance.
- Anomaly pockets: DBSCAN’s “noise” points are candidates for review.
- Document topics: clustering embeddings of support tickets.
Playground
Three models on the same blobs. k-means is told k = 3. E-means starts with ten centroids and has to discover the number of clusters itself: watch the loss chart step down each time it merges two clusters or deletes one. DBSCAN needs no k either but counts dense regions. Raise the blob count to six, or switch the data to moons, and press Train again. Use Add model for a fourth.
Click the plot to add a point.
Press Train to compare.
Press Train to compare.
Train a model to see its loss curve.
| Step - | train ━ | test ┅ |
|---|
colour = cluster · ✕ = noise · ● train ○ test
Press Train. The verdict appears here.
In Python
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import StandardScaler
Xs = StandardScaler().fit_transform(X)
for k in range(2, 9): # there is no "right" k: look at the curve, then at the segments themselves
km = KMeans(n_clusters=k, n_init=10, random_state=42).fit(Xs)
print(k, round(km.inertia_), round(silhouette_score(Xs, km.labels_), 3))
Pitfalls
- k-means on non-round clusters. It cuts moons and rings in half, as the playground shows.
- Trusting one run. Different seeds can give different clusters; check stability.
- Picking k from a metric alone. The elbow is a hint; the test is whether the business can use the segments.
- DBSCAN with varying density. One ε cannot fit both a tight and a loose cluster.