Unsupervised learning

Unsupervised learning is a way of training AI on data that has no answers attached. The model gets, for example, a list of customers and their purchases, and looks for structure by itself: groups of similar customers, unusual orders or products that are often bought together. It does not know what the groups mean; a person has to name them and decide what to do with them.

How does it differ from supervised learning?

In supervised learning, every example comes with the right answer, such as an email marked as spam, and the model learns to predict that answer. Unsupervised learning gets only the examples. The textbook Deep Learning by Goodfellow, Bengio and Courville explains the name: in supervised learning an instructor or teacher shows the system what to do, and in unsupervised learning there is none, so the algorithm has to make sense of the data on its own. Google's machine learning course puts the practical difference this way: with clustering, unlike classification, you do not define the categories yourself.

What can it do?

• Clustering groups similar examples. K-means splits the data into a number of groups that you choose, each around its average; DBSCAN finds dense areas of points, in any shape, without a fixed number of groups.
• Dimensionality reduction compresses many columns into a few. PCA finds the directions that explain most of the variation in the data; t-SNE draws high-dimensional data as a two-dimensional map.
• Anomaly detection finds data points that do not fit, which can point to faulty equipment, human error or a security breach (IBM).
• Association rules find products that are often bought together, known as market basket analysis.
• Generative models, such as a GAN, learn what the data looks like as a whole. According to the Deep Learning textbook, that is usually the aim of unsupervised learning in deep learning: learning the probability distribution the data came from.

Where does self-supervised learning fit?

Language models blur the line. Their pre-training uses unlabelled text, and the GPT-2 paper (2019) describes language modelling as "unsupervised distribution estimation". Yet each training step is a supervised prediction of the next token, with the text itself as the answer. The Deep Learning textbook notes that the two terms are not formally defined and that this trick splits an unsupervised problem into a series of supervised ones. Hugging Face calls this self-supervised learning: the training objective is computed from the input itself, so nobody has to label the data.

Where does it go wrong?

• There is no answer key. According to scikit-learn, evaluating a clustering is not as simple as counting errors, and IBM lists a higher risk of inaccurate results and the need for people to validate the output.
• The method shapes the result. K-means needs the number of groups in advance and assumes compact, evenly spread clusters, so it handles elongated or irregular shapes poorly.
• Maps can mislead. In a t-SNE plot, cluster sizes and the distances between clusters may mean nothing (Wattenberg et al., 2016).

Example

Suppose a webshop describes each customer with two numbers: orders per year and average order value. This snippet runs k-means from scikit-learn on twelve made-up customers:

1
from sklearn.cluster import KMeans
2
from sklearn.pipeline import make_pipeline
3
from sklearn.preprocessing import StandardScaler
4
 
5
# orders per year, average order value in euros (made-up customers)
6
customers = [[24, 15], [20, 18], [26, 12], [22, 20],
7
             [2, 180], [3, 150], [1, 210], [2, 170],
8
             [8, 60], [10, 55], [7, 70], [9, 65]]
9
model = make_pipeline(StandardScaler(), KMeans(n_clusters=3, n_init=10, random_state=0))
10
print(model.fit_predict(customers))  # [1 1 1 1 0 0 0 0 2 2 2 2]

The scaler puts both columns on the same scale first, because k-means compares plain distances. The numbers 0, 1 and 2 are only labels. A person has to see that group 1 orders often and spends little per order, and give it a name such as "regulars", the naming step Google's course describes.