Supervised learning

Supervised learning is a way to teach an AI model with examples that come with the right answer. Show it thousands of emails marked spam or not spam, or past orders marked returned or kept, and it learns to give that answer for new cases itself. Google compares it to a student who prepares for an exam by studying old exams together with their answers.

How does it work?

In Google's introduction to machine learning, each training example has features, the input, and a label, the answer the model should predict. The learning is "supervised" because a person supplies data with the known correct results. During training the model makes a prediction for an example, compares it with the label and adjusts itself to shrink the difference, the loss, usually with gradient descent. It is then tested on labelled examples it has not seen, and after that it makes predictions for new, unlabelled data (inference).

Two of the most common uses are classification, which predicts a category such as spam or not spam, and regression, which predicts a number such as a sale price (regression and classification). The Python library scikit-learn lists the usual algorithms, from linear models and decision trees to neural networks.

Do language models use it?

Yes, after pre-training. That first phase learns from unlabelled text (self-supervised learning). For InstructGPT, OpenAI then had people write the desired answers to prompts and fine-tuned GPT-3 on them with supervised learning, before the RLHF step (Ouyang et al., 2022). This step is known as supervised fine-tuning.

Where does it go wrong?

• Labels cost time and money. IBM notes that labelled datasets need time-consuming annotation by human experts.
• Labels contain mistakes. Northcutt et al. (2021) estimated that on average at least 3.3% of the labels in the test sets of ten widely used datasets are wrong, including at least 6% of the ImageNet validation set.
• The examples must cover the cases the model will meet. Google's example: a hundred years of weather data from July alone gives poor rainfall predictions for January.
• A model copies the patterns in its labels, including human bias. Google advises checking the training data and the predictions for bias before a model goes live.

Example

Esteva et al. (Nature, January 2017) trained a neural network on 129,450 clinical photos covering 2,032 skin diseases, using only the pixels and the disease labels as input. On photos confirmed by biopsy, they compared it with 21 board-certified dermatologists on two yes-or-no questions, one of them melanoma versus a harmless mole. The network performed on par with all of the dermatologists on both. It could only do so because every training photo came with a diagnosis.