Transfer learning

Transfer learning means reusing what an AI model has learned on one task as the starting point for another. A model that has already learned from over a million photos or a huge amount of text needs only a small set of your own examples to learn a related job, such as sorting your customer emails. The large AI models behind chatbots are built on this idea.

How does it work?

Researchers at Stanford describe the idea as taking the knowledge learned on one task, such as recognising objects in images, and applying it to another, such as recognising activities in videos (Bommasani et al., 2021). In deep learning this usually means pre-training: a model first learns a general task on a large dataset and is then adapted to the task you care about. In their words, transfer learning makes foundation models possible, and scale makes them strong.

Stanford's CS231n course notes describe the classic image case. A network pre-trained on ImageNet, 1.2 million images in 1,000 categories, is used in one of two ways. As a fixed feature extractor, it stays frozen and only a new final classifier is trained on your data. With fine-tuning, training continues through the network, usually with a smaller learning rate so the pre-trained weights do not change too much too quickly. Language models add two options: LoRA, which trains a few small added matrices, and in-context learning, which adapts the model with a prompt and no training at all.

Formally, transfer learning uses what was learned in a source domain and task to learn a target task where the domain or the task differs (Ruder, 2017, after Pan and Yang, 2010). Documents in another language are a case of cross-lingual adaptation; documents on other topics are a case of domain adaptation.

Why does it work?

In networks trained on photos, the first layers learn general features, such as detectors for edges and patches of colour, that are useful for many tasks, while the last layers become specific to the original task. Yosinski et al. (2014) measured this layer by layer. Features transferred less well as the two tasks grew further apart, but even features from a distant task beat starting from random weights.

How much data does it save?

Howard and Ruder (2018) fine-tuned a pre-trained language model to classify IMDb film reviews. With only 100 labelled examples, it matched a model trained from scratch on 10 times as many; with 50,000 unlabelled reviews added, it matched one trained on 100 times as many. CS231n gives a rule of thumb: with a small dataset that resembles the original one, train only a classifier on top, and with a large dataset, fine-tune the whole network.

When does it fail?

When the source is too far from the target, transfer can make the result worse than learning without it, which researchers call negative transfer (Wang et al., 2018). Fine-tuning can also erode what the model could do before (catastrophic forgetting).

Example

Suppose a webshop wants to sort incoming customer emails into returns, delivery questions and complaints. It has far too little text to train a language model from scratch. It takes a pre-trained model, labels a few hundred of its own emails and trains a small classifier on top, or fine-tunes the model. The pre-trained model already knows how language works; the shop's examples only have to teach it the three categories.