What is Transfer Learning?

What is Transfer Learning?

Transfer learning is a machine learning technique where a model trained on one task is reused as the starting point for a different but related task. Instead of training from scratch, the model transfers knowledge it already learned.

Why It Matters

Transfer learning is why you can fine-tune a powerful LLM on just a few hundred examples of your specific task and get excellent results. It's the economic engine of modern AI: the billions of dollars spent pre-training foundation models create reusable knowledge that millions of downstream users benefit from. Without transfer learning, every AI application would need its own massive training run.

How It Works

1. Source task — a model is trained on a large, general dataset (e.g., all of Wikipedia for language, ImageNet for vision).
2. Knowledge transfer — the trained weights (internal representations) capture general features that apply across tasks: language structure, visual edges, semantic concepts.
3. Target task — the pre-trained model is adapted to a specific task by:

• Fine-tuning all parameters on new data
• Freezing most layers and only training a small output head
• Prompting (for LLMs) — no weight updates needed
• LoRA/PEFT — efficient parameter-efficient fine-tuning

Transfer learning works because early model layers learn generic features (edges, grammar rules) while later layers learn task-specific features. The generic layers transfer well across tasks.

Example

A hospital fine-tunes a pre-trained language model on 500 medical consultation transcripts to build a clinical note summarizer. The model already understands language, grammar, and medical terminology from pre-training — it only needs to learn the specific summarization format.