What is Transfer Learning?
Transfer learning is a machine learning technique where a model trained on one task is reused as the starting point for a different but related task. Instead of training from scratch, the model transfers knowledge it already learned.
Why It Matters
Transfer learning is why you can fine-tune a powerful LLM on just a few hundred examples of your specific task and get excellent results. It's the economic engine of modern AI: the billions of dollars spent pre-training foundation models create reusable knowledge that millions of downstream users benefit from. Without transfer learning, every AI application would need its own massive training run.
How It Works
- Source task — a model is trained on a large, general dataset (e.g., all of Wikipedia for language, ImageNet for vision).
- Knowledge transfer — the trained weights (internal representations) capture general features that apply across tasks: language structure, visual edges, semantic concepts.
- Target task — the pre-trained model is adapted to a specific task by:
- Fine-tuning all parameters on new data
- Freezing most layers and only training a small output head
- Prompting (for LLMs) — no weight updates needed
- LoRA/PEFT — efficient parameter-efficient fine-tuning