
What is Transfer Learning?
Transfer learning is a machine learning technique where a model trained on one task is reused as the starting point for a different but related task. Instead of training from scratch, the model transfers knowledge it already learned.
Why It Matters
Transfer learning is why you can fine-tune a powerful LLM on just a few hundred examples of your specific task and get excellent results. It's the economic engine of modern AI: the billions of dollars spent pre-training foundation models create reusable knowledge that millions of downstream users benefit from. Without transfer learning, every AI application would need its own massive training run.
How It Works
- Source task β a model is trained on a large, general dataset (e.g., all of Wikipedia for language, ImageNet for vision).
- Knowledge transfer β the trained weights (internal representations) capture general features that apply across tasks: language structure, visual edges, semantic concepts.
- Target task β the pre-trained model is adapted to a specific task by:
- Fine-tuning all parameters on new data
- Freezing most layers and only training a small output head
- Prompting (for LLMs) β no weight updates needed
- LoRA/PEFT β efficient parameter-efficient fine-tuning