What is Catastrophic Forgetting?

Catastrophic forgetting (also called catastrophic interference) is a phenomenon where a neural network, when trained on new data or tasks, loses previously learned knowledge. The new training overwrites the weights that encoded earlier capabilities, causing dramatic performance degradation on the original tasks.

Why It Matters

Catastrophic forgetting is a fundamental challenge for AI systems that need to learn continuously. It explains why you can't simply keep fine-tuning the same model on new data without risk — the model may lose its general capabilities. This is a core concern for continual learning, personalization, and keeping models up-to-date without full retraining.

How It Works

Why it happens:

Neural networks store knowledge distributed across weights (parameters)
When trained on new data, gradient updates modify these shared weights
If the new task's optimal weights differ from the old task's, old knowledge is overwritten
Smaller models are more susceptible (fewer parameters to share)

When it occurs:

Fine-tuning — adapting a pre-trained model to a specific task may degrade general knowledge
Sequential task learning — training on task A, then task B, then task C causes forgetting of A
Data distribution shift — updating a model with data from a different distribution

Mitigation strategies:

Regularization methods (EWC, SI) — add a penalty for changing weights that were important for previous tasks
Replay methods — mix a small amount of old training data with new data
Architecture methods — allocate separate model capacity for different tasks
LoRA / adapters — add small trainable layers while freezing the base model (the dominant approach for LLM fine-tuning)
Model merging — merge separately fine-tuned models instead of sequential training

Example

A company fine-tunes GPT-4 to be an expert legal document reviewer. After training on thousands of legal documents, the model excels at legal analysis but has forgotten how to write marketing copy, do math, or answer general knowledge questions. The legal training catastrophically overwrote the general capabilities. Using LoRA adapters instead would preserve the base model's knowledge.