Catastrophic Forgetting

Catastrophic forgetting is the loss of previously learned behavior when a model is trained on new data or tasks.

Updates optimized for the new distribution can overwrite parameters important to older capabilities. Mixing old data, regularizing changes, replay, or modular adaptation can help preserve prior behavior.

Where Catastrophic Forgetting comes up

Adaption’s AutoScientist: Automating the Frontier of Model Training and Alignment

Training & Fine-tuning

Catastrophic Forgetting

Catastrophic forgetting is the loss of previously learned behavior when a model is trained on new data or tasks.

Ask Melo about this ← all terms

Where Catastrophic Forgetting comes up

Adaption’s AutoScientist: Automating the Frontier of Model Training and Alignment

Catastrophic Forgetting

Related terms

Where Catastrophic Forgetting comes up

Catastrophic Forgetting

Related terms

Where Catastrophic Forgetting comes up