Hybrid method for continual learning in convolutional neural networks / Basile Tousside, M.Sc. (TUM). Wuppertal, [2025]
Inhalt
- Introduction
- Theoretical Foundations and Related Work
- Notation and Formalism
- Fundamentals of Supervised Learning in Neural Networks
- Introduction to Catastrophic Forgetting in Supervised Learning
- Introduction to Continual Learning for Supervised Learning
- Definition and Challenge of CL
- History and Motivation of CL
- CL Desiderata
- Delimitation to Other ML Fields
- Foundational Principles and Overview of Regularization Based CL Approaches
- Foundational Principles and Methods of Architecture Based CL Approaches
- Interplay of Regularization- and Architectured-based CL Methods
- Overview of Neural Network Model Compression
- Datasets and Metrics
- Chapter Summary
- A New Regularization-based CL Method for Static Neural Networks
- Problem Statement
- Method: Group and Exclusive Sparse Regularization-based CL (GESCL)
- Preliminaries
- From Weights to Filter/Neuron Importance
- Estimating Filter/Neuron Importance
- Mitigating Forgetting in Sequential Task Learning
- Addressing Plasticity in Sequential Task Learning
- Combining Stability and Plasticity to Enhance CL Efficiency
- Solving the Resulting Optimization Problem
- Experimental Evaluation
- Ablation Study and Analysis
- Chapter Summary
- A New Hybrid Regularization- and Expansion-based CL Method
- Problem Statement
- Method: Sparsification and Expansion-based Continual Learning (SECL)
- Preliminaries
- SECL: Preventing Forgetting
- SECL: Sparsification/Plasticity
- SECL: Network Expansion
- Combining Stability, Plasticity and Expansion to Enhance CL Efficiency
- Solving the Resulting Optimization Problem
- Experimental Evaluation
- Baselines
- Datasets
- Training Details
- Overcoming Catastrophic Forgetting
- Overall Classification Accuracy
- Assessing Forward Transfer
- Ablation Study and Analysis
- In-depth Analysis of Network Expansion Effects
- Ablation Study
- Comparative Analysis of Used Capacity
- Training and Inference Time
- Limitations
- Chapter Summary
- Conclusions, Implications and Future Directions