Event-driven strategies for biologically plausible learning / by Jesús Andrés Espinoza-Valverde. Wuppertal, [2025]
Inhalt
- List of Figures
- List of Tables
- Introduction
- Research Questions
- Enhancing Biological Plausibility in Learning with Spiking Neural Networks
- Bridging Biologically Inspired Learning with Continual Learning
- Thesis Structure
- Background
- Biological Neurons
- Biological Synapses
- Biological Neural Networks
- Neural Plasticity and Learning in Biological Systems
- Artificial Spiking Neurons
- Artificial Synapses
- Spiking Neural Networks
- Gradient-Based Learning Methods and Their Biological Plausibility
- Supervised Learning
- Backpropagation Through Time (BPTT)
- Real-Time Recurrent Learning (RTRL)
- Biologically Plausible Learning Rules
- Simulation Technology for Spiking Neural Networks
- Event-driven Eligibility Propagation with Additional Biologically Inspired Features
- Eligibility Propagation (e-prop)
- Time-Driven Implementation of e-prop
- Event-driven Implementation of e-prop
- The Case for Event-Driven Synaptic Updates
- Main Challenge for Event-Driven e-prop: Instantaneous Transmissions
- Pipelining
- Design and Implementation
- Cleaning of e-prop History
- Event-Driven e-prop with Additional Biological Features
- Generalized Delays in e-prop
- Continuous Dynamics
- Dynamic Firing Rate Regularization
- Classification Without Normalization
- Eligibility Trace Filter Decoupled from Output Time Constant
- Surrogate Gradients: Smoothness and Flexibility
- Full Membrane Voltage Reset
- Biologically Inspired Connectivity and Weight Constraints
- Spike-Triggered Asynchronous Weight Updates
- Design and Implementation
- Cleaning of e-prop History
- Results
- Benchmark Tasks
- Proof of Concept: Event-Driven e-prop Reproduces Time-Driven Results
- Performance of Event-Driven e-prop with Biological Features
- Discussion
- Biologically Inspired Continual Learning through Online Saliency Traces
- Biological Mechanisms of Continual Learning
- Continual Learning in Machine Learning
- Elastic Weight Consolidation (EWC)
- Synaptic Intelligence (SI)
- Biological Plausibility of EWC and SI
- Proposed Approach
- Online Saliency Traces
- Novelty Neuron and Novelty Signal
- Bayesian Online Changepoint Detection
- Saliency Trace and Novelty Neuron Integration
- Results
- Discussion
- Conclusions and Outlook
- Appendix A - Convergence of the Exponential Moving Average
- Appendix B - Simulation Parameters
- Bibliography