Improving SpikeProp’s Training Efficiency in Spiking Neural Networks for Large Language Models Through Innovative Weight Initialization
کۆلیژا زانست

Improving SpikeProp’s Training Efficiency in Spiking Neural Networks for Large Language Models Through Innovative Weight Initialization

کۆلیژا زانست

Spiking neural networks (SNNs) use individual temporal spikes for computation and communication, simulating the actions of biological neurons. SNNs have long been disregarded because they were considered intricate and difficult to analyze. In this work, we investigate the improvement of SpikeProp, a supervised learning model specifically tailored for SNNs. Three distinct models are proposed and investigated, including the proposed model 1, the proposed model 2, and the proposed model 3, each providing unique improvements to the SpikeProp algorithm. To accelerate convergence and achieve adaptive learning rates, particle swarm optimization (PSO) and momentum factors are integrated into the proposed model 1. In the proposed model 2, a rate dependency is introduced based on angle-driven learning. By incorporating PSO and learning rates, model 3 combines the strengths of both models 1 and 2. We believe that SNNs can be trained and classified more efficiently and accurately using these models. Furthermore, we examine how large language models (LLMs) might inform the design and interpretability of neural architectures and learning methodologies while also enhancing SNN training. Through the use of LLMs, we seek to improve model transparency and encourage more Responsible AI (RAI) principles. A thorough evaluation and comparison of the proposed models with traditional methods confirms that these models consistently outperform traditional methods across various real-world datasets. Consequently, they have high potential for practical applications in neural network training in real-world settings and LLM-informed development, contributing to the advancement of AI systems.

PDF Attachments