Introduction ¦ Curriculum Vitae ¦ Web Design ¦ Portfolio ¦ Research Interests
Back-propagation in Spiking Neural Networks 2002
Back-propagation in Spiking Neural Networks
This dissertation attempted to replicate the findings of Bohte [2002]. Bohte derives a supervised learning rule, SpikeProp, for a spiking neural network akin to traditional error-back-propagation. Bohte found that networks of spiking neurons, with biologically reasonable action potentials, could perform complex non-linear classification in fast temporal coding just as well as rate-coded networks.
After initial failure to replicate the results of Bohte, further experiments found that the network did converge on a solution to XOR in a similar number of presentations of the input to Bohte but only with the high learning rate of 1, a learning rate which was reportedly too large to converge in Bohte [2002].
Investigative experiments, into the differences between Bohte and these findings, discovered a relationship between the random initialisation values of the weights, the initial (fixed) threshold and the learning rate. Changing these values led to networks that could learn to solve XOR in a similar number of presentations of the input but interestingly only with very different learning rates.
The conclusion drawn from the experiments is that the reason given in Bohte for the small learning rates needed is possibly incorrect. Additionally it is concluded form the experiments that, during back-propagation the delta values are too large, in inverse proportion with the scaling down carried out on the initial weights in order that a specified threshold of one could be used.
A copy of this dissertation is (or will shortly be) available from the University of Bath library.