A parallel algorithm for simulation of large neural networks

The simulation of biologically realistic neural networks requires the numerical solution of very large systems of differential equations. Variables within the system can be changing at rates that vary by orders of magnitude, not only at different times of the solution, but at the same time in different parts of the network. Therefore, an efficient implementation must be able to vary the solution step size, and do so independently in different subsystems. A single processor algorithm is presented in which each neuron can be solved with its own step size by using a priority queue to integrate them in the correct order. But this leaves the problem of how communication and synchronisation between neurons should be managed when executing in parallel. The proposed solution uses an algorithm based on waveform relaxation, which allows groups of neurons on different processors to be solved independently and hence in parallel, for substantial parts of the computation. Realistic test problems were run on a distributed memory parallel computer and results show that speedups of 10 using 16 processors are achievable, and indicate that further speedups may be possible.