By: Lukasz Kusmierz, RIKEN Center for Brain Science
We propose an analytically tractable neural connectivity model with power-law distributed synaptic strengths. When threshold neurons with biologically plausible number of incoming connections are considered, our model features a continuous transition to chaos and can reproduce biologically relevant low activity levels and scale-free avalanches. In contrast, the Gaussian counterpart exhibits a discontinuous transition to chaos and thus cannot be poised near the edge of chaos. We validate our predictions in simulations of networks of binary as well as leaky integrate-and-fire neurons. Our results suggest that heavy-tailed synaptic distribution may form a weakly informative sparse-connectivity prior that can be useful in biological and artificial adaptive systems.
Experimental studies report heavy tails of the distributions of synaptic strengths. However, theoretical network models commonly rely on the assumption that the total synaptic input current of each neuron can be approximated by the normal distribution. With such an assumption, synaptic heavy tails are effectively ignored. Although some numerical studies explore the effects of the log-normal synaptic efficacy distribution, the lack of a fundamental mathematical theory limits our understanding of the functional roles of such synaptic distributions. We propose a novel, analytically tractable connectivity model with power-law distributed synaptic strengths. For simplicity of the analysis we study a simple discrete-time dynamics of threshold units.
Our model, in contrast to the Gaussian counterpart, features a continuous transition between quiescent and active states. Because the transition in the Gaussian network is discontinuous, it cannot reproduce biologically relevant low activity levels. Numerical simulations confirm our theoretical predictions.
The active state of the Cauchy network is chaotic. Thus, the critical point corresponds to an edge of chaos. Moreover, at the critical point we observe scale-free avalanches, i.e. bursts of activity with power-law distributions of sizes and lifetimes.
We tested both networks on various reservoir computing tasks. Here we show results of the delayed XOR task, in which two bits are presented to the network at times t=0 and t=T. The output (readout) of network at time t=2T should corresponds to the exclusive or of these two bits. For longer delays Cauchy networks perform better than Gaussian networks because they can utilize slow time scales generated around the edge of chaos.
Inspiring posters! Thanks and congrats 😉