Skip to content

npvoid/neural_heterogeneity

Repository files navigation

Code for Neural heterogeneity promotes robust learning

DOI

Perez-Nieves, N., Leung, V.C.H., Dragotti, P.L. et al. Neural heterogeneity promotes robust learning. Nat Commun 12, 5791 (2021). https://doi.org/10.1038/s41467-021-26022-3

Requirements

For surrogate gradient descent based methods:

For FORCE training based methods:

For plotting experimentally observed time constant distributions:

Running Surrogate based experiments:

In order to run an experiments you simply need to navigate to SuGD_code and run

python main.py

This will run with the default parameters, which were set for SHD dataset. You can specify them, for instance to initialise the network with heterogeneous time constants following a uniform distribution and training the time constants:

python main.py --het_ab 1 --dist_prms uniform --train_ab 1

Note that the number of inputs have to be adjusted for each dataset as the table below. A comprehensive list of parameters and a description is provided within main.py.

dataset nb_inputs
SHD & SSC 700
F-MNIST 784
N-MNIST 2312 = 34 × 34 × 2
DVS128 32768 = 128 × 128 × 2

All datasets can be downloaded from the links provided below. Running the N-MNIST and DVS dataset require an extra step to convert the data into .h5 format, This can be done by navegating to SuGD_code/NMNIST or SuGD_code/DVS and running save_data_nmnist.py or save_data_dvs.py respectively. You can specify the original dataset directory with --data_dir and where you want to save the converted dataset --output_filename arguments.

Running FORCE based experiments:

Navigate to FORCE_code. This directory contains the script LIFSONGBIRD_HET.m. This script runs FORCE in the three regimes specified in the paper and then plots the results. You may want to comment the last section which runs the gridsearch as it takes very long.

Plotting experimentally observed time constants (Allen and Manis datasets):

Go to Experiment_taudist_code. Since it is problematic to access the time constants in the Paul Manis dataset, you can first run manis_data.py to extract and save them in a text file. You may then run time_constant_distributions.py to plot the distributions of Paul Manis dataset, as well as Allen Atlas dataset. The plots will also include their corresponding Gamma and Log-normal fits.

Datasets:

The following links will direct you to the place where you can download all datasets/data: