A compiled graph learns

#exp075 · SNNLANG · data v35.4.0
Created
Updated

Abstract

We asked whether a compiled graph can participate in gradient-based classifier training rather than only forward simulation. We trained an excitatory–inhibitory MNIST classifier through the compiled graph interface and selected a checkpoint using held-out validation loss.

Training produced coherent loss and validation trajectories, showing that optimization traversed the compiled representation. This demonstrates the learning interface on a bounded example, not competitive classification performance or broad generalization.

Results

Compiled classifier topology

Figure 1: Classifier with 128 excitatory and 32 inhibitory neurons. Input and readout projections were trainable; excitatory–inhibitory recurrence remained fixed.

Training and validation trajectories

The loss-selected checkpoint came from epoch 4 it need not be the epoch with maximum accuracy (Fig. 2).

Figure 2: (A) Training and validation cross-entropy and (B) validation accuracy over 4 epochs. Validation values average 3 fixed stochastic encodings. The dashed line marks ten-class chance accuracy.

Methods

We tested optimisation of a compiled network using a small, fixed handwritten-digit classification task.

  1. Prepare data and network. We selected 1000 images from the official MNIST training partition with a fixed subset seed, then made a stratified 900/100 training/validation split. The classifier contained 128 excitatory and 32 inhibitory neurons and a ten-class mean-voltage readout.
  2. Encode and train. Pixel intensities controlled spike probability, with a maximum input rate of 25 Hz. We simulated 100 ms per image at 0.5 ms resolution and trained for 4 epochs in batches of 64, using seed 75. AdamW[1] minimised unit-weight cross-entropy with learning rate 0.001, weight decay 0.0001 and gradient-norm clipping at one; only input and readout weights changed.
  1. Select the checkpoint. After each epoch we evaluated the validation set using 3 fixed encoder draws. Selection minimised mean validation cross-entropy, breaking ties by mean accuracy and then the earliest epoch. The official test partition did not participate in selection; both selected and final states were retained.
  1. Display retained learning evidence. We displayed the recorded learning curves and selected-versus-final checkpoint outputs with the validation selection rule stated above.

References

  1. Ilya Loshchilov and Frank Hutter: Decoupled Weight Decay Regularization. ICLR, 2019. doi:10.48550/arXiv.1711.05101