A bundle checkpoint replays

#exp076 · SNNLANG · data v35.4.0
Created
Updated

Abstract

We asked whether a trained graph bundle can be checkpointed and replayed through compiled and explicit descriptions of the same network. We compared loaded parameters, forward outputs, loss, gradients and an optimizer update under a deterministic equivalence gate.

The replay routes agreed exactly; differing classification scores came from different datasets and encoding aggregation rather than checkpoint corruption. This supports checkpoint compatibility for this network family, not equivalence between arbitrary graph implementations.

Results

Compiled replay protocol

Figure 1: Protocol schematic for compilation, training, checkpoint reload and a separate one-step equivalence test; the arrows describe operations, not measurements.

Training-validation trajectories

The validation-selected checkpoint came from epoch 2 (Fig. 2).

Figure 2: (A) Training and validation cross-entropy and (B) validation accuracy across 2 epochs. Validation averages 3 encoder draws on 16 images.

Checkpoint evaluations

The selected bundle checkpoint loaded through the explicit route achieved 33.75%. A separately trained explicit-network checkpoint loaded through the bundle route achieved 16.25% (Fig. 2).

CheckpointValidationOfficial-test replay
Selected31.25%33.75%
Final31.25%33.75%
Table 2: Validation used 16 images and multiple encoder draws; replay used 160 official-test images and one fixed encoding.

One-step route equivalence

ComparisonResult
Initial parametersexact
Forward logitsexact
Cross-entropyexact
Gradientsexact
AdamW updateexact
Table 3: Separately executed deterministic test with identical encoded spikes, labels and seeded initialisation. Relative and absolute tolerances were both zero. This gate does not compare accuracies obtained on different datasets.

Methods

We tested checkpoint interchange and numerical equivalence as separate properties of the same supported classifier family.

  1. Train and select states. We split 160 MNIST training images into 144 optimisation and 16 validation examples. The network used a mean-voltage readout, 0.5 ms timestep and 40 ms duration. We trained input and readout weights for 2 epochs with AdamW[1], learning rate 0.001, weight decay 0.0001 and gradient-norm clipping at one, while recurrent weights remained fixed. Selection minimised cross-entropy averaged over 3 validation encodings, with accuracy and earliest epoch as tie-breakers.
  1. Reload and evaluate. We evaluated selected and final states on 160 sampled official-test images using one fixed spike encoding. We also loaded the selected compiled-network state through the explicit network route, then trained an explicit network for one epoch with a separate seed and loaded that state through the compiled route. Checkpoint inspection compared parameter names and shapes; accuracy comparisons retained their dataset and encoding context.
  1. Test one-step equality. In a separate deterministic fixture we seeded both descriptions identically and supplied the same encoded spikes and labels. We compared initial parameters, forward outputs, cross-entropy, gradients and one AdamW update with zero numerical tolerance. This bounded test isolates implementation equivalence from stochastic evaluation.

References

  1. Ilya Loshchilov and Frank Hutter: Decoupled Weight Decay Regularization. ICLR, 2019. doi:10.48550/arXiv.1711.05101