Decoder Accuracy Improves with Input Rate

#exp080 · Gamma-gated sparsity · data v35.4.0
Created
Updated

Abstract

This experiment asked which input rates preserve enough information in synaptically and membrane-filtered MNIST images for classification. In the retained calibration, nonlinear decoders were trained across the tested rates and evaluated on held-out digits.

Decoder accuracy improved with input rate, yielding a practical tested range for later variable-rate PING experiments. The result calibrates the filtering and decoding pipeline; it does not measure PING-network accuracy or predict performance between tested rates.

Results

Mixed-rate validation trajectories

The retained calibration contains three independently initialized nonlinear decoders trained on fresh encoding draws sampled across the complete input-rate grid (Fig. 1).

Mixed-rate validation accuracy over training epochs, with one curve per decoder.
Figure 1: Each curve shows validation accuracy by epoch for one of 3 training replicates. Accuracy is the fraction correct across 5000 validation presentations, with input rate sampled uniformly and a fresh encoding draw used for every presentation and epoch.

If mixed-rate training learned digit structure that survived stochastic filtering, validation accuracy should improve across all three training replicates rather than only one trajectory.

Each replicate reached its maximum late in training. The selected epochs were 46, 42, 44, with validation accuracies 76.4%, 76.3%, 77.2%.

Notes.

Filtered digit features

A reused illustration shows one MNIST digit after the same finite-window synaptic and membrane filtering at three input rates (Fig. 2).

One MNIST input digit and filtered feature images at 0.5, 5 and 25 Hz.
Figure 2: (A) Reused illustrative input digit and its originally simulated features at (B–D) 0.5, 5 and 25 Hz, respectively. Simulations used 1.2 μS conductance increments, 200 ms presentations and independent encoding draws; feature panels share a 0–65 mV scale. The illustration was carried forward unchanged, not regenerated.

Because sparse Bernoulli input produces a small, random number of events, reducing the rate should remove responses from different pixels rather than attenuate the whole digit uniformly.

At 0.5 Hz only isolated fragments remained. The digit became progressively more complete at 5 and 25 Hz, consistent with stochastic event loss rather than uniform contrast scaling.

Held-out rate-accuracy curve

The three selected decoders received identical held-out feature vectors and encoding draws. The practical floor required every decoder to reach 50% accuracy at a tested rate (Fig. 3).

Held-out accuracy across tested input rates, showing the decoder mean and minimum-to-maximum range.
Figure 3: Points show mean accuracy across 3 training replicates at each maximum-pixel encoding rate; every replicate received the same 5000 held-out images and encoding draws. Shading spans the minimum–maximum replicate accuracy, not a confidence interval. Rules mark 10% chance and the 50% criterion.

If higher input rates preserved more digit structure, held-out accuracy should rise before approaching a plateau.

Mean accuracy across training replicates increased monotonically from 27.5% at 0.1 Hz to 96.2% at 25 Hz. The selected floor was 0.5 Hz: all three decoders first crossed the criterion there, and mean accuracy was 62.9%. The resulting interval is a decoder-relative calibration without interpolation; it does not establish PING-network performance.

Methods

  1. Retained computation. We reused a completed calibration containing three trained decoders, their validation histories and their held-out correctness records. We did not rerun feature simulation or decoder training for this article.

  2. MNIST partitions. Of the 60000 official training images, the first 20000 trained the decoders and the next 5000 selected checkpoints; the remaining 35000 were unused. Evaluation used the first 5000 images from the separate official test partition, with no overlap.

  3. Generate input events. Each normalized pixel intensity 𝑥𝑖∈[0,1] generated an independent binary event 𝑠𝑖[𝑘] at integration timestep Δ𝑡sim=0.1 ms:

    𝑝event,𝑖=𝑟input,max𝑥𝑖Δ𝑡sim1000,𝑠𝑖[𝑘]∼Bernoulli(𝑝event,𝑖).
    (1)

    Here 𝑖 indexes pixels, 𝑘 is the simulation-step index, 𝑝event,𝑖 is event probability, and 𝑟input,max is maximum-pixel encoding rate in spikes/s; 1000 converts milliseconds to seconds.

  4. Filter synaptic conductance. Excitatory conductance 𝑔𝑖[𝑘], in μS, decayed each step by exp(−Δ𝑡sim𝜏AMPA) before an event added 𝑤event. The AMPA time constant was 𝜏AMPA=2 ms and event strength was 𝑤event=1.2 μS.

  5. Integrate membrane voltage. During simulation step 𝑘, the updated conductance 𝑔𝑖[𝑘] was held fixed while each non-spiking membrane voltage 𝑉𝑚,𝑖(𝑡) obeyed

    𝐶𝑚𝑑𝑉𝑚,𝑖𝑑𝑡=𝑔𝐿(𝐸𝐿−𝑉𝑚,𝑖)+𝑔𝑖[𝑘](𝐸𝑒−𝑉𝑚,𝑖).
    (2)

    Capacitance was 𝐶𝑚=1 nF, leak conductance 𝑔𝐿=0.05 μS, leak reversal 𝐸𝐿=−65 mV and excitatory reversal 𝐸𝑒=0 mV. Starting at zero conductance and 𝐸𝐿, voltage advanced by the exact exponential solution for that step. Simulation and decoder arithmetic used single precision.

  6. Form pixel features. The feature 𝑧feature,𝑖, in mV, averaged post-update voltages above rest:

    𝑧feature,𝑖=1𝑁𝑡∑𝑘=1𝑁𝑡(𝑉𝑚,𝑖(𝑘Δ𝑡sim)−𝐸𝐿)≈1𝑇present∫0𝑇present(𝑉𝑚,𝑖(𝑡)−𝐸𝐿)d𝑡.
    (3)

    Here 𝑇present=200 ms, 𝑁𝑡=𝑇presentΔ𝑡sim is the timestep count, and physical time at step 𝑘 is 𝑡𝑘=𝑘Δ𝑡sim. Fresh encoding draws retained finite-window shot-noise effects without a stationary Gaussian approximation[1].

  7. Train mixed-rate decoders. At every epoch, we sampled the input rate for each training and validation presentation uniformly from 0.1, 0.25, 0.5, 1, 2, 5, 10, 25 Hz and generated a fresh encoding draw. Each 784–1024–10 ReLU decoder was trained for 50 epochs using cross-entropy, Adam with learning rate 0.001, no weight decay and batch size

  8. Independent training replicates. Stochastic-stream identifiers 42, 43, 44 defined independent model initializations, rate assignments and encoding draws.

  9. Select checkpoints. At each of the 50 eligible epochs, validation accuracy was the fraction correct across 5000 validation presentations. The earliest epoch attaining the maximum validation accuracy supplied the selected checkpoint for each training replicate.

  10. Evaluate shared test features. Every held-out image was simulated once at each tested rate, and all selected decoders received the same feature vector for that image and rate; the feed-forward decoder had no state across presentations. The predicted class 𝑦̂ was the class 𝑐 with the largest output logit 𝑧𝑐, and accuracy was measured per training replicate and rate.

  1. Aggregate held-out accuracy. For each rate and training replicate, we averaged correctness across 5000 held-out images. We then recorded the mean, minimum and maximum accuracy across the 3 training replicates at each rate.

  2. Select the tested interval. The practical floor was the lowest tested rate where every decoder reached 50% accuracy, and the ceiling was the highest tested rate. No interpolation was used; an empty qualifying set was reported as right-censored.

  1. Present retained evidence. We redrew the validation trajectories and rate-accuracy summary from recorded measurements, using the aggregation defined above without interpolation. We reused the original finite-window feature illustration unchanged rather than implying a new simulation.

Dataset

Appendix: Finite-window filtering and interpretation

The Bernoulli input events form shot noise: each produces a discrete conductance jump followed by exponential AMPA decay. These pulses change both the voltage toward which the membrane moves and how quickly it moves there. A spike’s An event’s effect therefore depends on the voltage and conductance left by earlier events, rather than adding a fixed voltage increment.

At low input rates, a finite presentation may contain no events, one event, or a few arriving at different times. Response statistics can change during the presentation (nonstationary), and their distribution can be asymmetric or concentrated around a few outcomes (non-Gaussian)[1]. Direct simulation retained these count and timing effects rather than replacing them with a steady, bell-shaped approximation.

Neural decoding measures information accessible to a specified readout, not absolute information content or the mechanism by which a biological population uses it[2]. The criterion therefore selects a practical interval for this representation and decoder. Transfer to a PING network requires a separate evaluation; decoding accuracy here does not establish gamma timing benefits.

References

  1. Brigham & Destexhe: Nonstationary Filtered Shot-Noise Processes and Applications to Neuronal Membranes. Physical Review E, 2015. doi:10.1103/PhysRevE.91.062102
  2. Quian Quiroga & Panzeri: Extracting Information from Neuronal Populations: Information Theory and Decoding Approaches. Nature Reviews Neuroscience, 2009. doi:10.1038/nrn2578