Using the control#

--v-grad-dampen changes the backward pass through the legacy biophysical neuron’s membrane increment. It is intended to reduce gradient amplification without intentionally changing the forward model. It does not divide the entire voltage gradient by a constant, repair a non-finite forward pass, or guarantee convergence.

In tools/snnsim/models.py, both lif_step and lif_step_expeuler apply _scale_grad(dv, 1.0 / v_grad_dampen) to the voltage increment dv. The legacy training CLI defaults to 80; the exp006 — Training explicitly uses 1000. Treat either value as a configuration choice, not a universal threshold. A value of 1 disables the scaling. Use positive values, and values at least 1 when the intention is damping rather than amplification.

Damping changes the optimization problem’s supplied gradients. Keep it fixed when reproducing a recipe; record it when comparing training runs. Graph-native training has its own recipe contract: see exp088 — Training recipes and graph-native learning.

Check the primitive#

This small check exercises the actual helper without training a network or downloading data. Run it from the repository root:

import torch
from tools.snnsim.models import _scale_grad

x = torch.tensor([2.0], dtype=torch.float64, requires_grad=True)
y = _scale_grad(x, 0.01)
y.sum().backward()
torch.testing.assert_close(y, x)
torch.testing.assert_close(x.grad, torch.tensor([0.01], dtype=x.dtype))

Expected result: both assertions pass. The forward value is approximately 2 and its derivative is approximately 0.01. The helper is private; this is an implementation check, not a new public API.

The helper computes

Here $𝑥$ is the input tensor, $𝑐$ is a dimensionless gradient scale, and detach retains the value while removing its autograd dependency. In exact arithmetic ${𝐹}_{𝑐\left(𝑥\right)}=𝑥$, but autograd returns derivative $𝑐$. Floating-point multiplication and addition can introduce rounding: this expression does not establish bitwise identity of full trajectories. PyTorch documents the dependency boundary in Tensor.detach.

The implemented update#

The following equations describe the local exponential-Euler membrane update with noise and active voltage clamps excluded. They are an implementation derivation, not a proof of global network stability. The exp100 — COBANet page covers the surrounding dynamics.

The synapse helper decays the previous conductance and then adds the spike kick:

Here ${𝑔}^{𝑘}$ is a row of conductances in μS at step $𝑘$, $𝑠\left[𝑘\right]$ the supplied dimensionless presynaptic spike row, $𝑊$ the stored weight matrix in μS, $\mathrm{\Delta }{𝑡}_{\text{sim}}$ the integration timestep in ms, and ${𝜏}_{\text{syn}}$ the pathway’s synaptic decay time in ms. In particular, the new kick is not multiplied by ${𝛽}_{\text{syn}}$. Network scheduling determines which spike row reaches each pathway.

Using the updated excitatory and inhibitory conductances ${𝑔}_{𝑒}$ and ${𝑔}_{𝑖}$, define

Here ${𝑔}_{𝐿}$ is leak conductance, ${𝐶}_{𝑚}$ capacitance in nF, and ${𝐸}_{𝐿}$, ${𝐸}_{𝑒}$, ${𝐸}_{𝑖}$ the leak, excitatory, and inhibitory reversal potentials in mV. ${𝑔}_{\text{tot}}$ is total conductance, ${𝑉}_{\infty }$ the frozen-conductance equilibrium voltage, and ${𝛼}_{\text{mem}}$ the dimensionless membrane decay.

For current voltage ${𝑉}_{𝑚}$, the increment and candidate voltage are

Here ${𝑑}_{\text{grad}}$ is the dimensionless damping divisor configured by v_grad_dampen, and ${𝑉}_{\text{candidate}}$ is the candidate voltage before noise, clamps, thresholding, and reset. The implementation advances the voltage before testing its spike threshold. On a spiking or refractory neuron, torch.where replaces the retained voltage with the reset value; the emitted spike remains a separate output with its surrogate derivative.

What the derivatives say#

Holding conductances fixed, the backward derivative of the candidate voltage is

Thus damping the increment preserves the direct ${𝑉}_{𝑚}$ pathway; it does not replace the full derivative by $\frac{{𝛼}_{\text{mem}}}{{𝑑}_{\text{grad}}}$. For ${𝑑}_{\text{grad}}\ge 1$ and positive conductances this local derivative lies between ${𝛼}_{\text{mem}}$ and 1.

Define the undamped conductance sensitivity for channel $𝑞\in \left\{𝑒,𝑖\right\}$ as

${𝐸}_{𝑞}$ is that channel’s reversal potential. The damped candidate-voltage derivative is $𝜕\frac{{𝑉}_{\text{candidate}}}{𝜕}{𝑔}_{𝑞}=\frac{{𝜅}_{𝑞}}{{𝑑}_{\text{grad}}}$. At small timesteps, ${𝜅}_{𝑞}\approx \left(\mathrm{\Delta }\frac{{𝑡}_{\text{sim}}}{{𝐶}_{𝑚}}\right)\left({𝐸}_{𝑞}-{𝑉}_{𝑚}\right)$. This is where the control reduces sensitivity to both recurrent and feedforward conductance inputs.

These are local derivatives before reset and active clamps. Reset gates every derivative of the retained reset voltage, not just its membrane self-term. Gradients through the emitted spike can still propagate along other paths. The full backward pass combines these paths across cells and time; time-accumulated readouts also inject gradients at multiple steps.

A scalar loop-gain estimate can be a heuristic, but does not prove that gradients grow once per gamma cycle or that dividing an estimated gain by ${𝑑}_{\text{grad}}^2$ makes the entire network contractive. Such claims require the actual trajectory, stored fan-in-scaled weights, surrogate normalization, gates, and full coupled Jacobians. Neither successful forward simulation nor removal of the inhibitory loop guarantees stable learning.

Diagnosing a training failure#

1. Locate the first non-finite value. Check inputs, forward states, loss, and then gradients. If the forward pass is invalid, changing the backward derivative is not the repair.
2. Inspect the optimizer diagnostics. The legacy trainer records gradient norms and skipped updates. It clips the assembled gradient to norm 1 and skips an update if that norm is non-finite. Clipping cannot make an already invalid gradient informative.
3. Change one control at a time. Compare positive damping values while keeping the seed, input, duration, timestep, initialization, and optimizer settings fixed. Lower learning rate and stronger damping are different interventions.
4. Check learning as well as finiteness. Stronger damping also reduces useful conductance-input sensitivities. A finite loss with negligible learning is not sufficient evidence of a good setting.
5. Keep the scope of the check explicit. The helper assertion verifies its local derivative. Establishing training stability or accuracy requires a separately authorized experiment, with recorded diagnostics and validation results.

exp006 — Training · exp003 — SNNSIM API Reference