Moment equations for an autoregressive point process with an absolute refractory period

These notes extend the autoregressive point-process moment closures derived in Rule and Sanguinetti (2018) to models that include an absolute refractory period via a gating term that sets the rate to zero after a spike.

[get notes as PDF]

An autoregressive point-process with an absolute refractory period

We model neuronal spike trains as a sequence of impulses (Dirac $\delta$ distributions) in time:

$$ y(t) = \textstyle \sum_i \delta(t-t_i), $$

where $t_i$ are the spike times. Autoregressive point-process GLMs predict the instantaneous firing rate $\lambda(t)$ of a neuron as a linear-nonlinear function of spiking history $h(t,\tau)$ and external input $u(t)$. One way to write a model for the output spike train $y(t)$ of a neuron is:

\begin{equation}\begin{aligned} \lambda(t) &= \phi(\text H h(t,\tau) + u(t)) \\ y(t) &\sim \operatorname{Poisson}(\lambda(t) \cdot d t) \\ h(t,\tau) &= y(t-\tau). \end{aligned}\end{equation}

Here, $\phi(\cdot)$ is a rectifying nonlinearity that reflects that the firing rate must be positive, and $\text H$ is a spiking history filter. The function $\phi$ can be any rectifying nonlinearity, although it is usually exponential, logistic, or rectified polynomial. Above, we have denoted the history filter as a linear operator, defined for a single neuron as:

$$ \text H h(t,\tau) = \lim_{\epsilon \downarrow 0} \textstyle \int_{\infty}^{\epsilon} \eta(\tau) h(t,\tau) d\tau, $$

where $\eta(\tau)$ are the history filter weights. The limit $\epsilon \downarrow 0$ simply means that we exclude the present time-point ($\tau = 0$) from the spiking history. In single neurons, the spike history filter captures phenomena like resonance, bursting, or a relative refractory period. In population models, it also captures how different neurons excite or inhibit one another.

After a neuron spikes, there is typically a brief 5-10 ms time window in which it cannot spike again, called the absolute refractory period. The firing rate is zero during this time. Depending on the choice of nonlinearity $\phi$, it can be hard for the linear weights to drive the rate to zero. This can be fixed by adding a gating term that goes to zero if the cell has spiked recently:

\begin{equation}\begin{aligned} \lambda(t) &= g[h(t,\tau)] \cdot \phi(\text H h(t,\tau) + u(t)) \end{aligned}\end{equation}

The gating function $g[h(t,\tau)]$ is defined as:

\begin{equation}\begin{aligned} g[h(t)] &= 1 - \lim_{\epsilon \downarrow 0} \textstyle \int_{\epsilon}^{1} \rho(\tau) h(\tau) d\tau. \end{aligned}\end{equation}

The coefficients $\rho(\tau)$ are a refractory history filter. For example, to constrain a neuron to fire at most once per unit time, set $\rho(\tau)=1$ for $\tau\in(0,1]$ (and zero otherwise). If we define a linear operator $R$ as

\begin{equation}\begin{aligned} \text R h(t,\tau) &= \lim_{\epsilon \downarrow 0} \textstyle \int_{\epsilon}^{1} \rho(\tau) h(t,\tau) d\tau, \end{aligned}\end{equation}

then we may write the firing rate as

\begin{equation}\begin{aligned} \lambda &= (1-\text Rh) \cdot \phi(\text H h + u). \end{aligned}\end{equation}

In this notation, the history process $h$ evolves as

\begin{equation} \begin{aligned} \partial_t h &= -\partial_\tau h + \delta_{\tau=0} y \\ y &\sim\operatorname{Poisson}(\lambda\cdot \Delta t) \\ \lambda &= (1-\text Rh) \cdot \phi(\text H h + u)\end{aligned} \end{equation}

We are interested in deriving equations for the mean and covariance of the auxiliary history process $h$. To incorporate corrections from covariance effects, we need to estimate the expected (mean) rate $\left<\lambda\right>$, based on the mean and covariance of the spiking history:

\begin{equation} \begin{aligned} \left<\lambda\right> &= \left<(1-Rh) \phi(Hh + u)\right>. \end{aligned} \end{equation}

We'll the a Gaussian moment closure on a local, second-order Taylor expansion of the (nonlinear) intensity described in Rule and Sanguinetti (2018).

If we assume that $h$ is Gaussian, then correcting for higher moments amounts to correcting for the interaction between the covariance $\Sigma$ and the curvature of $\lambda$.

This moment closure is derived for models without absolute refractoriness, $\lambda = \phi(\text Hh+u)$, in Rule and Sanguinetti (2018). In these notes, we therefore focus on the additional term $\text Rh\cdot \phi(\text Hh + u)$ arising from the absolute refractory period.

Covariance corrections to the mean

To derive a covariance correction to the mean, we need to calculate the curvature of the term $\text Rh \phi(\text Hh + u)$ about the current mean $\mu(t,\tau)=\left< h(t,\tau) \right>$.

In the following derivations, we denote functions of time-lag $\tau$ $h(\tau_1,t)$, $h(\tau_2,t)$ as $h_i$, $h_j$, etc., for compactness. Using the abbreviations

$$ \begin{aligned} \phi & = \phi(\text Hh+u), \\ \phi' & = \phi'(\text Hh+u), \end{aligned} $$

etc. The derivatives are

\begin{equation} \begin{aligned} \partial_{h_i} [\text Rh \phi] &= (\text Rh) \phi' \eta_i + \rho_i \phi \\ \partial_{h_j}\partial_{h_i} [\text Rh \phi] &= \rho_j \phi' \eta_i + (\text Rh) \phi'' \eta_j \eta_i + \rho_i \phi' \eta_j \end{aligned} \end{equation}

Or in vector notation:

\begin{equation} \begin{aligned} \nabla \left[\text Rh \phi\right] &= (\text Rh) \phi' \circ \text H + \phi \circ \text R \\ \nabla \nabla^\top \left[\text Rh \phi\right] &= \phi' \circ [\text R \text H^\top + \text H \text R^\top ] + (\text Rh) \phi'' \circ \text H \text H^\top, \end{aligned} \end{equation}

where $\circ$ denotes element-wise multiplication.

The covariance corrections to the mean come from how the noise (covariance) interacts with the second derivatives derived above:

$$ \tfrac 1 2 \textstyle\sum_{ij} \Delta h_i \Delta h_j \cdot \partial_{h_j}\partial_{h_i}[\dots]. $$

In vector notation, this can be abbreviated as

$$ \tfrac 1 2 \operatorname{tr}\left[ \Sigma \cdot \nabla \nabla^\top \left(\dots\right) \right]. $$

The covariance correction without absolute refractoriness is (Rule and Sanguinetti, 2018):

\begin{equation} \begin{aligned} \tfrac 1 2 \phi'' \circ \text H\Sigma \text H^\top, \end{aligned} \end{equation}

and the covariance corrections for the absolute-refractory term $\left<(\text Rh) \phi\right>$ are

\begin{equation} \begin{aligned} & \tfrac 1 2 \operatorname{tr}\left[ \Sigma \cdot \left(\phi' \circ [\text R \text H^\top + \text H \text R^\top ] + (\text R\mu) \phi'' \circ \text H \text H^\top \right)\right] \\=& \tfrac 1 2 \left\{ \phi' \circ [\text R \Sigma \text H^\top + \text H \Sigma \text R^\top ]+(\text R\mu) \phi'' \circ \text H \Sigma \text H^\top \right\}. \end{aligned} \end{equation}

Combining these, we get the overall correction to the mean of

\begin{equation} \begin{aligned} \tfrac 1 2 \left[ (1-\text R\mu) \phi'' \circ \text H\Sigma \text H^\top + \phi' \circ \left(\text R \Sigma \text H^\top + \text H \Sigma \text R^\top \right) \right]. \end{aligned} \end{equation}

I personally find the notation a bit more readable if we define the variance of the input $\sigma_a^2 = \text H\Sigma \text H^\top$, the gating term $g = 1-\text R\mu$, and a variable $K =\text R \Sigma \text H^\top$ that summarizes interactions between the refractory gating and autohistory filter:

\begin{equation} \begin{aligned} \tfrac 1 2 \left[g \phi'' \circ \sigma_a^2 + \phi' \circ \left(K+K^\top \right)\right] \end{aligned} \end{equation}

In words: the variance in the activation, $\sigma_a^2=\text H\Sigma \text H^\top$, interacts with the curvature of the firing rate nonlinearity $\phi''$. This is similar to the model without the absolute refractory period, just multiplied by the mean gating term $g=1-\text R\mu$. We also get an additional correction from the slope of the nonlinearity, $\phi'$. This is related to how terms in the absolute refractory filter $\text R$ correlate with terms in the history filter $\text H$, $K = \text R \Sigma \text H^\top$.

Time evolution of the covariance

Following Rule and Sanguinetti (2018), the covariance evolves according to the the Jacobian of the mean system, similar to the extended-time Kalman-Bucy filter. Refer to Rule and Sanguinetti (2018) for the linear terms. The introduction of an absolute refractory filter only changes the nonlinear terms in the Jacobian, $\nabla \lambda$. This is given by:

\begin{equation} \begin{aligned} \nabla \lambda &= g \phi' \circ \text H - \phi \circ \text R. \end{aligned} \end{equation}

These moment equations generalize to population models, where the history and refractory filters are replaced by matrices of filters for each member of the population, and history moments now refer to the joint moments of the population history.