Limit of an infinite chain of first-order exponential filters

First-order exponential filter

The simplest model how the voltage $x$ at a synapse responds to input $u$ is a first-order filter:

$$\tau \dot{x}=-x+u.$$

This corresponds to convolving signal $u(t)$ with exponential filter $\mathrm{H}(t)\exp (-t/\tau )$, where $\mathrm{H}(\cdot )$ is the Heaviside step function:

$$\begin{array}{rl}x(t)& =h(t)\ast u(t)\\ h(t)& =\mathrm{H}(t)\exp (-t/\tau ).\end{array}$$

The alpha function

A first-order filter has a discontinuous jump in response to an abrupt inputs (like spikes). A more realistic response is the "alpha function"  $t\cdot \exp (-t)$. The alpha function can be obtained by convolving two first decay functions (i.e. chaining together two first-order filters):

$$\begin{array}{rl}\tau {\dot{x}}_1& =-x_1+u\\ \tau {\dot{x}}_2& =-x_2+x_1.\end{array}$$

This is sometimes written in the compact notation

$${(\tau \frac{d}{dt}+1)}^{2}x=u.$$

Higher orders

You can repeat this operation many times, obtaining responses with increasing smoothness. The family $t^n\cdot \exp (-t)$ reflects $n+1$ feed-forward variables coupled by exponential decay ${\dot{x}}_n=x_{n-1}-x_n$. The integral of $t^n\exp (-t)$ grows with $n$. To normalize, divide by $n!=\mathrm{\Gamma }(n+1)$: 

$$h(t)=\mathrm{H}(t)\frac{t^n}{\mathrm{\Gamma }(n+1)}{e}^{-t}.$$

The response of $t^n\exp (-t)$ peaks at time $t=n$. Rescale time with $t\leftarrow nt$ to get a peak response at $t=1$. To keep the integral of the response normalized when rescaling, multiply by $n$.

$$h(t)=\mathrm{H}(t)n\frac{(nt{)}^n}{\mathrm{\Gamma }(n+1)}{e}^{-nt}$$

This is equivalent to choosing a time constant $\tau =1/n$ for each of the $n$ filtering stages. To place the peak response at time $t_0$, set $\tau =t_0/n$. 

This corresponds to a gamma distribution with $k=n+1$ and $\theta =1/n$. For large $n$ this approximates a Gaussian with $\mu =\frac{n+1}{n}$ and ${\sigma }^2=\frac{n+1}{n^2}$. As $n\to \mathrm{\infty }$ this converges to a Dirac delta (impulse) distribution. 

To stabilize the variance instead of time-to-peak, rescale time by $1/\sqrt{n+1}$. This corresponds to a gamma distribution with $k=n+1$ and $\theta =1/\sqrt{k}$. The time-to-peak in this case diverges as $n\to \mathrm{\infty }$.