Let $\mathcal M$ be a manifold

(old graphics for a presentation; neural manifolds can drift in many ways. Some are disruptive, some are not. Some are easier to compensate for than others)

Things that lead to tangent variability (affect coding):

• Variability within the embedding space
• In-band variability: changes in neuronal tuning to behavior
• Coordinate distortions: Generally disruptive
• Contraction or expansion, rigid transformations: Require compensation, some component will disrupt coding (how much?)

Things that do not lead to tangent variability:

• Variations that are in the null space of the embedding space
• Some changes in manifold expansion/contraction is orthogonal to coding direction

Things that lead to correctable variability:

• Rigid transformation (+translation): generally, these changes can be corrected for by re-alignment.
• Sufficiently large-scale variability looks like rigid transformation (+translation) locally
• High-frequency / fine-scale variability can be removed by filtering

Brief manifesto on control and neuroscience

I've recently joined the "Control Lab" at the university of Cambridge, but I'll still be researching neuroscience. This is a fanciful attempt to summarize the current dogma in neuroscience about how things like "free will" and "homeostasis" might relate to technical concepts like "optimal control". None of this is new; it is mostly motivated by old ideas of cybernetics, as best explored by Todorov and colleagues. 

Computation in the brain employs predictive negative feedback. This inverts (i.e. controls) a system by cancelling a prediction error (residual). Learning rules employ this on slow timescales, consolidating these inverse representations into feed-forward circuits. Recurrent dynamics complement this, by generating the spiking output required to cancel any error not cancelled by the forward network. In turn, these recurrent dynamics refine the forward weights and reduce the amount of feedback needed in the future (assuming the environment is stationary). This then bring us back to the general idea of learning, perception, and control. Todorov's papers are a good treatment of this. 

Local learning rules to attenuate forgetting in neural networks

Another paper from our work on Restricted Boltzmann Machines (RBMs) from the Hennig lab. 

[get paper PDF]

Main points:

• We noticed that measures of synaptic importance were available from local firing statistics (at least in Boltzmann machines)
• We look at an artificial neural network that stores memories and is easy to analyze. (Hopfield nets are the zero temperature limit of a Boltzmann machine).
• We evaluated whether this local measure of synaptic importance could help stabilize important weights when networks learn multiple things that interfere with each-other
• Intuition: biological variables, like synapse size, can correlate with useful statistical quantities. This provides tricks for biologically-plausible approximations of algorithms.
• Intuition: in systems that learn, if a parameter takes on an unusual or surprising value, it is likely that this value was set through learning—and you might want to leave it fixed.

Approximations of the measurement update and model likelihood for nonlinear spatiotemporal Cox processes

Epilogue: These notes were part of a project to infer unobserved ("latent") states in a neural field model, as well as the parameters of that model, from spiking observations. It has since been published. Ultimately, for speed we ended up selecting the Laplace approximation for the measurement update, solved via Newton-Raphson.

[get PDF]

We are interested in approximations to the measurement update for a spatially-extended latent-variable point-process, where the latent variables are also spatial fields that undergo dynamics similar to chemical reaction-diffusion systems. 

The latent variables or fields are concentrations, activations, or some similar physical quantity. They are therefore constrained to be non-negative, and also typically must obey conservation laws. Additionally, the observed point-process intensity field must also be constrained to be non-negative. 

Such systems arise in chemical reaction diffusion systems, epidemiological models, and neural field models, where the measurement is a point-process that is coupled indirectly to the latent spatiotemporal system.