[CTN] Talk today: Yan Wu (DeepMind)

[Bryan Tripp]

Hi everyone,

Just a final reminder about the talk, 3:30 this afternoon in E5 2004.
Apologies if you didn't receive previous announcements. I'm not sure the
mailing list is working reliably.

Bryan

Learning Attractor Dynamics for Generative Memory
Yan Wu (Google DeepMind)

A central challenge faced by memory systems is the robust retrieval of a
stored pattern in the presence of interference due to other stored patterns
and noise. A theoretically well-founded solution to robust retrieval is
given by attractor dynamics, which iteratively cleans up patterns during
recall. However, incorporating attractor dynamics into modern deep learning
systems poses difficulties: attractor basins are characterized by vanishing
gradients, which are known to make training neural networks difficult. In
this work, we exploit recent advances in variational inference and avoid
the vanishing gradient problem by training a generative distributed memory
with a variational lower-bound-based Lyapunov function. The model is
minimalistic with surprisingly few parameters. Experiments shows it
converges to correct patterns upon iterative retrieval and achieves
competitive performance as both a memory model and a generative model.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://artsservices.uwaterloo.ca/pipermail/ctn/attachments/20181211/c2872ae7/attachment.html>