[CTN] CTN seminar: Yan Wu (DeepMind) 3:30 Tuesday Dec 11, E5 2004

[Bryan Tripp]

Hi everyone,

Just a reminder about the talk tomorrow.

As usual, CTN grad students are invited to lunch. Please meet in the CTN
area at 11:30. Shamak Dutta (s7dutta at uwaterloo.ca) is organizing lunch.

Bryan


On Fri, Nov 30, 2018 at 1:01 PM Bryan Tripp <bptripp at gmail.com> wrote:

> Hi everyone,
>
> Please join us for the last CTN talk of the term, by Yan Wu. The title and
> abstract follow.
>
> Bryan
>
> Learning Attractor Dynamics for Generative Memory
> Yan Wu (Google DeepMind)
>
> A central challenge faced by memory systems is the robust retrieval of a
> stored pattern in the presence of interference due to other stored patterns
> and noise. A theoretically well-founded solution to robust retrieval is
> given by attractor dynamics, which iteratively cleans up patterns during
> recall. However, incorporating attractor dynamics into modern deep learning
> systems poses difficulties: attractor basins are characterised by vanishing
> gradients, which are known to make training neural networks difficult. In
> this work, we exploit recent advances in variational inference and avoid
> the vanishing gradient problem by training a generative distributed memory
> with a variational lower-bound-based Lyapunov function. The model is
> minimalistic with surprisingly few parameters. Experiments shows it
> converges to correct patterns upon iterative retrieval and achieves
> competitive performance as both a memory model and a generative model.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://artsservices.uwaterloo.ca/pipermail/ctn/attachments/20181210/1f6c3755/attachment.html>