<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
tt
{mso-style-priority:99;
font-family:"Courier New";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
color:black;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-AU" text="#000000" link="blue" vlink="purple"
bgcolor="white">
<div class="WordSection1"><tt><br>
</tt>
<tt><span style="font-size:10.0pt">Hi Terry,</span></tt><tt><span
style="font-size: 10pt;"><br>
<br>
Thank you very much for the fast response. You are right, I
should have explained the problem more carefully! However, I'm
a new Nengo user and don't have much experience.
<br>
<br>
What I want to do is similar to the simulation Brian Fischer
did about 10 years ago, that is to simulate the auditory
system of a barn owl (B.J. Fischer, and J.L. Pena, (2011)
Owl’s behaviour and neural representation predicted by
Bayesian inference, Nature Neuroscience 14 (8): 1061-1067). To
find the direction from which a sound comes from, the auditory
system has a field of neurons, each associated with a
different angle. A first approximation would be to test which
neuron has the highest activity: for example, if it is the
neuron which is associated with 40 degrees, this gives the
direction.
<br>
<br>
However, a better approximation would be to multiply the
activities of all the neurons with the associated angles and
divide it by the total activity of all neurons.
<br>
<br>
But the problem already appears in a simpler example. Here I
have implemented two ways to divide two numbers, the first is
direct division and the second is using the logarithm. Using
the logarithm is better, but in both cases I can get crazy
results, quite often also negative, depending on the radius of
the ensembles. (At one point I used the LIFRate neurons to get
rid of too many fluctuations, but the results doesn't depend
on the kind on neurons).
<br>
<br>
The code for this example is:<br>
<br>
--------------------------------------<br>
import numpy as np<br>
import matplotlib.pyplot as plt<br>
import nengo<br>
<br>
model = nengo.Network(label='Log sums')<br>
with model:<br>
<br>
input_1 = nengo.Node(output=10)<br>
input_2 = nengo.Node(output=2)<br>
v = nengo.Ensemble(50, dimensions=2, radius=12.0)<br>
nengo.Connection(input_1, v[0])<br>
nengo.Connection(input_2, v[1])<br>
<br>
# Create a 2-population with representing the log of the
numbers<br>
def logf(x):<br>
if x == 0:<br>
return 0<br>
else:<br>
return np.log(x)<br>
vlog = nengo.Ensemble(50, dimensions=2, radius=8.0)<br>
nengo.Connection(input_1, vlog[0],function=logf)<br>
nengo.Connection(input_2, vlog[1],function=logf)<br>
<br>
# compute the division directly<br>
def quot(x):<br>
if x[1] == 0:<br>
return 0<br>
else:<br>
return float(x[0])/float(x[1])<br>
out = nengo.Ensemble(800,dimensions=1,radius=10)<br>
nengo.Connection(v,out,function=quot,synapse=0.1)<br>
<br>
#compute the division using the logarithm<br>
def diff(x):<br>
return x[0]-x[1]<br>
<br>
outlog =
nengo.Ensemble(800,dimensions=1,neuron_type=nengo.neurons.LIFRate(tau_rc=0.01,
tau_ref=0.002),radius=6)<br>
nengo.Connection(vlog,outlog,function=diff,synapse=0.1)<br>
<br>
def exp(x):<br>
return np.exp(x)<br>
<br>
out_exp = nengo.Ensemble(800,dimensions=1,radius=12)<br>
nengo.Connection(outlog,out_exp,function=exp,synapse=0.1)<br>
-----------------------------------------<br>
<br>
Unfortunately I am departing on leave for 5 weeks from the end
of this week. As I’m travelling, I won’t be able to work on
the problem over the next couple of weeks, so please excuse
the delay in my responses until my return.<br>
<br>
Thank you very much for your help, which I greatly appreciate.<br>
<br>
Best regards,<br>
<br>
Peer</span></tt><tt><o:p></o:p></tt><br>
<tt>
</tt>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<br>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier New"">
</span><br>
<br>
-------- Weitergeleitete Nachricht -------- <o:p></o:p></p>
<div>
<table class="MsoNormalTable" border="0" cellpadding="0"
cellspacing="0">
<tbody>
<tr>
<td style="padding:0cm 0cm 0cm 0cm" nowrap="nowrap"
valign="top">
<p class="MsoNormal" style="text-align:right"
align="right"><b>Betreff: <o:p></o:p></b></p>
</td>
<td style="padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal">Re: [nengo-user] normalized average
of a variable x over n ensembles of neurons<o:p></o:p></p>
</td>
</tr>
<tr>
<td style="padding:0cm 0cm 0cm 0cm" nowrap="nowrap"
valign="top">
<p class="MsoNormal" style="text-align:right"
align="right"><b>Datum: <o:p></o:p></b></p>
</td>
<td style="padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal">Tue, 9 Feb 2016 10:31:04 -0500<o:p></o:p></p>
</td>
</tr>
<tr>
<td style="padding:0cm 0cm 0cm 0cm" nowrap="nowrap"
valign="top">
<p class="MsoNormal" style="text-align:right"
align="right"><b>Von: <o:p></o:p></b></p>
</td>
<td style="padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal">Terry Stewart <a
href="mailto:terry.stewart@gmail.com"><a class="moz-txt-link-rfc2396E" href="mailto:terry.stewart@gmail.com"><terry.stewart@gmail.com></a></a><o:p></o:p></p>
</td>
</tr>
<tr>
<td style="padding:0cm 0cm 0cm 0cm" nowrap="nowrap"
valign="top">
<p class="MsoNormal" style="text-align:right"
align="right"><b>An: <o:p></o:p></b></p>
</td>
<td style="padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal">Peer Ueberholz <a
href="mailto:peer@ueberholz.de"><a class="moz-txt-link-rfc2396E" href="mailto:peer@ueberholz.de"><peer@ueberholz.de></a></a><o:p></o:p></p>
</td>
</tr>
<tr>
<td style="padding:0cm 0cm 0cm 0cm" nowrap="nowrap"
valign="top">
<p class="MsoNormal" style="text-align:right"
align="right"><b>Kopie (CC): <o:p></o:p></b></p>
</td>
<td style="padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal"><a
href="mailto:nengo-user@ctnsrv.uwaterloo.ca"><a class="moz-txt-link-abbreviated" href="mailto:nengo-user@ctnsrv.uwaterloo.ca">nengo-user@ctnsrv.uwaterloo.ca</a></a>
<a href="mailto:nengo-user@ctnsrv.uwaterloo.ca"><nengo-user@ctnsrv.uwaterloo.ca></a><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<pre>Hello Peer,<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Thank you for the question! I'm not quite sure exactly what you're<o:p></o:p></pre>
<pre>trying to compute here, though -- if the neurons are representing some<o:p></o:p></pre>
<pre>vector x, then that means that their activity a_i should itself be a<o:p></o:p></pre>
<pre>function of x. And if that's the case, then I'm not quite sure what<o:p></o:p></pre>
<pre>it would mean to compute a weighted average when the weighting factor<o:p></o:p></pre>
<pre>a_i is itself a function of x. Do you want to control a_i<o:p></o:p></pre>
<pre>independently of x_i? Or are they supposed to be so tightly coupled?<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Could you give a little more context of what you're trying to do here?<o:p></o:p></pre>
<pre> What are the inputs and outputs that you want? One possible thing<o:p></o:p></pre>
<pre>that's coming to mind is that you've got two inputs: the vector x and<o:p></o:p></pre>
<pre>the vector a, and you want to compute \sum_i^n a_i x_i / \sum_1^n a_i.<o:p></o:p></pre>
<pre>This isn't quite what you asked, as I now have a_i and x_i as just two<o:p></o:p></pre>
<pre>different things being represented by the neurons, rather than a_i<o:p></o:p></pre>
<pre>being a direct measure of the activity of the neurons. But if that's<o:p></o:p></pre>
<pre>what you wanted, then this is how it could be done in nengo:<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>----------<o:p></o:p></pre>
<pre>import nengo<o:p></o:p></pre>
<pre>import numpy as np<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>model = nengo.Network()<o:p></o:p></pre>
<pre>with model:<o:p></o:p></pre>
<pre> D = 4<o:p></o:p></pre>
<pre> stim_x = nengo.Node([0]*D) # the values to average<o:p></o:p></pre>
<pre> stim_a = nengo.Node([0.3]*D) # the weights to apply when doing the average<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> ens = nengo.Ensemble(n_neurons=2000, dimensions=D*2)<o:p></o:p></pre>
<pre> nengo.Connection(stim_x, ens[:D])<o:p></o:p></pre>
<pre> nengo.Connection(stim_a, ens[D:])<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> result = nengo.Ensemble(n_neurons=50, dimensions=1)<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> def weighted_average(x):<o:p></o:p></pre>
<pre> a = x[D:]<o:p></o:p></pre>
<pre> x = x[:D]<o:p></o:p></pre>
<pre> return sum(a*x)/sum(a)<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> # this should give sample data that you want the neurons<o:p></o:p></pre>
<pre> # to be good at<o:p></o:p></pre>
<pre> def make_sample():<o:p></o:p></pre>
<pre> x = np.random.uniform(-1,1,size=D)<o:p></o:p></pre>
<pre> a = np.random.uniform(0,1,size=D)<o:p></o:p></pre>
<pre> return np.hstack([x, a])<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> nengo.Connection(ens, result, function=weighted_average,<o:p></o:p></pre>
<pre> eval_points=[make_sample() for i in range(4000)])<o:p></o:p></pre>
<pre> --------------<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>One slightly uncommon thing that's being done in that code is the<o:p></o:p></pre>
<pre>eval_points, which I'm using to make sure that nengo doesn't try to<o:p></o:p></pre>
<pre>optimize the neural connections across the whole space, which would<o:p></o:p></pre>
<pre>include *negative* values for a_i. Trying to optimize over the whole<o:p></o:p></pre>
<pre>space is often too difficult and unneeded, so we tend to try to just<o:p></o:p></pre>
<pre>optimize over the part of the space that's needed (as given by the<o:p></o:p></pre>
<pre>make_sample() function).<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Still, I'm not quite sure this solves your problem, but maybe it's a<o:p></o:p></pre>
<pre>step in the right direction. Let me know how I'm misinterpreting<o:p></o:p></pre>
<pre>things and I can make another attempt at solving the problem!<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>:)<o:p></o:p></pre>
<pre>Terry<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre><o:p> </o:p></pre>
<pre><o:p> </o:p></pre>
<pre><o:p> </o:p></pre>
<pre>On Mon, Feb 8, 2016 at 6:49 PM, Peer Ueberholz <a href="mailto:peer@ueberholz.de"><peer@ueberholz.de></a> wrote:<o:p></o:p></pre>
<pre>> Hi<o:p></o:p></pre>
<pre>><o:p> </o:p></pre>
<pre>> I want to compute a normalized average of a variable x over n ensembles of<o:p></o:p></pre>
<pre>> neurons<o:p></o:p></pre>
<pre>><o:p> </o:p></pre>
<pre>> \bar{x} = \sum_i^n a_i x_i / \sum_1^n a_i<o:p></o:p></pre>
<pre>><o:p> </o:p></pre>
<pre>> where a_i is the activity of each ensemble, \sum_1^n a_i the sum over the<o:p></o:p></pre>
<pre>> activities of the n ensembles and x_i the value of a variable x assigned to<o:p></o:p></pre>
<pre>> ensemble i.<o:p></o:p></pre>
<pre>><o:p> </o:p></pre>
<pre>> To implement the sum is possible, although not entirely straightforward.<o:p></o:p></pre>
<pre>> However, the division seems to represent a more serious problem. A possible<o:p></o:p></pre>
<pre>> solution that I've tried is to avoid division is to use logarithms, but this<o:p></o:p></pre>
<pre>> doesn't appear to help much either. Does anyone know a simple method to<o:p></o:p></pre>
<pre>> compute this quantity in Nengo?<o:p></o:p></pre>
<pre>><o:p> </o:p></pre>
<pre>><o:p> </o:p></pre>
<pre>> Thanks,<o:p></o:p></pre>
<pre>><o:p> </o:p></pre>
<pre>> Peer<o:p></o:p></pre>
<pre>><o:p> </o:p></pre>
<pre>> _______________________________________________<o:p></o:p></pre>
<pre>> nengo-user mailing list<o:p></o:p></pre>
<pre>> <a href="mailto:nengo-user@ctnsrv.uwaterloo.ca">nengo-user@ctnsrv.uwaterloo.ca</a><o:p></o:p></pre>
<pre>> <a href="http://ctnsrv.uwaterloo.ca/mailman/listinfo/nengo-user">http://ctnsrv.uwaterloo.ca/mailman/listinfo/nengo-user</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>