<div dir="ltr">Hi Aditya,<div><br></div><div>The process we normally use for solving for decoders involves some mathematically intensive operations like inverting matrices. Those matrices grow by the number of neurons, so the more neurons (and dimensions) you use, the slower it's going to be.</div><div><br></div><div>However, it definitely should not take that long to solve for decoders. We've found that the way in which you install NumPy can make a huge difference in decoder solving speed.</div><div><br></div><div>If you're in Linux, then we've found that compiling OpenBLAS and using that gives the best results; one of the Nengo developers, Eric Hunsberger, wrote a nice post on doing this: <a href="https://hunseblog.wordpress.com/2014/09/15/installing-numpy-and-openblas/">https://hunseblog.wordpress.com/2014/09/15/installing-numpy-and-openblas/</a></div><div><br></div><div>If you're on Windows, I believe the Numpy-mkl binaries at <a href="http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy">http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy</a> will be quite fast.</div><div><br></div><div>If you're Mac OS X, then Mac's built in accelerate framework is quite speedy, so you can just pip install numpy.</div><div><br></div><div>Once you've reinstalled NumPy, you should see a huge improvement in decoder solving speed.</div><div><br></div><div>- Trevor</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Oct 1, 2015 at 1:53 PM, Aditya Gilra <span dir="ltr"><<a href="mailto:aditya_gilra@yahoo.com" target="_blank">aditya_gilra@yahoo.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style="color:#000;background-color:#fff;font-family:times new roman,new york,times,serif;font-size:14px"><div dir="ltr"><span>Thanks Trevor. for the nice notebook. Yes, I overlooked the seed issue. I was setting the numpy seed at the beginning, but not again after the network mod. Setting the seed in the Network() is much cleaner.</span></div><div dir="ltr"><br></div><div dir="ltr">An observation: The time taken for the construction of encoders and decoders by Simulator() seems very non-linear. It is pretty fast for 1250 neurons in an Exc-Inh configuration (<10 seconds). Making this ~6250 neurons takes about 3 minutes. With ~12500 neurons this takes almost 30 minutes. There's no thrashing to disk, swap is not used at all, so that's not the reason for the slowness. Must be the decoders calculation for the functions and transformations in the connections? Is this expected?</div><span class="HOEnZb"><font color="#888888"><div dir="ltr"><br></div><div dir="ltr">-Aditya.<br></div></font></span><div><div class="h5"> <br><div><br><br></div><div style="display:block"> <div style="font-family:times new roman,new york,times,serif;font-size:14px"> <div style="font-family:HelveticaNeue,Helvetica Neue,Helvetica,Arial,Lucida Grande,Sans-Serif;font-size:16px"> <div dir="ltr"> <font size="2" face="Arial"> On Wednesday, 30 September 2015 10:00 PM, Trevor Bekolay <<a href="mailto:tbekolay@gmail.com" target="_blank">tbekolay@gmail.com</a>> wrote:<br> </font> </div> <blockquote style="border-left:2px solid rgb(16,16,255);margin-left:5px;margin-top:5px;padding-left:5px"> <br><br> <div><div><div><div dir="ltr">Hi Aditya,<div><br clear="none"></div><div>I made a quick notebook to demonstrate how to do this: <a rel="nofollow" shape="rect" href="https://gist.github.com/tbekolay/3fa35e84423e3bb52f14" target="_blank">https://gist.github.com/tbekolay/3fa35e84423e3bb52f14</a></div><div><br clear="none"></div><div>In words:</div><div><br clear="none"></div><div>The main point is that decoders are solved for in the build process based on the random distributions that are sampled during the build process. For that reason, you normally can't use decoders from one simulator in the next simulator; and, `nengo.builder.solve_for_decoders` only works if you've fully specified all of the random factors ahead of time, which makes the model inflexible.</div><div><br clear="none"></div><div>To solve these issues, you can set the seed on the network so that the random factors will be sampled deterministically for that seed, and use the same seed when you build the model again. If you want to get a different network, then change the seed.</div><div><br clear="none"></div><div>Also, we incorporate the radius into the weights, so once you multiply the encoders and decoders together, you have to divide by the radius for neuron-to-neuron connections.</div><div><br clear="none"></div><div>Finally, It's possible to remove connections completely by doing `model.connections.remove(my_connection)`.</div><div><br clear="none"></div><div>- Trevor</div></div><div><div><br clear="none"><div>On Wed, Sep 30, 2015 at 1:16 PM, Aditya Gilra <span dir="ltr"><<a rel="nofollow" shape="rect" href="mailto:aditya_gilra@yahoo.com" target="_blank">aditya_gilra@yahoo.com</a>></span> wrote:<br clear="none"><blockquote style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style="color:#000;background-color:#fff;font-family:times new roman,new york,times,serif;font-size:14px"><span></span><div dir="ltr"><span>Thanks Trevor,</span></div><div dir="ltr"><br clear="none"></div><div dir="ltr">Am able to access the encoders and decoders pre-simulation now.</div><div dir="ltr"><br clear="none"></div><div dir="ltr">Further queries:</div><div dir="ltr">1)</div><div dir="ltr">How can I compute the weights between neurons from the encoders and decoders? Currently, I doing this as below but doesn't work:</div><div dir="ltr">WEE = dot(dot(Eencoders,W),EtoEdecoders) for Nengo2 via pip<br clear="none"></div><div dir="ltr">WEE = dot(Eencoders,EtoEweights) for Nengo2 dev version from github<br clear="none"></div><div dir="ltr"><br clear="none"></div><div dir="ltr">Details:</div><div dir="ltr">My network:</div><div dir="ltr"><div dir="ltr"> rator = nengo.Ensemble( Nexc, dimensions=N, radius=reprRadius)</div><div dir="ltr"><span></span><div dir="ltr"> EtoEfake = nengo.Connection(rator, rator,</div><div dir="ltr"> transform=W, synapse=tau) # synapse is tau_syn for filtering</div><div dir="ltr"><div dir="ltr"><br clear="none"></div><div dir="ltr">I get encoders and decoders to compute the weight matrix between neurons:<br clear="none"></div><div dir="ltr"> # Generate the encoders and decoders, so that I can access these</div><div dir="ltr"> # and use them for calculating WEE</div><div dir="ltr"> sim = nengo.Simulator(model)</div><div dir="ltr"> Eencoders = sim.data[rator].encoders</div><div dir="ltr"> Eintercepts = sim.data[rator].intercepts</div><div dir="ltr"> Emax_rates = sim.data[rator].max_rates</div><div dir="ltr"> Egains = sim.data[rator].gain</div><div dir="ltr"> #EtoEdecoders = sim.data[EtoEfake].decoders # only for nengo2 release using pip</div><div dir="ltr"> #WEE = dot(dot(Eencoders,W),EtoEdecoders) # weights includes W</div><div dir="ltr"> EtoEweights = sim.data[EtoEfake].weights # for nengo2 dev from github</div><div dir="ltr"> WEE = dot(Eencoders,EtoEweights) # weights = dot(W,decoders)</div><div dir="ltr"><br clear="none"></div></div></div></div><div dir="ltr">Now, I should be able to use WEE as below "between neurons" and get the same output:</div><div dir="ltr"><div dir="ltr"> # having computed the decoders, remove the EtoEfake connection</div><div dir="ltr"> # can't delete, only setting weights to zero</div><div dir="ltr"> EtoEfake.transform = zeros(shape=(N,N))</div><div dir="ltr"> # and replace it by the EtoE connection below with modified weights WEEexc</div><div dir="ltr"> EtoE = nengo.Connection(rator.neurons, rator.neurons,</div><div dir="ltr"> transform=WEE, synapse=tau)</div><div dir="ltr"><br clear="none"></div><div dir="ltr">However, including the last third part doesn't have give the same output as not including it.</div></div><div dir="ltr"><br clear="none"></div><div dir="ltr">2)</div><div dir="ltr">Above, I "remove" the EtoEfake connection by setting transform to 0.</div><div dir="ltr">Is there a way to 'delete' this from the network after calling nengo.Simulator()</div><div dir="ltr"><br clear="none"></div><div dir="ltr">Or is there a utility function that can compute the decoders without having to create this 'fake' connection, maybe nengo.builder.solve_for_decoders() but I still need a solver instance. Any example?</div><div dir="ltr"><br clear="none"></div><div dir="ltr">Thanks,</div><div dir="ltr">Aditya.</div> <br clear="none"><div><br clear="none"><br clear="none"></div><div style="display:block"> <div style="font-family:times new roman,new york,times,serif;font-size:14px"> <div style="font-family:HelveticaNeue,Helvetica Neue,Helvetica,Arial,Lucida Grande,Sans-Serif;font-size:16px"><span> </span><div dir="ltr"> <font size="2" face="Arial"> On Monday, 28 September 2015 10:09 PM, Trevor Bekolay <<a rel="nofollow" shape="rect" href="mailto:tbekolay@gmail.com" target="_blank">tbekolay@gmail.com</a>> wrote:<br clear="none"> </font> </div> <div><div><blockquote style="border-left:2px solid rgb(16,16,255);margin-left:5px;margin-top:5px;padding-left:5px"> <br clear="none"><br clear="none"> <div><div><div><div dir="ltr">Hi Aditya,<div><br clear="none"></div><div>The items that you're interested in are accessible in the step between creating the model and simulating it. When you create a simulator with nengo.Simulator(network), Nengo builds the model, which mostly fills in all of the details you're asking about. It exposes the results of building through the same sim.data dictionary as probed information, keyed with the object. I don't think this is documented anywhere though, sorry about that!</div><div><br clear="none"></div><div>1) After you build the simulator, that data should be available with sim.data[my_ensemble].encoders, sim.data[my_ensemble].intercepts, and sim.data[my_ensemble].max_rates. That should tell you what Nengo's using.</div><div><br clear="none"></div><div>Nengo does a bit of scaling internally if you pass in a distribution, compared to sampling the distributions and passing them in manually. That might be one source of why these are different for you, but it's hard to know without seeing the exact network.</div><div><br clear="none"></div><div>2) You need to use probes to get access to the decoders and synaptic weights in a connection as learning is occuring. But, if you want to see the values that are used at the start of the simulation, you can use sim.data[my_connection].decoders (if you're using the latest version from Github, this has recently changed to sim.data[my_connection].weights). If you aren't already doing so, you'll need to store a handle to your connection; e.g., my_connection = nengo.Connection(pre, post).</div><div><br clear="none"></div><div>Hope that helps,</div><div>Trevor</div></div><div><br clear="none"><div>On Mon, Sep 28, 2015 at 3:57 PM, Aditya Gilra <span dir="ltr"><<a rel="nofollow" shape="rect" href="mailto:aditya_gilra@yahoo.com" target="_blank">aditya_gilra@yahoo.com</a>></span> wrote:<br clear="none"><blockquote style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div style="color:#000;background-color:#fff;font-family:times new roman,new york,times,serif;font-size:14px"><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px">Hi,</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px"><br clear="none"></div><div dir="ltr" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px">I've just started using Nengo (version 2). I would like to know the encoders and decoders used by Nengo2 as below. Or construct them with a utility function.</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px"><br clear="none"></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px">1) How can I access the encoders, intercepts and max_rates instantiated in an Ensemble? These are not in the probeable list. And in any case, I would like to access them before the simulation is run.</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px"><br clear="none"></div><div dir="ltr" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px">When I construct and set encoders using UniformHypersphere( surface=True ), and similarly ensembles and maxrates using Uniform(), as per the Ensemble defaults, the variables represented by the population are not as smooth as when Nengo instantiates the encoders itself. How do I know what Nengo used?</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px"><br clear="none"></div><div dir="ltr" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px">2) How can I access the decoders and synaptic weights learnt/instantiated in a Connection given a transformation / function? decoders are in probeable, but ideally I'd like to know them while constructing my network, not after the simulation. Or maybe there's a utility function I can call that returns these without actually making a Connection.</div><div dir="ltr" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px"><br clear="none"></div><div dir="ltr" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px">Thanks,</div><div dir="ltr" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px">Aditya.</div><div dir="ltr" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px"><br clear="none"></div></div></div></div><br clear="none">_______________________________________________<br clear="none">
nengo-user mailing list<br clear="none">
<a rel="nofollow" shape="rect" href="mailto:nengo-user@ctnsrv.uwaterloo.ca" target="_blank">nengo-user@ctnsrv.uwaterloo.ca</a><br clear="none">
<a rel="nofollow" shape="rect" href="http://ctnsrv.uwaterloo.ca/mailman/listinfo/nengo-user" target="_blank">http://ctnsrv.uwaterloo.ca/mailman/listinfo/nengo-user</a><br clear="none">
<br clear="none"></blockquote></div><br clear="none"></div></div></div><br clear="none"><br clear="none"></div> </blockquote> </div></div></div> </div> </div></div></div></blockquote></div><br clear="none"></div></div></div></div><br><br></div> </blockquote> </div> </div> </div></div></div></div></div></blockquote></div><br></div>