[nengo-user] obtaining encoders and decoders used by Ensemble / Connection in Nengo2?
Trevor Bekolay
tbekolay at gmail.com
Thu Oct 1 14:54:16 EDT 2015
Hi Aditya,
The process we normally use for solving for decoders involves some
mathematically intensive operations like inverting matrices. Those matrices
grow by the number of neurons, so the more neurons (and dimensions) you
use, the slower it's going to be.
However, it definitely should not take that long to solve for decoders.
We've found that the way in which you install NumPy can make a huge
difference in decoder solving speed.
If you're in Linux, then we've found that compiling OpenBLAS and using that
gives the best results; one of the Nengo developers, Eric Hunsberger, wrote
a nice post on doing this:
https://hunseblog.wordpress.com/2014/09/15/installing-numpy-and-openblas/
If you're on Windows, I believe the Numpy-mkl binaries at
http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy will be quite fast.
If you're Mac OS X, then Mac's built in accelerate framework is quite
speedy, so you can just pip install numpy.
Once you've reinstalled NumPy, you should see a huge improvement in decoder
solving speed.
- Trevor
On Thu, Oct 1, 2015 at 1:53 PM, Aditya Gilra <aditya_gilra at yahoo.com> wrote:
> Thanks Trevor. for the nice notebook. Yes, I overlooked the seed issue. I
> was setting the numpy seed at the beginning, but not again after the
> network mod. Setting the seed in the Network() is much cleaner.
>
> An observation: The time taken for the construction of encoders and
> decoders by Simulator() seems very non-linear. It is pretty fast for 1250
> neurons in an Exc-Inh configuration (<10 seconds). Making this ~6250
> neurons takes about 3 minutes. With ~12500 neurons this takes almost 30
> minutes. There's no thrashing to disk, swap is not used at all, so that's
> not the reason for the slowness. Must be the decoders calculation for the
> functions and transformations in the connections? Is this expected?
>
> -Aditya.
>
>
>
> On Wednesday, 30 September 2015 10:00 PM, Trevor Bekolay <
> tbekolay at gmail.com> wrote:
>
>
>
> Hi Aditya,
>
> I made a quick notebook to demonstrate how to do this:
> https://gist.github.com/tbekolay/3fa35e84423e3bb52f14
>
> In words:
>
> The main point is that decoders are solved for in the build process based
> on the random distributions that are sampled during the build process. For
> that reason, you normally can't use decoders from one simulator in the next
> simulator; and, `nengo.builder.solve_for_decoders` only works if you've
> fully specified all of the random factors ahead of time, which makes the
> model inflexible.
>
> To solve these issues, you can set the seed on the network so that the
> random factors will be sampled deterministically for that seed, and use the
> same seed when you build the model again. If you want to get a different
> network, then change the seed.
>
> Also, we incorporate the radius into the weights, so once you multiply the
> encoders and decoders together, you have to divide by the radius for
> neuron-to-neuron connections.
>
> Finally, It's possible to remove connections completely by doing
> `model.connections.remove(my_connection)`.
>
> - Trevor
>
> On Wed, Sep 30, 2015 at 1:16 PM, Aditya Gilra <aditya_gilra at yahoo.com>
> wrote:
>
> Thanks Trevor,
>
> Am able to access the encoders and decoders pre-simulation now.
>
> Further queries:
> 1)
> How can I compute the weights between neurons from the encoders and
> decoders? Currently, I doing this as below but doesn't work:
> WEE = dot(dot(Eencoders,W),EtoEdecoders) for Nengo2 via pip
> WEE = dot(Eencoders,EtoEweights) for Nengo2 dev version from github
>
> Details:
> My network:
> rator = nengo.Ensemble( Nexc, dimensions=N, radius=reprRadius)
> EtoEfake = nengo.Connection(rator, rator,
> transform=W, synapse=tau) # synapse is
> tau_syn for filtering
>
> I get encoders and decoders to compute the weight matrix between neurons:
> # Generate the encoders and decoders, so that I can access these
> # and use them for calculating WEE
> sim = nengo.Simulator(model)
> Eencoders = sim.data[rator].encoders
> Eintercepts = sim.data[rator].intercepts
> Emax_rates = sim.data[rator].max_rates
> Egains = sim.data[rator].gain
> #EtoEdecoders = sim.data[EtoEfake].decoders # only for
> nengo2 release using pip
> #WEE = dot(dot(Eencoders,W),EtoEdecoders) # weights
> includes W
> EtoEweights = sim.data[EtoEfake].weights # for nengo2
> dev from github
> WEE = dot(Eencoders,EtoEweights) # weights =
> dot(W,decoders)
>
> Now, I should be able to use WEE as below "between neurons" and get the
> same output:
> # having computed the decoders, remove the EtoEfake connection
> # can't delete, only setting weights to zero
> EtoEfake.transform = zeros(shape=(N,N))
> # and replace it by the EtoE connection below with modified
> weights WEEexc
> EtoE = nengo.Connection(rator.neurons, rator.neurons,
> transform=WEE, synapse=tau)
>
> However, including the last third part doesn't have give the same output
> as not including it.
>
> 2)
> Above, I "remove" the EtoEfake connection by setting transform to 0.
> Is there a way to 'delete' this from the network after calling
> nengo.Simulator()
>
> Or is there a utility function that can compute the decoders without
> having to create this 'fake' connection, maybe
> nengo.builder.solve_for_decoders() but I still need a solver instance. Any
> example?
>
> Thanks,
> Aditya.
>
>
>
> On Monday, 28 September 2015 10:09 PM, Trevor Bekolay <tbekolay at gmail.com>
> wrote:
>
>
>
> Hi Aditya,
>
> The items that you're interested in are accessible in the step between
> creating the model and simulating it. When you create a simulator with
> nengo.Simulator(network), Nengo builds the model, which mostly fills in all
> of the details you're asking about. It exposes the results of building
> through the same sim.data dictionary as probed information, keyed with the
> object. I don't think this is documented anywhere though, sorry about that!
>
> 1) After you build the simulator, that data should be available with
> sim.data[my_ensemble].encoders, sim.data[my_ensemble].intercepts, and
> sim.data[my_ensemble].max_rates. That should tell you what Nengo's using.
>
> Nengo does a bit of scaling internally if you pass in a distribution,
> compared to sampling the distributions and passing them in manually. That
> might be one source of why these are different for you, but it's hard to
> know without seeing the exact network.
>
> 2) You need to use probes to get access to the decoders and synaptic
> weights in a connection as learning is occuring. But, if you want to see
> the values that are used at the start of the simulation, you can use
> sim.data[my_connection].decoders (if you're using the latest version from
> Github, this has recently changed to sim.data[my_connection].weights). If
> you aren't already doing so, you'll need to store a handle to your
> connection; e.g., my_connection = nengo.Connection(pre, post).
>
> Hope that helps,
> Trevor
>
> On Mon, Sep 28, 2015 at 3:57 PM, Aditya Gilra <aditya_gilra at yahoo.com>
> wrote:
>
> Hi,
>
> I've just started using Nengo (version 2). I would like to know the
> encoders and decoders used by Nengo2 as below. Or construct them with a
> utility function.
>
> 1) How can I access the encoders, intercepts and max_rates instantiated in
> an Ensemble? These are not in the probeable list. And in any case, I would
> like to access them before the simulation is run.
>
> When I construct and set encoders using UniformHypersphere( surface=True
> ), and similarly ensembles and maxrates using Uniform(), as per the
> Ensemble defaults, the variables represented by the population are not as
> smooth as when Nengo instantiates the encoders itself. How do I know what
> Nengo used?
>
> 2) How can I access the decoders and synaptic weights learnt/instantiated
> in a Connection given a transformation / function? decoders are in
> probeable, but ideally I'd like to know them while constructing my network,
> not after the simulation. Or maybe there's a utility function I can call
> that returns these without actually making a Connection.
>
> Thanks,
> Aditya.
>
>
> _______________________________________________
> nengo-user mailing list
> nengo-user at ctnsrv.uwaterloo.ca
> http://ctnsrv.uwaterloo.ca/mailman/listinfo/nengo-user
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://artsservices.uwaterloo.ca/pipermail/nengo-user/attachments/20151001/111fc39d/attachment-0002.html>
More information about the nengo-user
mailing list