8 ms·

Given a network topology found using this technique, I wonder how it would compare with the same topology with fit parameters?

hardmaru 7y ago
Hi, So in the article we also reported in the experimental results section, the performance increase when we fit the individual parameters using the network topology found, and compared it to the non-tuned parameters.
- brechmos 7y ago
  Awesome, I was just reading the paper when you sent this. It looks like a really interesting direction of work. I have a couple astrophysics CNN problems for which I am not sure the best architecture and I am now curious to try this out.
- afabisch 7y ago
  This reminds me a lot of the work on compressed neural network from Jan Koutnik and his colleagues. They don't evolve topology of a NN, but they learn weights of a neural network in some compressed space. That seems to be very similar to weight sharing. Here are some related papers: - original idea: http://people.idsia.ch/~tino/papers/koutnik.gecco10.pdf http://people.idsia.ch/~tino/papers/koutnik.gecco10.pdf - vision-based TORCS: http://repository.supsi.ch/4548/1/koutnik2013fdg.pdf http://repository.supsi.ch/4548/1/koutnik2013fdg.pdf - backpropagation with compressed weights: http://www.informatik.uni-bremen.de/~afabisch/files/2013_NN_LCS.pdf http://www.informatik.uni-bremen.de/~afabisch/files/2013_NN_... For example, in the case of the cart pole (without swing up) benchmark a simple linear controller with equal positive weights is required which can easily be encoded with this approach.
  - hardmaru 7y ago
    Hi, Thanks for the references. The GECCO paper on compressed network search has been a big influence on previous projects I worked on, see: https://news.ycombinator.com/item?id=16694153 https://news.ycombinator.com/item?id=16694153 https://news.ycombinator.com/item?id=14883694 https://news.ycombinator.com/item?id=14883694 it’s a small community!

hardmaru 7y ago

Hi, So in the article we also reported in the experimental results section, the performance increase when we fit the individual parameters using the network topology found, and compared it to the non-tuned parameters.

brechmos 7y ago
Awesome, I was just reading the paper when you sent this. It looks like a really interesting direction of work. I have a couple astrophysics CNN problems for which I am not sure the best architecture and I am now curious to try this out.
afabisch 7y ago
This reminds me a lot of the work on compressed neural network from Jan Koutnik and his colleagues. They don't evolve topology of a NN, but they learn weights of a neural network in some compressed space. That seems to be very similar to weight sharing. Here are some related papers: - original idea: http://people.idsia.ch/~tino/papers/koutnik.gecco10.pdf http://people.idsia.ch/~tino/papers/koutnik.gecco10.pdf - vision-based TORCS: http://repository.supsi.ch/4548/1/koutnik2013fdg.pdf http://repository.supsi.ch/4548/1/koutnik2013fdg.pdf - backpropagation with compressed weights: http://www.informatik.uni-bremen.de/~afabisch/files/2013_NN_LCS.pdf http://www.informatik.uni-bremen.de/~afabisch/files/2013_NN_... For example, in the case of the cart pole (without swing up) benchmark a simple linear controller with equal positive weights is required which can easily be encoded with this approach.
- hardmaru 7y ago
  Hi, Thanks for the references. The GECCO paper on compressed network search has been a big influence on previous projects I worked on, see: https://news.ycombinator.com/item?id=16694153 https://news.ycombinator.com/item?id=16694153 https://news.ycombinator.com/item?id=14883694 https://news.ycombinator.com/item?id=14883694 it’s a small community!