12 ms·

Let's look at the most important section of the paper. He estimates the processing power of the brain: The human brain contains about 10^11 neurons. Each neuro

by ctl 11y ago

Let's look at the most important section of the paper. He estimates the processing power of the brain: The human brain contains about 10^11 neurons. Each neuron has about 5 • 10^3 synapses, and signals are transmitted along these synapses at an average frequency of about 10^2 Hz. Each signal contains, say, 5 bits. This equals 10^17 ops. The true value cannot be much higher than this, but it might be much lower. In other words, there are 5 * 10^14 synapses in the brain, and each synapse transmits up to 100 signals per second, and we can probably encode each signal with 5 bits. That's ~10^17 bits per second. So, uh... does anybody else notice that that's not an estimate of processing power? That's an estimate of the rate of information flow between neurons, across the whole brain. The level of confused thinking here is off the charts. Does this guy not understand that in order to simulate the brain, you not only have to keep track of information flows between neurons, you also need to simulate the neurons themselves? That's not merely a flaw in his argument. It indicates that he has no idea what he's talking about, at all. Needless to say, this paper and its conclusions are complete nonsense.

themgt 11y ago
Yup. I want to again quote a paper recommended to me on HN a while back [1] However, the relevance of Turing model is questioned even in case of present-day computing [33] [34]. Indeed, any computing machine that follows a Turing model would be highly inefficient to simulate the activity of biological neurons and experience an increased slowdown. Since the super-Turing computing power of the brain has its origins in these ‘strong’ interactions that occur inside neurons, current models have missed the most important part. Simply, Nature doesn’t care if the N-body problem has analytical solutions [36] or can be simulated in real time on a Turing machine [37]. ... While previous models have attempted to represent Hamiltonians using Turing machines [35] the paper [1] shows that the Hamiltonian model of interaction can represent itself a far more powerful model of computation. Turing made an important step forward; however, there is no need to limit natural models of computation to Turing models. In this sense, the new framework of computation using interaction is universal in nature and provides a more general description of computation than the formal Turing model. In other words God was unaware of Turing's work and has put forward a better model for physical computation in the brain. http://arxiv.org/ftp/arxiv/papers/1210/1210.1983.pdf http://arxiv.org/ftp/arxiv/papers/1210/1210.1983.pdf
davmre 11y ago
Neural nets require a couple of FLOPs per synapse. The processing power required is a direct function of the number of synapses. Each neuron is essentially applying a particular logical op, and counting the neurons and their inputs gives you the number of ops. I don't get why this seems so objectionable. Sure, real neurons in the brain might be doing something a couple of orders of magnitude more complicated than the nodes in an ANN, so you could tack on another 10^2 factor to those estimates if you like. But fundamentally, counting synapses is a reasonable way to get a Fermi estimate of the brain's processing power, and Bostrom's estimates are not significantly different from those others have arrived at by similar methods.
- ctl 11y ago
  You’re right. I didn’t read the paper very carefully, and was myopically focused on the emulating-a-real-brain AI strategy. As in, let’s slice up a real human brain, map the neurons and synapses, and then simulate them as faithfully as possible. To do that you need a great deal of fidelity in your simulations of neurons, which are enormously complex. But there is an argument to be made that neuronal complexity is incidental to the brain’s overall “computational capacity”; that you could replace the neurons in a human brain with much simpler nodes and still end up with a functional intelligence, after sufficient rewiring. I don’t think that claim is obvious, but it’s definitely possible. And if it’s true, you can have human-level intelligence for 10^19 ops, given suitable software. So I apologize for my post. It was over the top and unfair. All that said, I still disagree with Bostrom’s conclusions. I think he enormously understates the difficulty of creating intelligent software, if we’re not just copying an existing brain.
reasonattlm 11y ago
Interestingly, recent research suggests synaptic variability does come to about 5 bits. http://www.salk.edu/news-release/memory-capacity-of-brain-is-10-times-more-than-previously-thought/ http://www.salk.edu/news-release/memory-capacity-of-brain-is... “We were amazed to find that the difference in the sizes of the pairs of synapses were very small, on average, only about eight percent different in size. No one thought it would be such a small difference. This was a curveball from nature.” Because the memory capacity of neurons is dependent upon synapse size, this eight percent difference turned out to be a key number the team could then plug into their algorithmic models of the brain to measure how much information could potentially be stored in synaptic connections. It was known before that the range in sizes between the smallest and largest synapses was a factor of 60 and that most are small. But armed with the knowledge that synapses of all sizes could vary in increments as little as eight percent between sizes within a factor of 60, the team determined there could be about 26 categories of sizes of synapses, rather than just a few. “Our data suggests there are 10 times more discrete sizes of synapses than previously thought.” In computer terms, 26 sizes of synapses correspond to about 4.7 “bits” of information. Previously, it was thought that the brain was capable of just one to two bits for short and long memory storage in the hippocampus.
api 11y ago
I formally studied biology not CS, partly out of an interest in AI. Everyone who thinks superintelligence or even just human or higher-animal level intelligence is right around the corner needs to study genomics, proteomics, molecular biology, and neuroscience. Study them with an open mind and think about what's really going on. A neuron is not a switch. A neuron is an organism. It contains a gene regulatory network more complex than the entire network topology of Amazon's entire web services stack, and that's just looking at the aspects of gene regulation and enzyme (a.k.a. nanomachine) operation that we understand. There are about 100 billion of these in the brain and every one of them is running in parallel and communicating constantly. There are also about 10 glial cells for every one neuron, and glia are involved in neural computation in ways we know are there but don't yet fully understand. (Seems to be related to longer term regulation of synapse behavior, etc.) Each glial cell also contains a massive gene regulatory network and so on. The CS and AI fields suffer from a lot of Dunning-Kreuger effect when they talk about biology. The level of processing power and the parallelism that's going on in the brain of a living thing is simply mind numbing. It's as incredible as the sense you get of the scale of the universe when looking at the Hubble Deep Field. Our present-day computers are toys. We are not even close. It would at least take advances equivalent to the ones that took us from vacuum tube ENIAC to here. Edit: I don't write off superintelligence categorically though. I think we could achieve forms of it not through pure AI but by deeply augmenting biological intelligence. Genetic and biochemical performance enhancement could also play a role. Imagine having more working memory, perfect motivational control, the ability to regulate your own desire/motivational structure, and needing only a few hours of sleep. Cyborg superintelligence is a possibility in the foreseeable future and it does raise issues similar to those the superintelligence folks raise. So I don't dismiss an intelligence explosion. I just very strongly doubt it would be purely solid state.
- baddox 11y ago
  > A neuron is an organism. It contains a gene regulatory network more complex than the entire network topology of Amazon's entire web services stack, and that's just looking at the aspects of gene regulation and enzyme (a.k.a. nanomachine) operation that we understand. Can you give some more details about this? How are you quantifying the complexity of a neuron and of the AWS stack?
sawwit 11y ago
In a recent estimate of the bits per synapse they found it to be an order of magnitude higher than previous estimates: http://www.eurekalert.org/pub_releases/2016-01/si-mco012016.php http://www.eurekalert.org/pub_releases/2016-01/si-mco012016....
tim333 11y ago
Bostrom's paper is based on Hans Moravec's thinking and Moravec's paper is pretty well argued http://www.transhumanist.com/volume1/moravec.htm http://www.transhumanist.com/volume1/moravec.htm Bostrom as a philosopher may be fuzzy on processing power but Moravec who was actually building robots has a pretty good grasp.

by ctl 11y ago

themgt 11y ago

Yup. I want to again quote a paper recommended to me on HN a while back [1] However, the relevance of Turing model is questioned even in case of present-day computing [33] [34]. Indeed, any computing machine that follows a Turing model would be highly inefficient to simulate the activity of biological neurons and experience an increased slowdown. Since the super-Turing computing power of the brain has its origins in these ‘strong’ interactions that occur inside neurons, current models have missed the most important part. Simply, Nature doesn’t care if the N-body problem has analytical solutions [36] or can be simulated in real time on a Turing machine [37]. ... While previous models have attempted to represent Hamiltonians using Turing machines [35] the paper [1] shows that the Hamiltonian model of interaction can represent itself a far more powerful model of computation. Turing made an important step forward; however, there is no need to limit natural models of computation to Turing models. In this sense, the new framework of computation using interaction is universal in nature and provides a more general description of computation than the formal Turing model. In other words God was unaware of Turing's work and has put forward a better model for physical computation in the brain. http://arxiv.org/ftp/arxiv/papers/1210/1210.1983.pdf http://arxiv.org/ftp/arxiv/papers/1210/1210.1983.pdf

davmre 11y ago

Neural nets require a couple of FLOPs per synapse. The processing power required is a direct function of the number of synapses. Each neuron is essentially applying a particular logical op, and counting the neurons and their inputs gives you the number of ops. I don't get why this seems so objectionable. Sure, real neurons in the brain might be doing something a couple of orders of magnitude more complicated than the nodes in an ANN, so you could tack on another 10^2 factor to those estimates if you like. But fundamentally, counting synapses is a reasonable way to get a Fermi estimate of the brain's processing power, and Bostrom's estimates are not significantly different from those others have arrived at by similar methods.

ctl 11y ago
You’re right. I didn’t read the paper very carefully, and was myopically focused on the emulating-a-real-brain AI strategy. As in, let’s slice up a real human brain, map the neurons and synapses, and then simulate them as faithfully as possible. To do that you need a great deal of fidelity in your simulations of neurons, which are enormously complex. But there is an argument to be made that neuronal complexity is incidental to the brain’s overall “computational capacity”; that you could replace the neurons in a human brain with much simpler nodes and still end up with a functional intelligence, after sufficient rewiring. I don’t think that claim is obvious, but it’s definitely possible. And if it’s true, you can have human-level intelligence for 10^19 ops, given suitable software. So I apologize for my post. It was over the top and unfair. All that said, I still disagree with Bostrom’s conclusions. I think he enormously understates the difficulty of creating intelligent software, if we’re not just copying an existing brain.

reasonattlm 11y ago

Interestingly, recent research suggests synaptic variability does come to about 5 bits. http://www.salk.edu/news-release/memory-capacity-of-brain-is-10-times-more-than-previously-thought/ http://www.salk.edu/news-release/memory-capacity-of-brain-is... “We were amazed to find that the difference in the sizes of the pairs of synapses were very small, on average, only about eight percent different in size. No one thought it would be such a small difference. This was a curveball from nature.” Because the memory capacity of neurons is dependent upon synapse size, this eight percent difference turned out to be a key number the team could then plug into their algorithmic models of the brain to measure how much information could potentially be stored in synaptic connections. It was known before that the range in sizes between the smallest and largest synapses was a factor of 60 and that most are small. But armed with the knowledge that synapses of all sizes could vary in increments as little as eight percent between sizes within a factor of 60, the team determined there could be about 26 categories of sizes of synapses, rather than just a few. “Our data suggests there are 10 times more discrete sizes of synapses than previously thought.” In computer terms, 26 sizes of synapses correspond to about 4.7 “bits” of information. Previously, it was thought that the brain was capable of just one to two bits for short and long memory storage in the hippocampus.

api 11y ago

I formally studied biology not CS, partly out of an interest in AI. Everyone who thinks superintelligence or even just human or higher-animal level intelligence is right around the corner needs to study genomics, proteomics, molecular biology, and neuroscience. Study them with an open mind and think about what's really going on. A neuron is not a switch. A neuron is an organism. It contains a gene regulatory network more complex than the entire network topology of Amazon's entire web services stack, and that's just looking at the aspects of gene regulation and enzyme (a.k.a. nanomachine) operation that we understand. There are about 100 billion of these in the brain and every one of them is running in parallel and communicating constantly. There are also about 10 glial cells for every one neuron, and glia are involved in neural computation in ways we know are there but don't yet fully understand. (Seems to be related to longer term regulation of synapse behavior, etc.) Each glial cell also contains a massive gene regulatory network and so on. The CS and AI fields suffer from a lot of Dunning-Kreuger effect when they talk about biology. The level of processing power and the parallelism that's going on in the brain of a living thing is simply mind numbing. It's as incredible as the sense you get of the scale of the universe when looking at the Hubble Deep Field. Our present-day computers are toys. We are not even close. It would at least take advances equivalent to the ones that took us from vacuum tube ENIAC to here. Edit: I don't write off superintelligence categorically though. I think we could achieve forms of it not through pure AI but by deeply augmenting biological intelligence. Genetic and biochemical performance enhancement could also play a role. Imagine having more working memory, perfect motivational control, the ability to regulate your own desire/motivational structure, and needing only a few hours of sleep. Cyborg superintelligence is a possibility in the foreseeable future and it does raise issues similar to those the superintelligence folks raise. So I don't dismiss an intelligence explosion. I just very strongly doubt it would be purely solid state.

baddox 11y ago
> A neuron is an organism. It contains a gene regulatory network more complex than the entire network topology of Amazon's entire web services stack, and that's just looking at the aspects of gene regulation and enzyme (a.k.a. nanomachine) operation that we understand. Can you give some more details about this? How are you quantifying the complexity of a neuron and of the AWS stack?

sawwit 11y ago

In a recent estimate of the bits per synapse they found it to be an order of magnitude higher than previous estimates: http://www.eurekalert.org/pub_releases/2016-01/si-mco012016.php http://www.eurekalert.org/pub_releases/2016-01/si-mco012016....

tim333 11y ago

Bostrom's paper is based on Hans Moravec's thinking and Moravec's paper is pretty well argued http://www.transhumanist.com/volume1/moravec.htm http://www.transhumanist.com/volume1/moravec.htm Bostrom as a philosopher may be fuzzy on processing power but Moravec who was actually building robots has a pretty good grasp.