7 ms·
Neural network chip built using memristors
- noobiemcfoob 11y agoI have nothing to say but that this is a cool application of memristors. My graduate research is in cognitive computing and the thought of using circuitry to represent the synaptic weights as opposed to hardware/software based adders and multipliers is pretty awesome.
- superfx 11y agoLink to original paper: http://rdcu.be/cJ0x http://rdcu.be/cJ0x
- forketta 11y agohttp://arxiv.org/abs/1412.0611 http://arxiv.org/abs/1412.0611
- Balgair 11y agoTHANK YOU! Also, Dec 1 2014, this has been out a while then
- walterbell 11y agoMore recent papers at https://scholar.google.com/scholar?&q=leon+chua+memristor https://scholar.google.com/scholar?&q=leon+chua+memristor
- andyl 11y ago"Even on a 30 nm process, it would be possible to place 25 million cells in a square centimeter, with 10,000 synapses on each cell. And all that would dissipate about a Watt." Wow - seems like a lot. Human brain by comparison (sourced by google): - 12 watts - 100 billion neurons - 1000 trillion connections Computing with memsisters is going to be very interesting.
- deleted 11y ago[deleted]
- alphydan 11y agoput that on a 20cm x 30cm surface (laptop) ... and you have 25m x 600 = 15bn cells, 150 trillion synapses using 600W. If they can ever fabricate such a thing ... those neural networks are going to compute some scary stuff!
- venomsnake 11y ago> If they can ever fabricate such a thing ... those neural networks are going to compute some scary stuff! Do we have to pay extra for Austrian accent?
- thangalin 11y agoThe human brain has between 100 and 500 trillion synapses and consumes a lowly 12 watts. (In contrast, a 12.6 megawatt supercomputer, in 2013, took 40 minutes to simulate one second of biological brain activity.) The article cites 250 billion synapses per watt. For the same 12 watts as a human brain eats up, a set of memristors could simulate three trillion synapses. A cat, in comparison, has 10 trillion. To get 100 trillion synapses, multiply those 12 watts by 33.3 to get 400 watts (the draw of nearly seven 60-watt incandescent bulbs). Since one watt bags us 250 billion synapses and 400 watts is equivalent to a memristor-based human brain, then 400 cm^2 is the area needed to emulate the meekest of human minds. That's nearly the same area as half of a medium Domino's pizza. Certainly is... food for thought.
- e12e 11y agoThe thought of a drone with the intelligence of a cat is a scary thought... The numbers are interesting, though. 400 square centimetres sounds to me to be in the ballpark of a human brain (accounting for several layers).
- rbanffy 11y ago> The thought of a drone with the intelligence of a cat is a scary thought... And I immediately imagined drones going around the landing gears of bigger planes demanding their attention...
- deleted 11y ago[deleted]
- CamperBob2 11y agoLemme know when I can order a few on cut tape from DigiKey.
- badsock 11y agoI know it's not what you're asking, but for the record you can buy memristor-memory-based microcontrollers in single quantities from Mouser today. The reason I mention it is that memristors are being accused of being vaporware when really they're in production already.
- Balgair 11y agoLINK! You just can't say that and not provide a link to the catalog for us!
- leohutson 11y agohttp://www.mouser.com/ProductDetail/Panasonic/MN101LR05DXW/?qs=OeBdveGBEcQzg65tbPujiw%3D%3D http://www.mouser.com/ProductDetail/Panasonic/MN101LR05DXW/?...
- lowglow 11y agoFor anyone interested: http://en.wikipedia.org/wiki/Resistive_random-access_memory http://en.wikipedia.org/wiki/Resistive_random-access_memory
- bloaf 11y agoI think this is it: http://www.mouser.com/Search/Refine.aspx?Keyword=reram http://www.mouser.com/Search/Refine.aspx?Keyword=reram
- p1esk 11y agoMemristor crossbars are exciting even outside of neural network applications: it can be used as a very dense, non-volatile memory. If this design can be scaled up, it could potentially replace both flash memory storage, and RAM.
- Balgair 11y agoThese things are really not well respected for what they can do. IBM introduced their TrueNorth tech late last year. Those used a 'neural' design for the chips to overcome the von Newman barriers for computer design. Along the way, the TrueNorth chips also reduced power consumption by a LOT. However, those designs are still digital. With a memristor in the TrueNorth set-up you can have a similar system for input, processing, storage, and output in an analog system. I feel that I need to emphasize that. The memristor is the component that will easily allow for analog logic to occur at digital speeds and with digital logic type systems (very grossly speaking). What these little guys can do is under-sold.
- p1esk 11y agoThere have been dozens of analog neuromorphic chips built in the last 30 years. Latest ones use floating gate transistors for synapses. Memristors, in theory, are better devices that flash memory (faster, lower power, more dense), however that's just in theory. In practice, they are very hard to scale. This crossbar is 12x12. No one knows how to build anything much larger than that, on a mass scale.
- themeek 11y agoThis is one of the exotic devices in DARPA's UPSIDE competition for exascale computing. This initiative seeks to find non-state (non-transistor) based approaches to computation: exploitation of nanoscale response properties of discrete components to perform some restricted, non-binary, forms of computation. Essentially, exotic ways to abuse silicon lithography to get analog computation. The idea, and this can be seen on DARPA's slides (http://www.darpa.mil/workarea/downloadasset.aspx?id=2147485714 http://www.darpa.mil/workarea/downloadasset.aspx?id=21474857...), is to get computation that is several orders of magnitude higher for their specialized sets of problems than what can theoretically be reached by traditional computing models even if Moore's law continues. DARPA would like to first apply this technology to ARGUS drone systems (https://www.youtube.com/watch?v=QGxNyaXfJsA https://www.youtube.com/watch?v=QGxNyaXfJsA) and related technology because streaming video can't be done to the ground, tracking and decision making must be done on board - yet traditional processing platforms can only track a few orders of magnitude fewer targets that what the military would like. In a more advanced phase, if memristor or coupled oscillator (etc) approaches to building inference models become possible, then programs written in DARPA's other initiative (Probablistic Programming) could be programmed into these exotic solid state devices to compute in a way more analogous to today's generic computation. And indeed, eventually the adoption of Probablistic Programming will train programmers to write code for quantum computers - while more complicated, replacing Probablistic Programming's PDFs with probability amplitudes almost get one there. I hope to see more journalistic coverage of some of the other exotic devices.
- chm 11y agoHow exactly is a memristor not a state device? And about journalistic coverage... you seem to be knowledgeable about these programs, so there's an opportunity for you :)
- Synaesthesia 11y agoA lot of it is shrouded in secrecy I'm afraid, unless you're doing the research. I'm also very interested in memristors, I think it's a quantum leap forward for computing, in many respects. But there's very little information one can get out there. Would love to know where I can find out more.
- meric 11y agoA neural network chip semi-conductor startup: http://brainchipinc.com/technology/ http://brainchipinc.com/technology/ They "backdoor listed" on to an Australian mining company, share price went from 1 cent to 27 cents: https://www.google.com/finance?cid=11163357 https://www.google.com/finance?cid=11163357 Valued at $57m.
- modeless 11y agoMan, why are all the silicon people so fixated on spiking nets? Maybe in 20 years once we figure out how the brain works they'll be great but if you built a convnet chip instead then it could be smashing records in real problems like speech recognition, translation, image identification, etc, today. Where are the convnet chip startups?
- p1esk 11y agoFirst of all, what makes you think a hardware convnet will perform better than a software convnet? Second, you can implement a convnet with a spiking circuit: http://link.springer.com/article/10.1007%2Fs11263-014-0788-3 http://link.springer.com/article/10.1007%2Fs11263-014-0788-3
- modeless 11y agoThat paper does not implement training, only testing. Low-power testing is good to have e.g. for mobile applications, but it doesn't increase the capabilities of our algorithms. What we need to advance the field is faster training of larger nets. That's where the really interesting applications will be discovered. A convnet-optimized chip could clearly be much faster and more power efficient than a convnet running on a CPU or GPU. The move from CPU to GPU brought a 10x speedup already, but GPUs are hardly ideal for running convnets. For one thing, they have tons of graphics-specific hardware that's useless for convnets and could just be deleted in a convnet chip. For another, GPUs are much more flexible than necessary for convnets. The main operation you need to perform is convolution and you could make fixed-function convolution units that would be much more power and area efficient than generalized GPU shader cores. For yet another thing, there's no reason to believe that 32-bit IEEE 754 floating point is the best power/precision tradeoff for convnets. I'm willing to bet that you could go much lower. You could even experiment with approximate arithmetic; 0.5 ULP precision is probably not necessary.
- FractalNerve 11y agoI'm very excited about this! Is there any possibility for me to work in a company working on similar stuff? That would make a dream come true for me! I live in Germany and would love to write my masters thesis a related topic! This is so amazing! Btw. I have found an HP Invent sign in my town, but the security guard didn't answer any questions about it. The only thing he said was that I won't find any address or telephone number for it. That made me curious, because HP is working on a memristor based Computer, but I doubt that they produce it in Germany.
- p1esk 11y agoAre you an EE? What exactly would you like to work on?
- FractalNerve 11y agoI've previous EE experience, but I study CS. I'm open for project suggestions and would love to do research in self-assembly for mass fabrication and study/develop AI Models.
- p1esk 11y agoWell, the obvious project suggestion is to read about challenges of building a larger crossbar, then work on overcoming those challenges. Literature list is provided in the paper. This type of work all about mass fabrication, but has nothing to do with AI models. Which direction you want to go?
- FractalNerve 11y agoI think given my background I'm a better match for AI, than for mass fabrication. That's what I'd really enjoy working on.
- chimtim 11y agoThis looks cool but I'm somewhat skeptical. I would be more interested in seeing what problem the system solves better or decently (say even MNIST) rather than how it was built using memristors. There is a lesson from IBM trying to mimic a rat's brain -- that is you try to solve a problem rather than just burn power.
- p1esk 11y ago>>>I would be more interested in seeing what problem the system solves better Better than what?
- modeless 11y agoBetter than previous state of the art on a standard machine learning dataset. Check some leaderboards here: http://rodrigob.github.io/are_we_there_yet/build/#datasets http://rodrigob.github.io/are_we_there_yet/build/#datasets
- p1esk 11y agoWhat are you talking about? They build a 12x12 crossbar. The best you can do with it is to implement a single layer perceptron to classify 3x3 pixel patterns. Once they figure out how to scale it up, they will implement a larger network.
- vegedor 11y agobetter than anything else, obviously.
- return0 11y agoThe technology sounds very promising but if the goal is to simulate the brain, the ANN models we have today are inadequate. Current evidence suggests that it needs to incorporate dendritic dynamics and , soon, molecular computation.
- neolefty 11y agoIs the goal simulation or functional equivalence? For example, to simulate a horse, is it necessary to create legs, or is it okay to build a road and use wheels?
- return0 11y agopossibly, we don't know yet, but ANNs are not isomorphic to real neurons. If we are talking about a bottom-up approach to intelligence, we have to opt for realistic simulation. If, on the other hand we knew what intelligence is, we can simulate it any way we like.
- peter303 11y agoMemristers are supposed to be the main memory of HP's future computing project called The Machine. It is supposed to be as fast as register memory and compact as flash.
- Lozi 11y agoThe general theory of memristors, meminductors, memcapacitors of any order (first order memristor is the genuine one invented in 1971 by Professor Leon Chua, however he generalized recently his discover to second, third order memristor, etc.)is published in a paper I wrote with him on september 2014 in International Journal of Bifurcation and Chaos. One can download it freely from the site https://www.researchgate.net/publication/261676241_MEMFRACTANCE_A_MATHEMATICAL_PARADIGM_FOR_CIRCUIT_ELEMENTS_WITH_MEMORY https://www.researchgate.net/publication/261676241_MEMFRACTA...