7 ms·
The comparison doesn't really hold. He is comparing energy spend during inference in humans with energy spend during training in LLM's. Humans spend their lif
by cynusx 2y ago
The comparison doesn't really hold.
He is comparing energy spend during inference in humans with energy spend during training in LLM's.
Humans spend their lifetimes training their brain so one would have to sum up the total training time if you are going to compare it to the training time of LLM's.
At age 30 the total energy use of the brain sums up to about 5000 Wh, which is 1440 times more efficient.
But at age 30 we didn't learn good representations for most of the stuff on the internet so one could argue that given the knowledge learned, LLMs outperform the brain on energy consumption.
That said, LLM's have it easier as they are already learning from an abstract layer (language) that already has a lot of good representations while humans have to first learn to parse this through imagery.
Half the human brain is dedicated to processing imagery, so one could argue the human brain only spend 2500 Wh on equivalent tasks which makes it 3000x more efficient.
Liked the article though, didn't know about HNSW's.
Edit: made some quick comparisons for inference
Assuming a human spends 20 minutes answering in a well-thought out fashion.
Human watt-hours: 0.00646
GPT-4 watt-hours (openAI data): 0.833
That makes our brains still 128x more energy efficient but people spend a lot more time to generate the answer.
Edit: numbers are off by 1000 as I used calories instead of kilocalories to calculate brain energy expense.
Corrected:
human brains are 1.44x more efficient during training and 0.128x (or 8x less efficient) during inference.
- greenthrow 2y agoThe article is a bit of a stretch but this is even more of a stretch. Humans can do way more than an LLM, humans are never in only learning mode, our brains are always at least running our bodies as well, etc.
- glenstein 2y agoExactly right - we are obviously not persistently in all-out training mode over the course of our lifetimes. I suppose they intended that as a back-of-the-envelope starting point rather than a strict claim however. But even so, gotta be accountable to your starting assumptions, and I think a lot changes when this one is reconsidered.
- jryan49 2y agoHow about the fact that llm's don't work unless humans generate all that data in the first place. I'd say the llm's energy usage is the amount it takes to train plus the amount to generate all that data. Humans are more efficient at learning with less data.
- Closi 2y agoHumans also learn from other humans (we stand on the shoulders of giants), so we would need to account for all the energy that has gone into generating all of human knowledge in the 'human' scenario too. i.e. not many humans invent calculus or relativity from scratch. I think OP's point stands - these comparisons end up being overly hand-wavey and very dependent on your assumptions and view.
- jryan49 2y agoYes I agree. The whole concept of trying to compare energy usage is incredibly complicated.
- Closi 2y agoI think you would probably have to take into account the full functioning power of a human too. We don't know how to fully operate a human brain when it's fully disconnected from eyes, a mouth, limbs, ears and a human heart.
- wanderingmind 2y agoNot just that the brain of a newborn comes pretrained with billions of years of evolution. There is an energy cost associated with that which must be taken into account
- eloeffler 2y agoThen you must also take that cost into account when calculating the cost of training LLMs, as well as the cost humans operating the devices and their respective individual brain development. LLMs are always an additional cost, never more efficient because they add to the calculation, if you look at it that way.
- Closi 2y agoOnly if we are counting the cost to generate all the inputs to training, and not just the training itself - it just depends on the scope of the analysis. (i.e. taken to the extreme, as humans learn from their environment, do we have to count all energy that has gone into creating the world as we know it?)
- daflip 2y agoIf taken to the extreme I can't help but quote Carl Sagan :-) "If you wish to make an apple pie from scratch you must first invent the universe"
- coldtea 2y agoWell, LLMs also pressupose humans and evolution, since they needed us to create them, so their tally is even higher by definition...
- thfuran 2y agoBrains are only about half a billion years old.
- eru 2y agoAlso our brains and our language are co-optimised to be compatible. ChatGPT has to deal with the languages we already created, it doesn't get to co-adapt.
- freehorse 2y ago> representations for most of the stuff on the internet Yes we have learnt far more complex stuff, ffs.
- londons_explore 2y ago> At age 30 the total energy use of the brain sums up to about 5000 Wh, That doesn't sound right... 30 years * 20 Watts = 1.9E10 Joules = 5300 kWh.
- cynusx 2y agoWhere did you get the 20 Watt from? My number is based on calorie usage
- cynusx 2y agooh ok, I used 400 calories/day and not 400 kcal/day. Yea, then the numbers are off by 1000
- 1992spacemovie 2y agoI respect that you replied to the comment and owned your math error :) The rest of your comment is an interesting observation. Never thought about it starting out at the cal level.
- mirekrusin 2y agoAlso you can't cp human brain.
- marginalia_nu 2y agoYou kinda can do a sort of LoRA though. Reading the right book can not only change what you hold true, but how you think.
- phantompeace 2y agoNot yet, anyway.
- vasco 2y agoWe can clone humans at current level of technology, otherwise there wouldn't be agreements about not doing it due to the ethical implications. Of course its just reproducing the initial hardware and not the memory contents or the changes in connections that happen at runtime.
- mirekrusin 2y agoWell, we know how to make kids, but then cp takes 20 years and rarely works.
- Rinzler89 2y agoThe plot of The Matrix would beg to differ.
- dist-epoch 2y agoFor every calorie a human consumes, hundreds or thousands more are used by external support systems. So yeah, you do use 2000 calories a day, but unless you live in an isolated jungle tribe, vast amounts of energy are consumed on delivering you food, climate control, electricity, water, education, protection, entertainment and so on.
- unyttigfjelltol 2y agoIncluding support from ChatGPT. It really is a comparison of calories without ChatGPT and calories with, and that gets to the real issue of whether ChatGPT justifies its energy intensity or not. History suggests we won't know until the technology exits the startup phase.
- b112 2y agoBy that metric, the electricity is only part of it. The cost of building the harsware, the cost of building the roof and walls for the datacentre, the cost of clearing the land, cost of humans maintaining the hardware, the cost of all the labour making the linux kernel, libc6, etc, etc. Lots of additionals here too.
- bamboozled 2y agoIt's almost like...nothing exists in a vacuum.
- greenthrow 2y agoAre you going to include all the externalities to build and power the datacenters behind LLMs then? Because i guarantee those far outweigh what it takes to feed one human.
- CuriouslyC 2y agoYou're doing apples and oranges. Humans who spend a long time doing inference have not fully learned the thing being inferred - unlike LLMs, when we are undertrained, rather than a huge spike in error rate, we go slower. When humans are well trained, human inference absolutely destroys LLMs.
- cheema33 2y ago> When humans are well trained, human inference absolutely destroys LLMs. This isn't an apt comparison. You are comparing a human trained in a specific field to an LLM trained on everything. When an LLM is trained with a narrow focus as well, human brain cannot compete. See Garry Kasparov vs Deep Blue. And Deep Blue is very old tech.
- CuriouslyC 2y ago1. Deep blue isn't a LLM. I don't care how well you train a LLM, it's not going to be more efficient than an optimally trained human, not even close. It's actually arrogant as hell to assume that we can achieve a higher level of energy efficiency than billions of years of evolution, particularly so early in the game. 2. Chess is a closed form system with a finite and relatively small number of position compared with the real world.
- eru 2y ago> It's actually arrogant as hell to assume that we can achieve a higher level of energy efficiency than billions of years of evolution, particularly so early in the game. You are right that LLMs are still far off from the performance of the human brain. Both in absolute terms, and also relative to the power used. However, I don't see anything arrogant here. We have lots of machines that can do many tasks more energy efficient (and better) than humans. Both mechanical and intellectual tasks.
- CuriouslyC 2y agoIt's not arrogance to think you can create a tool that does one thing the brain does better than the brain for less power. It's arrogance to think that you can do everything the brain does for less power. Living organisms have been relentlessly honed for the ability to efficiently solve varied problems across ~10^40 experiments over the age of the earth. If some marginally intelligent monkeys think they can build an error corrected, digital system that encompasses all of that functionality while using less power, I'd say that's obviously arrogance, particularly if it hasn't been the subject of a civilizational drive for a few millennia already.
- bamboozled 2y agoHumans spend their lifetimes training their brain I don't think this is true personally, ideally as children, we spend out time having fun and learning about the world is a side effect. This borg like thinking applied to intelligence because we have LLMs is unusual to me. I learned surfing through play and enjoyment, not through training like a robot. We can train for something with intention, but I think that is mostly a waste of energy, albeit necessary on occasion.
- Jensson 2y ago> we spend out time having fun and learning about the world is a side effect What do you think "play" is? Animals play to learn about themselves and the world, you see most intelligent animals play as kids with the play being a simplification of what they do as adults. Human kids similarly play fight, play build things, play cook food, play take care of babies etc, it is all to make you ready for an adult life. Playing is fun since playing helps us learn, otherwise we wouldn't evolve to play, we would evolve to be like ants that just work all day long if that was more efficient. So the humans who played around beat those who worked their ass off, otherwise we would all be hard workers.
- bamboozled 2y agoBut I just play because it's fun, I roll dice for fun, are you trying to tell me all this is a secret front for "training" ?
- Jensson 2y agoFun is your brain rewarding you for something it thinks is practice on a useful skill. You get bored once you mastered it enough. Some people continue playing a game even when it stops being fun, they are addicted to the reward mechanism in the game, and now the brain thinks that playing the game is a good way to work and provide for itself. I don't call that "play", its work, just not productive work. Why is dice fun? Because your brain wants to map the pattern of the dice, trying to figure out how to get good rolls. You see that in most dice players, they develop a lot of superstition about what is good and bad dice, or how they always roll bad in critical moments etc. I'd assume that is from nature where you try to figure out what is a good nut to crack or where to find prey etc, basically a way to figure out useful patterns from random events.
- pama 2y agoThanks. So after your corrected energy estimate and more reasonable assumptions it appeaars that the clickbaity title of the article is off by more than 7 orders of magnitude. With the upcoming NVidia inference chips later this year it will be off by another log unit. It is hard for biomatter to compete with electrons in silicon and copper.
- bognition 2y agoIf we’re going to exclude the cortical areas associated with vision, you also need to exclude areas involved in motor control and planning. Those also account for a huge percent of the total brain volume. We probably need to exclude the cerebellum as well (which is 50% of the neurons in the brain) as it’s used for error correction in movement. Realistically you probably just need a few parts of the lambic system. Hippocampus, amygdala, and a few of the deep brain dopamine centers.
- philipov 2y agoA lot of our cognition is mapped to areas that are used for something else, so excluding areas simply because they are used for something else is not valid. They can still be used for higher-level cognition. For example, we use the same area of the brain to process the taste of disgusting food as we do for moral disgust.
- bufferoverflow 2y agoAlso, human brains come pre-trained by billions of years of evolution. It doesn't start as a randomly-connected structure. It already knows how to breathe, how to swallow, how to lean new things.