7 ms·

I would really like it if an LLM tool would show me the power consumption and environmental impact of each request I’ve submitted.

preciz 1y ago
And each toilet flush you make should also have a Co2 calculation which should go against your daily carbon allowance.
- evrimoztamur 1y ago
  Spending drinking water for toilet flushes is indeed a problem. Perhaps not CO2 measurements directly, but informing people in general of how much high quality water is wasted on flushes alone will hopefully bring more momentum into more efficient flushing mechanism and introducing grey water systems to new and old buildings alike. Good idea!
- j-pb 1y ago
  people downvote your sarcasm, but if you do the calculations you're kinda right. 1Kg of Beef costs: - The energy equivalent of 60.000 ChatGPT queries. - The water equivalent of 50.000.000 ChatGPT queries. Applied to their metric Mistral Large 2 used: - The water equivalent of 18.8 Tons of Beef. - The CO2 equivalent of 204 Tons of Beef. France produces 3836 Tons of Beef per day, and one large LLM per 6 months. So yeah, maybe use ChatGPT to ask for vegan recipes. People will try to blame everything else they can get a hold on before changing the stuff that really has an impact, if it means touching their lifestyle. The LLMs are not the problem here.
  - bluefirebrand 1y ago
    The difference is that food is important and live-giving and LLMs are a very fancy magic 8-ball
    - j-pb 1y ago
      You don't need beef, beef is a lifestyle choice. I use LLMs to do all of my coding these days, it's certainly more essential for feeding me than beef.
      - deleted 1y ago
        [deleted]
      - leksak 1y ago
        Also a lifestyle choice
        j-pb 1y ago
        Yes but one with a much much much smaller impact as we just demonstrated. This is exactly the kind of cognitive dissonance in people that I meant. You literally see the math and go "but I like my meat, why should I give that up if you got your AI". Because, as I just demonstrated, my AI takes a infinitesimal fraction of your meat. It literally takes you only going vegan for a day to offset your entire AI usage of a year.
        jrflowers 1y ago
        This is spot on because there can’t be two issues that exist simultaneously. There can only be one thing that wastes enormous amounts of energy and that thing is beef
        j-pb 1y ago
        You can try to misconstrue and ridicule the argument, but that won't change the math that if you have one thing that causes 1 unit of damage, and another thing that causes 100.000 units of damage, then for all intents and purposes the thing that produces 1 unit of damage is irrelevant. And any discussion that tries to frame them as somewhat equally important issues is dishonest and either malicious or delusional. My guess, as I've expressed earlier in the comment chain, is that it's emotionally easier for people to bike-shed about the 0.01% of their environmental impact, than to actually tackle things that make up 20%. And no it's not only beef (which is a stand-in for meat and diary), another low hanging fruit is also transport, like switching your car for a bike. But switching from meat and diary to a vegan diet would reduce up to 20% of your personal environmental impact, in terms of CO2. And about 80-90% of rainforest deforestation is driven directly or indirectly by livestock production. So it's simply the easiest most impactful thing everyone can do. (Switching your car for a bike isn't possible for people in rural areas for example.)
  - plants 1y ago
    Those are incredible stats. As a vegan who uses LLMs at work frequently, I would love to have the source as well :)
    - j-pb 1y ago
      - https://mistral.ai/news/our-contribution-to-a-global-environmental-standard-for-ai - https://www.statista.com/statistics/1201677/greenhouse-gas-emissions-of-major-food-products/ - https://www.statista.com/chart/9483/how-thirsty-is-our-food/ - https://blog.samaltman.com/the-gentle-singularity - https://x.com/sama/status/1890820962993533232 You got to do the calculations yourself though.
      - plants 1y ago
        Wow, thanks. I’m even coming up with 500K chatGPT queries for the amount of energy consumed as a KG of beef, though I might have moved a decimal place somewhere - feel free to check my math :) “average query uses about 0.34 watt-hours of energy” - or 0.00034MWH Using this calculator: https://www.epa.gov/energy/greenhouse-gas-equivalencies-calculator#results https://www.epa.gov/energy/greenhouse-gas-equivalencies-calc... - in my zip, 0.0002KG of CO2 per MWH. (Though, I suppose it depends more on the zip where they’re doing inference, however this translation didn’t seem to vary much when I tried other zips) Then, 99.48KG/0.0002KG= 497,400 chatGPT queries worth of CO2 per KG of beef? Thanks for sharing!
        j-pb 1y ago
        `0.00034MWH`should probably be kilo not mega, but I think you still did it correctly :D I think I used Btu (thermal units) in my calculation, so I only calculated actual energy expenditure for my 60.000 queries result. But you are right, a better metric might be to use CO2-equivalent, because cows emit a lot of methane, whereas chatbots don't. 99.48KG / 0.228kg/kq = 436,315 queries per kg of beef. Yup checks out!
    - JimDabell 1y ago
      This is a good article on the subject: > Using ChatGPT is not bad for the environment — https://andymasley.substack.com/p/individual-ai-use-is-not-bad-for https://andymasley.substack.com/p/individual-ai-use-is-not-b... He’s done some good followup articles as well: https://andymasley.substack.com/s/ai-and-the-environment https://andymasley.substack.com/s/ai-and-the-environment
- jrflowers 1y ago
  This is a good point because being curious about energy usage is the same thing as advocating for an imaginary rule about energy usage
- stonogo 1y ago
  Toilets are already labeled with their usage rate.
- aziaziazi 1y ago
  I don’t flush my toilet, I Kildwick [0] but J-pd has a more interesting comparaison [0] https://www.kildwick.com/ https://www.kildwick.com/ [1] https://news.ycombinator.com/threads?id=j-pb https://news.ycombinator.com/threads?id=j-pb
jiehong 1y ago
Let’s call it GreenOps
jeffbee 1y ago
That would be ... thousands of time less useful than giving you the same information at the motor fuel pump. Unfortunately this isn't one of those situations where every little bit counts. There are 2 or 3 things you can do to reduce your environmental impact and not using chatbots isn't one of the things.
kingstnap 1y ago
You can assume the API price is roughly proportional to electricity usage. If you buy $10 in tokens, that probably folds into ~$3 to $5 dollars in electricity. Which would be around 30 to 90 kWhr in electricity. Depending on the source, it could be anywhere from ~500g/kWhr (for natural gas) and ~24g/kWhr for hydroelectric. It's a really wide spread, but I'd say for $10 in tokens, you'd probably be in the neighbourhood of 1 kg to 40 kg of emissions. What's a good thing is that a lot of the spread comes from the electricity source. So if we can get all of these datacenters on clean energy sources it could change emissions by over an order of magnitude compared to gas turbines (like XAi uses).
- dijit 1y ago
  I don’t think that you can make this assumption. People are selling AI at a loss right now.
- chessgecko 1y ago
  if you bought a nvidia h100 at wholesale prices (around $25k) and ran it 24/7 at commercial electric rates (lets say $0.1 per kwh), then it would take you over 40 years to spend the purchase price of the gpu in electricity. Maybe bump it down to 20 for data center cooling. I don't think the cost of the ai is close to converging to the price of power yet. Right now its mostly the price of hardware and data center space minus subsidies.

preciz 1y ago

And each toilet flush you make should also have a Co2 calculation which should go against your daily carbon allowance.

evrimoztamur 1y ago
Spending drinking water for toilet flushes is indeed a problem. Perhaps not CO2 measurements directly, but informing people in general of how much high quality water is wasted on flushes alone will hopefully bring more momentum into more efficient flushing mechanism and introducing grey water systems to new and old buildings alike. Good idea!
j-pb 1y ago
people downvote your sarcasm, but if you do the calculations you're kinda right. 1Kg of Beef costs: - The energy equivalent of 60.000 ChatGPT queries. - The water equivalent of 50.000.000 ChatGPT queries. Applied to their metric Mistral Large 2 used: - The water equivalent of 18.8 Tons of Beef. - The CO2 equivalent of 204 Tons of Beef. France produces 3836 Tons of Beef per day, and one large LLM per 6 months. So yeah, maybe use ChatGPT to ask for vegan recipes. People will try to blame everything else they can get a hold on before changing the stuff that really has an impact, if it means touching their lifestyle. The LLMs are not the problem here.
- bluefirebrand 1y ago
  The difference is that food is important and live-giving and LLMs are a very fancy magic 8-ball
  - j-pb 1y ago
    You don't need beef, beef is a lifestyle choice. I use LLMs to do all of my coding these days, it's certainly more essential for feeding me than beef.
    - deleted 1y ago
      [deleted]
    - leksak 1y ago
      Also a lifestyle choice
      - j-pb 1y ago
        Yes but one with a much much much smaller impact as we just demonstrated. This is exactly the kind of cognitive dissonance in people that I meant. You literally see the math and go "but I like my meat, why should I give that up if you got your AI". Because, as I just demonstrated, my AI takes a infinitesimal fraction of your meat. It literally takes you only going vegan for a day to offset your entire AI usage of a year.
        jrflowers 1y ago
        This is spot on because there can’t be two issues that exist simultaneously. There can only be one thing that wastes enormous amounts of energy and that thing is beef
        j-pb 1y ago
        You can try to misconstrue and ridicule the argument, but that won't change the math that if you have one thing that causes 1 unit of damage, and another thing that causes 100.000 units of damage, then for all intents and purposes the thing that produces 1 unit of damage is irrelevant. And any discussion that tries to frame them as somewhat equally important issues is dishonest and either malicious or delusional. My guess, as I've expressed earlier in the comment chain, is that it's emotionally easier for people to bike-shed about the 0.01% of their environmental impact, than to actually tackle things that make up 20%. And no it's not only beef (which is a stand-in for meat and diary), another low hanging fruit is also transport, like switching your car for a bike. But switching from meat and diary to a vegan diet would reduce up to 20% of your personal environmental impact, in terms of CO2. And about 80-90% of rainforest deforestation is driven directly or indirectly by livestock production. So it's simply the easiest most impactful thing everyone can do. (Switching your car for a bike isn't possible for people in rural areas for example.)
- plants 1y ago
  Those are incredible stats. As a vegan who uses LLMs at work frequently, I would love to have the source as well :)
  - j-pb 1y ago
    - https://mistral.ai/news/our-contribution-to-a-global-environmental-standard-for-ai - https://www.statista.com/statistics/1201677/greenhouse-gas-emissions-of-major-food-products/ - https://www.statista.com/chart/9483/how-thirsty-is-our-food/ - https://blog.samaltman.com/the-gentle-singularity - https://x.com/sama/status/1890820962993533232 You got to do the calculations yourself though.
    - plants 1y ago
      Wow, thanks. I’m even coming up with 500K chatGPT queries for the amount of energy consumed as a KG of beef, though I might have moved a decimal place somewhere - feel free to check my math :) “average query uses about 0.34 watt-hours of energy” - or 0.00034MWH Using this calculator: https://www.epa.gov/energy/greenhouse-gas-equivalencies-calculator#results https://www.epa.gov/energy/greenhouse-gas-equivalencies-calc... - in my zip, 0.0002KG of CO2 per MWH. (Though, I suppose it depends more on the zip where they’re doing inference, however this translation didn’t seem to vary much when I tried other zips) Then, 99.48KG/0.0002KG= 497,400 chatGPT queries worth of CO2 per KG of beef? Thanks for sharing!
      - j-pb 1y ago
        `0.00034MWH`should probably be kilo not mega, but I think you still did it correctly :D I think I used Btu (thermal units) in my calculation, so I only calculated actual energy expenditure for my 60.000 queries result. But you are right, a better metric might be to use CO2-equivalent, because cows emit a lot of methane, whereas chatbots don't. 99.48KG / 0.228kg/kq = 436,315 queries per kg of beef. Yup checks out!
  - JimDabell 1y ago
    This is a good article on the subject: > Using ChatGPT is not bad for the environment — https://andymasley.substack.com/p/individual-ai-use-is-not-bad-for https://andymasley.substack.com/p/individual-ai-use-is-not-b... He’s done some good followup articles as well: https://andymasley.substack.com/s/ai-and-the-environment https://andymasley.substack.com/s/ai-and-the-environment
jrflowers 1y ago
This is a good point because being curious about energy usage is the same thing as advocating for an imaginary rule about energy usage
stonogo 1y ago
Toilets are already labeled with their usage rate.
aziaziazi 1y ago
I don’t flush my toilet, I Kildwick [0] but J-pd has a more interesting comparaison [0] https://www.kildwick.com/ https://www.kildwick.com/ [1] https://news.ycombinator.com/threads?id=j-pb https://news.ycombinator.com/threads?id=j-pb

jiehong 1y ago

Let’s call it GreenOps

jeffbee 1y ago

That would be ... thousands of time less useful than giving you the same information at the motor fuel pump. Unfortunately this isn't one of those situations where every little bit counts. There are 2 or 3 things you can do to reduce your environmental impact and not using chatbots isn't one of the things.

kingstnap 1y ago

You can assume the API price is roughly proportional to electricity usage. If you buy $10 in tokens, that probably folds into ~$3 to $5 dollars in electricity. Which would be around 30 to 90 kWhr in electricity. Depending on the source, it could be anywhere from ~500g/kWhr (for natural gas) and ~24g/kWhr for hydroelectric. It's a really wide spread, but I'd say for $10 in tokens, you'd probably be in the neighbourhood of 1 kg to 40 kg of emissions. What's a good thing is that a lot of the spread comes from the electricity source. So if we can get all of these datacenters on clean energy sources it could change emissions by over an order of magnitude compared to gas turbines (like XAi uses).

dijit 1y ago
I don’t think that you can make this assumption. People are selling AI at a loss right now.
chessgecko 1y ago
if you bought a nvidia h100 at wholesale prices (around $25k) and ran it 24/7 at commercial electric rates (lets say $0.1 per kwh), then it would take you over 40 years to spend the purchase price of the gpu in electricity. Maybe bump it down to 20 for data center cooling. I don't think the cost of the ai is close to converging to the price of power yet. Right now its mostly the price of hardware and data center space minus subsidies.