> And training is the main money sink, whereas inference is cheap. False. Training happens once for a time period, but inference happens again and again every

> And training is the main money sink, whereas inference is cheap. False. Training happens once for a time period, but inference happens again and again every time users use the product. Inference is the main money sink. "according to a report from Google, inference now accounts for nearly 60% of total energy use in their AI workloads. Meta revealed something even more striking: within their AI infrastructure, power is distributed in a 10:20:70 ratio among experimentation, training, and inference respectively, with inference taking the lion’s share." https://blogs.dal.ca/openthink/the-hidden-cost-of-ai-conversations-understanding-llm-inference-energy-consumption/ https://blogs.dal.ca/openthink/the-hidden-cost-of-ai-convers...