7 ms·

Google/Rulesofthumb

53 points by MurizS 3y ago

setheron 3y ago
It also treats SWEs as identical; I wish they were a commodity like hardware.... Unfortunately 1 Jeff Dean SWE != SWE
Jabbles 3y ago
> Given the choice between having a team of 3 people working under me or 1000 dedicated H100s for my work, I’d have to think for a little bit. But the exact numbers are important. An H100 is ~$2/hour [1], so 1000 is $16M/year (24/7). Even if Google gets a massive internal discount that's still way more than 3 people's total cost. If you have to choose between 16 (highly paid, senior) people or 1000 H100s would you have to think about the choice? Then again when you revisit this comment in a few years' time the original comparison may be correct. [1] https://gpus.llm-utils.org/h100-gpu-cloud-availability-and-pricing/ https://gpus.llm-utils.org/h100-gpu-cloud-availability-and-p...
- silvestrov 3y ago
  purchase price != sales price. Very unlikely that Google pays $16M for a single H100. Amazon has it at $44k: https://www.amazon.com/Tesla-NVIDIA-Learning-Compute-Graphics/dp/B0C3XH4QSJ https://www.amazon.com/Tesla-NVIDIA-Learning-Compute-Graphic...
  - inexcf 3y ago
    Better reread the post. It was about renting GPUs. 16M for 1000 H100 per year. So i guess 44k at Amazon would make that 44M Dollar.
- ExoticPearTree 3y ago
  I don’t think the author meant for this to be treated as an absolute, more like food for thought. And if you really insist on 24/7 comparison, you would come to need about 12 people, as the expected productivity per person is of about 6 hrs/day. Factor in the fact that people need vacations, sick days, weekends off… it looks like 1000 H100 might actually be a good trade-off in the very near future.
insecteblond14 3y ago
I wish this internal website existed at my company. This would help throw crazy requests from the business out the window more easily
- phyrex 3y ago
  What keeps you from writing it up?
- bananapub 3y ago
  make a trivial spreadsheet to do it then, or fifty lines of javascript. it's an excellent tool for sharpening thinking early on, as long as people don't come to think of it as anything more than a rule of thumb.
j2kun 3y ago
> costs of fixed assets Minor quibble unrelated to the main content of the post: the measures are not fixed costs of assets, but a blend of depreciation and the operating cost of power usage for those assets. Sort of a regular snapshot of an average daily accounting cost, so to speak (which is reasonable). And this was cost to Google, not taking into account what Google could make by charging Cloud customers for the use of those assets (opportunity cost). My understanding is that the main use of this tool was actually for engineers to give reasonable-ish impact statements for their performance work. I hadn't heard of anyone using it to make serious trade-offs in project planning, since at the level where that matters, the capacity planning teams had more precise costs related to their actual budgets, as well as short term goals like "RAM has a supply chain shock so we can't get any more than X for the next quarter."
- progbits 3y ago
  Yup that's right. Also a pet peeve of mine, people constantly screwed up the units. "10 SWEs" is rate (cost per time), same as "10 TB of RAM", but "SWE-years" is cost (ie dollars). Many design documents use these inconsistently.
the_duke 3y ago
There is nothing new or dystopian about this, a tradeoff between capital expenditure and labour has existed since ... forever. Do I buy expensive tool X or get buy with cheap ones that make the work take longer or be less precise? Do I buy fancy machine Y or pay 10 people to do it manually? The only new development with AI is that this was traditionally limited to relatively manual or repetitive processes, but is now expanding to knowledge work as well. In the medium term the question will be do I just pay for people, or do I spend resources on collecting data and training a model?
- novagameco 3y ago
  Dystopia is when white collar workers are no longer secure in the labor force. Basic economics is when blue collar workers are no longer secure in the labor force
  - ExoticPearTree 3y ago
    You are never secure in your job. Its just that it happens less often to white collar work. You know, for other people you are the other people that things happen to.
    - novagameco 3y ago
      Nobody is ever secure in their particular job, but white collar workers used to be secure in the economy. Now, AI threatens to upend a lot of white collar work without creating any new kinds of jobs to replace them. Anybody who says that AI will create more jobs than it replaces is just divining the future based on the impact of totally unrelated technologies (steam engine, electric motors) on the economy hundreds of years ago
      - ExoticPearTree 3y ago
        Used to, just as I imagine some other jobs were secure in the economy 100 years ago. I personally don’t know what impact AI will have on the job market, but is is not going to be an overnight revolution.
galoisscobi 3y ago
[flagged]
- cqqxo4zV46cp 3y ago
  [flagged]
deleted 3y ago
[deleted]

53 points by MurizS 3y ago

It also treats SWEs as identical; I wish they were a commodity like hardware.... Unfortunately 1 Jeff Dean SWE != SWE

> Given the choice between having a team of 3 people working under me or 1000 dedicated H100s for my work, I’d have to think for a little bit. But the exact numbers are important. An H100 is ~$2/hour [1], so 1000 is $16M/year (24/7). Even if Google gets a massive internal discount that's still way more than 3 people's total cost. If you have to choose between 16 (highly paid, senior) people or 1000 H100s would you have to think about the choice? Then again when you revisit this comment in a few years' time the original comparison may be correct. [1] https://gpus.llm-utils.org/h100-gpu-cloud-availability-and-pricing/ https://gpus.llm-utils.org/h100-gpu-cloud-availability-and-p...

silvestrov 3y ago
purchase price != sales price. Very unlikely that Google pays $16M for a single H100. Amazon has it at $44k: https://www.amazon.com/Tesla-NVIDIA-Learning-Compute-Graphics/dp/B0C3XH4QSJ https://www.amazon.com/Tesla-NVIDIA-Learning-Compute-Graphic...
- inexcf 3y ago
  Better reread the post. It was about renting GPUs. 16M for 1000 H100 per year. So i guess 44k at Amazon would make that 44M Dollar.
ExoticPearTree 3y ago
I don’t think the author meant for this to be treated as an absolute, more like food for thought. And if you really insist on 24/7 comparison, you would come to need about 12 people, as the expected productivity per person is of about 6 hrs/day. Factor in the fact that people need vacations, sick days, weekends off… it looks like 1000 H100 might actually be a good trade-off in the very near future.

insecteblond14 3y ago

I wish this internal website existed at my company. This would help throw crazy requests from the business out the window more easily

phyrex 3y ago
What keeps you from writing it up?
bananapub 3y ago
make a trivial spreadsheet to do it then, or fifty lines of javascript. it's an excellent tool for sharpening thinking early on, as long as people don't come to think of it as anything more than a rule of thumb.

j2kun 3y ago

> costs of fixed assets Minor quibble unrelated to the main content of the post: the measures are not fixed costs of assets, but a blend of depreciation and the operating cost of power usage for those assets. Sort of a regular snapshot of an average daily accounting cost, so to speak (which is reasonable). And this was cost to Google, not taking into account what Google could make by charging Cloud customers for the use of those assets (opportunity cost). My understanding is that the main use of this tool was actually for engineers to give reasonable-ish impact statements for their performance work. I hadn't heard of anyone using it to make serious trade-offs in project planning, since at the level where that matters, the capacity planning teams had more precise costs related to their actual budgets, as well as short term goals like "RAM has a supply chain shock so we can't get any more than X for the next quarter."

progbits 3y ago
Yup that's right. Also a pet peeve of mine, people constantly screwed up the units. "10 SWEs" is rate (cost per time), same as "10 TB of RAM", but "SWE-years" is cost (ie dollars). Many design documents use these inconsistently.

the_duke 3y ago

There is nothing new or dystopian about this, a tradeoff between capital expenditure and labour has existed since ... forever. Do I buy expensive tool X or get buy with cheap ones that make the work take longer or be less precise? Do I buy fancy machine Y or pay 10 people to do it manually? The only new development with AI is that this was traditionally limited to relatively manual or repetitive processes, but is now expanding to knowledge work as well. In the medium term the question will be do I just pay for people, or do I spend resources on collecting data and training a model?

novagameco 3y ago
Dystopia is when white collar workers are no longer secure in the labor force. Basic economics is when blue collar workers are no longer secure in the labor force
- ExoticPearTree 3y ago
  You are never secure in your job. Its just that it happens less often to white collar work. You know, for other people you are the other people that things happen to.
  - novagameco 3y ago
    Nobody is ever secure in their particular job, but white collar workers used to be secure in the economy. Now, AI threatens to upend a lot of white collar work without creating any new kinds of jobs to replace them. Anybody who says that AI will create more jobs than it replaces is just divining the future based on the impact of totally unrelated technologies (steam engine, electric motors) on the economy hundreds of years ago
    - ExoticPearTree 3y ago
      Used to, just as I imagine some other jobs were secure in the economy 100 years ago. I personally don’t know what impact AI will have on the job market, but is is not going to be an overnight revolution.

galoisscobi 3y ago

[flagged]

cqqxo4zV46cp 3y ago
[flagged]

deleted 3y ago

[deleted]