My hope is that hardware improvements (better node densities every 2-3 years, better designs, etc) will pick up the majority of the savings for these companies

My hope is that hardware improvements (better node densities every 2-3 years, better designs, etc) will pick up the majority of the savings for these companies in the future, assuming LLM performance starts to taper off with diminishing returns.