There are people exploring pushing more computation down to the device layer to avoid shuffling the bits around: https://www.usenix.org/conference/fast23/presen

There are people exploring pushing more computation down to the device layer to avoid shuffling the bits around: https://www.usenix.org/conference/fast23/presentation/yang-zhe https://www.usenix.org/conference/fast23/presentation/yang-z... DPUs are also probably a piece you are missing. Vendors are starting to position these as almost PCIe arbiters with access to both nvme devices and GPUs without the need to involve the main cpu to move data between them.