5 ms·
Hardware requirements? I think I can hit the memory bandwidth building from parts I have in my house. Maybe even 2x. Asking for fun not profit.
by rainclouds 2y ago
Hardware requirements? I think I can hit the memory bandwidth building from parts I have in my house. Maybe even 2x. Asking for fun not profit.
- morphle 2y agoI'd love to visit your house then. You have 768-1400 GB DRAM with 6000 GB/s memory bandwidth? Nice house. In my house I currently have almost 900 GB/S memory bandwidth in aggregate but only 132 GB total DRAM.
- rainclouds 2y agoI’ve got a terabyte of DDR4, and a bunch of old thread rippers. They can take 256 each and 8 channel.
- morphle 2y agoYep, that's the right stuff. Now simply cross-connect all the free PCIe lanes of all thread rippers and you have a nice cluster for LLMs.