7 ms·
Yeah that’s definitely the smarter buy if you want to just have models running quickly. But the cost of 2 p150 and a 4090 was <$5000 for me. The main issue is
by ydj 3mo ago
Yeah that’s definitely the smarter buy if you want to just have models running quickly. But the cost of 2 p150 and a 4090 was <$5000 for me.
The main issue is the immature software, and somewhat baroque way of writing kernels. Please, buy one and join us.
- arjie 3mo agoWere you able to connect the two P150 using the qsfp-dd cable? They only sell 4x and 8x topologies so I’m curious if that worked for you. Are you able to run them tensor parallel?
- ydj 3mo agoYeah, I’m doing TP with two cards. The topology is configured based on yaml files, and if you are not using a predefined config you can just create a new config with your topology. I’m not even using a 800G cable since they are expensive and I don’t think I need the bandwidth, opting for 400G instead. This just needs a config change for the number of Ethernet links it uses internally. (Apparently these cables are just many 200G links put together.)
- arjie 3mo agoBrilliant, thank you. Maybe I'll get a couple in a bit.