57 ms·

I have a 1500 dollar machine that can run it at 50 tok/s (3 V100s)

• Dig1t 3mo ago
  How did you buy 3 V100's for $1500??
  • sixdimensional 3mo ago
    Not OP and just guessing, but probably SXM2 GPU modules for the V100. Those can be acquired fairly inexpensively, but there is work to do to get them working together and the V100 has some limitations on the types of models you can run.
    • georgeven 3mo ago
      PCIE 16 gig is <300 dollars these days a piece
  • georgeven 3mo ago
    on eBay you can buy 4 of them for 1000. PCIE 16 gigs.