4 ms·
SemiAnalysis estimates their profit margin to be 70%. To be losing money on inference implies that their costs are almost 4X higher than SemiAnalysis has calc
by bryanlarsen 12d ago
SemiAnalysis estimates their profit margin to be 70%. To be losing money on inference implies that their costs are almost 4X higher than SemiAnalysis has calculated. That's not credible.
- p-e-w 12d agoI don’t see how they could credibly estimate inference costs without knowing the model size.
- FuckButtons 12d agoBut we do have a reasonable estimate of model size.
- nbardy 11d agoYou can estimate the model size by looking at tokens per second and comparing to open source models