5 ms·
I disagree they will become commodities because most of my use cases are more sensitive to accuracy than cost. We typically have usage volume that isn't absurd
by TimPC 2y ago
I disagree they will become commodities because most of my use cases are more sensitive to accuracy than cost. We typically have usage volume that isn't absurd and for large enterprise customers our LLM budget is a rounding error. Meanwhile our product saves them many hours of a data engineer. If we can pay double to get a 10% performance boost we will do so gladly. You can already see this in LLM pricing where they have cheap models that deliver low performance a My bet is that 80% of profits will be made on the workloads that are sensitive to accuracy and workloads where running an LLM at all gets you most of the benefit will become very commoditized.