5 ms·
You'd be surprised how often people in enterprise can be left waiting months to get an API key approved for an LLM provider.
by xfalcox 1y ago
You'd be surprised how often people in enterprise can be left waiting months to get an API key approved for an LLM provider.
- diggan 1y agoAre you saying that it's faster for them to get the hardware to run the weights themselves? Otherwise I'm not sure what the relevancy is.
- ChromaticPanic 1y agoYes some have existing infra
- diggan 1y agoI'm having a somewhat hard time believing a corporation where getting a API key for a LLM service is very difficult, somehow has the (GPU) infrastructure already running for doing the same thing themselves, unless they happen to be a ML corporation, but I don't think we're talking about those in this context.
- oooyay 1y agoNah this is definitely a real scenario. Getting access to public models requires a lot of security review, but proving through Bedrock is much more simple. I may be spoiled in having worked for companies that have ML departments and developer XP departments though.
- diggan 1y agoNot sure Bedrock counts as self-hosting though, isn't it a managed service Amazon provides? > I may be spoiled in having worked for companies that have ML Sounds likely, yeah, how many companies have ML departments today? DS departments seem common, but ML i'm not too sure about
- deleted 1y ago[deleted]
- fourthark 1y agoA lot of companies think they do
- achierius 1y agoNo, this is very real. One reason why this can happen: a company has elaborate processes for protecting their internal data from leaking, but otherwise lets engineers do what they want with resources allocated to them.
- pegasus 1y agoUnless they are already in the possession of such hardware (like an M3 mac, for example).