5 ms·

Can you not use cross-region inference?

• smcleod 2y ago
  90% of our customers do not allow this due to data sovereignty. Bedrock here is lagging so far behind several customers assume AWS simply aren't investing here anymore - or if they are it's an afterthought - and a very expensive one at that. I've spoken with several account managers and SAs and they seem similarly frustrated with the continual response from above that useful models are "coming soon". You can't even BYO models here, we usually end up spinning up big ol' GPU EC2 instances and serving our own, or for some tasks running locally as you can get better openweight LLMs.
  • dheerkt 2y ago
    Hmm interesting, didn't realize that data sovereignty requirements were so stringent. Wonder how other cloud providers are doing in this sense considering GPU shortages across the board.