We had this same challenge in our own app builder, we ended up creating an internal LLM proxy with per-sandbox virtual keys (which the proxy maps to the real ke

We had this same challenge in our own app builder, we ended up creating an internal LLM proxy with per-sandbox virtual keys (which the proxy maps to the real key + calculates per-sandbox usage), so even if the sandbox leaks its key it doesn't impact anything else.