9 ms·

How do y'all deploy fine-tuned models? I have separate projects for staging and prod, but it doesn't seem like a fine-tune can be shared across projects. Am I

How do y'all deploy fine-tuned models? I have separate projects for staging and prod, but it doesn't seem like a fine-tune can be shared across projects. Am I wrong to split projects by env? Am I expected to run fine-tunes separately per env (surely not)? Am I missing an option to share fine-tunes across projects?

simonw 2y ago
Are you talking about fine-tuned models that you host yourself, or fine-tuned models from a hosted provider like OpenAI? What do you mean by an "env" here?
- underyx 2y ago
  I mean OpenAI hosted fine-tunes (same as referenced in OP.) I have a staging deployment and a production deployment. Ideally anything that I roll out to production, I can try on staging first — including switching from gpt-4o to a fine-tuned gpt-4o. I don't want the production API key to be accessible by my staging app, so I have two separate projects within the OpenAI dashboard. One is called my-app-prod, and the other is my-app-staging. To illustrate the problem further, I also have infrastructure to eval models before changing what production is running. The eval infrastructure also has its own OpenAI project, so that I can set separate billing limits on running evals. Any fine-tuned model needs to be benchmarked first, but again, I'm not sure how to make the same fine-tune available to both the eval project, and the production app project.
  - john_allard 2y ago
    Hey, engineer on the OpenAI fine-tuning team here. We know this is something of a pain and we’re trying to come up with a way to allow you to share / move models across projects. If you really need to share a model across projects right now the best way to do it is to train the model in the default project, it will then be available to use and all other projects. That’s not an ideal solution obviously but is the only mechanism available currently
    - underyx 2y ago
      Gotcha, thanks for the answer! We're a pretty heavy user of OpenAI (my company's called Semgrep) and I'd love to know if you ever run private betas or work with development partners for features like project permissioning. If so, feel free to email me at bence@semgrep.com — otherwise, appreciate your work on improving this!

simonw 2y ago

Are you talking about fine-tuned models that you host yourself, or fine-tuned models from a hosted provider like OpenAI? What do you mean by an "env" here?

underyx 2y ago
I mean OpenAI hosted fine-tunes (same as referenced in OP.) I have a staging deployment and a production deployment. Ideally anything that I roll out to production, I can try on staging first — including switching from gpt-4o to a fine-tuned gpt-4o. I don't want the production API key to be accessible by my staging app, so I have two separate projects within the OpenAI dashboard. One is called my-app-prod, and the other is my-app-staging. To illustrate the problem further, I also have infrastructure to eval models before changing what production is running. The eval infrastructure also has its own OpenAI project, so that I can set separate billing limits on running evals. Any fine-tuned model needs to be benchmarked first, but again, I'm not sure how to make the same fine-tune available to both the eval project, and the production app project.
- john_allard 2y ago
  Hey, engineer on the OpenAI fine-tuning team here. We know this is something of a pain and we’re trying to come up with a way to allow you to share / move models across projects. If you really need to share a model across projects right now the best way to do it is to train the model in the default project, it will then be available to use and all other projects. That’s not an ideal solution obviously but is the only mechanism available currently
  - underyx 2y ago
    Gotcha, thanks for the answer! We're a pretty heavy user of OpenAI (my company's called Semgrep) and I'd love to know if you ever run private betas or work with development partners for features like project permissioning. If so, feel free to email me at bence@semgrep.com — otherwise, appreciate your work on improving this!