8 ms·

I just want to add my $0.02 currently working at a games studio that is integrating AI generated art into our art pipelines. Midjourney definitely generates re

I just want to add my $0.02 currently working at a games studio that is integrating AI generated art into our art pipelines. Midjourney definitely generates really high quality art based on simple prompts, but the inability to really customize the output basically kills its utility. We heavily use Stable Diffusion with specific models and ControlNet to get customizable and consistent results. Our artists also need to extensively tweak and post-process the output, and re-run it again in Stable Diffusion. This entire workflow is definitely beyond a Discord-based interface to say the least.

tomng 3y ago
Really curious about the challenges you've been seeing as your art team's been trying to integrate AI generated art into your pipeline. Excited by the potential for artists to accelerate their creative process with advances like ControlNet, but makes sense there's a lot frustrating about the process today. Exploring tools that might help. If you might be interested in chatting, email me at tom@adobe.com. Thanks!
jononor 3y ago
If you would give a talk about this, I would watch it - despite being out of the graphics for almost 10 years now. Really want to hear from the trenches about the workflows, benefits and challenges you have.
- pjgalbraith 3y ago
  Here is a test I did the other day of rough sketch (hand drawn) -> clean line work (AI) -> coloured (AI). This workflow gives 100% control over the output because you can easily adjust the linework in the intermediary step. https://twitter.com/P_Galbraith/status/1649317290926825473?cxt=HHwWgoDT-c3VxuMtAAAA https://twitter.com/P_Galbraith/status/1649317290926825473?c... This is using Stable Diffusion and the Control Net Lineart Model. The coloured version is pretty rough but it was a quick test. In my opinion Stable Diffusion is vastly superior to Midjourney if you have the skill to provide input to img2img/ControlNet. I have some other earlier workflow experiments on Youtube if you're interested in this kind of thing https://www.youtube.com/pjgalbraith https://www.youtube.com/pjgalbraith
netdur 3y ago
use https://github.com/deep-floyd/IF https://github.com/deep-floyd/IF, it uses LLM to generate exact art you need.
- jamilton 3y ago
  The image quality of DeepFloyd is much lower than Stable Diffusion 1.5 though, it's a pretty major tradeoff. Can definitely be part of the workflow since it really is good at composition, but right now it's not a replacement.
- jelling 3y ago
  Deep Floyd doesn't allow commercial usage, such as a game studio using it.
  - zirgs 3y ago
    They said that they will change the licence for the final version IIRC.
exodust 3y ago
> AI generated art into our art pipelines I'd be interested to know where the art ends up in the game? Do you mean 2D backgrounds and billboards in-game? Or are we talking cut-scenes and menu screen art?

tomng 3y ago

Really curious about the challenges you've been seeing as your art team's been trying to integrate AI generated art into your pipeline. Excited by the potential for artists to accelerate their creative process with advances like ControlNet, but makes sense there's a lot frustrating about the process today. Exploring tools that might help. If you might be interested in chatting, email me at tom@adobe.com. Thanks!

jononor 3y ago

If you would give a talk about this, I would watch it - despite being out of the graphics for almost 10 years now. Really want to hear from the trenches about the workflows, benefits and challenges you have.

pjgalbraith 3y ago
Here is a test I did the other day of rough sketch (hand drawn) -> clean line work (AI) -> coloured (AI). This workflow gives 100% control over the output because you can easily adjust the linework in the intermediary step. https://twitter.com/P_Galbraith/status/1649317290926825473?cxt=HHwWgoDT-c3VxuMtAAAA https://twitter.com/P_Galbraith/status/1649317290926825473?c... This is using Stable Diffusion and the Control Net Lineart Model. The coloured version is pretty rough but it was a quick test. In my opinion Stable Diffusion is vastly superior to Midjourney if you have the skill to provide input to img2img/ControlNet. I have some other earlier workflow experiments on Youtube if you're interested in this kind of thing https://www.youtube.com/pjgalbraith https://www.youtube.com/pjgalbraith

netdur 3y ago

use https://github.com/deep-floyd/IF https://github.com/deep-floyd/IF, it uses LLM to generate exact art you need.

jamilton 3y ago
The image quality of DeepFloyd is much lower than Stable Diffusion 1.5 though, it's a pretty major tradeoff. Can definitely be part of the workflow since it really is good at composition, but right now it's not a replacement.
jelling 3y ago
Deep Floyd doesn't allow commercial usage, such as a game studio using it.
- zirgs 3y ago
  They said that they will change the licence for the final version IIRC.

exodust 3y ago

> AI generated art into our art pipelines I'd be interested to know where the art ends up in the game? Do you mean 2D backgrounds and billboards in-game? Or are we talking cut-scenes and menu screen art?