8 ms·

Show HN: Real-time image generation with SDXL Lightning

444 points by treesciencebot 3y ago

tiborsaas 3y ago
Insanely cool :) One bug though: when I increment the seed, it renders two image and it takes a bit of jumping up and down in seed numbers to get to the same image.
minimaxir 3y ago
For posterity, SDXL Lightning is open source on Hugging Face with a relatively-permissive license: https://huggingface.co/ByteDance/SDXL-Lightning https://huggingface.co/ByteDance/SDXL-Lightning There are a few other UIs for it, e.g. https://replicate.com/lucataco/sdxl-lightning-4step https://replicate.com/lucataco/sdxl-lightning-4step
- treesciencebot 3y ago
  Yep, this is using SDXL Lightning underneath which is trained by ByteDance on top of Stable Diffusion XL and released as an open source model. In addition to that, it is using our inference engine and real-time infrastructure to provide a smooth experience compared to other UIs out there (which as far as I know, speed-wise, are not even comparable, ~370ms for 4 step here vs ~2-3 seconds in the replicate link you posted).
  - smallerfish 3y ago
    Any plans to make an API? I'm building a website to catalog fairly common objects, and could use images to spice it up. I was looking at pexels...but this is just so much better. EDIT - ah you have one. You're welcome. Sign up here folks. :) Couple of questions in that case: a) What is the avg price per 512x512 image? Your pricing is in terms of machine resources, but (for my use case) I want a comparison to pexels. b) What would the equivalent machine setup be to get inference to be as fast as the website demo? c) Is the fast-sdxl api using the exact same stack as the website?
    - drochetti 3y ago
      There's no hidden magic in the playground and in the demo app, we use the same API available for all customers and also the same JS client and best practices available in our docs. To all your questions, I recommend playing with it in the API playground, you'll be able to test different image sizes, parameters, and have an idea of the cost per inference. If you have any other questions, say hello on our Discord and I'm happy to help you. https://fal.ai/models/stable-diffusion-xl-lightning https://fal.ai/models/stable-diffusion-xl-lightning
- 01HNNWZ0MV43FF 3y ago
  What's the RAM and speed like for local inference?
  - r-k-jo 3y ago
    It's using ~15GB VRAM
    - lonk 3y ago
      What is the speed with CPU+16GB ram without GPU?
      - whywhywhywhy 3y ago
        SDXL normally takes 40-60+ minutes per image on CPU so considering this is 1-4 steps instead of 20-25 steps you can make a guess.
- r-k-jo 3y ago
  I also made a demo with Gradio, but it's 2x slower than fal.ai! Using stable-fast compilation running on a single A10G https://huggingface.co/spaces/radames/Real-Time-Text-to-Image-SDXL-Lightning https://huggingface.co/spaces/radames/Real-Time-Text-to-Imag... I you have GPU/cuda/Docker you can try it locally docker run -it -p 7860:7860 --platform=linux/amd64 --gpus all -e SFAST_COMPILE="1" -e USE_TAESD="0" registry.hf.space/radames-real-time-text-to-image-sdxl-lightning:latest python app.py
  - smcleod 3y ago
    This is fantastic, thank you so much for taking the time to make a local demo available.
actionfromafar 3y ago
How ... can it be so fast? And what is "blob:https://blbahblah https://blbahblah" image? By the way the racoon is very prone to get two tails if you change the prompt a little. :)
- lights0123 3y ago
  A temporary URL that represents a buffer in JS: https://developer.mozilla.org/en-US/docs/Web/API/URL/createObjectURL_static https://developer.mozilla.org/en-US/docs/Web/API/URL/createO...
- moxvallix 3y ago
  For second question, its an object URL. https://developer.mozilla.org/en-US/docs/Web/API/URL/createObjectURL_static https://developer.mozilla.org/en-US/docs/Web/API/URL/createO... It's a way to turn a file, or blob, into a URL usable with an image element etc.
Reubend 3y ago
I love this demo. It's easily accessible, fast, and intuitive. It's stunning that we can get this level of quality so easily.
- winkelmann 3y ago
  This and Groq were really surprising to see. I still remember, not too long ago, waiting around for ages just to get a messed up image generation from some site where you got like 20 generations for free with an account. The fact that we're at the point where you can just go to a website and get lightning-fast text and image generation without sign-ups or captcha solving is amazing. I didn't have this type of performance uplift in the cards for early 2024, especially to the extent that they (Groq and fal.ai) can afford to open up their demos completely. Side note: In my opinion, the fast generations also make up a lot for the shortcomings in image generation quality. I find that even if it messes up, a good result is usually just a seed or small prompt change away.
hoc 3y ago
"bug with long antennas and chrome legs" you only end there because it's so fast. nice. really a new explorative quality.
pugworthy 3y ago
It's curious what it does with single letters. Seems for me to often settle on a small rather detailed building. The more I repeat the letter (e.g., "111" vs "11111111" the odder the building gets. Which I can see now seems pretty sensitive to the seed.
- samus 3y ago
  A word or a concept that is unknown has simply no impact on the output. Try to replace "baby raccoon" with "maxolhx" in the prompt, and it will ignore the word and render an Italian priest instead. Strictly speaking it still has an impact, but nothing we could easily describe. You're pretty much just playing with the seed.
altryne1 3y ago
I've used this + Groq yesterday to augment (with a chrome extension) the infinite fun game from Neal Agrawal, but generate actual images and not only emojis. This feels like the future, near real time image and LLM generation (using Mixtral from Groq as my prompt writer) and Fal API for read time generation! https://x.com/altryne/status/1760561501096575401?s=20 https://x.com/altryne/status/1760561501096575401?s=20
- lakshyaag 3y ago
  Woah, that's pretty cool. Wondering if it can be converted into a card-based game
  - pitherpather 3y ago
    > a game Competitive prompting?
    - Zircom 3y ago
      I've had an idea for a cards vs humanity style game but using image generation instead, there's a central card for the round and you add something from your hand and then pick from 5-10 images to submit.
      - ben_w 3y ago
        Sounds more like Dixit? https://en.wikipedia.org/wiki/Dixit_(board_game) https://en.wikipedia.org/wiki/Dixit_(board_game)
    - jcims 3y ago
      Prompt Fighter!
- deleted 3y ago
  [deleted]
- airstrike 3y ago
  that was a pretty cool video / demo / idea. good stuff!
refulgentis 3y ago
_Really_ impressive demo but it'd be oh-so-much-more-impressive if it was smooth, right now ex. deleting a word or adding a space causes 4 inferences in quick session so it feels janky (EDIT: maybe intentional? step by step displayed?) Btw this is from fal.ai, I first heard of them when they posted a Stable Cascade demo the morning it was released. They're *really* good, I *highly* recommend them for any inferencing you're doing outside OpenAI. Been in AI for going on 3 years, and on it 24/7 since last year. Fal is the first service that sweats the details to get it to the point it runs _this_ fast in practice, not just in papers. ex. web socket connection, use short-lived JWTs to avoid having to go through an edge function to sign a request with an API key, etc.
- jameshart 3y ago
  Good point. If it’s this fast, maybe it should generate intermediate images along a smooth path through the latent space, rather than just jumping right to the target
  - refulgentis 3y ago
    It's sort of the inverse if I'm seeing it correctly: adding one character triggers one inference, but you see steps 1, 2, 3, and 4 of the inference the latent space stuff became popular through it being a visual allegory, which accidentally confused the technical term it originated from. there's nothing visually smooth about it, it's not a linear interpolation in 3D space, it's a chaotic journey through 3 billion dimension space
    - jameshart 3y ago
      Well, it ends up being a journey through different images pulled from the same noise, so yes, any smoothness results more from the degree to which the sampling approach produces similar features when pulled towards slightly different target embeddings than from intrinsically the images being 'neighbors'. These low-step approaches probably preserve a lot less of the 'noise' features in the final image so latent space cruising is probably less fun.
    - 3y ago
doodlebugging 3y ago
"A cinematic shot of a coelacanth armed with a yellow corinthian leather hand cannon" Needs work.
- barnabyjones 3y ago
  The neat thing about this speed though is you can flip through the seeds quickly. Seed 626925 is giving me a fish holding some kind of gun, with what I guess are leather gloves. This has always been the main problem with SD imo, it can't really parse sentence structure so adjective descriptions often don't affect the thing you want.
  - doodlebugging 3y ago
    Yeah, I like the speed of the renders. It feels relatively smooth. OP's post to me feels like a marketing post where the output image is a really close representation of the product they hope to sell. We always called these types of things "carefully selected, random examples", in short they are cherry-picked for their adherence to a standard. In that same vein mine is also a carefully selected, random example of the output you get when the algorithms don't work well, therefore the "Needs work" qualification. Both are useful since you need to understand the limitations of the tools that you are employing. In my case I stepped thru animals until I found one that it could not render accurately. It did know that a coelacanth is a fish but it couldn't produce an accurate image of one. Then I added modifiers that it could not place in context. It's a bit like searching the debris field of a tornado for perfectly rounded debris particles and holding those up as a typical result without mentioning that you end up having to ignore all the splintery debris scattered from hell to breakfast around it.
krasin 3y ago
Great speed! As for the quality, I borrowed the query ([1]) that people used to test Stable Diffusion 3 and other models today: "Photo of a red sphere on top of a blue cube. Behind them is a green triangle, on the right is a dog, on the left is a cat". Here is what I got: https://imgur.com/a/XrAuqCB https://imgur.com/a/XrAuqCB To compare it with Stable Diffusion 3: https://pbs.twimg.com/media/GG8mm5va4AA_5PJ?format=jpg&name=large https://pbs.twimg.com/media/GG8mm5va4AA_5PJ?format=jpg&name=... 1. https://news.ycombinator.com/item?id=39467526 https://news.ycombinator.com/item?id=39467526
- treesciencebot 3y ago
  Spatial prompt adherence is a general missing piece is SDXL (or previous versions of the SD). Hoping that the SD will get it into a good shape as your examples! Test the example on Stable Cascade as well (latest open-weight stability model), and yeah, even that is not great at it https://fal.ai/models/stable-cascade?share=eab44060-690b-4973-91c5-23d445db7d67 https://fal.ai/models/stable-cascade?share=eab44060-690b-497....
- Xaiph_Rahci 3y ago
  What I got is pretty accurate: https://imgur.com/a/vH0zq5b https://imgur.com/a/vH0zq5b (seed: 3919562)
- nomel 3y ago
  Cycling through different seeds gives very different results.
airstrike 3y ago
Absolutely love this. Wish URLs were shareable! `late 90s movie poster, 24 hour clock movie "2: Electric Boogaloo" dan aykroyd1` turned out great
- scottmf 3y ago
  providing the seed would have allowed it to be shared
  - airstrike 3y ago
    alas, I was overworked and underslept and forgot. my bad...
- drochetti 3y ago
  We just added share. Let me know what you come up with!
  - psychoslave 3y ago
    https://fastsdxl.ai/share/8qu3p2z47xzy https://fastsdxl.ai/share/8qu3p2z47xzy
  - airstrike 3y ago
    love it! I have to log off but I should let you know it seems like the generation is different depending on whether you arrow up or arrow down into the seed when the focus is on the seed input (i.e. going up from 5 to 6 will have a different result than going from 7 to 6)
- psychoslave 3y ago
  What a normal person looks like: https://fastsdxl.ai/share/1mb3d5lo5ic9 https://fastsdxl.ai/share/1mb3d5lo5ic9 what a hero looks like: https://fastsdxl.ai/share/x9jxax4pnljd https://fastsdxl.ai/share/x9jxax4pnljd what a terrorist looks like: https://fastsdxl.ai/share/ejtyvv9ahpfs https://fastsdxl.ai/share/ejtyvv9ahpfs what the person I wish to be looks like: https://fastsdxl.ai/share/8ekkecm5rqsr https://fastsdxl.ai/share/8ekkecm5rqsr This is very interesting as the fast pace can help to quickly evaluate the biases incorporated just changing the seed.
Glyptodon 3y ago
Speed is impressive, but it doesn't seem to know what a pappenheimer rapier is. Keep getting a guy with weird bladed staff.
- jachee 3y ago
  I’m a 46-year old former LARPer, D&D nerd, etc, and I don’t know what a pappenheimer rapier looks like… it might be a bit niche. :)
  - _ache_ 3y ago
    My test is Axolotl. That it doesn't seem to have seen enough in the training set. The color is right (young one). Doesn't sound too niche.
  - pmontra 3y ago
    https://en.wikipedia.org/wiki/Pappenheim-hilt_rapier https://en.wikipedia.org/wiki/Pappenheim-hilt_rapier
a1o 3y ago
Yeah it can't do pixelart or any other thing that I tried. But yey speed.
- treesciencebot 3y ago
  pixel art is a particularly hard thing for these models to do, especially without further fine tunes or loras. but I'm pretty sure you should be able to get that quality with one of nerijs's loras [0]. But for now, i'd do some prompt templating and try some variations of this: 'pixel-art picture of a cat. low-res, blocky, pixel art style, 8-bit graphics' [0]: https://huggingface.co/nerijs/pixel-art-xl https://huggingface.co/nerijs/pixel-art-xl
hansonpeter 3y ago
[dead]
SeanAnderson 3y ago
wow! this is really fast. this is actually at the speed I want image generation to be at. it makes me want to explore and use the tool. everything else has felt like begrudgingly throwing a prompt and eventually giving up because the iteration cycle is too slow. of course the quality of what is being generated is not competitive with SOTA, but this is going in a really good direction!
albert_e 3y ago
Idea: convert this into a side scrolling game where the background gradually and seamlessly transitions into a rendering of the words we are dealing with as we progress. I am imagining the green lush landscape from early parts of the demo to slowly transform into the dry mountainous landscape from later images while new characters appear in the foreground.
_ache_ 3y ago
Still doesn't pass the Axolotl test. :/ I guess it need more memory.
ultimoo 3y ago
inferencing images at the speed of typing is such a fantastic way to show off what it's capable of. kudos!
- lelag 3y ago
  And also what it is not capable of doing too... Simply try to get it to output an image with a female that's not a beauty queen. Even when specifically prompted to produce an image of ugly people, it can only generate beautiful people.
  - CapricornNoble 3y ago
    I got some unattractive results easily: https://fastsdxl.ai/share/dfxtkr3r0w3z https://fastsdxl.ai/share/dfxtkr3r0w3z https://fastsdxl.ai/share/e70xgiudn8j0 https://fastsdxl.ai/share/e70xgiudn8j0 But when I switched the prompt to include "portrait", yeah it produces women who are too attractive, like an actress wearing makeup to appear ugly for a role.
diego_sandoval 3y ago
This is amazing. How much does it cost per image?
thyrox 3y ago
Wow this is super impressive but does somebody know a way to generate consistent characters with stable diffusion? What I mean is if my first prompt is a girl talking to cat and second prompt is girl playing with that cat, I want the girl and cat to be the same in both pictures. Is that possible? If so any links or tutorials will be super helpful to learn.
- ppsreejith 3y ago
  IIRC Dashtoon studio allows you to create comics with consistent characters using stable diffusion: https://dashtoon.com/create https://dashtoon.com/create
- OKRainbowKid 3y ago
  Check out IP-Adapter, FaceID, and Reactor.
- padolsey 3y ago
  Check out https://scenario.gg https://scenario.gg - they let you train your own LoRAs on custom images of a character (you need around 20 or so images from different angles for good consistency). A bit simpler, and actually still pretty decent is IP-Adapter, which they also support. Having the cat be consistent is going to be challenging without a custom LoRA I reckon. See this for guidance: https://help.scenario.com/training-a-character-lora https://help.scenario.com/training-a-character-lora
- Zetobal 3y ago
  It's usually enough to just use names "Maria Smith" will almost always look like "Maria Smith" in good SD models.
- jumboking63 3y ago
  You can do this on Dashtoon Studio. They let you upload just one image and train a consistent character Lora. It's a software for AI comic creation. Found this video on their youtube https://www.youtube.com/watch?v=EEQwEvKQGvE https://www.youtube.com/watch?v=EEQwEvKQGvE Lora is by far the most versatile because you can get your character consistently in any pose and in any camera angle. IP adapter replicate too many traits from the input image, and you can't choose what not replicate like the pose. So getting a character from a portrait input to do anything else can become. For Reactor you need a generated image into which you can swap in a face. Works very well for realistic images, for stylized images the style is not maintained. Also hairstyle won't get copied. So Dashtoon is the most reliable thing and easy thing I've found so far because collecting 20 images of a new character is hard and the properties of the images in a Lora training set are really important like how many close ups, how many expressions etc.
danielecook 3y ago
This is incredible. The reduction in latency has a profound effect in terms of the way I interact with this type of tool. The speed benefit is more than just more image generation. The speed here helps me easily keep the same train of thought moving along as I try different things.
fareesh 3y ago
Nancy Pelosi as a ninja without a mask = Nancy Pelosi as a ninja wearing a mask
supermatt 3y ago
What is the difference between SDXL turbo (released last november) and lightning (released 2 days ago?)? I havent seen any discussion of lightning on here the last few days and hnsearch only shows a few posts with no comments.
- supermatt 3y ago
  OK - I found the lightning paper and it seems the difference (to an end user) is turbo is max 512px (with no lora support) and this is 1024px (with lora support). The example images in the paper are also subjectively better quality and composition - but appear more stylised to my eye. link: https://arxiv.org/html/2402.13929v1 https://arxiv.org/html/2402.13929v1
tdudhhu 3y ago
Wow this is fast! The results also look great, but the more I see AI generated images the more I believe it is not going to eat jobs. Almost none of the results are production ready. They all contain strange parts. And maybe more important: the look and feel is always the same. It is amazing this works as fast as it does, but I think AI is still in it's 'hype' stage.
- jjbinx007 3y ago
  Midjouney is more advanced and some of the output is definitely job stealingly good. It can do excellent photo real, sketch, pixel art, diagrams, painting, digital painting, 3d, all to a high standard. I think of current ai/ml stuff right now as a very fast intern assistant that will get you 85% of the way there but needs supervision. But the technology is still so new it will only get better.
- ben_w 3y ago
  > the look and feel is always the same. This is an important thing, but not a consistent thing between humans. For example, while I can tell half of these are AI (and not just because I typed in the prompt), they have very different looks and feels to me: • https://fastsdxl.ai/share/6djh0dlat0s6 https://fastsdxl.ai/share/6djh0dlat0s6 "Will Smith facing a white plastic robot, close up, side view, renaissance masterpiece oil painting by da Vinci" • https://fastsdxl.ai/share/ctwqegl5i3xq https://fastsdxl.ai/share/ctwqegl5i3xq "a hand stitched embroidery of a cute tiger-racoon playing in woodland" • https://fastsdxl.ai/share/mkfrx33xc4ee https://fastsdxl.ai/share/mkfrx33xc4ee "a selfie shot of a furry in a furry convention" • https://fastsdxl.ai/share/mphyrzzjsces https://fastsdxl.ai/share/mphyrzzjsces "Simple sketch of Mordor, Mount Doom, dark and moody, despair, dense fog" • https://fastsdxl.ai/share/hgjwx6avyx0h https://fastsdxl.ai/share/hgjwx6avyx0h "coffee mug stain on paper" But there are many others like yourself who apparently have higher standards than I do. (And there are also many who have lower standards than me, who were happy to print huge posters where the left and right eyes of subject didn't match).
victorbjorklund 3y ago
Damn that is fast.
edf13 3y ago
Still issues with limbs when using AI generative artwork... https://fastsdxl.ai/share/0y54bhvalknv https://fastsdxl.ai/share/0y54bhvalknv
kennyadam 3y ago
Incredibly impressive how fast it is. Very disturbing results are just a second away! https://fastsdxl.ai/share/q5oe57udw953 https://fastsdxl.ai/share/q5oe57udw953
monkeydust 3y ago
This is nuts, in a good way. Way more fun to have fast inference on images than text.
jansan 3y ago
Very cool. How about putting the images that were created along the way into a carousel that you can swipe through, just like a flipbook?
- albumen 3y ago
  This is a great idea. Being able to scan through previous images would encourage freer concept exploration, since you could then jump back to your favoured branch-off point easily.
ozfive 3y ago
It's so fast I question the quality of all other models that cost money.
deleted 3y ago
[deleted]
padolsey 3y ago
This is so quick! The demo being publicly consumable is powerful, though SDXL is not immune to abuse or NSFW (and possibly illegal) generations. I wonder who ends up being held accountable for such generations.
- whelp_24 3y ago
  That's a strange question. Why is nudity something that an image generator shouldn't be able to create? Are genitals a fiction that should never ve present in human art?
  - sebzim4500 3y ago
    Probably the fear is about deepfakes rather than generic porn.
  - padolsey 3y ago
    Oh indeed. I believe these models should be uncensored. But as we’ve seen with the latest SD model, and with the much more locked down ‘LLM-fronted’ image generators from OpenAI and Google, safety is a massive concern and so they’ve been ridiculously cautious. Not only with the outputs but also with the training material. (‘Porn-In-Porn-Out’) Regardless of how we feel, lawyers and regulators wait at the door. We should expect new legal precedent within the next year re the generation of copyright-infringing, deepfake, and pornographic material.
ritzaco 3y ago
This is so impressive. Is it an ad for fal.ai? Or who is paying for it?
bsenftner 3y ago
Seems like with this kind of speed, the only reason this is not a video tool is the lack of consistency controls?
timnetworks 3y ago
That's very fast... typing in SDXL as the prompt gives a chilling self-portrait though.
petermcneeley 3y ago
Who is gonna tell George Lucas? https://fastsdxl.ai/share/o8d16q87qkbt https://fastsdxl.ai/share/o8d16q87qkbt
- TehShrike 3y ago
  "star wars businessman" https://fastsdxl.ai/share/iag9nqd6vasw https://fastsdxl.ai/share/iag9nqd6vasw
rcarmo 3y ago
Ooooookaaaaaay.... This is faster than autocomplete. Amazing.
rcarmo 3y ago
Note: doesn't seem to work on iPad.
- rcarmo 3y ago
  Why the downvote? It doesn’t work on iOS browsers, period. It’s a fact.
  - winkelmann 3y ago
    Maybe it's an iOS thing? I just spent about an hour generating stuff from my iPad. *Edit:* The previous poster specifically said they use an iPad, that's odd. Worked fine on my 12,9" iPad Pro, Display Zoom is set to "more space", i.e. more content shown. In case it has an issue with not enough screen space?
    - rcarmo 3y ago
      They’ve since added a refresh button if you check the GitHub history. Both my previous comments were valid at the time of posting them, so again I don’t understand the downvotes-I suspect it’s a personal thing as it’s happened to many of my recent comments.
      - winkelmann 3y ago
        To be clear: I did not downvote any comment, and I don't agree with anyone who did, since I don't believe the reports of the website not working for someone on i(Pad) OS to be made-up. That being said, I don't think the refresh button (actually a seed randomizer) is related. I was definitely already playing around with it on my iPad when you posted your comment. However, I am not sure about the timeline in relation to the first comment (above yours).
tobeagram 3y ago
I had some fun inputting the American Psycho transcript into this. Made a little video [1]. Insane how fast this is, great work! [1]: https://x.com/TobiasWhetton/status/1761038697074466868 https://x.com/TobiasWhetton/status/1761038697074466868
Workaccount2 3y ago
Mind blowing how fast this is
ongytenes 3y ago
Looking forward to it once it is refined. It still creates some strange artifacts. Within the HN message, the demo image of the baby raccoon in priest robes, one can count five paws.
superasn 3y ago
What would be the cost per month to host the same demo yourself?

Insanely cool :) One bug though: when I increment the seed, it renders two image and it takes a bit of jumping up and down in seed numbers to get to the same image.

minimaxir 3y ago

For posterity, SDXL Lightning is open source on Hugging Face with a relatively-permissive license: https://huggingface.co/ByteDance/SDXL-Lightning https://huggingface.co/ByteDance/SDXL-Lightning There are a few other UIs for it, e.g. https://replicate.com/lucataco/sdxl-lightning-4step https://replicate.com/lucataco/sdxl-lightning-4step

treesciencebot 3y ago
Yep, this is using SDXL Lightning underneath which is trained by ByteDance on top of Stable Diffusion XL and released as an open source model. In addition to that, it is using our inference engine and real-time infrastructure to provide a smooth experience compared to other UIs out there (which as far as I know, speed-wise, are not even comparable, ~370ms for 4 step here vs ~2-3 seconds in the replicate link you posted).
- smallerfish 3y ago
  Any plans to make an API? I'm building a website to catalog fairly common objects, and could use images to spice it up. I was looking at pexels...but this is just so much better. EDIT - ah you have one. You're welcome. Sign up here folks. :) Couple of questions in that case: a) What is the avg price per 512x512 image? Your pricing is in terms of machine resources, but (for my use case) I want a comparison to pexels. b) What would the equivalent machine setup be to get inference to be as fast as the website demo? c) Is the fast-sdxl api using the exact same stack as the website?
  - drochetti 3y ago
    There's no hidden magic in the playground and in the demo app, we use the same API available for all customers and also the same JS client and best practices available in our docs. To all your questions, I recommend playing with it in the API playground, you'll be able to test different image sizes, parameters, and have an idea of the cost per inference. If you have any other questions, say hello on our Discord and I'm happy to help you. https://fal.ai/models/stable-diffusion-xl-lightning https://fal.ai/models/stable-diffusion-xl-lightning
01HNNWZ0MV43FF 3y ago
What's the RAM and speed like for local inference?
- r-k-jo 3y ago
  It's using ~15GB VRAM
  - lonk 3y ago
    What is the speed with CPU+16GB ram without GPU?
    - whywhywhywhy 3y ago
      SDXL normally takes 40-60+ minutes per image on CPU so considering this is 1-4 steps instead of 20-25 steps you can make a guess.
r-k-jo 3y ago
I also made a demo with Gradio, but it's 2x slower than fal.ai! Using stable-fast compilation running on a single A10G https://huggingface.co/spaces/radames/Real-Time-Text-to-Image-SDXL-Lightning https://huggingface.co/spaces/radames/Real-Time-Text-to-Imag... I you have GPU/cuda/Docker you can try it locally docker run -it -p 7860:7860 --platform=linux/amd64 --gpus all -e SFAST_COMPILE="1" -e USE_TAESD="0" registry.hf.space/radames-real-time-text-to-image-sdxl-lightning:latest python app.py
- smcleod 3y ago
  This is fantastic, thank you so much for taking the time to make a local demo available.

actionfromafar 3y ago

How ... can it be so fast? And what is "blob:https://blbahblah https://blbahblah" image? By the way the racoon is very prone to get two tails if you change the prompt a little. :)

lights0123 3y ago
A temporary URL that represents a buffer in JS: https://developer.mozilla.org/en-US/docs/Web/API/URL/createObjectURL_static https://developer.mozilla.org/en-US/docs/Web/API/URL/createO...
moxvallix 3y ago
For second question, its an object URL. https://developer.mozilla.org/en-US/docs/Web/API/URL/createObjectURL_static https://developer.mozilla.org/en-US/docs/Web/API/URL/createO... It's a way to turn a file, or blob, into a URL usable with an image element etc.

Reubend 3y ago

I love this demo. It's easily accessible, fast, and intuitive. It's stunning that we can get this level of quality so easily.

winkelmann 3y ago
This and Groq were really surprising to see. I still remember, not too long ago, waiting around for ages just to get a messed up image generation from some site where you got like 20 generations for free with an account. The fact that we're at the point where you can just go to a website and get lightning-fast text and image generation without sign-ups or captcha solving is amazing. I didn't have this type of performance uplift in the cards for early 2024, especially to the extent that they (Groq and fal.ai) can afford to open up their demos completely. Side note: In my opinion, the fast generations also make up a lot for the shortcomings in image generation quality. I find that even if it messes up, a good result is usually just a seed or small prompt change away.

hoc 3y ago

"bug with long antennas and chrome legs" you only end there because it's so fast. nice. really a new explorative quality.

pugworthy 3y ago

It's curious what it does with single letters. Seems for me to often settle on a small rather detailed building. The more I repeat the letter (e.g., "111" vs "11111111" the odder the building gets. Which I can see now seems pretty sensitive to the seed.

samus 3y ago
A word or a concept that is unknown has simply no impact on the output. Try to replace "baby raccoon" with "maxolhx" in the prompt, and it will ignore the word and render an Italian priest instead. Strictly speaking it still has an impact, but nothing we could easily describe. You're pretty much just playing with the seed.

altryne1 3y ago

I've used this + Groq yesterday to augment (with a chrome extension) the infinite fun game from Neal Agrawal, but generate actual images and not only emojis. This feels like the future, near real time image and LLM generation (using Mixtral from Groq as my prompt writer) and Fal API for read time generation! https://x.com/altryne/status/1760561501096575401?s=20 https://x.com/altryne/status/1760561501096575401?s=20

lakshyaag 3y ago
Woah, that's pretty cool. Wondering if it can be converted into a card-based game
- pitherpather 3y ago
  > a game Competitive prompting?
  - Zircom 3y ago
    I've had an idea for a cards vs humanity style game but using image generation instead, there's a central card for the round and you add something from your hand and then pick from 5-10 images to submit.
    - ben_w 3y ago
      Sounds more like Dixit? https://en.wikipedia.org/wiki/Dixit_(board_game) https://en.wikipedia.org/wiki/Dixit_(board_game)
  - jcims 3y ago
    Prompt Fighter!
deleted 3y ago
[deleted]
airstrike 3y ago
that was a pretty cool video / demo / idea. good stuff!

refulgentis 3y ago

_Really_ impressive demo but it'd be oh-so-much-more-impressive if it was smooth, right now ex. deleting a word or adding a space causes 4 inferences in quick session so it feels janky (EDIT: maybe intentional? step by step displayed?) Btw this is from fal.ai, I first heard of them when they posted a Stable Cascade demo the morning it was released. They're *really* good, I *highly* recommend them for any inferencing you're doing outside OpenAI. Been in AI for going on 3 years, and on it 24/7 since last year. Fal is the first service that sweats the details to get it to the point it runs _this_ fast in practice, not just in papers. ex. web socket connection, use short-lived JWTs to avoid having to go through an edge function to sign a request with an API key, etc.

jameshart 3y ago
Good point. If it’s this fast, maybe it should generate intermediate images along a smooth path through the latent space, rather than just jumping right to the target
- refulgentis 3y ago
  It's sort of the inverse if I'm seeing it correctly: adding one character triggers one inference, but you see steps 1, 2, 3, and 4 of the inference the latent space stuff became popular through it being a visual allegory, which accidentally confused the technical term it originated from. there's nothing visually smooth about it, it's not a linear interpolation in 3D space, it's a chaotic journey through 3 billion dimension space
  - jameshart 3y ago
    Well, it ends up being a journey through different images pulled from the same noise, so yes, any smoothness results more from the degree to which the sampling approach produces similar features when pulled towards slightly different target embeddings than from intrinsically the images being 'neighbors'. These low-step approaches probably preserve a lot less of the 'noise' features in the final image so latent space cruising is probably less fun.
  - 3y ago

doodlebugging 3y ago

"A cinematic shot of a coelacanth armed with a yellow corinthian leather hand cannon" Needs work.

barnabyjones 3y ago
The neat thing about this speed though is you can flip through the seeds quickly. Seed 626925 is giving me a fish holding some kind of gun, with what I guess are leather gloves. This has always been the main problem with SD imo, it can't really parse sentence structure so adjective descriptions often don't affect the thing you want.
- doodlebugging 3y ago
  Yeah, I like the speed of the renders. It feels relatively smooth. OP's post to me feels like a marketing post where the output image is a really close representation of the product they hope to sell. We always called these types of things "carefully selected, random examples", in short they are cherry-picked for their adherence to a standard. In that same vein mine is also a carefully selected, random example of the output you get when the algorithms don't work well, therefore the "Needs work" qualification. Both are useful since you need to understand the limitations of the tools that you are employing. In my case I stepped thru animals until I found one that it could not render accurately. It did know that a coelacanth is a fish but it couldn't produce an accurate image of one. Then I added modifiers that it could not place in context. It's a bit like searching the debris field of a tornado for perfectly rounded debris particles and holding those up as a typical result without mentioning that you end up having to ignore all the splintery debris scattered from hell to breakfast around it.

krasin 3y ago

Great speed! As for the quality, I borrowed the query ([1]) that people used to test Stable Diffusion 3 and other models today: "Photo of a red sphere on top of a blue cube. Behind them is a green triangle, on the right is a dog, on the left is a cat". Here is what I got: https://imgur.com/a/XrAuqCB https://imgur.com/a/XrAuqCB To compare it with Stable Diffusion 3: https://pbs.twimg.com/media/GG8mm5va4AA_5PJ?format=jpg&name=large https://pbs.twimg.com/media/GG8mm5va4AA_5PJ?format=jpg&name=... 1. https://news.ycombinator.com/item?id=39467526 https://news.ycombinator.com/item?id=39467526

treesciencebot 3y ago
Spatial prompt adherence is a general missing piece is SDXL (or previous versions of the SD). Hoping that the SD will get it into a good shape as your examples! Test the example on Stable Cascade as well (latest open-weight stability model), and yeah, even that is not great at it https://fal.ai/models/stable-cascade?share=eab44060-690b-4973-91c5-23d445db7d67 https://fal.ai/models/stable-cascade?share=eab44060-690b-497....
Xaiph_Rahci 3y ago
What I got is pretty accurate: https://imgur.com/a/vH0zq5b https://imgur.com/a/vH0zq5b (seed: 3919562)
nomel 3y ago
Cycling through different seeds gives very different results.

airstrike 3y ago

Absolutely love this. Wish URLs were shareable! `late 90s movie poster, 24 hour clock movie "2: Electric Boogaloo" dan aykroyd1` turned out great

scottmf 3y ago
providing the seed would have allowed it to be shared
- airstrike 3y ago
  alas, I was overworked and underslept and forgot. my bad...
drochetti 3y ago
We just added share. Let me know what you come up with!
- psychoslave 3y ago
  https://fastsdxl.ai/share/8qu3p2z47xzy https://fastsdxl.ai/share/8qu3p2z47xzy
- airstrike 3y ago
  love it! I have to log off but I should let you know it seems like the generation is different depending on whether you arrow up or arrow down into the seed when the focus is on the seed input (i.e. going up from 5 to 6 will have a different result than going from 7 to 6)
psychoslave 3y ago
What a normal person looks like: https://fastsdxl.ai/share/1mb3d5lo5ic9 https://fastsdxl.ai/share/1mb3d5lo5ic9 what a hero looks like: https://fastsdxl.ai/share/x9jxax4pnljd https://fastsdxl.ai/share/x9jxax4pnljd what a terrorist looks like: https://fastsdxl.ai/share/ejtyvv9ahpfs https://fastsdxl.ai/share/ejtyvv9ahpfs what the person I wish to be looks like: https://fastsdxl.ai/share/8ekkecm5rqsr https://fastsdxl.ai/share/8ekkecm5rqsr This is very interesting as the fast pace can help to quickly evaluate the biases incorporated just changing the seed.

Glyptodon 3y ago

Speed is impressive, but it doesn't seem to know what a pappenheimer rapier is. Keep getting a guy with weird bladed staff.

jachee 3y ago
I’m a 46-year old former LARPer, D&D nerd, etc, and I don’t know what a pappenheimer rapier looks like… it might be a bit niche. :)
- _ache_ 3y ago
  My test is Axolotl. That it doesn't seem to have seen enough in the training set. The color is right (young one). Doesn't sound too niche.
- pmontra 3y ago
  https://en.wikipedia.org/wiki/Pappenheim-hilt_rapier https://en.wikipedia.org/wiki/Pappenheim-hilt_rapier

a1o 3y ago

Yeah it can't do pixelart or any other thing that I tried. But yey speed.

treesciencebot 3y ago
pixel art is a particularly hard thing for these models to do, especially without further fine tunes or loras. but I'm pretty sure you should be able to get that quality with one of nerijs's loras [0]. But for now, i'd do some prompt templating and try some variations of this: 'pixel-art picture of a cat. low-res, blocky, pixel art style, 8-bit graphics' [0]: https://huggingface.co/nerijs/pixel-art-xl https://huggingface.co/nerijs/pixel-art-xl

hansonpeter 3y ago

[dead]

SeanAnderson 3y ago

wow! this is really fast. this is actually at the speed I want image generation to be at. it makes me want to explore and use the tool. everything else has felt like begrudgingly throwing a prompt and eventually giving up because the iteration cycle is too slow. of course the quality of what is being generated is not competitive with SOTA, but this is going in a really good direction!

albert_e 3y ago

Idea: convert this into a side scrolling game where the background gradually and seamlessly transitions into a rendering of the words we are dealing with as we progress. I am imagining the green lush landscape from early parts of the demo to slowly transform into the dry mountainous landscape from later images while new characters appear in the foreground.

_ache_ 3y ago

Still doesn't pass the Axolotl test. :/ I guess it need more memory.

ultimoo 3y ago

inferencing images at the speed of typing is such a fantastic way to show off what it's capable of. kudos!

lelag 3y ago
And also what it is not capable of doing too... Simply try to get it to output an image with a female that's not a beauty queen. Even when specifically prompted to produce an image of ugly people, it can only generate beautiful people.
- CapricornNoble 3y ago
  I got some unattractive results easily: https://fastsdxl.ai/share/dfxtkr3r0w3z https://fastsdxl.ai/share/dfxtkr3r0w3z https://fastsdxl.ai/share/e70xgiudn8j0 https://fastsdxl.ai/share/e70xgiudn8j0 But when I switched the prompt to include "portrait", yeah it produces women who are too attractive, like an actress wearing makeup to appear ugly for a role.

diego_sandoval 3y ago

This is amazing. How much does it cost per image?

thyrox 3y ago

Wow this is super impressive but does somebody know a way to generate consistent characters with stable diffusion? What I mean is if my first prompt is a girl talking to cat and second prompt is girl playing with that cat, I want the girl and cat to be the same in both pictures. Is that possible? If so any links or tutorials will be super helpful to learn.

ppsreejith 3y ago
IIRC Dashtoon studio allows you to create comics with consistent characters using stable diffusion: https://dashtoon.com/create https://dashtoon.com/create
OKRainbowKid 3y ago
Check out IP-Adapter, FaceID, and Reactor.
padolsey 3y ago
Check out https://scenario.gg https://scenario.gg - they let you train your own LoRAs on custom images of a character (you need around 20 or so images from different angles for good consistency). A bit simpler, and actually still pretty decent is IP-Adapter, which they also support. Having the cat be consistent is going to be challenging without a custom LoRA I reckon. See this for guidance: https://help.scenario.com/training-a-character-lora https://help.scenario.com/training-a-character-lora
Zetobal 3y ago
It's usually enough to just use names "Maria Smith" will almost always look like "Maria Smith" in good SD models.
jumboking63 3y ago
You can do this on Dashtoon Studio. They let you upload just one image and train a consistent character Lora. It's a software for AI comic creation. Found this video on their youtube https://www.youtube.com/watch?v=EEQwEvKQGvE https://www.youtube.com/watch?v=EEQwEvKQGvE Lora is by far the most versatile because you can get your character consistently in any pose and in any camera angle. IP adapter replicate too many traits from the input image, and you can't choose what not replicate like the pose. So getting a character from a portrait input to do anything else can become. For Reactor you need a generated image into which you can swap in a face. Works very well for realistic images, for stylized images the style is not maintained. Also hairstyle won't get copied. So Dashtoon is the most reliable thing and easy thing I've found so far because collecting 20 images of a new character is hard and the properties of the images in a Lora training set are really important like how many close ups, how many expressions etc.

danielecook 3y ago

This is incredible. The reduction in latency has a profound effect in terms of the way I interact with this type of tool. The speed benefit is more than just more image generation. The speed here helps me easily keep the same train of thought moving along as I try different things.

fareesh 3y ago

Nancy Pelosi as a ninja without a mask = Nancy Pelosi as a ninja wearing a mask

supermatt 3y ago

What is the difference between SDXL turbo (released last november) and lightning (released 2 days ago?)? I havent seen any discussion of lightning on here the last few days and hnsearch only shows a few posts with no comments.

supermatt 3y ago
OK - I found the lightning paper and it seems the difference (to an end user) is turbo is max 512px (with no lora support) and this is 1024px (with lora support). The example images in the paper are also subjectively better quality and composition - but appear more stylised to my eye. link: https://arxiv.org/html/2402.13929v1 https://arxiv.org/html/2402.13929v1

tdudhhu 3y ago

Wow this is fast! The results also look great, but the more I see AI generated images the more I believe it is not going to eat jobs. Almost none of the results are production ready. They all contain strange parts. And maybe more important: the look and feel is always the same. It is amazing this works as fast as it does, but I think AI is still in it's 'hype' stage.

jjbinx007 3y ago
Midjouney is more advanced and some of the output is definitely job stealingly good. It can do excellent photo real, sketch, pixel art, diagrams, painting, digital painting, 3d, all to a high standard. I think of current ai/ml stuff right now as a very fast intern assistant that will get you 85% of the way there but needs supervision. But the technology is still so new it will only get better.
ben_w 3y ago
> the look and feel is always the same. This is an important thing, but not a consistent thing between humans. For example, while I can tell half of these are AI (and not just because I typed in the prompt), they have very different looks and feels to me: • https://fastsdxl.ai/share/6djh0dlat0s6 https://fastsdxl.ai/share/6djh0dlat0s6 "Will Smith facing a white plastic robot, close up, side view, renaissance masterpiece oil painting by da Vinci" • https://fastsdxl.ai/share/ctwqegl5i3xq https://fastsdxl.ai/share/ctwqegl5i3xq "a hand stitched embroidery of a cute tiger-racoon playing in woodland" • https://fastsdxl.ai/share/mkfrx33xc4ee https://fastsdxl.ai/share/mkfrx33xc4ee "a selfie shot of a furry in a furry convention" • https://fastsdxl.ai/share/mphyrzzjsces https://fastsdxl.ai/share/mphyrzzjsces "Simple sketch of Mordor, Mount Doom, dark and moody, despair, dense fog" • https://fastsdxl.ai/share/hgjwx6avyx0h https://fastsdxl.ai/share/hgjwx6avyx0h "coffee mug stain on paper" But there are many others like yourself who apparently have higher standards than I do. (And there are also many who have lower standards than me, who were happy to print huge posters where the left and right eyes of subject didn't match).

victorbjorklund 3y ago

Damn that is fast.

edf13 3y ago

Still issues with limbs when using AI generative artwork... https://fastsdxl.ai/share/0y54bhvalknv https://fastsdxl.ai/share/0y54bhvalknv

kennyadam 3y ago

Incredibly impressive how fast it is. Very disturbing results are just a second away! https://fastsdxl.ai/share/q5oe57udw953 https://fastsdxl.ai/share/q5oe57udw953

monkeydust 3y ago

This is nuts, in a good way. Way more fun to have fast inference on images than text.

jansan 3y ago

Very cool. How about putting the images that were created along the way into a carousel that you can swipe through, just like a flipbook?

albumen 3y ago
This is a great idea. Being able to scan through previous images would encourage freer concept exploration, since you could then jump back to your favoured branch-off point easily.

ozfive 3y ago

It's so fast I question the quality of all other models that cost money.

deleted 3y ago

[deleted]

padolsey 3y ago

This is so quick! The demo being publicly consumable is powerful, though SDXL is not immune to abuse or NSFW (and possibly illegal) generations. I wonder who ends up being held accountable for such generations.

whelp_24 3y ago
That's a strange question. Why is nudity something that an image generator shouldn't be able to create? Are genitals a fiction that should never ve present in human art?
- sebzim4500 3y ago
  Probably the fear is about deepfakes rather than generic porn.
- padolsey 3y ago
  Oh indeed. I believe these models should be uncensored. But as we’ve seen with the latest SD model, and with the much more locked down ‘LLM-fronted’ image generators from OpenAI and Google, safety is a massive concern and so they’ve been ridiculously cautious. Not only with the outputs but also with the training material. (‘Porn-In-Porn-Out’) Regardless of how we feel, lawyers and regulators wait at the door. We should expect new legal precedent within the next year re the generation of copyright-infringing, deepfake, and pornographic material.

ritzaco 3y ago

This is so impressive. Is it an ad for fal.ai? Or who is paying for it?

bsenftner 3y ago

Seems like with this kind of speed, the only reason this is not a video tool is the lack of consistency controls?

timnetworks 3y ago

That's very fast... typing in SDXL as the prompt gives a chilling self-portrait though.

petermcneeley 3y ago

Who is gonna tell George Lucas? https://fastsdxl.ai/share/o8d16q87qkbt https://fastsdxl.ai/share/o8d16q87qkbt

TehShrike 3y ago
"star wars businessman" https://fastsdxl.ai/share/iag9nqd6vasw https://fastsdxl.ai/share/iag9nqd6vasw

rcarmo 3y ago

Ooooookaaaaaay.... This is faster than autocomplete. Amazing.

rcarmo 3y ago

Note: doesn't seem to work on iPad.

rcarmo 3y ago
Why the downvote? It doesn’t work on iOS browsers, period. It’s a fact.
- winkelmann 3y ago
  Maybe it's an iOS thing? I just spent about an hour generating stuff from my iPad. *Edit:* The previous poster specifically said they use an iPad, that's odd. Worked fine on my 12,9" iPad Pro, Display Zoom is set to "more space", i.e. more content shown. In case it has an issue with not enough screen space?
  - rcarmo 3y ago
    They’ve since added a refresh button if you check the GitHub history. Both my previous comments were valid at the time of posting them, so again I don’t understand the downvotes-I suspect it’s a personal thing as it’s happened to many of my recent comments.
    - winkelmann 3y ago
      To be clear: I did not downvote any comment, and I don't agree with anyone who did, since I don't believe the reports of the website not working for someone on i(Pad) OS to be made-up. That being said, I don't think the refresh button (actually a seed randomizer) is related. I was definitely already playing around with it on my iPad when you posted your comment. However, I am not sure about the timeline in relation to the first comment (above yours).

tobeagram 3y ago

I had some fun inputting the American Psycho transcript into this. Made a little video [1]. Insane how fast this is, great work! [1]: https://x.com/TobiasWhetton/status/1761038697074466868 https://x.com/TobiasWhetton/status/1761038697074466868

Workaccount2 3y ago

Mind blowing how fast this is

ongytenes 3y ago

Looking forward to it once it is refined. It still creates some strange artifacts. Within the HN message, the demo image of the baby raccoon in priest robes, one can count five paws.

superasn 3y ago

What would be the cost per month to host the same demo yourself?