8 ms·
Show HN: Offline sketch to image geneartor in a whiteboard
I'm excited to share my indie product, https://drawing.pics https://drawing.pics
In this post, I'll share all the technical details behind it.
So, what exactly is DrawingPics?
It's an image-to-image and sketch-to-image generator that runs in a whiteboard, allowing you to draw your draft directly on the canvas and easily generate the desired image.
How does it work from a technical perspective?
TLDR: Miniconda + Diffusers + Electron + Excalidraw
One of the most important aspects of creating this tool was finding a stable diffusion locally inference solution. There are many solutions available, such as https://github.com/leejet/stable-diffusion.cpp https://github.com/leejet/stable-diffusion.cpp and https://github.com/apple/ml-stable-diffusion https://github.com/apple/ml-stable-diffusion. I tested the C++ version, but the inference speed was very slow, and Metal GPU support still had issues (you can find relevant issues in their repo). Ultimately, I decided to use python to run it because PyTorch is mature and MPS support is well-established. And I chose Miniconda, because it can create a small, isolated Python environment to run the program.
The AI model should run in the background so that we can continuously produce images while you draw. We need to find an RPC method to enable communication between the Python process and Electron's Node.js process. The easiest way is to run a Python HTTP server, but the memory usage is too high. We need a more lightweight solution, so I used xmlrpc for memory efficiency, although there might be better alternatives that I'm unaware of. The AI inference part is handled by diffusers, which is great, but I had to apply some custom patches to make it work in this situation. This can be a bit challenging if you're not familiar with Python.
For the frontend, I initially used a low-level canvas library and tried to implement a drawing pad from scratch. However, it had too many details, so I chose a more mature option: Excalidraw. It's fantastic, with the only shortcoming being limited API support.
Finally, I combined all these technologies in Electron, ensuring they work smoothly on both the main process and the renderer process.
Ok, Is DrawingPics free to use?
80% of the project is free to use. I only charge a lifetime license for the effort I've put in, as I've spent 4 months on it and made numerous improvements based on user feedback. Currently, only Mac is supported. Windows support will be added later.
- kender 2y ago[flagged]
- endofreach 2y ago"Write reply to this HN post"-prompt fail?
- playingalong 2y agoForgotten dollar sign or whatever token is a reference to variable. text vs. $text And testing in production.
- BeatRipperz 2y ago[dead]
- ffitch 2y agoI wish someone came up with a rough sketch to neat pretty sketch generator, that’d be more practical for whiteboarding.
- phlsa 2y agoUnfortunately, the model download keeps failing for me. That being said, it looks promising and I love how simple the setup looks.
- arctanx 2y agoSorry for that, could you please provide screenshots or description in here: https://drawingpics.canny.io/feedback https://drawingpics.canny.io/feedback So that I can investigate the issue
- _lvbh 2y agoI’m confused. How is this different from ControlNets?
- arctanx 2y agoThe Controlnets model is too large(maybe 2GB? extra download), which means it requires more memory. The line control model that I used in drawingpics is about 300MB, which is suitable for light users
- xnx 2y agoI here a lot of people really like Stable Diffusion for Krita (https://github.com/Acly/krita-ai-diffusion https://github.com/Acly/krita-ai-diffusion). It's pretty easy to set up, powerful, local, and totally free.
- arctanx 2y agoKrita is a great drawing app, but the Krita SD plugin is not easy to set up. I think DrawingPics installation is more friendly, I wrote a blog about it https://docs.drawing.pics/index/getting-started/drawingpics-onboarding https://docs.drawing.pics/index/getting-started/drawingpics-...
- panki27 2y agoTo setup the Krita SD plugin, literally all you have to do is download it and import it into Krita. It installs the backend all by itself.
- arctanx 2y agoHi, I found the Krita SD plugin docs here: https://www.interstice.cloud/plugin https://www.interstice.cloud/plugin It seems it needs Docker to be installed? It might consume more power than running in a local Python environment. By the way, the plugin is great.
- s-dsds 2y agothat's "dockers" as in a "panel in the Krita interface" (which is somewhat modular), not Docker as in "Docker the container environment"
- mgkimsal 2y agoMinor typo: "Check You Email" should be "Check Your Email". Congrats on launch.
- arctanx 2y agoThx, fixed now.
- playingalong 2y agoThe pricing link in hamburger doesn't open for me. Chrome Android
- michaelmior 2y ago> The easiest way is to run a Python HTTP server, but the memory usage is too high. We need a more lightweight solution, so I used xmlrpc for memory efficiency Doesn't XML-RPC operate over HTTP?
- arctanx 2y agoIt does, but I used Flask previously for comparison, so I came to that conclusion.
- michaelmior 2y agoIt seems like you've already proved your conclusion false though if you ended up using an HTTP server anyway. The conclusion seems to be that Flask wasn't suitable for the task.
- arctanx 2y agoYou are right, the conclusion should be revised to: "I ended up using an HTTP server anyway, but I choosed the more efficient one", because Flask's http server is for testing only. Sorry for the unclear. I think using a Unix Socket could be more efficient but I don't want to be over-engineering :)
- artur_makly 2y agoCongrats on the launch.. this will inspire more young budding artists (kids especially) .. will email you from @visualsitemaps.com for more ideas and questions.