varunshenoy

109 ms·

1.
by varunshenoy 2y ago
I've been playing with AI agents for months, and most of them are pretty bad. They often get stuck in loops, which is frustrating. This happens in MultiOn, AutoGPT, and others. I've used Devin a few times (see: https://
2.
Where are the good AI products?(varunshenoy.substack.com)
2 points by varunshenoy 3y ago|0 comments
3.
Super JSON Mode: Up to 20x Faster JSON Generation from LLMs(github.com)
2 points by varunshenoy 3y ago|0 comments
4.
Show HN: Generate Cover Art for Spotify Playlists(coverart.lol)
2 points by varunshenoy 3y ago|1 comments
5.
by varunshenoy 3y ago
Hey guys! Just wanted to share a fun side project. Code is here: https://github.com/varunshenoy/latentverse
6.
Generating VR worlds with Stable Diffusion in < 5s(latentverseai.com)
6 points by varunshenoy 3y ago|2 comments
7.
by varunshenoy 3y ago
Slightly different set of trade-offs, but similar mental model. You always use large batch sizes (compute bound) and the bottleneck usually ends up communication between GPUs/nodes.
8.
by varunshenoy 3y ago
Good question. Yes, the 10GB available for batching is in the HBM. In a single forward pass, you move the entire model from HBM -> SRAM exactly once. In a batched forward pass, this is still the case, so you end up doing more compute for
9.
by varunshenoy 3y ago
Thanks! vLLM for quick set up, TRT-LLM for best performance. Both available on https://baseten.co/ .
10.
by varunshenoy 3y ago
Absolutely. Looks like the M1 Ultra has 800GB/s of memory bandwidth and ~20 TFLOPS of compute. The same calculations from the post should hold, except with these new values.
11.
by varunshenoy 3y ago
:)
12.
A guide to open-source LLM inference and performance(baseten.co)
113 points by varunshenoy 3y ago|14 comments
13.
Show HN: Misgif – AI Generated GIFs(misgif.app)
2 points by varunshenoy 3y ago|0 comments
14.
by varunshenoy 3y ago
Awesome job guys, and thank you for creating it. Curious if you guys have any insights on long-term memory and if there are better ways to do retreivel apart from top-k. Seems weird that every RAG app uses top-k especially since you might p
15.
Why Open Source AI Will Win(varunshenoy.substack.com)
1 points by varunshenoy 3y ago|0 comments
16.
How we got Stable Diffusion XL inference to under 2 seconds(baseten.co)
51 points by varunshenoy 3y ago|5 comments
17.
by varunshenoy 3y ago
You can write an extension to support LoRA (~10 lines of Python HF Diffusers code). If you get to this before me, please create a PR!
18.
by varunshenoy 3y ago
LoRAs can be handled as a straight-forward Python extension!
19.
by varunshenoy 3y ago
If anyone wants to add SDXL support, all you have to do is create a new extension with the correct SDXL logic (loading from HF diffusers, etc.). You could parameterize `num_inference_steps`, for example, to delegate decisions to the user of
20.
by varunshenoy 3y ago
done :)
21.
Opendream: A layer-based UI for Stable Diffusion(github.com)
472 points by varunshenoy 3y ago|137 comments
22.
How do embeddings work? (in under 10 minutes)(generativemagic.substack.com)
3 points by varunshenoy 3y ago|0 comments
23.
Natural language is an unnatural interface(varunshenoy.substack.com)
154 points by varunshenoy 3y ago|88 comments
24.
Coauthor: Open-Source Text to LaTeX in Overleaf(github.com)
5 points by varunshenoy 4y ago|0 comments
25.
Please Build More Silly Things(varunshenoy.substack.com)
1 points by varunshenoy 4y ago|1 comments
26.
by varunshenoy 4y ago
Agreed. Definitely a drawback of this technique — you might not get the exact specificity you want. In general, GraphGPT tends to be very conservative in adding nodes/relationships. Not sure why, but probably deserves more investigatio
27.
by varunshenoy 4y ago
If anyone wants to play around without having to set everything up: https://graphgpt.vercel.app/ Just bring your own OpenAI API Key.
28.
by varunshenoy 4y ago
We've been messing with a Vercel deployment so you might've seen that :) https://graphgpt.vercel.app/ It's not quite battle tested, but think I gotta sleep and take a look at it tomorrow.
29.
by varunshenoy 4y ago
Handling state (especially long-term) is really a struggle for LLMs right now. This issue should become easier to work with as context windows scale up in the next couple years (or months, who knows!).
30.
by varunshenoy 4y ago
Yup, this is correct. Huge fan of Arc!

More ›