edunteman

5 ms·

1.
by edunteman 3mo ago
Nice work! Had to do similar serverside for stripping named entities and ended up burning way too much LLM cash
2.
by edunteman 4mo ago
I hate to be that guy, but you can just choose to do these things. Learn to set SMART goals. Then set just one, so there's no excuses around priorities. Obviously, choose well. Make that goal your life.
3.
by edunteman 6mo ago
I’ve been a big fan of “what’s the thinnest this could be” interpretations of sandboxes. This is a great example of that. On the other end of the spectrum there’s just-bash from the Vercel folks.
4.
by edunteman 7mo ago
The llm detector in my brain went off too
5.
by edunteman 7mo ago
Your repo was actually a major point of reference! Thank you for open sourcing it. Ironically when I first got into zig I built a similar generator for python bridging which your project reminded me of https://github.com/eri
6.
by edunteman 7mo ago
Exactly, that’d be the intention. For compute-heavy or long running jobs you’d still probably want a dedicated VM like on E2B but for quick stuff, bVisor
7.
by edunteman 7mo ago
Correct, your PATH resolves to your local tools as if it was unprotected bash, but syscalls are filtered/virtualized
8.
by edunteman 7mo ago
Hell yeah, love to hear it! Happy to answer any questions or issues you run into
9.
Show HN: BVisor – An Embedded Bash Sandbox, 2ms Boot, Written in Zig(github.com)
24 points by edunteman 7mo ago|9 comments
10.
by edunteman 8mo ago
The part that most resonates with me is the lingering feeling of “oh but it must be my fault for underspecifying” which blocks the outright belief that models are just still sloppy at certain things
11.
Just-in-Time Thanksgiving Cookbook(cookwithbutter.com)
2 points by edunteman 10mo ago|0 comments
12.
by edunteman 11mo ago
Good question, I imagine you’d need to set up an ngrok endpoint to tunnel to local LLMs. In those cases perhaps an open source (maybe even local) version would make more sense. For our hosted version we’d need to charge something, given sto
13.
by edunteman 11mo ago
Ah yes that makes sense, have heard of those cases too but hadn’t put much thought into it. Thanks for pointing it out!
14.
by edunteman 11mo ago
I’d love your opinion here! Right now, we assume first call is correct, and will eagerly take the first match we find while traversing the tree. One of the worst things that could currently happen is we cache a bad run, and now instead of o
15.
by edunteman 11mo ago
Very, very common approach! Wrote more on that here: https://blog.butter.dev/the-messy-world-of-deterministic-age...
16.
by edunteman 11mo ago
Awesome to hear you’ve done similar. JSON artifacts from runs seem to be a common approach for building this in house, similar to what we did with the muscle mem. Detecting cache misses is a bit hard without seeing what the model sees, part
17.
by edunteman 11mo ago
I feel the same - we’ll use it as long as we can since it’s customer aligned but I wouldn’t be surprised if competitive or COGs costs force us to change in the future.
18.
by edunteman 11mo ago
It’s bring-your-own-key, so any calls proxied to OpenAI just end up billing directly to your account as normal. You’d only pay Butter for calls that don’t go to the provider. That’d be a separate billing account with butter.
19.
by edunteman 11mo ago
I couldn’t see how it wouldn’t be, as it’s a free market opt-in decision to use Butter
20.
Show HN: Butter – A Behavior Cache for LLMs(butter.dev)
50 points by edunteman 11mo ago|22 comments
21.
The Messy World of Deterministic Agents(blog.butter.dev)
3 points by edunteman 11mo ago|0 comments
22.
by edunteman 1y ago
I've got a blog on this from the launch of Muscle Mem, which should paint a better picture https://erikdunteman.com/blog/muscle-mem Computer use agents (as an RPA alternative) is the easiest example to reach to: U
23.
by edunteman 1y ago
Thanks! For langchain you can repoint your base_url in the client. Autogpt I'm not as familiar with. Closed loop robotics using LLMs may be a stretch for now, especially since vision is a heavy component, but theoretically the patterns
24.
by edunteman 1y ago
An interesting alternative product to offer is injecting prompt cache tokens into requests where they could be helpful; not bypassing generations but at least low hanging fruit for cost savings
25.
Show HN: Butter, a muscle memory cache for LLMs(docs.butter.dev)
23 points by edunteman 1y ago|11 comments
26.
by edunteman 1y ago
I'd argue that understanding disassembled assembly could be considered reverse engineering, which would logically extend to source code unless we draw the line at compilation
27.
by edunteman 1y ago
Love your sleep time stuff! It's an inspiration for Muscle Mem
28.
by edunteman 1y ago
Great to see you here :) thanks for the kind words!
29.
by edunteman 1y ago
totally follows! thanks for sharing, will noodle on it
30.
by edunteman 1y ago
Definitely, but there's lots of API surface area to figure out first, so will stick strictly to python for now. I can see a future where this ends up with a native storage/query core and with python/js bindings on top.

More ›