hellovai

7 ms·

1.
by hellovai 9mo ago
(on of the creators of BAML here) yep! exactly! that workaround we've found works quite well, but the problem is that its not sufficient to just retry in the case of failed schema matches (its both inefficient and also imo incorrect).
2.
A Cautionary Tale on Vibes(boundaryml.com)
1 points by hellovai 10mo ago|1 comments
3.
BAML is hiring compilers/rust engineers (YC W23)(github.com)
1 points by hellovai 10mo ago
4.
by hellovai 1y ago
if you haven't tried the research -> plan -> implementation approach here, you are missing out on how good LLMs are. it completely changed my perspective. the key part was really just explicitly thinking about different levels of
5.
by hellovai 1y ago
Have you tried techniques that don’t require modifying the LLM and the sampling strategy for structure outputs? For example, schema aligned passing, where you build error tolerance into the parser instead of coercing to a grammar. https:&#
6.
by hellovai 1y ago
if you share your prompt with me on promptfiddle.com i can play around with it and see how i can make it better!
7.
by hellovai 1y ago
its a bit more nuanced than applicative lifting. parts of of SAP is that, but there's also supporting strings that don't have quotation marks, supporting recursive types, supporting unescaped quotes like: `"hi i wanted to say
8.
by hellovai 1y ago
appreciate you tyring it. the reason it dropped the day was due to your type system not being understood by the LLM you're using. the model replied with { "Text": "coffee liqueur", &quot
9.
by hellovai 1y ago
have you tried schema-aligned parsing yet? the idea is that instead of using JSON.parse, we create a custom Type.parse for each type you define. so if you want a: class Job { company: string[] } And the LLM happens to output: {
10.
by hellovai 1y ago
really cool to see BAML on here :) 100% align on so much of what you've said here. its really about treating LLMs as functions.
11.
by hellovai 2y ago
yea! even deepseek. Calling an external function / tool calling is really just a data extraction problem. say you have a tool: def calculator(expr: str) -> float then the model just needs to say: { "function": "calc
12.
Deepseek R1: Adding Function Calling/Tool use(boundaryml.com)
2 points by hellovai 2y ago|0 comments
13.
Show HN: GitHub Wrapped – analyzing every file change in every commit(wrapped.dev)
6 points by hellovai 2y ago|0 comments
14.
Show HN: PromptFiddle – Open-source WASM-based LLM playground(promptfiddle.com)
2 points by hellovai 2y ago|0 comments
15.
by hellovai 2y ago
i 100% percent agree. people get so caught up on trying to do everything 90% right with AI, but they forget there's a reason most websites offer at least 2 9's of uptime.
16.
by hellovai 2y ago
We have some preliminary data with llama3.1 and we find that the smaller model gets to around 70% with BAML (+20% from base), but we'll update this dashboard with llama3.1 by end of week!
17.
Show HN: Beating OpenAI's structured outputs on cost, accuracy and speed(boundaryml.com)
8 points by hellovai 2y ago|2 comments
18.
by hellovai 2y ago
Take a look at BAML (boundaryml.com) Its a different take that leverages a DSL to make prompting cleaner and fixes a few other ergonomic issues along the way. you can try it online at promptfiddle.com
19.
by hellovai 2y ago
nothing specific, but you can try our prompt / datamodel out on https://www.promptfiddle.com or if you're open to share your prompt / data model with, I can send over my best guess of a good prompt! We've fou
20.
by hellovai 2y ago
we recently added dynamic type support with this snippet! (docs coming soon!) Python: https://github.com/BoundaryML/baml/blob/413fdf12a0c8c1ebb75c... Typescript: https://github.com/BoundaryML&
21.
by hellovai 2y ago
oh thats really interesting, how often do you get errors like that? fyi, we actually fix those specific errors in our parser :)
22.
by hellovai 2y ago
The main drawback is really when you attempt to do more advanced prompting techniques like chain-of-thought or reasoning. forcing those parts to be json, can be hard and unnecessarily constrain the model. e.g. https://www.promptf
23.
by hellovai 2y ago
;) https://www.promptfiddle.com/structured-summary-66myE (sorry bad syntax highlighting when including baml code in baml code) { author: "Sam Lijin" key_points: [ "Structured output from LLMs, like JSON, is
24.
by hellovai 2y ago
that's a great question, there's three main benefits: 1. seeing the full prompt, even though that python code feels leaner, somehow you need to convert it to a prompt. a library will do that in some way, BAML has a VSCode playgrou
25.
by hellovai 2y ago
the main one is that most people don't own the model. so if you use openai / anthropic / etc then you can't use token masking. in that case, reprompting is pretty much the only option
26.
by hellovai 2y ago
thats pretty cool! We'll update the page after taking a look at the library!
27.
by hellovai 2y ago
our paid product is still in Beta actually as we're continuing to build it out, but BAML itself is and always will be open source (runs fully locally as well - no extra network calls). in terms of parsing, I do think we're likely
28.
by hellovai 2y ago
Thanks! We should add that to the docs haha. But the here's a few: - keys without strings - coercing singular types -> arrays when the response requires an array - removing any prefix or suffix tags - picking the best of many JSON c
29.
by hellovai 2y ago
not a noob question, here's how the LLM works: ``` prompt = "..." output = [] do: token_probabilities = call_model(prompt) best_token = pick_best(token_probabilities) if best_token == '<END>': b
30.
by hellovai 2y ago
XML is also a great option, but there are a few trade offs: > XML is a many more tokens (much slower + $$$ for complex schemas) > regardless of if you're looking for } or </output> its really a matter of "does you

More ›