FlyingLawnmower

6 ms·

1.
by FlyingLawnmower 8mo ago
Very nicely written guide! If the authors or readers are interested in some of the more technical details of how we optimized guidance & llguidance, we wrote up a little paper about it here: https://guidance-ai.github.io&#x2F
2.
by FlyingLawnmower 1y ago
Good point re: documentation... We have support for Huggingface Transformers, llama.cpp, vLLM, SGLang, and TensorRT-LLM, along with some smaller providers (e.g. mistral.rs). Using any of these libraries as an inference host means you can us
3.
by FlyingLawnmower 1y ago
guidance can handle many context-free grammars. We use an Earley parser under the hood ( https://en.wikipedia.org/wiki/Earley_parser ) which gives us significant flexibility boosts over alternative approaches that use we
4.
by FlyingLawnmower 1y ago
We did quite a thorough benchmarking of various structured decoding providers in one of our papers: https://arxiv.org/abs/2501.10868v3 , measuring structured outputs providers on performance, constraint flexibility, do
5.
by FlyingLawnmower 1y ago
Thanks :) Great question re: adoption...it's definitely dominated by JSON. Most API providers have standardized on JSON outputs, so application teams have started building shims that map other formats to JSON and back. Similarly, with
6.
by FlyingLawnmower 1y ago
If you can screen tokens against your grammar fast enough, you can build a bitmask over the entire token vocabulary and apply it right before sampling. As vocabulary sizes grow, this gets more complex to do in real time, but we (and other l
7.
by FlyingLawnmower 1y ago
If your masking is fast enough, you can make it easily work with spec dec too :). We manage to keep this on CPU. Some details here: https://github.com/guidance-ai/llguidance/blob/main/docs/opt...
8.
by FlyingLawnmower 1y ago
OpenAI has started to (at least for tool calls): https://platform.openai.com/docs/guides/function-calling#con...
9.
by FlyingLawnmower 1y ago
We do enable forcing these sequences of tokens in guidance, and find that it significantly speeds up structured generation. There are tricky alignment issues to make sure you pick the right sequence of tokens, but you can often proxy this w
10.
by FlyingLawnmower 1y ago
I spent a couple years building a high performance, expressive library for structured outputs in LLMs. Our library is used by OpenAI for structured outputs on the hosted API. Happy to answer questions on how this works: User friendly librar
11.
by FlyingLawnmower 1y ago
Sidenote, but the scholarship on distillation always makes me a bit sad. The Original work, cited in the abstract of the Hinton, Vinyals, and Dean paper that is cited everywhere, was the model compression work from Caruana, Buciluǎ, and Nic
12.
by FlyingLawnmower 1y ago
Yes. They are about entirely different patient reports.
13.
Sequential Diagnosis with Language Models(arxiv.org)
2 points by FlyingLawnmower 1y ago|1 comments
14.
by FlyingLawnmower 1y ago
Artificial intelligence holds great promise for expanding access to expert medical knowledge and reasoning. However, most evaluations of language models rely on static vignettes and multiple-choice questions that fail to reflect the complex
15.
by FlyingLawnmower 2y ago
https://chatgpt.com/share/6760a122-0ec4-8008-8b72-3e950f0288... My first try with o1. Seems right to me…what does this teach us about LLMs :)?
16.
by FlyingLawnmower 2y ago
Interesting paper, but their reason for dismissing constrained decoding methods seems to be that they want to academically study the in-context setting. For practitioners, using a framework like Guidance which forces the models to write val
17.
by FlyingLawnmower 2y ago
Last I checked, physician pay represents a fairly small (single digit) fraction of healthcare expenditure. If I recall right, administrative overhead and insurance is significantly more of a contributor to pricing.
18.
Prompt Engineered GPT-4 Beats Gemini on all of Google's text benchmarks(microsoft.com)
21 points by FlyingLawnmower 3y ago|3 comments
19.
by FlyingLawnmower 3y ago
Late to the thread here, but the paper announcing Med-PaLM ( https://arxiv.org/abs/2212.13138 ) does not report many benchmark results on Med-PaLM and is instead mostly about Flan-PaLM 540B (which is compared against in
20.
GPT-4 beats ChatGPT by over 30 points on Medical Challenge Problems(microsoft.com)
1 points by FlyingLawnmower 3y ago|1 comments
21.
by FlyingLawnmower 3y ago
"We present a comprehensive evaluation of GPT-4, a state-of-the-art LLM, on medical competency examinations and benchmark datasets. GPT-4 is a general-purpose model that is not specialized for medical problems through training or engin
22.
by FlyingLawnmower 5y ago
Do you have some sources where I can read up more on this? I have always believed that the cost of front line worker salaries (Doctors/Nurses) was a relatively small % of the total cost of healthcare. Studies like this [1] led me to be
23.
by FlyingLawnmower 5y ago
Have you thought about using central/global differential privacy (which tends to have much less noise) on the "high level aggregates" or "aggregated datasets" that persist after the research study ends? E.g. from th
24.
by FlyingLawnmower 6y ago
Check out this work from Rich Caruana & collaborators on model compression: http://www.cs.cornell.edu/~caruana/compression.kdd06.pdf which was a precursor to the model distallation work from Geoff Hinton: https:&#
25.
by FlyingLawnmower 6y ago
Hats off to people like Carlos Guestrin & John Giannandrea, who seem to have pushed a cultural shift through Apple. I didn't think I'd see a site like this a few years ago, based on Apple's historic reputation of strongly
26.
by FlyingLawnmower 7y ago
I don't understand how Facebook got a $5B fine, yet Equifax gets a ~$650m fine. The data breached in the Equifax case seems to cause far more direct harm, and affected many more Americans. It feels like the 10x difference should go the
27.
by FlyingLawnmower 8y ago
Can I ask what your business model or pathway to monetization is? Thanks for making it run locally, I can finally give it a real try now.
28.
by FlyingLawnmower 8y ago
I like the sentiment, but unfortunately predefining a set of categories doesn't prevent the issues. Even without access to a specific attribute like "wealth" or "race", the learning algorithms used here can still be
29.
by FlyingLawnmower 8y ago
Thanks for the site! Do you have thoughts on removing all the print/console.logs calls on every line? It feels like there's a lot of visual noise there, that can still be met with the comments showing state/output.
30.
by FlyingLawnmower 8y ago
Thank you for sharing your experience and journey with us. It's disheartening to see that your courage in speaking up and sharing is rewarded with such a negative attitude. I appreciate your attitude.

More ›