madiator

7 ms·

1.
by madiator 3mo ago
Check out OpenThoughts. It has a widely used dataset, a model that beats the deepseek's smaller reasoning models, and a paper that talks in detail about the data curation methodology. https://www.open-thoughts.ai/
2.
Finetuning Is So Back(madiator.substack.com)
1 points by madiator 11mo ago|0 comments
3.
by madiator 11mo ago
I wrote about this recently as well: https://madiator.substack.com/p/finetuning-is-so-back
4.
by madiator 1y ago
Congrats! Great work, Dan! Indeed, Terminal bench is awesome :)
5.
by madiator 1y ago
Right. The title conjures dreamy images in my mind that I was dying to see images!
6.
by madiator 2y ago
That's the hope, but that's never how things go. Of course researchers and research will be tremendously affected, and that's a shame.
7.
by madiator 2y ago
Yeah this was quite an interesting finding, given how Mistral has marketed this (OCR solved!)
8.
What DeepSeek Means for the World(madiator.substack.com)
2 points by madiator 2y ago|0 comments
9.
Accidentally decensoring DeepSeek-R1 distilled models(twitter.com)
2 points by madiator 2y ago|0 comments
10.
Open Thoughts: Curating the best reasoning datasets(github.com)
8 points by madiator 2y ago|0 comments
11.
Bespoke-Stratos-17k: Open Reasoning Dataset by Distilling DeepSeek-R1(huggingface.co)
2 points by madiator 2y ago|0 comments
12.
by madiator 2y ago
The blogpost is linked here: https://news.ycombinator.com/item?id=42826392
13.
Bespoke-Stratos: The unreasonable effectiveness of reasoning distillation(bespokelabs.ai)
13 points by madiator 2y ago|2 comments
14.
by madiator 2y ago
Thanks! We created bespoke-stratos-32B - let me know if you have any questions.
15.
by madiator 2y ago
Synthetic data got a bad reputation last year, but it is now an important component for all modern LLMs! In fact, we had also trained one model for -- ironically -- detecting hallucinations, and it was also trained on synthetic data. Say if
16.
by madiator 2y ago
We are working on it!
17.
Show HN: Curator – an open-source library for synthetic data generation(github.com)
13 points by madiator 2y ago|6 comments
18.
by madiator 2y ago
Not sure why you are so upset about a small and neat study ("please, please stop"). If you ask it to summarize (without feeding the entire bible), it needs to know the bible. Knowledge and reasoning are not entirely disconnected
19.
by madiator 2y ago
For the specific form of hallucination, which is called grounded factuality, we have trained a pretty good model that can detect if a claim is supported by a context. This is super useful for RAG. More info at https://bespokelabs
20.
by madiator 2y ago
Now, it will be nice to get the code snippets for these!
21.
by madiator 2y ago
There are several types of hallucinations, and the most important one for RAG is grounded factuality. We built a model to detect this, and it does pretty well! Given a context and a claim, it tells how well the context supports the claim. Y
22.
by madiator 2y ago
That's great, but it did not really write the program that the human asked it to do. :)
23.
by madiator 3y ago
I am sure most people never asked these questions to a human doing this research.
24.
by madiator 4y ago
I will be typing this comment on my Nokia's keyboard.
25.
by madiator 4y ago
Yeah it wouldn't fit. GPT3 is 175B params, so even if you use 8 bit for each weight, you need 175×10^9÷2^30 = 163GiB of memory.
26.
by madiator 4y ago
I am the author of the blog and I want to thank for all the feedback here. Because I now do realize the article starts with a high premise but fails short. Will do a better job next time! What I meant to talk about is as follows: we have en
27.
Get Uncomfortable(newsletter.smarter.blog)
64 points by madiator 4y ago|65 comments
28.
Ask HN: Should I separate out my Twitter posting with different accounts?
2 points by madiator 4y ago|2 comments
29.
by madiator 4y ago
Here's an example I can think of. Suppose you have a bunch of text documents, and you know that some documents are similar but not identical (e.g. plagiarized and slightly modified). You want to find out which documents are similar. Yo
30.
by madiator 4y ago
And I am completely surprised by people who think this tech today is not good, and fail to account that it can get way better in the future.

More ›