Tananon

13 ms·

1.
by Tananon 11mo ago
Nice, I actually read that Jina article when it was published, but forgot they use facility location as well! The saturated coverage algorithm looks pretty interesting, I'll have a look at how feasible it would be to add that to Pyvers
2.
by Tananon 11mo ago
That's indeed something I plan to add in the near future. I'll probably add a tutorial as well to showcase how you can use this with e.g. sentence transformers. There's some pretty good benchmarks in the paper that I used as
3.
by Tananon 11mo ago
True, I think that's also a great usecase! Though these algorithms likely won't scale to very large datasets (e.g. millions of samples), but for smaller datasets, like fine-tuning sets, I think this would work very well. I've
4.
Show HN: Pyversity – Fast Result Diversification for Retrieval and RAG(github.com)
86 points by Tananon 11mo ago|11 comments
5.
Show HN: Pyversity – Fast Result Diversification for Retrieval and RAG(github.com)
4 points by Tananon 11mo ago|0 comments
6.
by Tananon 1y ago
I think you are referring to for "batch in sentences.chunks(batch_size)"? This is not actually chunking sentences, chunks() is simply an iterator over a slice (in this case, a slice of all our input sentences of length batch_size)
7.
by Tananon 1y ago
Awesome to hear! It's great to see the Rust ML ecosystem growing, and we hope we can be a small part of it. Don't hesitate to reach out with any ideas or requests!
8.
by Tananon 1y ago
We support loading from both local as well as Hugging Face paths with from_pretrained! So let model = StaticModel::from_pretrained("my_custom_model", None, None, None)?; will work.
9.
by Tananon 1y ago
Thanks so much for the kind words, that's awesome to hear! If you have any ideas or requests, don't hesitate to reach out!
10.
by Tananon 1y ago
Indeed, I also didn't expect it to be so much faster! I think it's because most of the time is actually spent on tokenization (which also happens in Rust in the Python package), but there is some transfer overhead there between Ru
11.
by Tananon 1y ago
It depends a bit on the task and language, but my go-to is usually minishlab/potion-base-8M for every task except retrieval (classification, clustering, etc). For retrieval minishlab/potion-retrieval-32M works best. If performance
12.
Show HN: Model2vec-Rs – Fast Static Text Embeddings in Rust(github.com)
60 points by Tananon 1y ago|15 comments
13.
Show HN: SemHash – Semantic Text Deduplication, Outlier Filtering and Sampling(github.com)
7 points by Tananon 1y ago|0 comments