perone

7 ms·

1.
Where the wild Discovery Loops are(blog.christianperone.com)
1 points by perone 3mo ago|0 comments
2.
Gemma3n architecture: a short guide [slides](drive.google.com)
2 points by perone 11mo ago|0 comments
3.
Diffusion Elites: surprisingly good, simple and embarrassingly parallel(blog.christianperone.com)
9 points by perone 1y ago|0 comments
4.
by perone 1y ago
I share the same feeling, I think filesystems will have to reinvent themselves given the pace of how useful ML models became in the past years.
5.
by perone 1y ago
I'm planning to support MacOS, the only issue is with the encoders that I'm using now, I will probably work more on it next week to try to make a release that works on MacOS as well. Thanks !
6.
by perone 1y ago
Hi, there are no LLMs involved, it is all local and an embedding (vector representation) of the data is created and then that is used for search later, nothing is sent to cloud from your files and there are no local LLMs running as well, on
7.
by perone 1y ago
This is quite different than LanceDB. In VectorVFS I'm using the inodes directly to store the embeddings, there is no external file with metadata and db, the db is your filesystem itself, that's the key difference.
8.
by perone 1y ago
Hi, I think Rust won't bring much benefit here to be honest, the bottleneck is mainly the model and model loading. It would probably be a nightmare to load these models from Rust, I would have to use torch bindings and then convert eve
9.
by perone 1y ago
Hi, not sure if I understood what you meant by opaque embeddings as well, but the reason why files surface or not is due to the similarity score (which is basically the dot product of embeddings).
10.
by perone 1y ago
Hi, it is quite different, there is no LLM involved, we can certainly use it for a RAG for example, but what is currently implemented is basically a way to generate embeddings (vector representation) which are then used for search later, it
11.
by perone 1y ago
Thanks, I'm working on implementing the commands to clean the embeddings (you can now do that with Linux xattr command-line tool). I'm supporting CPU or GPU (NVIDIA) for the encoders and it only supports Linux at the moment.
12.
by perone 1y ago
Thanks. There is a bit of a nuance there, for example: you can build an index in first pass which will indeed be linear, but then later keep it in an open prompt for subsequent queries, I'm planning to implement that mode soon. But agr
13.
Show HN: VectorVFS, your filesystem as a vector database(vectorvfs.readthedocs.io)
279 points by perone 1y ago|138 comments
14.
Notes on Gilbert Simondon's "On the Mode of Existence of Technical Objects"(blog.christianperone.com)
2 points by perone 2y ago|0 comments
15.
by perone 2y ago
I'm not sure I agree about the data manifolds being too rigid. When we look at the quality score-based generative models and diffusion we can see a clear evidence of how flexible these representations are. We could say the same about s
16.
The geometry of data: the missing metric tensor and the Stein score [Part II](blog.christianperone.com)
64 points by perone 2y ago|7 comments
17.
Memory-Mapped CPU Tensor Between Torch, NumPy, Jax and TensorFlow(blog.christianperone.com)
1 points by perone 2y ago|0 comments
18.
by perone 2y ago
I wrote an article about it and S2 some time ago as well for those interested: https://blog.christianperone.com/2015/08/googles-s2-geometry...
19.
Large language model data pipelines and Common Crawl (WARC/WAT/WET) formats(blog.christianperone.com)
2 points by perone 3y ago|0 comments
20.
Appreciating the complexity of LLMs data pipelines(blog.christianperone.com)
1 points by perone 3y ago|0 comments
21.
by perone 3y ago
I find it interesting how everyone ignore EuclidesDB ( https://euclidesdb.readthedocs.io ) which came before Milvus and others in 2018, it is free and open-source. Same for all presentations from major DBs.
22.
PyTorch 2 Internals(slideshare.net)
4 points by perone 3y ago|0 comments
23.
PyTorch 2 Internals [slides](drive.google.com)
2 points by perone 3y ago|1 comments
24.
by perone 3y ago
https://feste.readthedocs.io/en/latest/design.html , this seems to be what Feste already does, and it offers automatic batching as well.
25.
by perone 3y ago
No.
26.
by perone 3y ago
I wrote an article about recent data pipelines as well if anyone is interested: https://blog.christianperone.com/2023/06/appreciating-llms-d...
27.
Appreciating the complexity of large language models data pipelines(blog.christianperone.com)
1 points by perone 3y ago|0 comments
28.
Appreciating the complexity of large language models data pipelines(blog.christianperone.com)
2 points by perone 3y ago|0 comments
29.
Show HN: Feste, an open-source framework to optimize and parallelize NLP tasks(feste.readthedocs.io)
2 points by perone 4y ago|0 comments
30.
Tutorial using LLVM to JIT PyTorch graphs to native code (x86/arm/RISC-V/WASM)(blog.christianperone.com)
2 points by perone 4y ago|0 comments

More ›