looobay

8 ms·

1.
by looobay 10mo ago
In the Pavel case, it involved child pornography groups on Telegram and the fact that they ignore a court order. But I agree with you for the authoritarian logics in Europe (even America) with Chat Control and other actions like the French
2.
by looobay 10mo ago
really cool!
3.
by looobay 10mo ago
That's an awesome project! It's literally a gold mine lol. Congrats and thank you for this!
4.
by looobay 11mo ago
It means something that is too out-of-data. For example if you try to make an LLM write a program in a strange or very new language it will struggle in non-trivial tasks.
5.
by looobay 11mo ago
There was research on LLMs training and distillation that if two models have a similar architecture (probably the case for Xai) the "master" model will distill knowledge to the model even if its not in the distillation data. So th
6.
by looobay 11mo ago
He received money from Libya for his presidential campaign [0], he's just a criminal ex-president... [0]: https://en.wikipedia.org/wiki/Libyan_financing_in_the_2007_F...
7.
by looobay 11mo ago
You should read the 6th page of the paper (and page 5 for architecture breakdown), they show that they are compressing the vision tokens with convolution to keep a strong semantic understanding and keep a small amount of tokens. But I think
8.
by looobay 11mo ago
Vision tokens are a good compression medium because with one vision token you have one vector of N elements, but with textual tokens you have M vectors of N elements, because one vision token represent multiple pixels (and possibly multiple
9.
by looobay 11mo ago
LLMs are compute heavy with quadratic scaling (in compute) per tokens. They are trying to compress text tokens into visual tokens with their VLM. Maybe they would render texts to an image before tokenizing to reduce the compute cost.