mryab

7 ms·

1. 1.▲
   by mryab 4y ago

   There is! See https://petals.ml/ for inference of models like BLOOM-176B over the internet or https://arxiv.org/abs/2301.11913 and https://arxiv.org/abs/2206.01288 that show you h
2. 2.▲
   by mryab 4y ago

   There absolutely are! Check out hivemind ( https://github.com/learning-at-home/hivemind ), a general library for deep learning over the Internet, or Petals ( https://petals.ml/ ), a system that leverages H
3. 3.▲
   by mryab 6y ago

   For now, the only separation we have is that each worker is responsible for its own weights, since network security has not been our top priority. Still, we've been thinking about adding some security measures like proof-of-work for ea
4. 4.▲
   by mryab 6y ago

   Thank you! This is definitely something we should look into in the future (hopefully with community help); as of now, training infrastructure and model convergence are the highest priorities. That said, we welcome all ideas of ways to moti
5. 5.▲
   by mryab 6y ago

   Sure! It's @m_ryabinin
6. 6.▲
   by mryab 6y ago

   Not directly related, but the Learning@home [1] project aims to achieve precisely that goal of public, volunteer-trained neural networks. The idea is that you can host separate "experts," or parts of your model (akin to Google&#x2
7. 7.▲
   by mryab 6y ago

   Here is a recent paper (disclaimer: I am the first author) named "Learning@home" which proposes something along these lines. Basically, we develop a system that allows you to train a network with thousands of "experts" d