sunshinesfbay

11 ms·

1. 1.▲
   by sunshinesfbay 2y ago

   I think this is not about new Pytorch features, although it requires the latest Pytorch and Executorch making me think that some features in pytorch and executorch got extended optimized for this use case? What makes this cool is that you c
2. 2.▲
   by sunshinesfbay 2y ago

   It's an end to end solution that supports the same model from server (including OpenAI API!) to mobile. To the extent that you just want to run on one specific platform, other solutions might work just as well?
3. 3.▲
   by sunshinesfbay 2y ago

   Pretty cool! What are the steps to use these on mobile? Stoked about using ollama on my iPhone!
4. 4.▲
   by sunshinesfbay 2y ago

   It would appear that Flash-3 is already something that exists for PyTorch based on this joint blog between Nvidia, Together.ai and Princeton about enabling Flash-3 for PyTorch: https://pytorch.org/blog/flashattention-3&