8 ms·
Any plans to support smaller models? I have a M4 Mac Mini with 16GB unified memory and an RTX 3060 (Laptop) with 6GB VRAM. My own product experiments all revolv
by brainless 1mo ago
Any plans to support smaller models? I have a M4 Mac Mini with 16GB unified memory and an RTX 3060 (Laptop) with 6GB VRAM. My own product experiments all revolve around small models and harness around them. Happy to contribute.
- mmastrac 1mo agoI've been pondering a smaller quantization and experts swapping for this! Happy to take on PRs if you want to experiment as well.