I am running Ornith 1.5 Q4 on my work issued laptop which surprisingly has 40gb of RAM and 20GB of “unified” VRAM. I am getting 50t/s of prefill and 5-6t/s of

I am running Ornith 1.5 Q4 on my work issued laptop which surprisingly has 40gb of RAM and 20GB of “unified” VRAM. I am getting 50t/s of prefill and 5-6t/s of inference. You can’t get crazy with it, but it’s pretty nice to just let it work away at simpler prompts overnight or during lunch break or while I am working on other things. Being able to literally a 30gb blob and then being able to talk with it while running on your machine it’s a different kind of magic compared to a far away magic in the cloud.