5 ms·
You have your numbers switched. 27B is the dense model and runs slowly on unified memory. 35B (A3B active) runs great on unified memory.
by cpburns2009 22d ago
You have your numbers switched. 27B is the dense model and runs slowly on unified memory. 35B (A3B active) runs great on unified memory.