8 ms·Speculative pre-positioning: off-path decode for stateful inference sessions1 points by logotype 3mo agologotype 3mo agoWith native support for sessions in an inference engine we can make use of idle GPUs... doing work!