4 ms·
You can't as far as I'm aware unless you control the entire batch during inference, or don't use batching which would require you to run your own inference.
by jmvoodoo 1y ago
You can't as far as I'm aware unless you control the entire batch during inference, or don't use batching which would require you to run your own inference.