4 ms·
If you prefer C++ and CUDA, then there's also tiny-vllm of mine [0] - recently we broke 1k gh stars [0] https://github.com/jmaczan/tiny-vllm https://github.com
by yu3zhou4 1mo ago
If you prefer C++ and CUDA, then there's also tiny-vllm of mine [0] - recently we broke 1k gh stars
[0] https://github.com/jmaczan/tiny-vllm https://github.com/jmaczan/tiny-vllm