6 ms·
Rust is a lot better than Python to run ML in production
- mectors 2y agoPython Docker images with CUDA, Python, Pytorch,... are 5GB to 10GB of third-party code. Here is an example of bringing this down to 1.13GB thanks to WasmEdge, 10MB of LlamaEdge API Server [compatible with ChatGPT] and 1.17GB for TinyLlama.
- verdverm 2y agoIf you are not actually running ML code, of course you can remove Python and ML libraries. You could get this down to MB by removing WASM and using a Go binary apples and oranges
- deleted 2y ago[deleted]