6 ms·
ggml started with focus on CPU inference, but lately we have been augmenting it with GPU support. Although still in development, it already has partial CUDA, Op
by ggerganov 3y ago
ggml started with focus on CPU inference, but lately we have been augmenting it with GPU support. Although still in development, it already has partial CUDA, OpenCL and Metal backend support
- qeternity 3y agoHi Georgi - thanks for all the work, have been following and using since the availability of Llama base layers! Wasn’t implying it’s CPU only, just that it started as a CPU optimized library.
- freedomben 3y agoAs a person burned by nvidia, I can't thank you enough for the OpenCL support
- ignoramous 3y ago(a novice here who knows a couple of fancy terms) > ...lately we have been augmenting it with GPU support. Would you say you'd then be building an equivalent to Google's JAX? Someone even asked if anyone would build a C++ to JAX transpiler [0]... I am wondering if that's something you may implement? Thanks. [0] https://news.ycombinator.com/item?id=35475675 https://news.ycombinator.com/item?id=35475675