pythongiant

7 ms·

1.
Bodhan.ai releases 1.2B ASR model that covers 22 Indian Languages(bodhan.ai)
2 points by pythongiant 10d ago|0 comments
2.
by pythongiant 1mo ago
Tenosra is all about making AI more efficient and resourceful you can read more abour it at: https://www.tenosra.com/proxima
3.
Show HN: Proxima serves 4x more requests with no hardware change on vLLM(github.com)
3 points by pythongiant 1mo ago|1 comments
4.
by pythongiant 2mo ago
Hey guys, i'm especially interested in feedback on the kernel design and integration onto mlx lm. Its on pypi as well as a simple pip install mlx-turboquant :P
5.
Show HN: TurboQuant for mlx-lm (Apple Silicon)(github.com)
1 points by pythongiant 2mo ago|1 comments
6.
by pythongiant 4mo ago
KVBoost is a drop-in replacement for AutoModelForCausalLM. Same API surface (KVBoost.from_pretrained(...), engine.generate(...)), but with cross-request KV reuse, FlashAttention-2, AWQ layer streaming, and speculative decoding bolted on.
7.
by pythongiant 4mo ago
Here's the repository incase anyone wants to have a look at the code. leave a star if you find it interesting :P https://github.com/pythongiant/KVBoost
8.
by pythongiant 4mo ago
Thanks! this is a weekend project that i am working on in the side just to learn more about ml engineering and custom cuda kernels. didnt think much about the website
9.
by pythongiant 4mo ago
Makes sense, fixing that. thanks!
10.
by pythongiant 4mo ago
my initial choice was to use Rust for this actually (Probably should've too :P) but i went with python for an initial mvp/skeleton for a future rewrite
11.
by pythongiant 4mo ago
KVBoost is a chunk-level KV cache reuse library for HuggingFace models (pip install kvboost). It supports two recompute strategies (selective boundary and CacheBlend), int8/int4 KV quantization for 2–4x RAM reduction, disk-backed cold
12.
Show HN: KVBoost – chunk-level KV cache reuse for HuggingFace, 5–48x faster TTFT(pythongiant.github.io)
20 points by pythongiant 4mo ago|18 comments
13.
CUDA Programming: From Zero to GPU Kernels – A Beginner's Guide(pythongiant.github.io)
1 points by pythongiant 8mo ago|1 comments
14.
by pythongiant 8mo ago
just published a beginner-friendly, hands-on guide to GPU programming with CUDA aimed at folks who are comfortable with basic programming but new to parallel computing. Instead of throwing API references and jargon at you, this guide takes
15.
Show HN: I built GPT from scratch to understand how it works(pythongiant.github.io)
7 points by pythongiant 8mo ago|0 comments