21 ms·
> vLLM and SGLang are complex, and bugs are common This for me is the heart of the issue. Feature creep will lead to the downfall of all these frameworks. Today
by ulam2 23d ago
> vLLM and SGLang are complex, and bugs are common
This for me is the heart of the issue. Feature creep will lead to the downfall of all these frameworks. Today, we can conjure our own bespoke inference engine for our own hardware in no time. It need only support a few modern model architectures. The code can be audited too. I think the article highlights an important gap area for the industry.