ducviet00

7 ms·

1. 1.▲
   by ducviet00 3mo ago

   > openvino uses ONNX internally OpenVINO only uses ONNX to parse the model, not to execute it. It runs computations through its own highly optimized inference engine specifically designed for Intel hardware. It doesn't rely on the O
2. 2.▲
   by ducviet00 3mo ago

   Unlike GPUs, CPUs aren't designed for massive parallelism. Because of this, batching inference won't necessarily give you a speed boost here. In fact, it can actually slow the process down. Instead, I'd recommend exploring CP
3. 3.▲
   by ducviet00 1y ago

   Maybe 41.8% is the score of Qwen3-235B-A22B-Thinking-2507, lol. 11% for the non-thinking model is pretty high
4. 4.▲
   by ducviet00 2y ago

   AMD has great hardware, but their software is a different story. It’s poorly documented, unstable, and doesn’t deliver good performance for end users. I’ve been working with the AMD MI300X for a few weeks, trying to get matrix multiplicatio