7 ms·
In my experience, LLMs tend to take noticeably longer to process images than text.
by ashed96 11mo ago
In my experience, LLMs tend to take noticeably longer to process images than text.
- psadri 10mo agoI wonder if these stay in the prefix cache?
- weird-eye-issue 10mo agoIt has to get the image data first, basically just IO time before processing it
- ashed96 10mo agoIIRC there's pre-processing (embedding/tokenization?) before feeding images to LLMs? Hit this issue optimizing LLM request times. Ending up lowering image resolution. Lost some accuracy but could bear that.