5 ms·
Fair warning, I have found local models and frontier models to be very bad at the specifics when it comes to cars. Small differences like month and year model
by ehnto 24d ago
Fair warning, I have found local models and frontier models to be very bad at the specifics when it comes to cars.
Small differences like month and year model can impact oil capacity, oil weight and things like that, the details that matter quite a bit.
I found frontier models couldn't get things like what engine was in a 1994 Nissan Skyline, one of the more infamous and talked about cars on internet forums for decades, with dedicated fan databases that would have been scraped.
Questions like "what air filter do I need for my 1994 Suzuki Swift?" are hit and miss.
- javier123454321 24d agoThis seems to be trained(? or referencing) on the specific cars' owner's manual.
- deleted 24d ago[deleted]
- efskap 24d agoYeah referencing is the way to go, as even finetuning probably captures style more than concrete facts. I know with large context windows we don't really RAG anymore, but for owner's manual lookup with a smaller model it seems ideal. Something every LLM user ends up learning is that they're far better used as search and summarization tools than as knowledge databases in themselves.
- HPsquared 24d agoThe hard part is finding the correct owners manual. Also in many cases the owners manual is generic and covers many options and trim levels. Edit: this applies to service manuals too
- Kim_Bruning 24d agoA model's internal knowledge is great! It's useful as initial priors to speed up the REAL search.
- whalesalad 24d agoJust a few hours ago I gave ChatGPT my window sticker and the installation manual for a new suspension setup. I asked for new hardware that would typically be replaced during this install, like torque-to-yield bolts and fasteners. I also asked for new oil filters. I got a comprehensive grid of the exact part numbers needed in a nice dense table. sol 5.6 high is my daily driver.
- ssl-3 24d ago> sol 5.6 high is my daily driver. Mine, too. Did you fact-check the output? I've found the bot to be particularly bad at stuff like producing accurate lists of parts to replace during automotive projects.
- whalesalad 24d agoSo far so good, hasn’t failed me yet. It’s done a stellar job chasing down parts for my cub cadet lawn mower too. Sorted out mid year model revisions and everything. I just gave it the sticker under the seat.
- ssl-3 24d agoIt makes stuff up about my car all the time, even when provided with factory and/or aftermarket service manuals to peruse in a custom GPT. That car is a ridiculously-common Honda Odyssey with an engine that is even more-common. I do sincerely wish you good luck with your suspension project.
- ygouzerh 24d agoFor prompts that needs fact-checking, I like these days to use Perplexity directly instead these days. It's way faster than the default websearch tool + give a link to the reference directly.
- HPsquared 24d agoYou really need something smart enough to browse through the parts etc suppliers and find links.
- alexandra_au 24d agoIt's nothing that tool calling/feeding it the correct information can't solve
- embedding-shape 24d agoEven so, in this case, author is using UD-Q3_K_S dynamic weights for Qwen3.6-35B-A3B, it will be dumb. Even the BF16 weights do stupid stuff like missing to confirm all parameters are defined when doing "rm -rf directory/$id", so it ends up deleting more than expected, I can't imagine the Q3 are actually useful for anything serious, even with tool calling or what not.
- petruspennanen 23d agoI've been very impressed by it's intelligence and lack of hallucinations. The dynamic Q3 is a good balance between accuracy and size keeping the 35B just below 16 GB. It is not supposed to know everything, it is your car. It actively disengages from off topic chatter (too slow for that anyway), better spend that time feeling the car. It keeps itself grounded on sensor input. One principle per wheel. assert only what you can sense, claim only what is verified, label anything interim loudly, and report failure plainly with no silver lining. Everything above those four patches is just suspension.
- embedding-shape 23d ago> . The dynamic Q3 is a good balance between accuracy and size keeping I'm having a hard time understanding how you find any sort of accuracy in Q3, when I use it with BF16 and it's hardly usable due to drastic hallucinations and inability for system prompt following. But, if it works for you, that's pretty good! Guess I'm jealous :)
- petruspennanen 23d agoHmm we must be doing something different, I think the official accuracy measurement for the Q3 is more than 90%. Maybe some bug affecting your test?