6 ms·
Not only are LLMs perfectly equipped for command-line interfaces, Linux is also the most performant OS for hosting the models locally.
by drak0n1c 26d ago
Not only are LLMs perfectly equipped for command-line interfaces, Linux is also the most performant OS for hosting the models locally.
- dionian 26d agoHow so? Compared to say Mac OS? (Just curious)
- tankenmate 26d agoThe biggest slow down on MacOS compared to Linux is the default filesystem APFS (but hey HFS and HFS+ were also slow). But not just that, the Linux dirent cache is optimised to within an inch of its life (it's deliberately sized to match CPU cachelines among other optimisations). And one of the chunks of that slow down is upper / lower case checks (Mac filesystems be default treat upper and lower case as the same thing when it comes to matching a filename). Because of that Linux works so much faster with small files, and text files are small compared to audio, video, etc.