37 ms·

when I try to do this (MBP M1 Max, Sonoma) I get 'killed' immediately

carbocation 3y ago
Same on an M1 Max 64G, Ventura. Xcode is installed[1]. 1 = ``` $ xcode-select --install xcode-select: error: command line tools are already installed, use "Software Update" in System Settings to install updates ```
- carbocation 3y ago
  For whatever it's worth, the SHA sum is correct. The killed message is uninformative, looks like what happens when I'm OOM (but I have 64GB RAM of which only 24 is used for anything at the moment). $ sha256sum < llamafile-server-0.1-llava-v1.5-7b-q4 a138c5db9cff3b8905dd6e579c2ab6c098048526b53ae5ab433ff1d1edb9de24 - $ ./llamafile-server-0.1-llava-v1.5-7b-q4 Killed: 9
  - carbocation 3y ago
    Looks like this may be due to crowdstrike, which I also have installed on this machine: https://github.com/Mozilla-Ocho/llamafile/issues/14#issuecomment-1834313659 https://github.com/Mozilla-Ocho/llamafile/issues/14#issuecom...
    - raajg 3y ago
      oh wow, I would have never imagined that 'that' would be preventing me from running llama + llava! Confirming that I have Crowdstrike running too.
derwiki 3y ago
On a Macbook Pro M2, I get $ ./llamafile-server-0.1-llava-v1.5-7b-q4 [2] 25224 illegal hardware instruction ./llamafile-server-0.1-llava-v1.5-7b-q4
- jart 3y ago
  Could you disable SIP and run `lldb -- $TMPDIR/.ape-1.8 ./llamafile-server-0.1-llava-v1.5-7b-q4` and give me (1) the name of the instruction that's illegal (or its hex value) and (2) the hex address of where that instruction is in memory? You're encouraged to file a GitHub issue about this too. Thanks!
  - derwiki 3y ago
    Closing the loop for anyone reading this thread -- see https://github.com/Mozilla-Ocho/llamafile/issues/11 https://github.com/Mozilla-Ocho/llamafile/issues/11 for the fix. Thanks jart!
fitted 3y ago
Same, process gets killed immediately for me.
raajg 3y ago
Yep, same issue. and the error message is unhelpful
- jart 3y ago
  We have an issue here tracking this: https://github.com/Mozilla-Ocho/llamafile/issues/14 https://github.com/Mozilla-Ocho/llamafile/issues/14 Please follow that issue for updates.
chatmasta 3y ago
Same, and then a few minutes later I got a Slack message from SecOps, LOL. Don't try this on a computer with CrowdStrike software running on it! It gets flagged because to a naive heuristic, the binary is indistinguishable from a virus. It appears to do some kind of magic self-extraction to an executable file in a temporary directory, and then that executable file executes the original file. And the CrowdStrike endpoint security product intercepts the suspicious execve, kills the process, and alerts the security team...

Same on an M1 Max 64G, Ventura. Xcode is installed[1]. 1 = ``` $ xcode-select --install xcode-select: error: command line tools are already installed, use "Software Update" in System Settings to install updates ```

carbocation 3y ago
For whatever it's worth, the SHA sum is correct. The killed message is uninformative, looks like what happens when I'm OOM (but I have 64GB RAM of which only 24 is used for anything at the moment). $ sha256sum < llamafile-server-0.1-llava-v1.5-7b-q4 a138c5db9cff3b8905dd6e579c2ab6c098048526b53ae5ab433ff1d1edb9de24 - $ ./llamafile-server-0.1-llava-v1.5-7b-q4 Killed: 9
- carbocation 3y ago
  Looks like this may be due to crowdstrike, which I also have installed on this machine: https://github.com/Mozilla-Ocho/llamafile/issues/14#issuecomment-1834313659 https://github.com/Mozilla-Ocho/llamafile/issues/14#issuecom...
  - raajg 3y ago
    oh wow, I would have never imagined that 'that' would be preventing me from running llama + llava! Confirming that I have Crowdstrike running too.

derwiki 3y ago

On a Macbook Pro M2, I get $ ./llamafile-server-0.1-llava-v1.5-7b-q4 [2] 25224 illegal hardware instruction ./llamafile-server-0.1-llava-v1.5-7b-q4

jart 3y ago
Could you disable SIP and run `lldb -- $TMPDIR/.ape-1.8 ./llamafile-server-0.1-llava-v1.5-7b-q4` and give me (1) the name of the instruction that's illegal (or its hex value) and (2) the hex address of where that instruction is in memory? You're encouraged to file a GitHub issue about this too. Thanks!
- derwiki 3y ago
  Closing the loop for anyone reading this thread -- see https://github.com/Mozilla-Ocho/llamafile/issues/11 https://github.com/Mozilla-Ocho/llamafile/issues/11 for the fix. Thanks jart!

fitted 3y ago

Same, process gets killed immediately for me.

raajg 3y ago

Yep, same issue. and the error message is unhelpful

jart 3y ago
We have an issue here tracking this: https://github.com/Mozilla-Ocho/llamafile/issues/14 https://github.com/Mozilla-Ocho/llamafile/issues/14 Please follow that issue for updates.

chatmasta 3y ago

Same, and then a few minutes later I got a Slack message from SecOps, LOL. Don't try this on a computer with CrowdStrike software running on it! It gets flagged because to a naive heuristic, the binary is indistinguishable from a virus. It appears to do some kind of magic self-extraction to an executable file in a temporary directory, and then that executable file executes the original file. And the CrowdStrike endpoint security product intercepts the suspicious execve, kills the process, and alerts the security team...