6 ms·

Unfortunately no BYOLLM. Brave supports bringing your own LLM e.g. through Ollama Besides that I'm using AI Summary Helper plugin for Chromium-based browsers h

• JFingleton 2y ago
  > Brave supports bringing your own LLM e.g. through Ollama It's a shame Brave is so far ahead of the game but no one seems to notice.
  • kbelder 2y ago
    I run Brave at home and a local LLM on the same machine, and didn't know this. I guess I'll be playing around this weekend.
• lhl 2y ago
  Sort of funny for the lack of local options considering that Mozilla funds llamafile even. Hopefully they allow some API integration, if they are using standard OpenAI API calls, it should be easy to enable swapping the endpoint. Also, while it's nice to have a service option for those without any spare compute, I think it's a bit of a shame on the model considering how even at the 7B class, models like Llama 3.1 8B, Qwen 2.5 8B or Tulu 3 8B, Falcon 3 7B, all clearly outclass Mistral 7B (Mistral 7B is also very bad at multilingual, and is particularly inefficient at multilingual tokenization). The current best fully open weights (Apache 2.0 or similar) small models currently are probably: OLMo 2 7B, Qwen 2.5 7B, Granite 3.1 8B, and Mistral Nemo Instruct (12B) There's been a recent launch of a "GPU-Poor" Chat Arena for those interested in scoping out some of the smaller models (not a lot of ratings so very noisy, take it with a grain of salt): https://huggingface.co/spaces/k-mktr/gpu-poor-llm-arena https://huggingface.co/spaces/k-mktr/gpu-poor-llm-arena