jmorgan

5 ms·

1.
by jmorgan 3mo ago
The larger models are available on Ollama's cloud as most folks don't have the hardware to run 500B-1T parameter models.
2.
by jmorgan 7mo ago
For local models I've been trying it with GLM-4.7-Flash and the new LFM2 24B model. I'm excited to try it with the new Qwen3.5 models that came out today as well.
3.
by jmorgan 7mo ago
I've been using Pi day to day recently for simple, smaller tasks. It's a great harness for use with smaller parameter size models given the system prompt is quite a bit shorter vs Claude or Codex (and it uses a nice small set of t
4.
by jmorgan 8mo ago
That's not good, sorry. I work on Ollama - shoot me an email (jeff@ollama.com) and we can help debug
5.
by jmorgan 8mo ago
It's available (with tool parsing, etc.): https://ollama.com/library/glm-4.7-flash but requires 0.14.3 which is in pre-release (and available on Ollama's GitHub repo)
6.
by jmorgan 9mo ago
The source is available here: https://github.com/ollama/ollama/tree/main/app
7.
by jmorgan 10mo ago
The gpt-oss weights on Ollama are native mxfp4 (the same weights provided by OpenAI). No additional quantization is applied, so let me know if you're seeing any strange results with Ollama. Most gpt-oss GGUF files online have parts of
8.
by jmorgan 1y ago
We did consider building functionality into Ollama that would go fetch search results and website contents using a headless browser or similar. However we had a lot of worries about result quality and also IP blocking from Ollama creating c
9.
Ollama Web Search(ollama.com)
348 points by jmorgan 1y ago|176 comments
10.
by jmorgan 1y ago
Amazing work. This model feels really good at one-off tasks like summarization and autocomplete. I really love that you released a quantized aware training version on launch day as well, making it even smaller!
11.
by jmorgan 1y ago
It should open ollama.com/connect – sorry about that. Feel free to message me jeff@ollama.com if you keep seeing issues
12.
by jmorgan 1y ago
Sorry about this. Re-downloading Ollama should fix the error
13.
by jmorgan 1y ago
Working on adding tool calling support to Magistral in Ollama. It requires a tokenizer change and also uses a new tool calling format. Excited to see the results of combining thinking + tool calling!
14.
UI-TARS-1.5: open-source computer use vision-language model(seed-tars.com)
4 points by jmorgan 1y ago|1 comments
15.
by jmorgan 2y ago
This is a great point. apt-get would definitely be a better install experience and upgrade experience (that's what I would want too). Tailscale does this amazing well: https://tailscale.com/download/linux The main
16.
by jmorgan 2y ago
Sorry this isn't more obvious. Ideally VRAM usage for the context window (the KV cache) becomes dynamic, starting small and growing with token usage, whereas right now Ollama defaults to a size of 2K which can be overridden at runtime.
17.
by jmorgan 2y ago
Phi-4's architecture changed slightly from Phi-3.5 (it no longer uses a sliding window of 2,048 tokens [1]), causing a change in the hyperparameters (and ultimately an error at inference time for some published GGUF files on Hugging Fa
18.
by jmorgan 2y ago
Thank you for writing this!
19.
Diffusion models are real-time game engines(gamengen.github.io)
1149 points by jmorgan 2y ago|409 comments
20.
by jmorgan 2y ago
Currently when the context limit is hit, there's a halving of the context window (or a "context shift") to allow inference to continue – this is helpful for smaller (e.g. 1-2k) context windows. However, not all models (especi
21.
by jmorgan 2y ago
Sorry about this – working on fixing the issue with hitting the context limit. Gemma 2 supports a 8192 context limit – which can be selected if you provide the `num_ctx` parameter in the API or via `ollama run` with `/set parameter num
22.
by jmorgan 2y ago
Sorry it's slow for you – happy to help debug why - shoot me an email at jeff@ollama.com
23.
by jmorgan 2y ago
Thank you for building htmx!
24.
by jmorgan 2y ago
Sorry it's taking so long to review and for the radio silence on the PR. We have been trying to figure out how to support more structured output formats without some of the side effects of grammars. With JSON mode (which uses grammars
25.
by jmorgan 2y ago
Pre-release versions are created to test new updates on bunch of different hardware setups (OS/GPUs) before releasing more broadly (and making new versions the default for the Linux/macOS/Windows installers – those pull from
26.
by jmorgan 2y ago
Not at the moment, although it is a highly requested feature (specifically fine-tuning). There are a few tools that you can (or will soon be able to) use to fine tune a model and then import the resulting adapter layers into Ollama: MLX [1]
27.
by jmorgan 2y ago
Shoot sorry about that. There's a few ways to keep the model loaded in memory: 1. If using `ollama run`: `ollama run llama3 --keepalive -1` 2. If running ollama serve directly, use `OLLAMA_KEEP_ALIVE=-1` ollama serve 3. If using the ap
28.
by jmorgan 2y ago
Codestral was just published here as well: https://ollama.com/library/codestral
29.
by jmorgan 2y ago
Yes, we are also looking at integrating MLX [1] which is optimized for Apple Silicon and built by an amazing team of individuals, a few of which were behind the original Torch [2] project. There's also TensorRT-LLM [3] by Nvidia optimi
30.
by jmorgan 2y ago
I think it's worth it, although it might be best to wait for the next iteration: there's rumors the M4 Macs will support up to 512GB of memory [1]. The current 128GB (e.g. M3 Max) and 192GB (e.g. M2 Ultra) Macs run these large mod

More ›