Patrick_Devine

6 ms·

1.
by Patrick_Devine 17d ago
How is Ollama pivoting to a "less open" direction? Everything is MIT licensed.
2.
by Patrick_Devine 4mo ago
Given the model was just republished by Google 15 minutes ago and we're going to have to redo everything (and everyone will have to redownload for all platforms -- not just Ollama), I'll just say that sometimes things don't w
3.
by Patrick_Devine 4mo ago
I realize this is a little confusing; we're working w/ the MLX team to bring MLX to other platforms, but we're not quite there yet. The `gemma4:12b-nvfp4` model is specifically for the MLX engine. For the GGUF 4bit variant (i
4.
by Patrick_Devine 4mo ago
I haven't yet pushed the MTP enabled gemma4 12b model for Ollama because in my testing I wasn't getting a performance bump. The other gemma4 MTP models should work OK right now, but there are some fixes we're just about to pu
5.
by Patrick_Devine 4mo ago
In my testing the Gemma 4 31b model had the biggest speed boost in Ollama w/ the MLX runner for coding tasks (at about 2x). Unfortunately you'll need a pretty beefy Mac to run it because quantization really hurts the acceptance ra
6.
by Patrick_Devine 5mo ago
I wish they would do this when you're boarding the plane. I get that there is essential information that everyone needs to know, but if you're a frequent flier you've probably heard the "put your larger carry-on in the o
7.
by Patrick_Devine 5mo ago
Isn't this why NASA is developing the Electrodynamic Dust Shield [1] system? [1] https://www.nasa.gov/image-article/nasas-dust-shield-success...
8.
by Patrick_Devine 5mo ago
If you're on a Mac, use the MLX backend versions which are considerably faster than the GGML based versions (including llama.cpp) and you don't need to fiddle with the context size. The models are `qwen3.6:35b-a3b-nvfp4`, `qwen3.6
9.
by Patrick_Devine 6mo ago
They are nvidia-fp4 weights, but CUDA support isn't _quite_ ready yet, but we've got that cooking.
10.
by Patrick_Devine 6mo ago
The 35b-a3b-coding-nvfp4 model has the recommended hyperparameters set for coding, not chatting. If you want to use it to chat you can pull the `35b-a3b-nvfp4` model (it doesn't need to re-download the weights again so it will pull qui
11.
by Patrick_Devine 6mo ago
Try it with mxfp8 or bf16. It's a decent model for doing tool calling, but I wouldn't recommend using it with 4 bit quantization.
12.
by Patrick_Devine 7mo ago
I noticed the same thing. I'm assuming they forgot to photoshop out the chinese characters.
13.
by Patrick_Devine 7mo ago
The Departing / Arrival airports plus a full track would be absolutely amazing.
14.
by Patrick_Devine 10mo ago
5 years is normal-ish depreciation time frame. I know they are gaming GPUs, but the RTX 3090 came out ~ 4.5 years before the RTX 5090. The 5090 has double the performance and 1/3 more memory. The 3090 is still a useful card even after
15.
by Patrick_Devine 10mo ago
The instruct models are available on Ollama (e.g. `ollama run ministral-3:8b`), however the reasoning models still are a wip. I was trying to get them to work last night and it works for single turn, but is still very flakey w/ multi-t
16.
by Patrick_Devine 10mo ago
The default ones on Ollama are MXFP4 for the feed forward network and use BF16 for the attention weights. The default weights for llama.cpp quantize those tensors as q8_0 which is why llama.cpp can eek out a little bit more performance at t
17.
by Patrick_Devine 1y ago
We uploaded gemma3:270m-it-q8_0 and gemma3:270m-it-fp16 late last night which have better results. The q4_0 is the QAT model, but we're still looking at it as there are some issues.
18.
by Patrick_Devine 1y ago
Ollama only uses llamacpp for running legacy models. gpt-oss runs entirely in the ollama engine. You don't need to use Turbo mode; it's just there for people who don't have capable enough GPUs.
19.
by Patrick_Devine 1y ago
I worked on the text portion of gemma3 (as well as gemma2) for the Ollama engine, and worked directly with the Gemma team at Google on the implementation. I didn't base the implementation off of the llama.cpp implementation which was d
20.
by Patrick_Devine 1y ago
Wait, what hosted APIs is Ollama wrapping?
21.
by Patrick_Devine 1y ago
The vision tower is 7GB, so I was wondering if you were loading it without vision?
22.
by Patrick_Devine 1y ago
Ollama has had vision support for Gemma3 since it came out. The implementation is not based on llama.cpp's version.
23.
by Patrick_Devine 2y ago
My point was multi-images and pan-and-scan. We haven't implemented those yet in Ollama, but soon!
24.
by Patrick_Devine 2y ago
There are some fixes coming to uniformly speed up pulls. We've been testing that out but there are a lot of moving pieces with the new engine so it's not here quite yet.
25.
by Patrick_Devine 2y ago
Not quite yet on Ollama, but hopefully we'll add this soon. Also, we didn't add the pan-and-scan algorithm yet for getting better clarity in the original image.
26.
by Patrick_Devine 2y ago
When Skytrain first came out it was touted about how quiet it was vs. other metro systems. [1] The problem (as others have pointed out) is just the Mark I trains are 40 years old and the maintenance hasn't kept up with the track and th
27.
by Patrick_Devine 2y ago
I still have my boxed copy (along with everything Loki produced) in a big box in the garage.
28.
by Patrick_Devine 2y ago
We ended up not publishing it as a library model just because it was leaked and not the official weights.
29.
by Patrick_Devine 2y ago
It also doesn't render unicode characters correctly (specifically unicode block chars). And they refuse to fix is, so d. is kinda a moot point.
30.
Structured Outputs with Ollama(ollama.com)
265 points by Patrick_Devine 2y ago|70 comments

More ›