mchiang

5 ms·

1.
by mchiang 8mo ago
hey, thanks for sharing. I had to go to the Twitter feed to find the GitHub link: https://github.com/21st-dev/1code
2.
by mchiang 11mo ago
this one is exciting. It'll enable and accelerate a lot of devices on Ollama - especially around AMD GPUs not fully supported by ROCm, Intel GPUs, and iGPUs across different hardware vendors.
3.
by mchiang 11mo ago
I am super hopeful! Hardware is improving, inference costs will continue to decrease, models will only improve...
4.
by mchiang 11mo ago
Qwen3-coder:30b is in the blog post. This is one that most users will be able to run locally. We are in this together! Hoping for more models to come from the labs in varying sizes that will fit on devices.
5.
by mchiang 11mo ago
ah! This must be downloaded from elsewhere and not from Ollama? So sorry about this. To help future optimizations for given quantizations, we have been trying to limit the quantizations to ones that fit for majority of users. In the case of
6.
by mchiang 11mo ago
Do you find yourself sticking with GLM 4.6 over Claude for some tasks? Or do you find yourself still wanting to reach for Claude?
7.
by mchiang 11mo ago
https://github.com/ollama/ollama?tab=readme-ov-file#supporte...
8.
by mchiang 11mo ago
Z.ai team is awesome and very supportive. I have yet to try synthetic.new. What's the reason for using multiple? Is it mainly to try different models or are you hitting some kind of rate limit / usage limit?
9.
by mchiang 11mo ago
but that is VC funded
10.
by mchiang 11mo ago
sorry, I don't use 4chan, so I don't know what's said there. May I ask what system you are using where you are getting memory estimations wrong? This is an area Ollama has been working on and improved quite a bit on. Latest v
11.
by mchiang 1y ago
It’s so much fun when you can mix passion and seeing students chat about what they want to build the most.
12.
by mchiang 1y ago
Oh yes! that is why I want to provide the names of the providers we use. I do believe in building in the open. The web search functionality has a very generous free tier (it is behind Ollama's free account to prevent abuse) that allows
13.
by mchiang 1y ago
No, Ollama is it's own project and separate. You can check it out via GitHub https://github.com/ollama/ollama
14.
by mchiang 1y ago
Sorry about this. We are working really hard on providing a usage based pricing. During the preview period we want to start offering a $20 / month plan tailored for individuals - and we are monitoring the usage and making changes as pe
15.
by mchiang 1y ago
We have relationships with many providers and I don't want to be seen as promoting or not promoting a specific provider. Some decent privacy-preserving vendors - Brave, Exa, Parallel Web Systems, DuckDuckGo etc We will continue to moni
16.
by mchiang 1y ago
To provide additional features or using Ollama's cloud hosted models, you can signup for an Ollama account. For starter, this is completely optional. It can be completely local too for you to publish your own models to ollama.com that
17.
by mchiang 1y ago
We work with search providers and ensure that we have zero data retention policies in place. The search results are yours to own and use. You are free to do what you want with it. Of course you are bound by local laws of the legal jurisdict
18.
by mchiang 1y ago
When we started Ollama, we were told how open-source (open-weight wasn't a term back then) will always be inferior to the close-sourced models. This was 2 years ago (Ollama's birthday is July 18th, 2023). Fast forward to now, open
19.
by mchiang 1y ago
Fair question. Some of the supported models are large and wouldn't fit on most local devices. This is just the beginning, and Ollama does not need to exclude cloud hosted frontier models either with the relationship we've built wi
20.
by mchiang 1y ago
I haven't tried SearXNG personally. How does it compare to Ollama's web search in terms of the search content returned?
21.
by mchiang 1y ago
I was pleasantly surprised on the model improvements when testing this feature. For smaller models, it can augment it with the latest data by fetching it from the web, solving the problem of smaller models lacking specific knowledge. For la
22.
by mchiang 1y ago
It’s a different repo. https://github.com/ggml-org/ggml The models are implemented by Ollama https://github.com/ollama/ollama/tree/main/model/models I can say as a fact, for th
23.
by mchiang 1y ago
I can say trying many inference tools after the launch, many do not have the models implemented well, and especially OpenAI’s harmony. Why does this matter? For this specific release, we benchmarked against OpenAI’s reference implementation
24.
by mchiang 1y ago
so sorry about this. We are learning. Possible to email, and we will first make it right while we improve Ollama's turbo mode. hello@ollama.com
25.
by mchiang 1y ago
thanks, I'll take that feedback, but I do want to clarify that it's not from llama.cpp/ggml. It's from ggml-org/ggml. I supposed it's all interchangeable though, so thank you for it.
26.
by mchiang 1y ago
hmm, I don't think so. This is more of, we want to keep improving Ollama so we can have a great core. For the users who want GPUs, which cost us money, we will charge money for it. Completely optional.
27.
by mchiang 1y ago
it's all open, and specifically, the new models are implemented here: https://github.com/ollama/ollama/tree/main/model/models
28.
by mchiang 1y ago
totally respect your choice, and it's a great project too. Of course as a maintainer of Ollama, my preference is to win you over with Ollama. If it doesn't meet your needs, it's okay. We are more energized than ever to keep i
29.
by mchiang 1y ago
OpenAI has only provided MXFP4 weights. These are the same weights used by other cloud providers.
30.
by mchiang 1y ago
hmm, how so? Ollama is open and the pricing is completely optional for users who want additional GPUs. Is it bad to fairly charge money for selling GPUs that cost us money too, and use that money to grow the core open-source project? At one

More ›