7 ms·

Show HN: Claude and ChatGPT need a datacenter. This runs on my phone

18 points by 12ziyad 17d ago

simon84 17d ago
This is very interesting. I have not tested it (yet) but I believe this is what will come next when talking about "AI will be everywhere". The self-contained disconnected mode is very convenient, though it could be useful to have an "online access" mode to allow it to access the Internet and perform lookup.
- SeriousM 17d ago
  The model is just the executing part. You can always give them tools via the harness to do various things.
- 12ziyad 15d ago
  Thanks. An optional online lookup mode is a nice idea and I've noted it — offline stays the default though, that's the whole point of it.
mdrzn 17d ago
"Download the apk from my website. Android will warn you because it did not come from the Play Store - that is normal for a direct download." mmmh not the best presentation. Established alternatives: - Google AI Edge Gallery - MLC Chat - PocketPal AI
- 12ziyad 15d ago
  Fair point on the wording — I've reworded it. And thanks for the list. AI Edge and MLC use the GPU, which I don't yet, so PocketPal is the closest comparison to mine. It was slower than PocketPal and that's fixed now.
andOlga 16d ago
The claim that these models "run on almost any phone" as well as the in-app indicator of performance that says it may "run comfortably" are some serious exaggeration. Tried running Qwen 0.5B on mine [1]. It took five minutes to produce "I am a large language model created by Anthropic" (lmao) and then stopped outputting anything at all. I am not expecting you to somehow make the models perform better or whatnot, I just believe that the performance claims need review. [1]: https://m.gsmarena.com/xiaomi_redmi_note_13_pro-12581.php https://m.gsmarena.com/xiaomi_redmi_note_13_pro-12581.php
- andOlga 15d ago
  Alright, I have since experimented with the other harnesses mentioned in this thread (AI Edge and PocketPal) and both run the same models much, much faster. Gemma 4 E2B is extremely usable, Qwen simply flies. Same device. I don't know what you are doing, but it's clearly not great...
  - 12ziyad 15d ago
    [dead]
eolexy 15d ago
[dead]

18 points by 12ziyad 17d ago

This is very interesting. I have not tested it (yet) but I believe this is what will come next when talking about "AI will be everywhere". The self-contained disconnected mode is very convenient, though it could be useful to have an "online access" mode to allow it to access the Internet and perform lookup.

SeriousM 17d ago
The model is just the executing part. You can always give them tools via the harness to do various things.
12ziyad 15d ago
Thanks. An optional online lookup mode is a nice idea and I've noted it — offline stays the default though, that's the whole point of it.

mdrzn 17d ago

"Download the apk from my website. Android will warn you because it did not come from the Play Store - that is normal for a direct download." mmmh not the best presentation. Established alternatives: - Google AI Edge Gallery - MLC Chat - PocketPal AI

12ziyad 15d ago
Fair point on the wording — I've reworded it. And thanks for the list. AI Edge and MLC use the GPU, which I don't yet, so PocketPal is the closest comparison to mine. It was slower than PocketPal and that's fixed now.

andOlga 16d ago

The claim that these models "run on almost any phone" as well as the in-app indicator of performance that says it may "run comfortably" are some serious exaggeration. Tried running Qwen 0.5B on mine [1]. It took five minutes to produce "I am a large language model created by Anthropic" (lmao) and then stopped outputting anything at all. I am not expecting you to somehow make the models perform better or whatnot, I just believe that the performance claims need review. [1]: https://m.gsmarena.com/xiaomi_redmi_note_13_pro-12581.php https://m.gsmarena.com/xiaomi_redmi_note_13_pro-12581.php

andOlga 15d ago
Alright, I have since experimented with the other harnesses mentioned in this thread (AI Edge and PocketPal) and both run the same models much, much faster. Gemma 4 E2B is extremely usable, Qwen simply flies. Same device. I don't know what you are doing, but it's clearly not great...
- 12ziyad 15d ago
  [dead]

eolexy 15d ago

[dead]