frontsideair

6 ms·

1.
by frontsideair 6mo ago
I was able to confirm that it actually runs on ANE, I'm impressed.
2.
by frontsideair 6mo ago
> Apple locked it behind Siri. apfel sets it free This doesn't feel truthful, it sounds like this tool is a hack that unlocks something. If I understand it correctly, it's using the same FoundationModels framework that powers A
3.
by frontsideair 11mo ago
Yeah, the initial experience with no colors doesn’t look great. I can implement this when I have some free time, if you feel like doing it please feel free to open a PR. Thanks!
4.
My Most Popular Application(blog.6nok.org)
3 points by frontsideair 11mo ago|4 comments
5.
by frontsideair 1y ago
14B Qwen was a good choice, but it became outdated a bit and seems like the new version of 4B surpassed it in benchmarks somehow. It's a balancing game, how slow a token generation speed can you tolerate? Would you rather get an answer
6.
by frontsideair 1y ago
Ollama adding a paid cloud version made me postpone this post for a few weeks at least. I don't object them to make money, but it was hard to recommend a tool for local usage and make the first instruction to go to settings and enable
7.
by frontsideair 1y ago
This is the command probably: sudo sysctl iogpu.wired_limit_mb=184320 Source: https://github.com/ggml-org/llama.cpp/discussions/15396
8.
by frontsideair 1y ago
I'm interested in this, my impression was that the newer chips have unified memory and high memory bandwidth. Do you do inference on the CPU or the external GPU?
9.
by frontsideair 1y ago
Good point, let me add a quick note.
10.
by frontsideair 1y ago
Thank you, it was the integral part of the whole post!
11.
Experimenting with Local LLMs on macOS(blog.6nok.org)
388 points by frontsideair 1y ago|262 comments
12.
by frontsideair 1y ago
According to the benchmarks, this one is improved in every one of them compared to the previous version, some better than 30B-A3B. Definitely worth a try, it’ll easily fit into memory and token generation speed will be pleasantly fast.
13.
by frontsideair 2y ago
This is the first time I’m hearing about Nebula. How does it compare to Tailscale?
14.
by frontsideair 2y ago
I saw from the comments that they provide smaller binaries, which may have worked on my Raspberry Pi. Maybe I’ll give it a try one day. https://tailscale.com/kb/1207/small-tailscale
15.
by frontsideair 2y ago
Thanks, this could’ve worked for my Raspberry Pi! I would’ve tried it if it was still in commission. Next time?
16.
by frontsideair 2y ago
That’s exactly why I put the spend limit, and this was the first time to confirm that it actually works.
17.
by frontsideair 2y ago
Exactly, I had a spend limit since I didn’t want to break the bank. It’s back up now.
18.
Tailscale Is Pretty Useful(blog.6nok.org)
9 points by frontsideair 2y ago|1 comments
19.
Building a semantic movie search demo with pgvector and Next.js(blog.6nok.org)
2 points by frontsideair 2y ago|0 comments
20.
by frontsideair 3y ago
Synthetic benchmarks should be retired. Build a real app, measure real impact. Unless your users insert 1000 rows, of course. Other than that, I agree with the general premise of React becoming the new IBM, that's totally fair.
21.
by frontsideair 3y ago
This looks like a treasure trove that I somehow missed! I'll give it a more thorough read later, but gave my star for now, thanks!
22.
by frontsideair 3y ago
I'm really interested in `home-manager` honestly, and while I agree about the benefits of declarative package management, I find it easier to type commands at a terminal like `nix profile upgrade pkg` or `nix registry pin nixpkgs`, wit
23.
I Use Nix on macOS(blog.6nok.org)
22 points by frontsideair 3y ago|16 comments
24.
Server-side rendering local dates without FOUC(blog.6nok.org)
1 points by frontsideair 3y ago|0 comments
25.
Best providers to deploy your full-stack app(blog.6nok.org)
2 points by frontsideair 3y ago|0 comments
26.
Buildless JavaScript(blog.6nok.org)
2 points by frontsideair 4y ago|0 comments
27.
The Suspense Is Killing Me: Part 2(blog.6nok.org)
2 points by frontsideair 4y ago|0 comments
28.
The Suspense Is Killing Me: Part 1(blog.6nok.org)
3 points by frontsideair 5y ago|0 comments
29.
You can't tie your shoes while running(blog.6nok.org)
2 points by frontsideair 6y ago|0 comments
30.
Run yarn install on Git checkout, pull, merge, rebase(github.com)
1 points by frontsideair 9y ago|0 comments

More ›