xaskasdf

6 ms·

1.
Show HN: The Simpsons Hit and Run Revamped(github.com)
4 points by xaskasdf 15d ago|0 comments
2.
Show HN: A LinkedIn Browser Gate Blocker(github.com)
2 points by xaskasdf 6mo ago|1 comments
3.
by xaskasdf 6mo ago
It didn't had any quality loss, since the PSNT as quantization it's mainly to convert the model over the console constraints (you can convert any model you want, even when i trained a model for this hw); it's q8 quantization,
4.
Show HN: I ran a language model on a PS2(github.com)
46 points by xaskasdf 6mo ago|12 comments
5.
by xaskasdf 7mo ago
Actually since they have direct GDS it should perform really well on professional gpus
6.
by xaskasdf 7mo ago
yeah, the mobo upgrade is something I gotta do anyway, so I'll cover that up more or less, the optane is something I didn't thought about
7.
by xaskasdf 7mo ago
I wondered the same, but the rendering seems right, the output was almost instant. I'll recheck the token counter; anyway as you say, fast isn't practical. Actually I had to develop my own tiny model https://huggingface
8.
by xaskasdf 7mo ago
Ya know, here on the local market there are a bunch of optanes hanging around, I'll try to manage one to check if there's any improvement
9.
by xaskasdf 7mo ago
Actually is purely bandwidth-bound, the major bottleneck of the whole process, for me in this case, is the B450 mobo I got that's only capable of pcie3 and 1x8 in the pcie lanes for gpu instead of 1x16; so I'm capped until I get a
10.
by xaskasdf 7mo ago
I thought about that, but idk if they allow me to modify the linux kernel and nvidia cuda kernel at all
11.
by xaskasdf 7mo ago
Actually I can't go full tdp with a 650w PSU, I got to upgrade it asap
12.
by xaskasdf 7mo ago
I updated the documentation to provide more info for the patching process, I added the patches themselves too and provided some risk info about the patches
13.
by xaskasdf 7mo ago
I did it, but with different quantization compressions, It ran into quality issues, I will try to rerun with the same quants if that fixes the issue, but the most that looks unused, its being used by rotating layers that are being swapped b
14.
by xaskasdf 7mo ago
Did you even read anything? hahaha
15.
by xaskasdf 7mo ago
Actually I'm thinking about buyin an AMD BC-250 that's bassically a ps5 with pcie factor format; and it's linux capable by default, maybe next month
16.
by xaskasdf 7mo ago
This was the experiment itself https://github.com/xaskasdf/ps2-llm The idea was basically to run a llm on a ps2, then I ran into some problems as the 32mb ram cap with 4mb vram cap; so I had to figure out a way to stre
17.
by xaskasdf 7mo ago
I got an m3, I will test it on metal and check how it goes
18.
by xaskasdf 7mo ago
Actually this idea was fueled by those since I went to check if there was anything near to what I wanted to achieve, pretty useful tho
19.
by xaskasdf 7mo ago
yeah, actually, I'm bottlenecked af since my mobo got pcie3 only :(
20.
by xaskasdf 7mo ago
That would be nice to see. Actually I was thinking about getting another 3090 and a mobo upgrade since I'm bottlenecked by pcie3 to tryna run glm 4.7 or 5 at q4_k_m, it should be possible.
21.
by xaskasdf 7mo ago
yeah, actually I wanted to see if this was possible at all. I managed to get around 3000 tokens/s on a ps2 with classic transformers, since the emotion engine is capable of 32 bit addresses, but it has like 32gb of ram. So I ran into t
22.
Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU(github.com)
395 points by xaskasdf 7mo ago|101 comments