23 ms·
Very excited for access to more RAM in a consumer form factor - by comparison, the most RAM you can get on a DDR5 motherboard is 128GB(actually, there may be 48
by idkyall 3y ago
Very excited for access to more RAM in a consumer form factor - by comparison, the most RAM you can get on a DDR5 motherboard is 128GB(actually, there may be 48 GB modules as well, so maybe 192), but at that capacity you're not getting anywhere near the rated speeds. The listed 1TB of capacity sounds super roomy by comparison
- jadbox 3y agoWhat are ye using all that ram for? -curious stranger
- itsboring 3y agoIsn’t it obvious? More browser tabs.
- taskforcegemini 3y agoIt can't be for tabs in chrome, that browser can't even use all 64gb I offer it ("putting tabs to sleep" is disabled on purpose as I need the tabs to stay active)
- nottorp 3y agoYou just don't have enough tabs open.
- taskforcegemini 3y agoI do, but they crash for memory insufficiency reasons while half of the RAM is still free. it's a bug, other people have commented on the issue with no solution so far. some suspect hardware acceleration to be a culprit, but I ruled that out. what I meant to convey in the first place: it doesn't matter how much hardware you throw at an issue if the software can't use it
- ghthor 3y agoSwitching to Firefox and using tree style tabs will help increase your total open tab count.
- tyfon 3y agoI have 128 GB in my computer with a 5850x, it allows me to run and load the 180B falcon and 70B llama2 LLMs in llama.cpp, although with different quantization. Speed is actually not that bad either.
- corn13read 3y ago[dead]
- mosselman 3y agoIs there some documentation on how to run this setup? How fast is your setup?
- rnk 3y agoI'm doing this on a mac studio with 128gb too. I'm using llama.cpp.
- acchow 3y agoSince you get GPU acceleration (because of the unified memory), I imagine this is probably much faster than the PC setup? Edit: Seems some people are getting 1-2.6 tokens/sec on Ryzen (no GPU acceleration), Llama 70B quantized https://www.reddit.com/r/LocalLLaMA/comments/15rqkuw/llama_2_q4_k_s_70b_performance_without_gpu/ https://www.reddit.com/r/LocalLLaMA/comments/15rqkuw/llama_2... Whereas Mac Studio gets 13 tokens/sec https://blog.gopenai.com/how-to-deploy-llama-2-as-api-on-mac-studio-m2-ultra-and-enable-remote-api-access-7c4e6423b2dd https://blog.gopenai.com/how-to-deploy-llama-2-as-api-on-mac...
- stoatmagoats 3y agoFriendly internet stranger’s input: - you don’t get GPU acceleration just by using unified memory. Llama.cpp still only uses the CPU on Apple Silicon chips. - the difference in tokens/sec is likely attributable to memory bandwidth. Mac Studios with the base Max chip have 400 GB/s memory bandwidth compared to around 50 GB/s for the Ryzen 5000 series CPUs
- swarnie 3y ago64gb -> OS, light gaming, 6 oracle DB's and a few middlewares.
- iopq 3y agoTry searching deep into the game tree in go. After a few million nodes you can't actually store them all in 16GB of RAM. That's just a day of searching on a 2060, you can get into the hundreds of millions with a faster GPU and a longer search horizon. But when you put it into swap, it won't be as fast...
- at_a_remove 3y agoOne underused angle for oodles of memory is the humble ramdisk. If you haven't run into these, you set aside a portion of memory to serve as a disk volume. If you have a temporary work product, some kind of intermediate stage bit you will not save, shoving it in a ramdisk provides some really amazing speedups. You can put a SQLite database in it on the fly, just for analysis, run at blazing speeds, keep the results. Image an optical disk into your ramdrive, chow away at it, keep the work product, and just clear the memory.
- Havoc 3y agoWhat software do you use to create it ?
- at_a_remove 3y agoThis is a good overview: https://en.wikipedia.org/wiki/List_of_RAM_drive_software https://en.wikipedia.org/wiki/List_of_RAM_drive_software
- dmitrygr 3y agoSoftware? mount none /ramdisk -t tmpfs
- ghthor 3y agoThere is some OSS for creating ramdisks on windows, but I don’t remember the name. In Linux it’s part of the kernel, and you can mount with type tempts. Not sure about OSX or bsds
- Washuu 3y agoSpillover from my video card's 24GB when doing huge renders in Blender.
- smcleod 3y agoDefinitely 192.