7 ms·
"Tinyboxes finally have a buy it now button"
- deleted 2y ago[deleted]
- sdfdsfsdfsdfs 2y agohttps://archive.md/RTvI8 https://archive.md/RTvI8
- WithinReason 2y agohttps://tinygrad.org/#tinybox https://tinygrad.org/#tinybox Looks like good value, but I wonder if it would get CPU/RAM bottlenecked, especially if you want to train something with a lot of preprocessing in the pipeline. Something comparable I've found with 7x4090 which comes to about $50k, but with much better CPU/RAM (3x CPU, 4x RAM, 5x SSD): https://www.overclockers.co.uk/8pack-supernova-mk3-amd-ryzen-threadripper-pro-extreme-pc-sys-8pk-00076.html https://www.overclockers.co.uk/8pack-supernova-mk3-amd-ryzen...
- coder543 2y agoIt’s weird how non-specific the CPU is there. Why wouldn’t they list a CPU part number? We don’t even know what generation of Epyc it is. (I get that it’s not the focus… but it is still important.)
- gpjt 2y agoCloser to $42k, i think, at least if you're comparing it to the Tinybox price -- the price in pounds on the site includes VAT, which you wouldn't pay as a business or if you were getting it for export outside the UK, whereas you'd need to add on VAT if you were getting a Tinybox in the UK.
- SushiHippie 2y ago> 6x PCIe 4.0 x16 (64 GB/s) Wikipedia [0] states that PCIe 4.0 x16 has a throughput of ~32GB/s, what does the (64 GB/s) indicate on the website, is this just a typo and you have 6x ~32GB/s or does it mean in total you can "only" expect a throughput of 64GB/s all lanes slots combined? If so, wouldn't you also be bottlenecked by the PCIe bandwidth (when moving data between CPU and GPU)? [0] https://en.wikipedia.org/wiki/PCI_Express#Comparison_table https://en.wikipedia.org/wiki/PCI_Express#Comparison_table
- arder 2y agoWell they're specifying the AMD EPYC and one of the things that the server line of AMD CPUs do that the consumer grade ones don't, is they have lots of connectivity. So for example an AMD EPYC 8324P is a 32 core CPU with 96 lanes of PCI Gen 5. Given that the 4090 GPU is PCI Gen 4, I think that's where you get the discrepancy. The 6 GPUS are connected in parallel to the CPU with 6 x16 connections (96 total lanes), the CPU could do this at Gen 5 (64GBs for each GPU) but the 4090 GPU is Gen4 only, so you'll only actually get 32GBps per connection.
- jasomill 2y agoMost EPYCs have 128 PCIe lanes, so I'd expect a full x16 link for all six GPUs. Pedantically, the combined bidirectional bandwidth of PCIe x16 is ~64 GB/s, as it's a full-duplex ~32 GB/s link, but that's an awfully misleading spec if this is the intent (akin to claiming Gigabit Ethernet is 2 Gb/sec).
- georgehotz 2y agoIt's the same way NVIDIA states bandwidth for PCIe and NVLink. https://www.nvidia.com/en-us/data-center/h100/ https://www.nvidia.com/en-us/data-center/h100/
- andersa 2y agoIt's 32GB/s in both directions. So when exchanging data two GPUs each can do this at 64GB/s. Is that a useful way to measure it? Who knows.
- sdfdsfsdfsdfs 2y ago[flagged]
- walterbell 2y agoTwitter lists reduce dependency on black-box feed algorithms.
- nwoli 2y agoIf you meant that last question sincerely: I’ve been able to improve the recommended tweets a ton (its basically good and useful now) by hammering on the “not interested” button a lot. It started out garbage as for you too though
- diggan 2y agoI'm guessing the algorithm put you in some bucket, as my (Spanish/European, if that matters) "For you" looks quite different, the 10 first tweets are ~4 coding things, 1 meme, 2 WIP animation things and 3 different angles on the ongoing Telegram story. Last time I opened Twitter seems to have been 3 months ago according to my browser history, so not really a big user of it either.
- ajb 2y agoSocial media sites are the collective equivalent of a "dollar auction"[1] In the dollar auction, each immediate choice isrational, but the overall process inevitably results in you losing value once you've decided to participate. But at each step, you lose more by quitting than by keeping going. Social media is the same, except that it's collective: if you want to keep in touch with certain people, you don't have a choice other than to use the social media site they are using, so leaving is a collective action problem. [1] https://en.m.wikipedia.org/wiki/Dollar_auction https://en.m.wikipedia.org/wiki/Dollar_auction
- robin_reala 2y ago> you don't have a choice other than to use the social media site they are using Hence federated social media. No one’s pretending the current implementations are perfect, but they’re far more interesting than the existing closed platforms.
- threeseed 2y agoVery unusual specs on paper. - Air cooling 6x4090 and a 32 core CPU for sustained peak workloads. - 3200W total power when a single 4090 can draw close to 600W. Maybe they are targeting startups who aren't interested in overclocking.
- WithinReason 2y agoA 4090 draws 450W unless you unlock the power limit
- Timshel 2y agoTransient might be an issue. GN discuss transients of ~30/40% over the nominal 450w (https://www.youtube.com/watch?v=j9vC9NBL8zo&t=616s https://www.youtube.com/watch?v=j9vC9NBL8zo&t=616s). And with a distributed training you can end-up with "synchronized" transients over all cards :(.
- jagrsw 2y agoI hold an electrical certification in the EU, though I'm not currently practicing. A quick point: transient surges are usually fine. Both cables and circuit breakers are designed to fail (trip or burn out) under sustained overloads. For example, a 16A Class C circuit breaker might take around an hour to trip with a constant 17A load, but a ~80A load would trip it ~instantly. PS: Of course, everything is a matter of integration over time (heat dissipation in cables mostly).
- zten 2y agoThe workstation equivalent, the RTX 6000 Ada, defaults to 300W. You can get most of the performance of a 4090 by capping the power.
- jagrsw 2y agoThis thing could run off a single US outlet (1600W) with some throttling, if I'm not mistaken? Shame it wasn't designed for EU sockets. 230V*16A = 3700W, or double that on separate breakers!
- BaculumMeumEst 2y agoSo is the plan for these to quietly update the hardware as better consumer hardware becomes available? This is a really interesting idea but as a small fry I would definitely be building myself if I went this route.
- a1o 2y agoThey have a real website, why not link to that instead of Twitter?
- deleted 2y ago[deleted]
- davidcollantes 2y agoAgree. See https://tinygrad.org/ https://tinygrad.org/.
- vessenes 2y agoI thought to myself this morning: "boy, that $15k pricetage is tempting." Then I thought to myself "how many times have I downloaded a github repo only to hand-replace cuda with mps, and then tried to figure out if there's a version of xformers that will work this week with my m3?" and then I thought "boy, that $25k is tempting." (15k: Radeon / 25k: Nvidia). For those wondering, 3200W power, in residential / low-end commercial in the US, they say you'll need two separate circuits, they have a built-in power limiting utility in the OS which will let you safely run on one circuit at reduced speed. The only part of this that gives me pause is interconnect -- over PCIe, 64GB/s stated. This is much, much lower than infiniband -- can any ML engineers comment on using this box for a full finetune of, say, LLama 3.1 / 70b?
- sidewndr46 2y agoa 220 volt 20 amp circuit should be good for over 3500 watts constant load in North America. Why is it requiring two circuits?
- hashhar 2y agoIIRC USA is 110V, not 220V?
- WoahNoun 2y agoMost US homes have at least one 220v split phase line for major appliances like stoves or AC.
- teraflop 2y agoYes, but most homes don't have extra 220v outlets except for the ones provided for the specific appliances that need them. So if you want to plug in a device like this "tinybox" at home, it's going to be a lot easier to find two separate 110v outlets on different circuits than to have a new 220v circuit added, or to unplug your stove every time you want to use it.
- plasticchris 2y agoI looked at the specs at the start of the year and just built something with the high end of consumer parts at around 4k usd. I was able to replicate the mlc 2x7900xtx results running some LLMs. Good enough to run most of the big models in gpu memory with a little quantization.
- roschdal 2y agoWhat is the most profitable thing I can do with 10 of these fine Tinybox boxes?
- x_may 2y agoSell them and invest the money
- onionisafruit 2y agoAt least that’s what geo decided is the most profitable thing to do
- 1970-01-01 2y agoLooks like the Mac cheese grater now has serious competition.
- bob1029 2y agoI had a preorder in for this but I canceled a few weeks back. My experience trying to run machines this powerful in residential settings has been extremely poor. All of the Seasonic power supplies that go beyond 1kW or so will trip my shitty (i.e. probably defective) Siemens AFCI breakers. Not even the same circuit all the time. Even after violating local electrical code, I have found that living with a 1500w+ monster inside my house during the summer at 100% utilization is a complete joke. Unless you live in the perfect datacenter climate (i.e. the people who designed the tiny box), this thing needs to be inside. All of that wattage is pure heat being dumped into your home. The HVAC solutions in most residences were not designed for this kind of heat load. It would be like running your oven with the door hanging open all day. For those of us in places like Texas, this machine simply would not be feasible to run for half the year.
- teruakohatu 2y ago> All of the Seasonic power supplies that go beyond 1kW or so will trip my shitty (i.e. probably defective) Siemens AFCI breakers. Not even the same circuit all the time. I don’t know much about US electrical standards but aren’t your residential circuits rated for 1800w or 2400w? Here in New Zealand they are 2400w and people regularly plug in 2400w fan heaters without issue. > The HVAC solutions in most residences were not designed for this kind of heat load. It would be like running your oven with the door hanging open all day. For those of us in places like Texas, this machine simply would not be feasible to run for half the year. Yes it wouldn’t be pleasant running this 24/7 in summer in any living space. But you could install a heatpump with 7kw of cooling capacity which should handle it (adding to the electricity bill).
- cptskippy 2y agoIn the US the National Electric Code caps draw at 80% of rated load. So a 15A circuit is permitted a 1440W load even though it should handle 1800W.
- bob1029 2y ago> I don’t know much about US electrical standards but aren’t your residential circuits rated for 1800w or 2400w? The residential AFCI issue I describe isn't about the wattage directly. It's about transient currents generated by large switch-mode power supplies being detected as arc faults. Similar concern as with induction motors.
- EncomLab 2y ago"tinybox red is 6x 7900XTX, tinybox green is 6x 4090" So the red is ~$5k in gpu's - where is the other $10k going?
- andersa 2y agoMotherboard and CPU, memory, NVMe drives, PSUs, SlimSAS cables and breakouts, a custom machined case, assembly, support. You're free to try building one yourself for cheaper. If you consider your time for researching/assembling/testing it to be worthless, and are happy with a contraption in a miner frame, then you can probably do it.
- zten 2y agoPC builds seem to short circuit everyone's pricing logic and drive any labor cost down to $0, just because they're willing to do it for free. Anything above that $0 is considered overpriced.
- aurareturn 2y agoIt’s not that they’re willing to do it for free. It’s that they’re doing it for fun. It’s a hobby, not work. Part of the fun is planning, researching, putting together the pieces and power it on.
- Dylan16807 2y agoThere are services that will build a PC for $200. It's entirely valid to ask where the money goes, and the answer is not the labor to put the pieces together. There's no reason to assume OP is being dismissive of that specific cost.
- deleted 2y ago[deleted]
- scoot 2y ago> The $15k tinybox red is the best perf/$ ML box in the world. It's fully networkable, so that's the metric that matters. No it isn't. Capex is only part of the equation. Opex (power and cooling amongst other things) is important. And networking at scale isn't cheap either.
- zelcon 2y agoUnless you're making money off it, $15k + however much you have to spend on installing a new breaker panel is too much to spend on hardware that will be outdated in 2 years. If you're making money off it, but you're still cheap, then buy a Supermicro + H100s and colo it in a datacenter. If you're not cheap, you'll just use Azure. So I'm not sure who this product is supposed to be for.
- justinclift 2y agoAnyone know how well this would compare to the Nvidia based workstations at the GPTShop.ai place? ie: things like this https://gptshop.ai/config/indexus.html https://gptshop.ai/config/indexus.html
- ddmf 2y agoGot confused with George Motz and thought this was some super burger making thing.