31 ms·
What Extropic is building
- kneel 3y agoThis guy spends an extraordinary amount of time posting memes and e/acc silliness. So much so I wonder what the hell they're doing with this company. Is he a prolific poster and an engineering genius? Or is he just another poster
- Bjorkbat 3y agoFor the longest time I thought the person behind the account was just some random guy who was probably very into crypto and decided to dabble in AI because of the parallels between e/acc and the whole "to the moon" messaging you find in crypto communities. Never would have guessed the guy was an actual physicist
- aaron695 3y ago[dead]
- fermionik 3y agoPhysical learning machines require noise to learn. They are also necessarily dissipative. See https://arxiv.org/abs/2209.11954 https://arxiv.org/abs/2209.11954. The key is to engineer the noise to maximise the learning rate. In classical devices, stochastic switching is controlled by temperature through the Kramers rate. This means kT controls energy loss. If you use dissipative quantum tunnelling this is not the true thermodynamic lower bound. Any quantum nonlinear dissipative system, with a far from equilibrium steady state, is a good case to consider. Dispersive optical bistability, realised in SC quantum circuits, is the way to go. And quantum error correction is unnecessary.
- jdulay19 3y agoCould someone smarter than me explain if this is a big deal or just hype? The work sound promising, but I wonder how long it would take to build and validate.
- deleted 3y ago[deleted]
- nfw2 3y agoI have a suspicion that a lot of people are nodding along because they don't want to seem like the village idiot.
- deleted 3y ago[deleted]
- shermantanktop 3y agonods vigorously
- zooq_ai 3y agoon the other hand, HN is filled with self-proclaimed critics that they dismiss everything and display their utter lack of imagination -- like AI, Metaverse, (success of) Snapchat, AirPods before
- api 3y agoDid HN dismiss AI? I saw some skepticism and still do but not dismissal. This site doesn’t get everything right. It tends to miss things that succeed in the consumer space because this is a pro audience not a mainstream audience. But it usually gets hard science right.
- digging 3y agoI'm similarly suspicious, and find it curious that this is the first I'm hearing about this at all. I don't have personal connections in physics or AI circles but I feel like I'd usually expect to have read mention of these ideas before finding this press release.
- jpm_sd 3y agoIt sounds like complete BS, unfortunately.
- 3y ago
- htrp 3y agothe engineering alone will be a nightmare
- rho4 3y agoimprove title pls
- brizzbuzz 3y agointeresting that a company w/ no public repositories has 1.1k github followers https://github.com/extropic-ai https://github.com/extropic-ai
- ozr 3y agoThe founder ('Beff Jezos') has a large twitter presence.
- echelon 3y agoIt's led by the e/acc [1] founder, BasedBeffJezos [2]. He has a huge cult following. It's turned into a lot of Twitter memes and shitposting [3]. [1] https://en.wikipedia.org/wiki/Effective_accelerationism https://en.wikipedia.org/wiki/Effective_accelerationism [2] https://twitter.com/BasedBeffJezos https://twitter.com/BasedBeffJezos [3] https://knowyourmeme.com/memes/cultures/eacc-effective-accelerationism https://knowyourmeme.com/memes/cultures/eacc-effective-accel...
- Bjorkbat 3y agoHonestly, as interesting as the the chip sounds, I'm admittedly kind of biased against the company's probability of success simply because the founder is basically the #1 e/acc meme account/shitposter on Twitter. Like, it's hard to take someone seriously when they spend tons of time shitposting on Twitter, it's even harder when it's revealed that they're behind one of the most popular shitposting accounts within a niche, almost cult-like community.
- arduanika 3y agoBack in the day, there was a saying that went something like: Steve Jobs was really good at what he did, and also an asshole to people. The former is really hard to replicate, so instead you'll find a lot of people going around and imitating the latter. Today, it's a very different situation. Now we have Elon Musk, who is really good at what he does, and also tweets a lot...
- 3y ago
- AbrahamParangi 3y agoMan, I am not a pessimist and I am very bullish on AI-the-field but my spidey sense is tingling that this is BS. - It is written in a way that sacrifices legibility for supposed precision but because the terms used can't really be applied precisely, it's equivalent to spurious digits in a scientific calculation. The usual reason this occurs is to obfuscate or to overawe the audience. - It is hard to overstate the difficulty of beating semiconductor with a wholly new branch of technology. They're so insanely good. People have been trying to beat them for decades and there's not even a solid theoretical thesis as to how to do so. Even the theoretical advantage of quantum computing is predicated on error correction being scalable which is a totally open question even theoretically.
- liveoneggs 3y agoIf room-temperature-stable bio-enhanced AI-specific-computer-powered chatbots don't seem like a realistic goal then maybe you should have clicked "play" on the linked spotify widget.
- 015a 3y agoFor me its the dichotomy between how absolutely impenetrable the blog post is, combined with the "Set the tone fam, play 'Entropy' by Noizinski on Spotify :)" widget in the bottom right. Like they're trying to check every box on the engagement farming list (something, to be sure, beff jezos is famous for). Very bad vibes. Hire someone who can communicate, and demonstrate what you're building.
- deleted 3y ago[deleted]
- 5cott0 3y agoso far the only thing they’ve built is more posts
- pphysch 3y agoTBF that's not a bad place to be in the current hype cycle. Better than releasing and being permanently written off as yet-another-ChatGPT-wrapper.
- spiantino 3y agoObvious grifty nonsense.
- thatguysaguy 3y agoUninmportant, but if you're citing Moore's paper I feel like you're just trying to pad out the references to make it look like you're serious
- gitfan86 3y agoAt a high level it is the right answer to the data center electricity demand problem. Which is that we need to make AI hardware more efficient. Pragmatically, it doesn't make much sense given that it would take years for this approach to have any real work use cases in a best case scenario. It seems way more likey that efficiency gains in digital chips will happen first making these chips less economically valuable.
- vipshek 3y agoI have no idea about the merits of this approach, but I found this interview with the founders a lot more sensical than the linked article: https://twitter.com/Extropic_AI/status/1767203839818781085 https://twitter.com/Extropic_AI/status/1767203839818781085
- blovescoffee 3y agoThis interview makes their product seem like BS. First, they literally cannot simply explain the problem or solution. Regardless, their pitch is that they're building a more power efficient probability distribution sampler. No one in AI research thinks that's a bottleneck. edit: btw the bottleneck in AI algos is matrix multiply and memory bandwith.
- ninjin 3y agoComputationally, yes, those are the bottlenecks. But I would also add supervised training data, as we can never get enough of that and it is one of few things that increases in compute are (to my mind, you could argue that by scaling unsupervised training further we could do away with it, but I am not yet convinced) not able to solve.
- blovescoffee 3y agoTheir startup is addressing computing bottlenecks so that's what I addressed. Supervised training dat isn't a bottleneck on LLMs, Diffusion models, or any of the hot areas at the moment.
- ninjin 3y agoI think the situation is less clear than that. While I have limited research experience with image generation, I believe I do have a fair understanding of large language models. From the publication of GPT-2 until ChatGPT, it was true that the argument always was that supervised training data was not a priority and that it all boiled down to scaling the amount of unsupervised training data. However, this all changed with preference tuning, etc. and I think there is also an argument to be made that the extensive training data curation that we see today (and is withheld from the "papers" we see for the models) is a form of supervision in its own right. It could be that we will see computational/data scaling dominate again, but I think it is equally possible that we will have the next few years dominated by data curation and exploring forms of supervision to "extract" value out of what was learnt at the unsupervised training stage. Still, you are correct that Extropic is looking at the computation rather than data. But, I wanted to chime in so as the discussion here would not leave the impression that we are still in the days of pure unsupervised scaling.
- Delumine 3y agoSeems like they're "passive" energy chips are only gonna be targeted $$$ towards big organizations, which make use of the Josephon effect. But if they're targeting transistor technology for the masses, how will they have an advantage against the incumbents
- empath-nirvana 3y agoSo, basically this seems to be a way to replace PRNGs with real randomness with some knobs so you can adjust the distribution. Let's assume for the sake of argument that this can replace every single PRNG call in inference and training, how much savings in cost/energy/run time would there actually be?
- rvz 3y agoToo early to tell about what this will be in the future. Either it turns out to be a foundational startup or a flash in a pan. But at least it is not the 5000th so-called AI-powered SaaS company that is using OpenAI API that has raised $20M+ to VCs and burning hundreds of thousands every month with little to no plan to generate revenue. Will be watching this one closely, but highly skeptical of this company.
- ac2u 3y agoHear hear, better to see someone go for broke trying something novel. At best they advance the field massively, at worst the backers lose their money but the tech/knowledge finds a home elsewhere and the knowledge in the field is nudged forward.
- jason-phillips 3y agoComments read like a confessional from out of the loop.
- sergiotapia 3y agoThe litepaper discusses Extropic's mission to develop a novel hardware platform that harnesses the natural fluctuations of matter as a computational resource for Generative AI. Key Points The demand for computing power in AI is increasing exponentially, but Moore's Law is slowing down due to fundamental physical limitations of transistors at the atomic scale. Biology hosts more efficient computing circuitry than current human-made devices by leveraging intrinsic randomness in chemical reaction networks. Energy-Based Models (EBMs) are a potential solution, as they are optimal for modeling probability distributions and require minimal data. However, sampling from EBMs is difficult on digital hardware. Extropic is implementing EBMs directly as parameterized stochastic analog circuits, which can achieve orders of magnitude improvement in runtime and energy efficiency compared to digital computers. Extropic's first processors are nano-fabricated from aluminum and run at low temperatures where they are superconducting, using Josephson junctions for nonlinearity. Extropic is also developing semiconductor devices that operate at room temperature, sacrificing some energy efficiency for scalability and accessibility. A software layer is being built to compile abstract specifications of EBMs to the relevant hardware control language, enabling Extropic accelerators to run large programs. --- Is this real or just theoretical?
- autonomousErwin 3y agoI wouldn't want to write this off because you get the feeling these guys are on to something that could be hugely important (ignoring quantum this thermodynamic that) - but surely it feels like they need to get to the point a lot faster e.g. "We're taking a new approach to building chips for AI because transistors can't get any smaller." I really don't know what they gain by convoluting the point and it's pretty hard to follow what the CEO is talking about half the time.
- riwsky 3y agoConvolutional neural networks were a huge advancement in their time
- autonomousErwin 3y agoI don't disagree. I just come away from the article feeling more confused as opposed to enlightened and excited about what they're building. It even makes me think that they don't understand what they're talking about which is why they're using complicated terminology to mask it but I'm hopeful I'm wrong and this is an engineering innovation that benefits everyone.
- Animats 3y agoI get that feeling, too. There may or may not be something there, but the article is mostly buzzword-slinging. They wrote "This will allow us to put an Extropic accelerator in every home, enabling everyone to partake in the thermodynamic AI acceleration." Huh? If they said something like "We are trying to cut the cost of stable diffusion by a factor of 100", that would sort of make sense. But then people would want to see a demo.
- autonomousErwin 3y agoA proof-of-concept would be amazing and that's what I thought they were releasing and would justify the hype (as opposed to a whitepaper). Maybe in a couple of months we'll see a HN post doing exactly this and we can eat our words (which I really hope is the case).
- DrDroop 3y agoI know everyone is calling BS on this, and I am just a simple web developer so what do I know but there are at least two priors that make me think that what is discussed here could have some validity. * The stochastic/random nature of processors is already used in cryptography for physically uncloneable functions. Dunno if this has any practical uses in industry, and it is crypto, so it is probably also BS, but it is the same phenomena you get if you log in into your BIOS and turn off ECC of your RAM. * The very first computer capable of MCMC was designed by von Neumann himself and used uranium as a source of randomness as part of the Manhattan project. Anyway semiconductors have never been my strong suit, but I guess this is more of a IP play then a consumer product business. Now let me get back to writing unit tests.
- Bnjoroge 3y agomeh, lmk when they actually ship something that's not bs
- binoct 3y agoI really hope this was an experiment in using gen AI: “Create a website for a new company that is building the next generation of computing hardware to power AI software. Make sure it sounds science-y but don’t be too specific.”
- horrysith 3y ago[flagged]
- swalsh 3y agoWhy make such a low effort pessimistic comment. What happened to HN?
- catchnear4321 3y agosame thing that is happening everywhere. cognitive effort is getting unbalanced, it used to be necessary to put effort in on write, and on read. now, it is hard to tell who put effort in at all. read or write. would you consider your own response to be optimistic or high effort?
- deleted 3y ago[deleted]
- adw 3y agoHN has always been a tense standoff between a few cliques, the first two being the ostensibly intended audience; * competent and curious engineers * entrepreneurs, who live on a continuum where one end is... * ...hucksters and snake-oil purveyors, of which there are plenty, and * (because this is the Internet) conspiracy theorists and other such loons and recently * political provocateurs You can make a thread work (for that group of people) if it self-selects who reads it. Unfortunately, AI is catnip to all five of these groups, so the average thread quality is exceptionally low – it serves all five groups badly. Whether some of these people _should_ be served well is a separate question.
- nfw2 3y ago
- dkarras 3y agosmells like snake oil. will probably end up becoming a cryptocurrency scam? or some other grift? time will tell. >Extropic is also building semiconductor devices that operate at room temperature to extend our reach to a larger market. funny stuff
- deepnotderp 3y agoI’m not a fan of Extropic, but I’m seeing a lot of misconceptions here. They’re not building “a better rng”- they’re building a way to bake probabilistic models into hardware and then run inference on them using random fluctuations. Theoretically this means much faster inference for things like PGMs. See here for similar things: https://arxiv.org/abs/2108.09836 https://arxiv.org/abs/2108.09836 There’s a company called Normal Computing that did something similar: https://blog.normalcomputing.ai/posts/2023-11-09-thermodynamic-inversion/thermo-inversion.html https://blog.normalcomputing.ai/posts/2023-11-09-thermodynam...
- winwang 3y agoSkimmed the litepaper. Has the flavor of: you can do "simulated" annealing by literally annealing. I like the idea of using raw physics as a "hardware" accelerator, i.e. analog computing. fwiw, quantum computing can be seen as a form of analog computing. I do think that a "better rng" can be interesting and useful in and of itself. Thanks for the Normal Computing post, it felt more substantial.
- pclmulqdq 3y agoI make a better RNG right now (https://arbitrand.com https://arbitrand.com). We experimented with doing ML training with it, but it's not clear that it trains any better than a non-broken PRNG. It might be fun to feed the output into stable diffusion and see how cool the pictures are, though.
- winwang 3y agoCloud RNG number streaming is interesting but costly, no? I did have the idea to serve truly random numbers via a quantum computer (trivial by just preparing the simplest state and measuring). Anything else can't be said to be truly random.
- pclmulqdq 3y agoYou don't need a quantum computer to sample noise from quantum processes. It's a prohibitively expensive way to go, and depending on how you built the quantum computer, it may be more susceptible to interference and non-quantum noise than using good circuits and custom systems.
- patcon 3y agoI believe this link is communicating within the family of thought from which this blog post also comes: https://knowm.org/thermodynamic-computing/ https://knowm.org/thermodynamic-computing/ It's a random, unassuming 7-year-old blog post from a DARPA-funded and defense-involved inventor. They happen to work in neuromorphic computing. Their other posts talk about some of that work. A cynical take is that it can seem like just hand-wavey garbage, but then again, it's been quietly getting tons of defense contractor money. I came across it years ago, and it has greatly accelerated my worldview, and has made me feel ahead of the curve in understanding what is going on in the universe. It's informed my community organizing. It's informed how I understand AI and consciousness and language, and the intersection of all these things. I'm inclined to believe that the people in this area are clued into something very substantial about how the universe works. EDIT: oops, shared the wrong link. This one is about thermodynamic evolution
- rabidsnail 3y agofund my new simulated annealing accelerator startup where we etch your model onto an aluminum flake and then hit it with a blowtorch
- dark_jensen 3y agofor all the hype around building alien tech, this is a bit underwhelming. the stuff from this startup feels more alien than what extropic is talking about - https://www.emergentia.tech/technology https://www.emergentia.tech/technology
- zachbee 3y agoThey're not wrong that sampling a complex, higher-dimensional probability distribution is hard to do efficiently. I'm not sure how useful it is to do it more efficiently, though. Also, the fact that they're using ultra-cold superconductors makes me wonder how much noise helps and how much it hurts. If your system is all about leveraging noise well, but you can only use super special well-behaved noise, then "bad noise" could easily ruin the quality of your generated solutions. It's cool to see something so wacky out there, though!
- ein0p 3y agoPeople need to read Hamming’s old papers in which he very clearly explains why analog circuits are not viable at scale. This is also why the brain uses spikes rather than continuous signals. The issue is noise, interference, and attenuation. There’s no way to get around this. If they have invented a way, I’d like to see it. But until it’s demonstrated, I’d take such things with a large grain of salt.
- sfnrm 3y agoSounds interesting. Do you have a link? (or at least a title?)
- ein0p 3y agoNot at the moment, but I do recall he has a chapter on this in his book “The Art of Doing Science and Engineering”, which I also recommend. He uses very long transmission lines to explain this, but the same thing applies at the nano scale, and perhaps to an even greater extent due to the much noisier environment and higher frequencies.
- Animats 3y agoYou can re-quantize analog signals into a finite number of levels to prevent noise accumulation. That's how TLC (8 levels) and QLC (16 levels) flash memory cells work. The cells store an analog value, but it's forced to a value close to one of N discrete values. The same approach is used in modems. Deep learning doesn't seem to need that much numerical precision. People started with 32-bit floats, then 16-bit floats, now sometimes 8-bit floats, and recently there are people talking up 2-bit trinary. The number of levels needed may not be too much for analog. If you have a regenerator once in a while to slot values back to the allowed discrete levels, you can clean up the noise. That's an analog to digital to analog conversion, of course. That's not what these guys are talking about, as far as I can tell.
- twobitshifter 3y agoanalog circuits are making a comeback because they are great for simulating the equations of the physical world more efficiently than a digital approach. https://spectrum.ieee.org/not-your-fathers-analog-computer https://spectrum.ieee.org/not-your-fathers-analog-computer
- Eliezer 3y agoI'm saddened to see the honorable name of Extropy and Extropianism, which carefully never descended to this level or anything like it, be stolen and captured by this nonsense.
- arduanika 3y agoIs this sarcasm? (Genuinely can't tell.) And also, are you the real Eliezer?
- Eliezer 3y agoNo, not sarcasm, and I am Eliezer Yudkowsky. I was around on the old Extropians mailing list starting in 1996, and their leadership did not talk like this. Max More (the founder of Extropianism) was a careful thinker then, and I haven't heard anything different about him more recently than that. "Extropy" is a term that was previously coined by a group of fairly nice people to describe themselves, and so far as I know is being stolen here without permission.
- arduanika 3y agoOh wow, hello! Seems like it was quite a time to be online. I mostly know of it through this version of events. Not sure how accurate you'd find it: https://aiascendant.substack.com/p/extropias-children-chapter-1-the-wunderkind https://aiascendant.substack.com/p/extropias-children-chapte... The word "extropy" itself seems to go back several decades before the mailing list, if I'm reading correctly here: https://en.wikipedia.org/wiki/Extropianism https://en.wikipedia.org/wiki/Extropianism. Still, I wouldn't be surprised if many/most of the original mailing list members found this usage a corruption.
- thom 3y agoI've thought for a while that what quantum computing will probably deliver is not going to be magical infinite processing power, but extremely fast, computational access to parameterizable physical processes. That is, a rock can simulate being a rock better than a computer can, but how do you hook it up to the rest of your system? But while I can imagine replacing a simple MCMC model, for example, with a stack of physics-based chips, is there a path all the way to designing, training and executing something LLM sized on top of that technology? I'm not smart enough to know, but as esoteric as it sounds, it feels like it's drawing on the less speculative end of the spectrum, and seems like a noble effort and not an actual scam.
- powera 3y agoNot only does this read like pure bullshit, it is bullshit on a website that crashes the Apple Vision Pro (and makes my laptop suffer). My prediction is that they will raise a nine-figure sum over the next decade, and never release a product that comes close to the performance of an NVIDIA card today.
- trzy 3y agoHard time believing this is legit given how much time the CEO spends goofing around on social media. If it were possible to short startups, this would be a top candidate.
- jp42 3y agohonestly, it would be too early to say this. Considering the people who invested in this startup, its better to assume CEO is capable. If he is not able to deliver in reasonable timeline then, we all are free to blame him for posting things on SM. actually many knows his company because he is goofing around on SM especially e/acc stuff.
- danielmarkbruce 3y agoIt's more interesting to see who passed on it. There isn't a single top tier VC here. This whole pitch sounds like the usual quantum computing babble.
- dark_jensen 3y agoexactly!! most of the large accounts hyping this stuff up are invested in it or dont seem to have a clear idea of what they invested in and/or part of the inner clique. also there doesn't seem to any serious hardware investors in that group - more folks who have bought into e/acc and all the scifi mythos around it.
- deleted 3y ago[deleted]
- danielmarkbruce 3y agoIt feels like serious people would have said something more like "we are going to improve the performance (measured in s), of the algorithms/models such X, Y, Z which are used in a, b, c." Can anyone name a company which used such absurd language to describe themselves and then actually delivered something valuable? There must be one.
- laserbeam 3y agoThe amount of buzzwords on this page should disqualify this from even getting votes on HN. Anyone who writes like this is trying to confuse and mislead the reader.