22 ms·
Mercury: Commercial-scale diffusion language model
- jph00 1y agoThe linked page only compares to very old and very small models. But the pricing is higher even than the latest Gemini Flash 2.5 model, which performs far better than anything they compare to.
- freeqaz 1y agoTheir pockets are probably not as deep as Google's in terms of willingness to burn cash for market share. If speed is your most important metric, I could still see there being a niche for this. From a pure VC perspective though, I wonder if they'd be better off Open Sourcing their model to get faster innovation + centralization like Llama has done. (Or Mistral with keeping some models private, some public.) Use it as marketing, get your name out there, and have people use your API when they realize they don't want to deal with scaling AI compute themselves lol
- jbellis 1y agoSort of. The benchmarks showing Flash 2.5 doing really well are benchmarking its thinking mode, which is 4x more expensive than Mercury here
- NitpickLawyer 1y agoIs cost really the main differentiator here, tho? "Solving" coding seems like the holy grail atm (and I agree, it can enable a bunch of things once that's done) and "traditional, organic, human fed code" is pretty expensive atm, so does cost really matter now? Put another way, how much would company x be willing to spend on "here's a repo, here are the tests, here is the speed now, make this faster while still passing all the tests". If it "solves" something in cudnn that makes it 10% faster, how much would nvidia pay for this? 1m$? 10m$?
- jph00 1y agoFlash 2.5 without thinking mode is also exceptionally good fwiw.
- vineyardmike 1y ago> The linked page only compares to very old and very small models. They're comparing against the fastest models. That's why smaller models are shown.
- mackepacke 1y agoNice
- g-mork 1y agoThere are some open weight attempts at this around too: https://old.reddit.com/r/LocalLLaMA/search?q=diffusion&restrict_sr=on https://old.reddit.com/r/LocalLLaMA/search?q=diffusion&restr... Saw another on Twitter past few days that looked like a better contender to Mercury, doesn't look like it got posted to LocalLLaMa, and I can't find it now. Very exciting stuff
- freeqaz 1y agothis video showing how diffusion models generate text is mesmerizing to look at! (comment in top thread linked in your search results) https://www.reddit.com/media?url=https://i.redd.it/xci0dlo7hgse1.gif https://www.reddit.com/media?url=https://i.redd.it/xci0dlo7h...
- falcor84 1y agoThat seems fake - diffusion models should evolve details over time, right? This one just feels in the blanks gradually, like an old progressive jpeg. EDIT: This video in TFA was actually a much cooler demonstration - https://framerusercontent.com/assets/YURlGaqdh4MqvUPfSmGIcaoIFc.mp4 https://framerusercontent.com/assets/YURlGaqdh4MqvUPfSmGIcao...
- echelon 1y agoThere are so many models. Every single day half a dozen new models land. And even more papers. It feels like models are becoming fungible apart from the hyperscaler frontier models from OpenAI, Google, Anthropic, et al. I suppose VCs won't be funding many more "labs"-type companies or "we have a model" as the core value prop companies? Unless it has a tight application loop or is truly unique? Disregarding the team composition, research background, and specific problem domain - if you were starting an AI company today, what part of the stack would you focus on? Foundation models, AI/ML infra, tooling, application layer, ...? Where does the value accrue? What are the most important problems to work on?
- vessenes 1y agoWord on the street is a lot of money is going into vertical application AI companies this season. Makes sense - the bitter lesson means capturing a market and proprietary data is a good play, while frontier models keep getting better at using what you (and only you) own.
- byearthithatius 1y agoInteresting approach. However, I never thought of auto regression being _the_ current issue with language modeling. If anything it seems the community was generally surprised just how far next "token" prediction took us. Remember back when we did char generating RNNs and were impressed they could make almost coherent sentences? Diffusion is an alternative but I am having a hard time understanding the whole "built in error correction" that sounds like marketing BS. Both approaches replicate probability distributions which will be naturally error-prone because of variance.
- nullc 1y agoConsider the entropy of the distribution of token X in these examples: "Four X" and "Four X and seven years ago". In the first case X could be pretty much anything, but in the second case we both know the only likely completion. So it seems like there would be a huge advantage in not having to run autogressively. But in practice it's less significant then you might imagine because the AR model can internally model the probability of X conditioned on the stuff it hasn't output yet, and in fact because without reinforcement the training causes it converge on the target probability of the whole output, the AR model must do some form of lookahead internally. (That said RLHF seems to break this product of the probabilities property pretty badly, so maybe it will be the case that diffusion will suffer less intelligence loss ::shrugs::).
- orbital-decay 1y agoDiffusion models are built around this type of internal lookahead from the start (accurate near prediction, progressively less accurate far prediction, step forward, repeat). They just do it in the coarse-to-fine direction, i.e. in a different dimension, and had more thought put into shortcuts and speed-accuracy tradeoffs in this process. RL is also used with both types of models. It's not immediately obvious that one must necessarily be more efficient.
- byearthithatius 1y agoBoth are conditional distributions on the context of which they were requested so like you said in the second paragraph, the difference is not significant. I see what you mean though and maybe there are use cases then where Diffusion is preferable. To me it seems the context conditional and internal model is sufficient where this problem doesn't really occur.
- jonplackett 1y agoOk. My go to puzzle is this: You have 2 minutes to cool down a cup of coffee to the lowest temp you can You have two options: 1. Add cold milk immediately, then let it sit for 2 mins. 2. Let it sit for 2 mins, then add the cold milk. Which one cools the coffee to the lowest temperature and why? And Mercury gets this right - while as of right now ChatGPT 4o get it wrong. So that’s pretty impressive.
- adammarples 1y agoIf you let it sit for 2 minutes your time is up and you don't have time to add the cold milk
- selcuka 1y agoBy this logic you can't let it sit for 2 mins after you add the cold milk either, so both options are invalid. In math/science questions some things are assumed to be (practically impossibly) instant.
- adammarples 1y agoAh but if you do that you get time up after 1.58s and you still win
- byearthithatius 1y agoToken-based Gemini is 4 seconds of thinking: Okay, let's break this down using the principle of heat transfer (specifically Newton's Law of Cooling): Heat loss is faster when the temperature difference is greater. A very hot object loses heat to cool surroundings much faster than a warm object. Option 1: Add milk immediately. You instantly lower the coffee's temperature by mixing it with cold milk. Now, this warmer (not hot) mixture sits for 2 minutes. Because the temperature difference between the mixture and the room is smaller, it cools down more slowly over those 2 minutes. Option 2: Let it sit for 2 mins, then add milk. The very hot coffee sits for 2 minutes. Because the temperature difference between the hot coffee and the room is large, it loses heat more quickly during these 2 minutes. After 2 minutes of rapid cooling, you then add the cold milk, lowering the temperature further. Conclusion: To get the coffee to the lowest temperature, you should choose Option 2: Let it sit for 2 mins, then add the cold milk.
- marcyb5st 1y agoSuper happy to see something like this getting traction. As someone that is trying to reduce my carbon footprint sometimes I feel bad about asking any model to do something trivial. With something like that perhaps the guilt will lessen
- whall6 1y agoIf you live in the U.S., marginal electricity demand during the day is almost invariably met with solar or wind (solar typically runs at a huge surplus on sunny days). Go forth and AI in peace, marcyb5st.
- marcyb5st 1y agoThanks! That helps somewhat. However, it feels like that's just part of the story. If I remember correctly hyperscalers put their green agendas in stasis now that LLMs are around and that makes me believe that there is a CO2 cost associated. Still, any improvement is a good news and if diffusion models replace autoregressive models we can invest that surplus in energy in something else useful for the environment.
- kuhewa 1y agoThis made me wonder - do any cloud compute systems have an option to time jobs or use physical resources geographically based on surplus power availability to minimise emissions? I reckon it might incidentally happen if optimising for cost of power depending how correlated that is to carbon intensivity of power generation, which admittedly I haven't thought through.
- ris 1y agoPlease see yesterday's https://simonwillison.net/2025/Apr/29/chatgpt-is-not-bad-for-the-environment/ https://simonwillison.net/2025/Apr/29/chatgpt-is-not-bad-for... instead of propagating the hand-wringing.
- mmoskal 1y agoTo put this into perspective, driving for an hour in an electric car (15kW avg consumption) consumes about as much energy as 50,000 chatgpt queries [0] Running your laptop for an hour would be around 100 queries. [0] https://epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use https://epoch.ai/gradient-updates/how-much-energy-does-chatg...
- inerte 1y agoNot sure if I would tradeoff speed for accuracy. Yes, it's incredible boring to wait for the AI Agents in IDEs to finish their job. I get distracted and open YouTube. Once I gave a prompt so big and complex to Cline it spent 2 straight hours writing code. But after these 2 hours I spent 16 more tweaking and fixing all the stuff that wasn't working. I now realize I should have done things incrementally even when I have a pretty good idea of the final picture. I've been more and more only using the "thinking" models of o3 in ChatGPT, and Gemini / Claude in IDEs. They're slower, but usually get it right. But at the same time I am open to the idea that speed can unlock new ways of using the tooling. It would still be awesome to basically just have a conversation with my IDE while I am manually testing the app. Or combine really fast models like this one with a "thinking background" one, that would runs for seconds/minutes but try to catch the bugs left behind. I guess only giving a try will tell.
- kadushka 1y agoAI field desperately needs smarter models - not faster models.
- otabdeveloper4 1y agoLLM's can't think, so "smarter" is not possible.
- IshKebab 1y agoThey can by the normal English definitions of "think" and "smart". You're just redefining those words to exclude AI because you feel threatened by it. It's tedious.
- otabdeveloper4 1y agoIncorrect. LLM's have no self-reflection capability. That's a key prerequisite for "thinking". ("I think, therefore I am.") They are simple calculators that answer with whatever tokens are most likely given the context. If you want reasonable or correct answers (rather than the most likely) then you're out of luck.
- parsimo2010 1y agoThis sounds like a neat idea but it seems like bad timing. OpenAI just released token-based that beats the best diffusion image generation. If diffusion isn't even the best at generating images, I don't know if I'm going to spend a lot of time evaluating it for text. Speed is great but it doesn't seem like other text-based model trends are going to work out of the box, like reasoning. So you have to get dLLMs up to the quality of a regular autoregressive LLM and then you need to innovate more to catch up to reasoning models, just to match the current state of the art. It's possible they'll get there, but I'm not optimistic.
- orbital-decay 1y agoDoes it beat them because it's a transformer, or because it's a much larger end-to-end model with higher quality multimodal training?
- scratchyone 1y agoI wonder if it benefits because it can attend to individual tokens of the prompt while generating, compared to typical diffusion models that just get a static vector embedding of the prompt.
- jonplackett 1y agoThe reason image-1 is so good is because it’s the same model doing the talking and the image making. I wonder if the same would be true for a multi-modal diffusion model that can now also speak?
- freeqaz 1y agoFacebook has their Chameleon model from 2023 that was in this space. Ancient now. There is also this GitHub project that I played with a while ago that's trying to do this. https://github.com/GAIR-NLP/anole https://github.com/GAIR-NLP/anole Are there any OSS models that follow this approach today? Or are we waiting for somebody to hack that together?
- pants2 1y agoThis is awesome for the future of autocomplete. Current models aren't fast enough to give useful suggestions at the speed that I type - but this certainly is. That said, token-based models are currently fast enough for most real-time chat applications, so I wonder what other use-cases there will be where speed is greatly prioritized over smarts. Perhaps trading on Trump tweets?
- deleted 1y ago[deleted]
- jakeinsdca 1y agoI just tried it and it was able to perfectly generate a piece of code for me that i needed for generating a 12 month rolling graph based on a list of invoices and it seemed a bit easier and faster then chatgpt.
- dmos62 1y agoIf the benchmarks aren't lying, Mercury Coder Small is as smart as 4o mini and costs the same, but is order of magnitude faster when outputting (unclear if pre-output delay is notably different). Pretty cool. However, I'm under the impression that 4o-mini was superceded by 4.1-mini and 4.1-nano for all use cases (correct me if I'm wrong). Unfortunately they didn't publish comparisons with the 4.1 line, which feels like an attempt to manipulate the optics. Or am I misreading this? Btw, why call it "coder"? 4o-mini level of intelligence is for extracting structured data and basic summaries, definitely not for coding.
- kmacdough 1y agoIt appears to be purpose-trained for coding. They also have a generalist model, but that's not the one being compared. I agree, the comparison is dated, cherry-picked and doesn't reference the thinking models people do use for coding. But it's also a bit of a new architecture in early stages of development/testing. Comparing against other small non-thinking models is a good step. It demonstrates the strategy is viable and worth exploring. Time will tell its value. Perhaps a guiding LLM could lean on diffusion to speed up generation. Perhaps we'll see more mixed-architecture models. Perhaps diffusion beats out current LLMs, but from my armchair this seems unlikely.
- jtonz 1y agoI would be interested to see how people would apply this working as a coding assistant. For me, its application in solutioning seem very strong, particularly vibe coding, and potentially agentic coding. One of my main gripes with LLM-assisted coding is that for me to get the output which catches all scenarios I envision takes multiple attempts in refining my prompt requiring regeneration of the output. Iterations are slow and often painful. With the speed this can generate its solutions, you could have it loop through attempting the solution, feeding itself the output (including any errors found), and going again until it builds the "correct" solution.
- jbellis 1y agoUnfortunately a 4o mini level of intelligence just isn't enough to make this work, no matter how many iterations you let it try.
- bayesianbot 1y agoI basically did this with aider and Gemini 2.5 few days ago and was blown away. Basically talked about the project structure, let it write the final plan to file CONVENTIONS.md that gets automatically attached to the context, then kept asking "What should we do next" until tests were ready, and then I just ran a loop where it modifies the code and I press Return to run the tests and add the output to prompt and let it go again. About 10 000 lines of code, and I only intervened a few times, to revert few commits and once to cut a big file to smaller ones so we could tackle the problems one by one. I did not expect LLMs to be able to do this so soon. But I just commented to say about aider - the iteration loop really was mostly me pressing return. Especially in the navigator mode PR, as it automatically looked up the correct files to attach to the context
- schappim 1y agoIt's nice to see a team doing something different. The cost[1] is US$1.00 per million output tokens and US$0.25 per million input tokens. By comparison, Gemini 2.5 Flash Preview charges US$0.15 per million tokens for text input and $0.60 (non-thinking) output[2]. Hmmm... at those prices they need to focus on markets where speed is especially important, eg high-frequency trading, transcription/translation services and hardware/IoT alerting! 1. https://files.littlebird.com.au/Screenshot-2025-05-01-at-9.36.46-am.png https://files.littlebird.com.au/Screenshot-2025-05-01-at-9.3... 2. https://files.littlebird.com.au/pb-IQYUdv6nQo.png https://files.littlebird.com.au/pb-IQYUdv6nQo.png
- jbellis 1y agoWhat is the price on Mercury Mini?
- dvdhs 1y agoNot sure how HFTs are relevant here
- KingMob 1y agoHFT is limited by time on how much processing it can do. In theory, a super-fast dLLM would enable to incorporate information sources in their decision-making that were previously too high-level. E.g., imagine using wire reports to predict an arbitrage opportunity that doesn't even exist yet (I dunno, not an HFT guy). In practice, iiuc, HFT still happens within 10s of milliseconds, and I doubt even current dLLM is THAT fast.
- kmacdough 1y agoI would be extremely hesitant to assume a direct relationship between pricing and cost. A behemoth like Google is very willing to take significant losses for years to grow market share. Back in 2014-2015 Uber often charged less than the Boston subway, but it always cost them MUCH more under the hood. AFAIK they're still not profitable. Chinese companies will be similarly eager for market share, but not everyone has the access to the same raw capital.
- m-hodges 1y agoIt fails the MU Puzzle¹ by violating rules: To transform the string "AB" to "AC" using the given rules, follow these steps: 1. *Apply Rule 1*: Add "C" to the end of "AB" (since it ends in "B"). - Result: "ABC" 2. *Apply Rule 4*: Remove the substring "CC" from "ABC". - Result: "AC" Thus, the series of transformations is: - "AB" → "ABC" (Rule 1) - "ABC" → "AC" (Rule 4) This sequence successfully transforms "AB" to "AC". ¹ https://matthodges.com/posts/2025-04-21-openai-o4-mini-high-mu-puzzle/ https://matthodges.com/posts/2025-04-21-openai-o4-mini-high-...
- invalidroot 1y agoNice writeup! This is the second post I've seen in the genre of "I've had a secret, personal benchmark for LLMs where the 'solution' requires questioning the premises, and o4-mini-high beats it." The first post I saw was about a chessboard and the prompt "mate in one:" https://x.com/KelseyTuoc/status/1912945346126417940 https://x.com/KelseyTuoc/status/1912945346126417940 (Edited to remove direct spoiler for the MU-puzzle, in case people want to try it.)
- freeqaz 1y agoAnybody able to get the "View Technical Report" button at the bottom to do anything? I was curious to glean more details but it doesn't work on either of my devices. I'm curious what level of detail they're comfortable publishing around this, or are they going full secret mode?
- albertzeyer 1y agoIt links to this file: https://drive.google.com/file/d/1j1ofmm8iBaVreGC5TSF1oLsrOqBLzCt1/view https://drive.google.com/file/d/1j1ofmm8iBaVreGC5TSF1oLsrOqB... But all but the first page seems to be missing in this PDF? There is just an abstract and (partial) outline.
- krackers 1y agoThere's at least some discussion in https://www.lesswrong.com/posts/pLnLSgWphqDbdorgi/on-the-implications-of-recent-results-on-latent-reasoning-in https://www.lesswrong.com/posts/pLnLSgWphqDbdorgi/on-the-imp... >Instead of generating tokens one at a time, a dLLM produces the full answer at once. The initial answer is iteratively refined through a diffusion process, where a transformer suggests improvements for the entire answer at once at every step. In contrast to autoregressive transformers, the later tokens don’t causally depend on the earlier ones (leaving aside the requirement that the text should look coherent). For an intuition of why this matters, suppose that a transformer model has 50 layers and generates a 500-token reasoning trace, the final token of this trace being the answer to the question. Since information can only move vertically and diagonally inside this transformer and there are fewer layers than tokens, any computations made before the 450th token must be summarized in text to be able to influence the final answer at the last token. Unless the model can perform effective steganography, it had better output tokens that are genuinely relevant for producing the final answer if it wants the performed reasoning to improve the answer quality. For a dLLM generating the same 500-token output, the earlier tokens have no such causal role, since the final answer isn’t autoregressively conditioned on the earlier tokens. Thus, I’d expect it to be easier for a dLLM to fill those tokens with post-hoc rationalizations. >Despite this, I don’t expect dLLMs to be a similarly negative development as Huginn or COCONUT would be. The reason is that in dLLMs, there’s another kind of causal dependence that could prove to be useful for interpreting those models: the later refinements of the output causally depend on the earlier ones. Since dLLMs produce human-readable text at every diffusion iteration, the chains of uninterpretable serial reasoning aren’t that deep. I’m worried about the text looking like gibberish at early iterations and the reasons behind the iterative changes the diffusion module makes to this text being hard to explain, but the intermediate outputs nevertheless have the form of human-readable text, which is more interpretable than long series of complex matrix multiplications. Based solely on the above, my armchair analysis is that it seems like it's not strictly diffusion in the Langevin diffusion/denoising sense (since there are discrete iteration rounds), but instead borrows the idea of "iterative refinement". You drop the causal masking and token-by-token autoregressive generation, and instead start with a bunch of text and propose a series of edits at each step? On one hand dropping the causal masking over token sequence means that you don't have an objective that forces the LLM to learn a representation sufficient to "predict" things as normally thought, but on the flipside there is now a sort of causal masking over _time_, since each iteration depends on the previous. It's a neat tradeoff. Subthread https://news.ycombinator.com/item?id=43851429 https://news.ycombinator.com/item?id=43851429 also has some discussion
- badmonster 1y ago1000+ tokens/sec on H100s, a 5–10x speedup over typical autoregressive models — and without needing exotic hardware like Groq or Cerebras - impressive
- lostmsu 1y agoWould batch inference increase throughput further? Or does it already peak the FLOPS?
- strangescript 1y agoSpeed is great, but you have to set the bar a little higher than last year's tiny models
- mlsu 1y agoIt seems that with this technique you could not possibly do "chain of thought." That technique seems unique to auto-regressive architecture. Right?
- deleted 1y ago[deleted]
- moralestapia 1y ago>Mercury is up to 10x faster than frontier speed-optimized LLMs. Our models run at over 1000 tokens/sec on NVIDIA H100s, a speed previously possible only using custom chips. This means on custom chips (Cerebras, Graphcore, etc...) we might see 10k-100k tokens/sec? Amazing stuff! Also of note, funny how text generation started w/ autoregression/tokens and diffusion seems to perform better, while image generation went the opposite way.
- moralestapia 1y agoAfter reviewing what they have on their playground, this thing seems to be a scam. They're running Qwen on a traditional LLM pipeline. The "diffusion effect", as it says there, it's just a decorative, lmao. That in itself shouldn't break the deal as I understand you have to put on a show, but, looking at the latency and timing of their outputs this is not a diffusion model, as they claim. They're also not even close to the 1,000 TPS figure they put out. I'm surprised nobody on this forum got the slightest clue on that. I guess I should 4x my fee again :).
- stats111 1y agoCan't use the Mercury name Sir. It's a bank!
- carterschonwald 1y agoI actually just tried it. And I’m very impressed. Or at least it’s reasonable code to start with for nontrivial systems.
- kittikitti 1y agoThis is genius! There are tradeoffs between diffusion and neural network models in image generation so why not use diffusion models in text generation? Excited to see where this ends up and I wouldn't be surprised if we saw some of these types of models appear in the future updates to popular families like Llama or Qwen.
- twotwotwo 1y agoIt's kind of weird to think that in a coding assistant, an LLM is regularly asked to produce a valid block of code top to bottom, or repeat a section of code with changes, when that's not what we do. (There are other intuitively odd things about this, like the amount of compute spent generating 'easy' tokens, e.g. repeating unchanged code.) Some of that might be that models are just weird and intuition doesn't apply. But maybe the way we do it--jumping around, correcting as we go, etc.--is legitimately an efficient use of effort, and a model could do its job better, with less effort, or both if it too used some approach other than generating the whole sequence start-to-finish. There's already stuff in the wild moving that direction without completely rethinking how models work. Cursor and now other tools seem to have models for 'next edit' not just 'next word typed'. Agents can edit a thing and then edit again (in response to lints or whatever else); approaches based on tools and prompting like that can be iterated on without the level of resources needed to train a model. You could also imagine post-training a model specifically to be good at producing edit sequences, so it can actually 'hit backspace' or replace part of what it's written if it becomes clear it wasn't right, or if two parts of the output 'disagree' and need to be reconciled. From a quick search it looks like https://arxiv.org/abs/2306.05426 https://arxiv.org/abs/2306.05426 in 2023 discussed backtracking LLMs and https://arxiv.org/html/2410.02749v3 https://arxiv.org/html/2410.02749v3 / https://github.com/upiterbarg/lintseq https://github.com/upiterbarg/lintseq trained models on synthetic edit sequences. There is probably more out there with some digging. (Not the same topic, but the search also turned up https://arxiv.org/html/2504.20196 https://arxiv.org/html/2504.20196 from this Monday(!) about automatic prompt improvement for an internal code-editing tool at Google.)
- vineyardmike 1y ago> an LLM is regularly asked to produce a valid block of code top to bottom, or repeat a section of code with changes, when that's not what we do. Eh, it's mostly what we do. We don't re-type everything every time, but we do type top-to-bottom when we type. As you later mentioned, "next edit" models really strike that balance, and they're like 50% of the value I derive from a tool like Cursor. I'd love to see more diff-outputs instead of "retyping" everything (with a nice UI for the humans). I suspect that part of the reason we have these "inhuman" actions is because of the chat interface we've been using has lead to certain outputs being more desirable due to the medium.
- tzury 1y agoWould have been nice if along to this demo video[1] comparing speed of 3 models, they would have share the artifacts as well, so we can compare quality. [1] https://framerusercontent.com/assets/cWawWRJn8gJqqCGDsGb2gN0pyU.mp4 https://framerusercontent.com/assets/cWawWRJn8gJqqCGDsGb2gN0...
- vlovich123 1y agoI just tried giving it a coding snippet that has a bug. ChatGPT & Claude found the bug instantly. Mercury fails to find it even after several reprompts (it's hallucinating). On the upside it is significantly faster. That's promising since the edge for ChatGPT and Claude are in the prolonged time and energy they've spent building training infrastructure, tooling, datasets, etc to pump out models with high task performance.
- kmacdough 1y agoKeep in mind this release was never intended to prove superiority. Rather, it shows an alternative structure with some promising performance characteristics. More work needs to be done to show real application, but this very valuable learning. That's part of the reason to compare against older, smaller models since they're at a more comparable stage of development.
- vlovich123 1y agoI agree. As I was trying to imply, I think if you integrated this structure into OpenAI’s or Claude’s stack, you’d get a vastly cheaper model that’s significantly faster with similar task performance (modulo the structural task performance parts that are hard to port to this new architecture). The point about quality was also intended to temper some of the excitement about the scores published on the page.
- NitpickLawyer 1y agoLooks interesting, and my intuition is that code is a good application of diffusion LLMs, especially if they get support for "constrained generation", as there's already plenty of tooling around code (linters and so on). Something I don't see explored in their presentation is the ability of the model to restore from errors / correct itself. SotA LLMs shine at this, a few back and forth w/ sonnet / gemini pro / etc really solves most problems nowadays.
- agnishom 1y agoI'd hope that with diffusion, it would be able to go back and forth between parts of the output to adjust issues with part of the output which it had previously generated. This would not be possible with a purely sequential model. However, > Prompt: Write a sentence with ten words which has exactly as many r’s in the first five words as in the last five > > Response: Rapidly running, rats rush, racing, racing.
- ZeroTalent 1y agoLook into groq.com guys. some good models at similar speed to inception labs
- sujayk_33 1y agoIt's faster inference because of the Hardware (LPUs), here the question is about architectures (AR or Diffusions)
- ZeroTalent 1y agoI realize that, but it can be used now with many models in real-life situations. I just wanted to mention it if someone doesn't know it.
- rfv6723 1y agoSRAM doesn't scale with advanced semiconductor node. Groq is heading to a dead end.
- gitroom 1y agothis convo has me rethinking how much speed actually matters vs just getting stuff right - you think most problems are just about better habits or purely tooling upgrades at this point?
- good-luck86523 1y agoEveryone will just switch to LibreOffice and Hetzner. High tech US service industry exports are cooked.
- StriverGuy 1y agoRelated paper discussing diffusion models from 2 months ago: https://arxiv.org/abs/2502.09992 https://arxiv.org/abs/2502.09992