8 ms·

Muse Spark 1.1

• deleted 2mo ago
  [deleted]
• deleted 2mo ago
  [deleted]
• redox99 2mo ago
  Very strong pricing, cheaper than Grok 4.5, particularly the cached reads. We'll have to wait to see if it's actually worth using (it's not on OpenRouter yet).
  • rpgbr 2mo ago
    That's what one does when its product and public perception is way behind competitors.
• Tiberium 2mo ago
  The pricing is insane: $1.25/$4.5 for 1M tokens, and $0.15 for cached input! https://dev.meta.ai/docs/getting-started/pricing-rate-limits https://dev.meta.ai/docs/getting-started/pricing-rate-limits
  • ignoramous 2mo ago
    Cheaper than Qwen 3.7 Max. Second indication, after Grok 4.5 ($2 in / $6 out), that the BigLabs are feeling the GLM 5.2 heat.
    • cedws 2mo ago
      [flagged]
  • fallingbananna 2mo ago
    Meta isn’t right now on the radar for most folks picking models. If they have a really good model, it makes sense to subsidise it, to gain users, before they align prices with competitors.
    • ycui7 2mo ago
      this is not subsidizing. this is way too expensive for a no-name model.
      • steinvakt2 2mo ago
        Depends on the quality
        • winfredJa 2mo ago
          just played around, it is pretty low quality. lower than sonnet.
  • ai_fry_ur_brain 2mo ago
    This is still ridiculously expensive imagine having to pay $10 for 100 search results on Google, thats essentially what this is. I really dont see how anyone's willing spend more than $1.50 per mm output. Let alone $15-50. Does anyone actually pay for usage based billing as a consumer?
    • 2mo ago
• kilroy123 2mo ago
  I personally do not like Meta, but I'll say this. The more competition, the better for regular consumers. (Enterprise too) - Chinese models - Grok - Meta - Google - OpenAI - Anthropic I think this is a win. I'm building like crazy to take advantage of all these subsidized tokens while I can.
  • alansaber 2mo ago
    Meta's local llama models used to be the face of open source AI. The scene has really changed.
    • cyanydeez 2mo ago
      they likely got the Peter Theil newsletter proclaiming open source models are the antichrist
      • bevekspldnw 2mo ago
        Only 666B models. Other sizes are fine.
      • alightsoul 2mo ago
        That person is Alexandr Wang. He made his money selling data annotation services to closed source companies: openai, anthropic, Google, even Meta
  • cpt100 2mo ago
    Yeah, I think it is definitely great. Having said that, I am still debating in my mind whether the volume of software engineers needed in the AI era is going to increase or decrease because of all of these advancements. On the one hand, because it is easy to build products, more and more people will build. And more and more products and features will be built. However, a lot of people who are non-technical will also try to build, but they get stuck, and then they will need engineers. The sheer volume of product built by both experienced technical companies and non-technical novice startups and founders and wannabe founders is going to be massive. That is the bull case for having more software engineers needed in the near future. On the other hand, in a year or so, people will build all these products, and most of them won't be able to market them, sell them and make money. Eventually, there won't really be a need for that many software engineers. I think overall the bull case is probably going to win net net.
• greenavocado 2mo ago
  Meta is back in the game, albeit not at the top. Impressive stuff, nonetheless.
  • qpricjalcbeu 2mo ago
    Weren't they caught multiple times gaming the benchmark even more so then the rest?
    • alansaber 2mo ago
      Let me assure you, literally everybody does this
      • sheepscreek 2mo ago
        I don't think it even matters. Because noone will continue to use an LLM that doesn't work well for them, whether or not it has a good bench result. So for their own sake, the correct representation can actually win them some loyalty: eg. Model X is weaker than Fable, but competes well with Opus/Sonnet and costs 1/5th as much etc - something similar playing out with Grok 4.5.
    • zmmmmm 2mo ago
      Yes and Zuck effectively disbanded the entire team that did that. Not saying we shouldn't cast a critical eye on it, but it probably does warrant a second chance.
      • qpricjalcbeu 2mo ago
        Zuck was part of that team.
  • cpt100 2mo ago
    They are not open source anymore, right?
• deleted 2mo ago
  [deleted]
• EgregiousCube 2mo ago
  Their published benchmarks seem to indicate that it's pretty good at coding and multimodal, but VERY good at successful tool calls. What kind of use case would be best for that shape?
  • alansaber 2mo ago
    This sounds... kind of useless? Really good JSON or similar constrained decoder performance is interesting, but normal decoder > tool validator loop with good error message > tool retry is almost always able to get a tool to work second try, and input is cached so it's not expensive.
    • aldanor 2mo ago
      Things are not always that simple, eg https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools/ https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools/
    • winstonp 2mo ago
      The avg coding session has hundreds or thousands of tool calls. Even a 5% failure rate noticeably notches up token use and cost. See Gemini.
      • alansaber 2mo ago
        Yes, but each tool call has a different failure %. The tool calls that make up the majority of volume like grep are going to have nowhere near a 5% failure. A custom user-defined skill having a 5% failure rate is probably fine.
  • xnorswap 2mo ago
    Debugging and diagnosis is very tool call heavy, whether that's grepping / transforming logs, calling out to profilers/tracers, or even just writing up incident reports. Bug diagnostics is about being okay at coding but better at tooling. Given a good diagnostic report, it can be handed to opus for the fix. Opus is okay at writing reports, but it still regularly gets table widths wrong in typst documents, leaving the last column full of text but only a handful of characters wide.
• deleted 2mo ago
  [deleted]
• throwaway613746 2mo ago
  [dead]
• lnenad 2mo ago
  Considering the DeepSWE result (imho if you're gonna give value to benchmarks this is one of the best) it's not good enough.
  • svantana 2mo ago
    It's a high quality benchmark for sure, but it being public means it's at risk of leaking into the models (unintentionally or not), right? For that reason I prefer to look at the private ones, like: HLE, SimpleBench, Kagi, ARC-AGI.
• qpricjalcbeu 2mo ago
  Yeah, no thanks. I cannot think of a worse company to trust with additional personal data.
  • niek_pas 2mo ago
    Me neither, though LLMs also provide services that don’t involve personal or sensitive data
  • ledak 2mo ago
    Exactly. I have zero trust in them, especially given their history of shady practices - like buying a VPN service (Onavo) and using it to track how people used competing apps.
• frangonf 2mo ago
  Is this the model trained on Meta "draftees"? Are we seeing this in the jump on JobBench?
• Jcampuzano2 2mo ago
  Competition for cheaper and efficient models is a good thing, regardless of if you don't like SpaceX, Meta, etc. Especially from US based labs I for one am really glad to get competitive models that will push the major labs to bring prices down. While Chinese open source labs are also great, unfortunately when it comes to US/Western political pressure it won't often have as much of a bearing on labs bringing prices down, especially for enterprises. Also if these numbers are true, this is truly breaking ground finally for Meta.
  • verdverm 2mo ago
    There are US companies hosting open weight models for enterprise, we just enabled Fireworks.ai for the devs
• zb3 2mo ago
  This is not open-weights, right?
  • EgregiousCube 2mo ago
    Correct
• phillipcarter 2mo ago
  My trust factor is gone with Meta right now. Has there been any independent analysis to confirm they didn't cheat on benchmarks again?
  • solarkraft 2mo ago
    They cheated again: https://news.ycombinator.com/item?id=48847019 https://news.ycombinator.com/item?id=48847019
    • meric_ 2mo ago
      No they didn't. Please read: https://www.anthropic.com/engineering/infrastructure-noise https://www.anthropic.com/engineering/infrastructure-noise
      • solarkraft 2mo ago
        Thanks for the read. It seems to confirm that resource limits are an important factor for terminal benchmarks: > The extra resources enable the agent to try approaches that only work with generous allocations, such as pulling in large dependencies, spawning expensive subprocesses, and running memory-intensive test suites. > An agent that writes lean, efficient code very fast will do well under tight constraints. An agent that brute-forces solutions with heavyweight tools will do well under generous ones. Both are legitimate things to test, but collapsing them into a single score without specifying the resource configuration makes the differences—and real-world generalizability—hard to interpret. So changing the resource limits changes the benchmark. Yet their score table claims their score to be for Terminal-Bench 2.1, not Terminal-Bench 2.1 with raised limits.
  • vehbiemiroglu 2mo ago
    Can't we use it even if we don't trust it?
    • phillipcarter 2mo ago
      I don't see a reason to use something that is, at best, about as good as models that are produced by labs who don't have a reputation of cheating.
• guluarte 2mo ago
  A lot of these benchmarks are unfamiliar. Are labs just choosing the ones that make them look best?
• deleted 2mo ago
  [deleted]
• carimura 2mo ago
  I missed the fact that Meta was developing and releasing closed-weights models... bummer. Would be great to see some more progress with American open-weights models.
• anthonypasq 2mo ago
  Everyone has been loving to shit on the Alexander Wang acquisition but this seems legitimately impressive to me? Meta's AI org when from a total mismanaged dumpster fire for multiple years to delivering a competitive model in less than a year on essentially their first try?
  • paxys 2mo ago
    How is it their first try? They were leading the race with Llama 3.x a few years ago.
    • rsstack 2mo ago
      They were leading the race in a niche category a few years ago. Now they are, according to some benchmarks, even on the right playing field.
    • anthonypasq 2mo ago
      As far as i remember, the entire AI org was essentially gutted and replaced with whoever Wang wanted to hire, and tbh that org completely failed to train llama 4 and I honestly doubt whatever techniques they used to ship llama 3 are at all relevant now. That was before reasoning models and the heavy emphasis on RL/post-training. so yeah, this is essentially their first try with a completely new org.
      • mchusma 2mo ago
        I agree with parent, Meta has been at this a long time and its only because they have recently fallen off that they pushed this "oh give us credit its really a new org" thing. Basically, if you can't actually "win" then try to fake a restart and say we are the fastest. Even given that, this is their second try (they had Spark 1.0). Spark 1.0 was uninteresting, this is potentially interesting, but we can't really try it yet it seems (at least not in Openrouter). Ultimately, competition is now fierce in this broad level of intelligence/cost: Spark 1.1, Grok 4.5, GPT 5.6 Luna, GLM 5.2 Sonnet not in the same ballpark of pricing (more expensive than Opus in many cases). Haiku has been basically abandoned.
• GodelNumbering 2mo ago
  Lot more details in the linked report https://ai.meta.com/static-resource/muse-spark-1-1-evaluation-report https://ai.meta.com/static-resource/muse-spark-1-1-evaluatio... From Terminal-bench-2.1 details, > We use a bash-tool-only agent harness to evaluate 89 Terminal-Bench 2.1 tasks from the official repository, where resources are capped at 6 CPU cores and 8GB RAM. This disqualifies the results. Each terminal bench task has a cpu upper limit and RAM upper limit. Overriding either is disqualification. For reference, in tbench-2.1, 1. 0 out of 89 task allow 6 cpu cores (highest is 4, and i think only 1 task) 2. 8 out of 89 tasks allow 8GB RAM This kind of shady benchmarking (I was talking about it just yesterday in a different context https://news.ycombinator.com/item?id=48838212 https://news.ycombinator.com/item?id=48838212) takes all joy out of building a harness to improve benchmark performance of a model because no matter what you do, you won't beat the headline (cheating) number. This is presumably why this model is not in the official benchmark leaderboard https://www.tbench.ai/leaderboard/terminal-bench/2.1 https://www.tbench.ai/leaderboard/terminal-bench/2.1 As an ex Meta employee, this is a little sad but not massively surprising. 'Number go up' is the core performance evaluation metric until PSC is done and you move on.
  • efromvt 2mo ago
    Out of curiosity, how often are the resource limits the bottlenecks? What do harnesses do to help here - limit parallelism? More efficient tools?
    • artrockalter 2mo ago
      The task could be verifiable in the environment so limiting its CPU and RAM could be to discourage brute forcing the answer.
  • kingstnap 2mo ago
    Why are resource limits considered at all aside from models accidentally fork bombing themselves? I thought the benchmark was supposed to be about terminal use and specifically chaining together lots of bash tool calls. Which test cases does this matter for?
• paxys 2mo ago
  How is every company able to show itself at the top of every benchmark?
  • adam_arthur 2mo ago
    Not much moat, incremental improvements, cherry picking models to compare. To be fair, seems more correct to compare against similar strength models if your main edge is pricing.
  • ffsm8 2mo ago
    They're being called "trust me bro benchmarks" for a reason ( ･ั ﹏ ･ั )
  • logicchains 2mo ago
    Wait to the exact moment your model is ahead on at least N benchmarks then publish.
  • morgengold 2mo ago
    First look what models are worse in a set of self selected benchmarks. Second, compare to older versions of competitor s models. Still does not look good? Compare to own previous models.
  • toephu2 2mo ago
    Anyone deep in the AI realm know which is the gold standard benchmark for coding?
    • deleted 2mo ago
      [deleted]
    • msdz 2mo ago
      It’s just inherently a difficult problem to solve, I think (just like with human individual contributors, where there famously also doesn’t exist one universal, automated process). “Good” benchmarks to gauge development skills at the moment seem to be: DeepSWE [0] by Datacurve FrontierCode [1] by Cognition And then there’s TerminalBench, which I’m certain has been saturated in post-training to no end, so I wouldn’t think of it as a gold standard anymore. But yeah, in general, it’s not going to get easier knowing which benchmarks are actually measuring “frontier” capability, and which are just getting results inflated by way of time/token budget [2], ever again. [0] https://deepswe.datacurve.ai https://deepswe.datacurve.ai [1] https://cognition.com/blog/frontier-code https://cognition.com/blog/frontier-code [2] https://xcancel.com/i/article/2064210146558136827 https://xcancel.com/i/article/2064210146558136827
• zmmmmm 2mo ago
  Good to see Meta finally back to releasing something at least worth evaluating. And it sounds like they did at least a bit skate to where the puck is going by focusing on tool and computer use.
• NitpickLawyer 2mo ago
  How are people trying this? I don't see it on openrouter. Any ways of testing this without subscribing to meta stuff?
  • maipen 2mo ago
    Probably need to wait some hours/1-2 days and openrouter will add it.
    • NitpickLawyer 2mo ago
      Thanks. I was asking because I couldn't find even their previous 1.0 model there.
• chvid 2mo ago
  Interesting that neither meta nor xai chose to do open source given that they are both clearly behind Google, OpenAI and anthropic - and a serious us open source offering would give them a clear foothold.
  • verdverm 2mo ago
    I suspect they have a brand problem from their social media ties and shady histories. I personally will never use their models, plenty of better alternatives. I'm now exclusively on open weight models
  • tpae 2mo ago
    Open source would make them an instant credible leader, major fumble (still can be fixed)
• bel8 2mo ago
  It seems to trade blows with GPT 5.5 and Opus 4.8 in performance while being cheaper than GLM 5.2.
• eugene3306 2mo ago
  > Model API is not available in your region. :( Well, Vietnam is not in the list of restricted territories. Anyway, what is "your region" ? Is this where I am now, or is it where I activated my Oculus 2 five years ago ?
  • steinvakt2 2mo ago
    Can’t you just use VPN?
    • bhc 2mo ago
      Your Meta account profile region has to be in the U.S., and they do Geo IP and SMS verification.
      • pranavchavda 2mo ago
        why?
  • redox99 2mo ago
    Same in Argentina. It's almost surely a region whitelist for now (it's the only reason Argentina ever gets blocked).
  • wyck 2mo ago
    Same in Canada. lol
• Sol- 2mo ago
  Interesting how the prevalent opinion until yesterday seems to have been that OpenAI & Anthropic are irreversibly ahead and now with xAI and Meta at least delivered something that's competitive with useful models and cheap too. Granted, the narrative that the two leading labs are ahead still holds with Fable (and perhaps an upcoming GPT6), but it's not as over as common knowledge by the opinion leaders would have us believe.
  • re-thc 2mo ago
    > Interesting how the prevalent opinion until yesterday seems to have been that OpenAI & Anthropic are irreversibly ahead Not the way you're implying? The GLM 5.2 hype was blowing way before this. Neither xAI nor Meta have really made a difference in a different way - similar results / similar pricing (to GLM 5.2).
  • logicchains 2mo ago
    People misinterpreted Google being behind as Anthropic and OpenAi being really ahead, when it was really just Google falling behind the same way it did with Tensorflow, Angular and GCP.
    • revolvingthrow 2mo ago
      > when it was really just Google falling behind the same way it did with Tensorflow, Angular and GCP Not sure I agree. Angular fell behind in popularity but was (is? unsure atm) still eminently usable. I gave gemini a test drive recently and it was horrendous, as in "picking dirt cheap Chinese model over gemini any day" bad, and with overzealous guardrails to boot. 3.1 pro feels a year behind and is extremely lazy. 3.5 flash feels like a model you’d run on your 128gb macbook, not something that was released a month ago and which costs a fair bit when used through api. In any case: as of right now I think that we went from a three horse race to anthropic / openai as premium choices vs whatever is the Chinese fotm for a fraction of the cost. 3.5 pro better be a miracle if google wants to hang out with the big boys, otherwise their only strategy is hoping that both US labs go broke and they remain the last man standing.
• whinvik 2mo ago
  Why are the plans and pricing for all these products so complicated. I don't know where I need to sign up to try it out. What is pricing? Is it API or subscription, what? I had the exact same experience with Grok 4.5 as well.
  • SwellJoe 2mo ago
    Nearly every model can be found on OpenRouter and used with a single key. Meta Spark is not among them, but Grok and almost every other model is. That's how I try models I don't already have an account for.
• minraws 2mo ago
  Tried to get access to the API, apparently the model API is not available in my region... I have questions regarding if I should even care but I don't so Meta please keep enjoying the irrelevance. lmao
  • cmrdporcupine 2mo ago
    Right, amazing because for me also... "My region" being Canada. I'm going to assume the only "region" that's permitted is the USA.
• IshKebab 2mo ago
  Haha their demo is AI spamming restaurants on Instagram. This is going to go really well.
• jacobgold 2mo ago
  Maybe Zuck should double down on his "spoiler" role with models rather than compete head-to-head. He doesn't have to match Anthropic or OpenAI model revenue if he can deflate theirs by 99%. All he has to do is keep spending a few billion dollars developing frontier models, release them as open weights, and turn coding models into a commodity. He also needs a good OSS reference harness to match. Very few people are in a position to do this and for it to make business sense. That's quite likely where things are headed regardless, and he could speed it up significantly. We should all hope models move from proprietary products to commodities the way compilers did. This may be one of the best things Zuck could do for the world.
  • odie5533 2mo ago
    If he deflates their revenues, who is going to rent the compute from Meta?
    • vineyardmike 2mo ago
      The goal is not for meta to take their market, the goal would be for meta to damage their competitors. If meta releases an open-weight LLM that is not Chinese made, cheaper to run than the SOTA premiums, etc, it would lower the number of people paying for frontier labs models. We saw with with early LLAMA models, but they didn’t keep up in the race with v4. Meta would benefit from this, not from increased revenue at the hands of open LLMs, but from reduced competition. Meta competes with Google for ad spend, and lowering the Google revenue (or increasing costs) from AI reduces the competitive advantage. OpenAI wants to build an ad engine, so same thing will apply there too - make it less-revenue-generating to compete. Meanwhile G, OpenAI, and Anthropic are huge talent sinks that they have to compete with, especially for ML talent which is core to Metas business goals (ads). Finally, Meta needs lots of GPUs to train their ad engine models. By reducing the revenue-per-GPU of these labs, they’re reducing demand on a core revenue generating supply they have to compete for.
      • AlotOfReading 2mo ago
        The problem is that Meta jumped headfirst into the circular financing of datacenter build out, so they're just as screwed if the infrastructure becomes less valuable.
• simonw 2mo ago
  I had a few days of preview access, which was long enough to put together a plugin for LLM. You can try the model out in the terminal like this: uv tool install llm llm install llm-meta-ai llm keys set meta-ai # paste API key here llm -m meta-ai/muse-spark-1.1 "Generate an SVG of a pelican riding a bicycle" Here's the result: https://tools.simonwillison.net/markdown-svg-renderer#url=https%3A%2F%2Fgist.github.com%2Fsimonw%2F4117330e4110279a172ed4876057816d https://tools.simonwillison.net/markdown-svg-renderer#url=ht... For comparison, here's the pelican I got from Muse Spark 1: https://simonwillison.net/2026/Apr/8/muse-spark/ https://simonwillison.net/2026/Apr/8/muse-spark/
  • edcrfv 2mo ago
    How do you find the time to “preview” so many models? It’s been a crazy time recently with the model releases. Does it ever feel like a chore?
    • gavinray 2mo ago
      His primary interest nowadays is LLM's. Unsure if he gets paid for it, but he seems to take it as a dayjob.
      • simonw 2mo ago
        My blog genuinely is a day job now, I make enough from the (unobtrusive, cookie-free) sponsorship banner that I no longer regret not having a proper software engineering job.
        • owebmaster 2mo ago
          [flagged]
          • pheaded_while9 2mo ago
            I see you're being blammed. "They hated him because he spoke the truth." Just because this online character is from your alma mater website doesn't mean he's not cynically and shamelessly self-promoting.
• fmind-dev 2mo ago
  Glad to see Meta back on track! Users will benefit greatly from this competition.
• jedisct1 2mo ago
  Not opensource.
• arizen 2mo ago
  I'm still confused is it available to public via some sort of subscription?
• ashish01 2mo ago
  Since it was making rounds yesterday here is what muse generated for the 3d rubik's cube prompt - https://ashish01.github.io/rubik-muse/ https://ashish01.github.io/rubik-muse/
  • Kuyawa 2mo ago
    Hold my beer... https://chat.z.ai/space/t19sx5kvw631-art https://chat.z.ai/space/t19sx5kvw631-art
• deleted 2mo ago
  [deleted]
• gavinray 2mo ago
  Despite Muse being relatively average, I've actually used the Meta AI webchat LLM since it released. The reason: Its writing style feels "unique", and I find it pleasant to read for science-based topics. I never ask _ONLY_ Meta AI, but the answer it gives is almost always in a distinctly different style than other frontier LLM's. I think this is because of the unique JEPA architecture they have, but that's a layman's hunch.
  • esafak 2mo ago
    I think it does not use JEPA.
• mchusma 2mo ago
  This not being available on Openrouter really makes it hard to test. I was going to compare vs Grok 4.5 and GPT-5.6 Luna, but I don't want to deal with signing up for Meta for it unless it checks out. Please Meta make this available.
• hintymad 2mo ago
  It's great that we have yet another competing models. The more models we have, the less likely we are subject to the ideologies and the controls thereof by the cults like Anthropic. And of course, it drives down the cost of tokens.
• Marciplan 2mo ago
  haha no thx
• dominotw 2mo ago
  Is building models on the frontier really easy now or something?
• deleted 2mo ago
  [deleted]
• lousken 2mo ago
  Still no weights? Useless
• weitendorf 2mo ago
  Just got it working with codex in a container! FYI I think there is a bug most others will run into at the Codex:Muse interface. It's some kind of parsing or integration error due to what I think is codex not anticipating server-side tool calling and how meta treats those ids... first couple times running codex with muse, it would fail on its first non-web search call. Got it fixed, not personally sold on the bespoke server-side tool calling and indefinite file storage yet, but also a very cool model that I'm enjoying using so far! https://github.com/accretional/awesome-muse-spark/blob/main/README.md#fix---reverse-proxy-shim-to-fix-id-replay https://github.com/accretional/awesome-muse-spark/blob/main/...
• bradfa 2mo ago
  Where is the data retention policy information for paid API per-token uses? Every other provider has one and makes it clear how they handle your data. A quick look doesn’t show one for this new offering.
  • bradfa 2mo ago
    Finally found it: https://dev.meta.ai/legal/ https://dev.meta.ai/legal/ Paid prompts are not used for training. Data retention is not entirely clear, they will keep it if they think you're breaking the rules, but it's not clear how long they keep it if they don't think you're breaking the rules.
• alightsoul 2mo ago
  Not open source, so not relevant. The only ones that can be relevant while being closed source are anthropic and openai. Not even Google except for search, not even spacexai. People use chatgpt more than Gemini for the average person who is not searching.
• dynm 2mo ago
  Perhaps on brand for Meta, there appears to be no mention anywhere on this page or any of the docs pages, of any data retention policy. Maybe this is hidden in an agreement you need to sign when making an account? Has anyone actually done that?