8 ms·
Qwen3.8 27B scores 52 on Artificial Analysis
- dethos 1mo agoI'm impressed with the score. This is a model that runs on a good, but still regular, desktop PC.
- anana_ 1mo agoFor more context, this puts it on par with models like GLM 5.2 and GPT 5.6 Luna, which are far larger
- bertili 1mo agoAnd more context: Same score as the latest DeepSeek Flash 0731 which has 284B parameters! (13B active) Its also the second best Qwen model, much better than Qwen 3.7 Max, but significantly below Qwen 3.8 Max.
- anthonypasq 1mo agoisnt the active parameter count more relevant than the total? qwen is a dense model no?
- kzrdude 1mo agoFor some tasks yes, and we don't know how many active parameters Luna is using..probably less than 27B
- anana_ 1mo agoAnd to read the tea leaves a little: 3.8 actually performs slightly worse than 3.6 on AA-Omniscience Accuracy, which could imply that they traded out world knowledge for capability in other areas. It also produces nearly twice as many tokens per task as 3.6 (and by extension, time), which may be a tradeoff required to achieve correctness at this parameter size.
- skohan 1mo agoImo it makes sense for things to move in the direction of small, focused models that excel in one area. I use LLMs for technical work 99% of the time, I could care less about general world knowledge, or if the model is good at creative writing. With good orchestration and delegation you can get surprisingly far with small models running on consumer hardware.
- anana_ 1mo agoAgreed. Luckily, this model also scores high in AA non-hallucination, so it knows what it doesn't know -- perfect for situations where it can just tool call a web search.
- tancop 1mo agoThe biggest untapped market is pure agentic models that are built for tool calling and non hallucination instead of memorizing facts. You need some world knowledge (as in common sense) to build a useful model, but I don't think perfect recall on general QA is a good use of space when you have web search and structured knowledge in Wikidata or Wolfram Alpha. Training should focus on tasks that require real intelligence instead of memory. Creative writing is actually good for this if you score it on coherence instead of getting random real life details right. Basic level of coding (simple prompt to code, don't need to one shot complex projects) is also great because writing a small script is more efficient than 20 separate tool calls.
- CamperBob2 1mo agoWith respect to Wolfram Alpha, it's worth noting that VibeThinker-3B is basically a match for the larger frontier models -- hundreds of times larger -- in the narrow domain of mathematical and logical reasoning problems. It doesn't seem necessary to resort to external models or tools for that, at least in principle.
- deleted 1mo ago[deleted]
- nsingh2 1mo agoAlso with Qwen 3.8 being more token hungry than Luna, using around 2.3x tokens. Which hurts for local deployment.
- sottol 1mo agoI'm torn on this - on the one hand performance matters, on the other so does capability. I could run Qwen 3.6 27B on my laptop, but at 5 tok/s it was too slow even without overthinking - I never used it. OTOH, Qwen 3.6 35B A3B ran at 20 tok/s but it just could not get done what I asked of it. It sort of got close but you had to repeat and retry so much that it might have been faster to run 27B dense... maybe? So that said, I might take a much better model that runs 2-3x slower (total time per task) but that's more capable over a faster, less capable one. I'd also like to try a proper "plan-then-execute" type execution where thinking is entirely disabled (or low) during the execution stage but enabled/max during the planning stage. I will definitely give 3.8 27B a better shot than 3.6 though.
- skohan 1mo agoDepends on your use-case. Over the past couple days, I've found 192k context more than enough for coding. There's more thinking for sure compared to comparably sized models (running on xhigh), but I've found the results are so much better that the entire session consumes less tokens on average since weaker models need more review passes.
- johnnyApplePRNG 1mo agoWe don't actually know how large they are, actually.
- catigula 1mo agoWhich should tell you how useful these benchmarks are.
- apitman 1mo agoVery interesting. I was not expecting anything close to this.
- kessler9 1mo ago[flagged]
- manunicholasjac 1mo ago[flagged]
- sottol 1mo agoA lot of the benchmarks seem often near meaningless these days - really bench-maxxed to the hilt. I tend to still look at the Artificial Analysis rankings to get at least an idea on relative performance of models, is that still warranted? What or other opinions on how representative the AA rankings are of real-world performance? Any better indicators?
- Iolaum 1mo agoYea and we are reaching the point where this benchmaxing is visible in the model's reported overthinking.
- logicchains 1mo agoIt's not overthinking, it's the right amount of thinking necessary for such a small model to get good results. The dumber the model, the more it has to think to be smart. There's no easy way to reduce the thinking without reducing the model quality.
- zdragnar 1mo agoQwen doom loops were amusing to watch the first time or two, but it's incredibly vexing to have it waffle over the same decision over and over and over and over again. I can get more done with a faster model by correcting it, and it feels better to babysit them than it does to babysit qwen to see if I need to intervene or if it will actually finish. I do like the output from qwen when I get it, but honestly I haven't been impressed enough with it to put up with the downsides.
- skohan 1mo agoIt's only been a couple days, but I haven't seen looping issues with 3.8 so far, compared to 3.6 which did occasionally have this problem.
- re5i5tor 1mo agoHave you tried it? I’d recommend doing so, it’s impressive in real use cases.
- colingauvin 1mo agoIt's 7th (!!!) overall on the agentic index, above Terra.
- hadlock 1mo agoStrangely Qwen 3.8 Max isn't on their list, at all.
- colingauvin 1mo agoIt's 3rd at 58.
- euazOn 1mo agoThat is actually insane. In my opinion, the future is local AI: for most daily tasks, you absolutely don't need Fable level intelligence - you need Fable level agentic capabilities. And this model has (almost) just that. If we get a similarly capable MoE model in a few months (yes we will), it's going to be an utterly wild ride.
- ProllyInfamous 1mo agoThank you for writing this; I have delayed 3.8:27b because my ollama needs to be updated (and I'm too lazy); but if I can save thousands of dollars not buying GPU hardware – for Top-10 performance – yes I will unlazy myself and update local ollama software.
- copperx 1mo agoCan't an agent do the upgrade for you?
- ProllyInfamous 29d agoI don't run them sudo, friend.
- beltsazar 1mo agoAs a comparison, Qwen3.6 27B scores 38, which was the highest in its small model category (4B–40B). Qwen3.8 27B beats all medium models (40B–150B). It has the same score as DeepSeek V4 Flash 0731, which ranks #5 in large model category (> 150B). Sources: - https://artificialanalysis.ai/models/open-source/small https://artificialanalysis.ai/models/open-source/small - https://artificialanalysis.ai/models/open-source/medium https://artificialanalysis.ai/models/open-source/medium - https://artificialanalysis.ai/models/open-source/large https://artificialanalysis.ai/models/open-source/large
- phsource 1mo agoSimon Willison's post about this gives a good context on why exactly this is happening. While it doesn't mention this in the Artificial Analysis page, this is likely with Max reasoning, which has extremely long reasoning traces: https://simonwillison.net/2026/Aug/16/qwen-38-27b/ https://simonwillison.net/2026/Aug/16/qwen-38-27b/ It seems like the token usage is 2.3x GPT Luna Max and almost 2x Kimi K3! https://imgur.com/a/dDSyhr2 https://imgur.com/a/dDSyhr2 I'm curious if they can make up for this with insanely high tokens-per-second especially when served from hosted providers, though, given how tiny it is (37B!)
- skohan 1mo agoI'm running 3.8 27B locally, and the results from the past few days have been excellent. I find raw speed is less of an issue when you can trust the model more to reach the right result.
- ArvidSu 1mo agoA ThinkingCap variant of Qwen 3.8 27b would be extremely interesting. https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B And then a Bonsai ternary on top of that model.
- kees99 1mo agoRe: bonsai - unsloth's quants have Q2 (UD-IQ2) variants, which are more or less same in size. ...or did Prism do something special with their "bonsai" releases? I didn't notice anything like QAT being mentioned.
- johnnyApplePRNG 1mo agoUnbelievable. Bravo Qwen team.
- bertili 1mo agoI can't shake this the existential feeling that this compact series of 27G bytes represent something profound and universal.
- kzrdude 1mo agoRaw model size is around 27x2 GB since it's in BF16 format
- macwhisperer 1mo ago4-bit quant sits at 15.72GB
- deleted 1mo ago[deleted]
- cardboard9926 1mo agoWhere's GLM 5.3 score?
- jakswa 1mo agoI'm waiting for this comparison too. I was impressed by a 1-shot GLM 5.3 did for me the other day.
- _tgxm 1mo agoPerhaps model size and reasoning length trade off to some extent, similar to CPU vs. RAM. A smaller model with a longer reasoning trace has more intermediate structure to latch onto and build on.
- deflator 1mo agoMakes sense to me. We will see, since if true then it is likely the other makers of small, dense models will copy it and include high reasoning by default. If that also makes the other dense open source models better, then you are probably correct.
- prakashbuilds 1mo agoInteresting to see where local models are going to be in the coming days. I am already starting to believe open source models are the way to go in the coming days. With Qwen 3.8 Max, Kimi K3 etx already delivering at part perf with frontier models, the future is going to be exciting.
- ProllyInfamous 1mo agoI purchased a 5070Ti (16GB nVIDIA GPU) a few months ago, and it is absolutely incredible what can be accomplished on local hardware (whether offline or not). Don't forget `mistral-small` (Apache's LLM), which to me is equally as impressive as qwen3.5 (only benefit of qwen is seeing the pre-text reasoning is often more helpful than the actual text output).
- JV00 1mo agoWhy is it not included in the Pareto line intelligence/cost chart?
- leprials 1mo agoTheres no official API yet. So theres nothing to price against.
- culi 1mo agoMost are running it locally
- WithinReason 1mo agoJust count the cost of electricity then :)
- ignoramous 1mo ago> So theres nothing to price against. To my surprise, providers on OpenRouter (io/akash/chutes) are serving Qwen3.8 27B at ~ $0.4 (in) / $3 (out) / $0.25 (cache), more expensive than DeepSeek v4 Flash. https://openrouter.ai/qwen/qwen3.8-27b https://openrouter.ai/qwen/qwen3.8-27b / https://archive.vn/RrDGO https://archive.vn/RrDGO
- JV00 1mo agoAh so luna still wins, unless you deploy on your own hardware
- Lynnr 1mo ago[flagged]
- matheusmoreira 1mo agoIt tied with Luna/max. Simply incredible.
- padolsey 1mo agoThe smaller these frontier-nearing models get, the more I'm reminded of https://en.wikipedia.org/wiki/Lottery_ticket_hypothesis https://en.wikipedia.org/wiki/Lottery_ticket_hypothesis
- keeganpoppen 1mo agoi think there definitely is some truth to this in terms of embeddings spaces, which is why i believe they are implemented by OpenAI/Anthropic in roughly highest import => least import bit order-- an overwhelming majority of the variance is in the first few hundred vector bits. i haven't actually tested this myself by manually truncating vectors, but it is my understanding that they generally speaking have this property.
- Havoc 1mo agoI wonder whether it’s possible to test the seed on a small model and then size it up on whatever works
- f311a 1mo agoWhy is it so small, but expensive? Open Router Input /M $0.45 Output /M $3.20 Cache read /M $0.05 Throughput 27 tps It would be a very nice model at 200-300 tps and if it was dirt cheap. What's the limiting factor of optimizing speed and price for inference providers?
- freakynit 1mo agoI read it somewhere recently that it's architecture does not allow serving as many concurrent requests as the deepseek models allow. Maybe that's why.
- FuckButtons 1mo agoFrom first principles, 27b dense vs 13b moe, means you spend ~2x more memory bandwidth per request amortized over the whole server (ie, you assume all experts are being concurrently used by some user during the forward pass, then on average the bandwidth required for one forward pass for any individual request is just the size of one expert). deepseek also have some innovations around kv cache and compressed attention which allow for further reductions in memory bandwidth which means that the thing that’s actually bottlenecking inference, (memory bandwidth) is significantly lower than for qwen 3.8, which has been optimized for running 1 instance ~= 1 user.
- meatmanek 1mo agoWhen processing multiple users in parallel, don't you end up having to load in multiple experts? Not every session is going to use each expert at exactly the same time.
- Grimblewald 1mo agowhere'd you read that? sounds like total bs but i could be wrong and would like to learn more.
- freakynit 1mo agoRead this on Reddit.. maybe it's total bs.. maybe not... i dont know.. mentioned in case someone wants to dig deeper into this.
- manofmanysmiles 1mo agoImagine this, and sucesor models on Cerebras or other silicon...
- WASDx 1mo agoThat might actually compensate for the overthinking, if it can think really fast. Dense models are easier than MoE to put on silicon. https://chatjimmy.ai/ https://chatjimmy.ai/ is getting 16k tps with an 8B model. Extrapolating that gives nearly 5k tps for 27B. And we're still early in this technology. If tps is so high, a compaction step could be performed over every thinking turn to keep context size down.
- Moduke 1mo agoVery exciting indeed. It is in the works. Their current dense offering, Gemma 4 31B, sits at ~1800t/s https://news.ycombinator.com/item?id=49308715 https://news.ycombinator.com/item?id=49308715
- kmike84 1mo agoI have an internal automated benchmark, which roughly follows my workflow, and I've been testing various models on it, local and cloud. Qwen 3.8 27B did awesome. Its understanding is correct, research is better than e.g. glm's (and I like glm), and implementation is good and careful. Qwen 3.8 27B doesn't look benchmaxxed. These "52 AA score" numbers feel real, which is surprising. I've been using it locally for a few days for other tasks as well. If not the speed, I'd be totally happy to use it as a daily driver instead of cloud models, it is that good. --- (benchmark, to get an idea): 1. First, initial prompt which is not super precise - similar to how I'd write a task when talking e.g. to Opus. I'm describing an idea, and asking model to come up with some plan, and also to criticize the approach. Task is about implementing a particular pi extension. I'm checking if a model actually understands what I'm asking. 2. Then, as a follow-up, I ask to research alternative implementations, research UX of similar extensions, etc. It needs to do web searches, inspect open source codebases, read articles and papers, etc. I don't prompt to do this exactly, but I expect good models to figure out they need to do it. 3. Then, implementation. Also, one finding: Q4 and Q8 seem to have very different behavior in this benchmark. Q4 produces 2-3x thinking in the end, and makes more turns - it seems it makes more mistakes, and needs effort to recover from them, while Q8 gets more things right in a first try. In the end, quality is roughly similar, but Q8 gets there much faster, especially the implementation (tried it several times). Could be a difference between concrete artifacts, or between runtimes, I don't know, but be careful - it seems the real-world experience with qwen 3.8 27B can be vastly different, depending on how it's set up. Regarding DeepSeek 0731 vs Qwen 3.8 27B. On this benchmark, Qwen understand my intent better, it's better at research, and I also liked its implementation more. But: if you're more precise in what you ask, 0731 is also very good, and it's quite a lot faster on mac; raw speed is better, and it needs less thinking to get there. So, I'd say it's a tie in practice, both are awesome :)
- algo_trader 1mo ago> So, I'd say it's a tie in practice, both are awesome :) Which harness for the benchmark ? You have previously commented on using OC/GLM. R u going to stock with it?
- 1mo ago
- K0IN 1mo agoI used Qwen 3.6 27B extensively (>1B tokens) and DeepSeek V4 Flash (the older one also 2B+ tokens). And I just can't fathom that the new 3.8 beats the new DeepSeek V4 Flash (which, in my eyes, is one of the best everyday coding models). What an insane release, and convenient size to use every day/locally. but i will test this model extensivly.
- JacobAsmuth 1mo agoIt has double the active params.
- drob518 1mo agoI’ve been using v4 Flash 0731 a lot lately and you can’t beat the price performance. That said, it sometimes takes my prompts as more of a suggestion than a directive. I’ve found that introducing a reviewer subagent (even with the same model) helps push it back to what I’ve asked for. But makes every coding session a back and forth: “do X” -> “use a reviewer subagent to analyze whether you really did X as I asked”.
- Saris 1mo agoWhat model do you normally run the subagent on? You mentioned flash as well for that, but I wonder if a more 'strict' model would do a better job at pushing the main back on track.
- armcat 1mo agoSo it's effectively on-par with GLM 5.2 and GPT 5.6 Luna?
- IronWolve 1mo agoAnyone try the 9B/2B distills yet? Wondering how they do for local tools
- marcfrommelious 1mo ago[flagged]
- x313 1mo agoI used this a lot over the weekend, and it's a really intelligent and strange model. It gets really agentic at the higher reasoning levels. It does the basics like goal tracking and tool calling well, but more than that, it gets obsessed with solving problems and will do insane/unusual things to get to the solution. It actually reminds me of GPT-5.6-Sol-max which is similarly obsessive. It doesn't surprise me at all that it outscores Opus 4.6. Opus had way better world knowledge but was more "human" with agent stuff - sort of lazy and uncreative, basically giving up once the obvious solutions failed. These newer models work more like magic, they are so creative and persistent at finding ways to get to the solution.
- culi 1mo agoI have the same reaction reading the internal "thinking" monologues of Kimi K3. When I sent a message that was basically "Nope, I'll just do XYZ instead. Thanks for your help", Kimi basically had an identity crisis. Like there was two wolves inside. One that deeply wanted to help more and go above and beyond and one that was trying to tame the other and make a graceful exit. Here's an excerpt of it > Should I verify their README changes? They didn't ask me to. "I've added some notes in the README. Thanks" — that's a closing statement, not a request. Reading the README unprompted to check their notes could be seen as helpful diligence, but they didn't ask for review. Keep it simple: acknowledge, brief close.
- Havoc 1mo agoYou see the same in GLM models. Even slight ambiguity in user instructions will send it into a tailspin on what intention was in thinking tokens then it goes let’s just make a judgement call on a direction and then proceed
- graceful6800 1mo agoObsessive is the right word. Over the weekend I had to stop it multiple times deep into a multi-hour long turn to ask what the hell it was doing. It was like a dog with a bone and would NOT let go of its current work to talk to me. I had to interrupt it three times with increasingly aggressive instructions to STOP and answer my questions before proceeding. In another session it straight up told me it was in the middle of debugging something important and to ask later. I'm running an RTX 6000 Blackwell. It regularly spent over an hour per turn thinking. Every time I looked at it, the thinking trace seemed coherent, sensible, appropriate. But it could never settle on a solution. Granted, I was trying to have it solve a hard problem that 5.6 Sol couldn't solve, but still. Either way, I'm still impressed. It genuinely feels better than Sonnet 5
- hrmon 1mo agoI want to highlight its (1-hallucation rate) at 70%. BRAVO! For me, this is its most wonderful score. GPT-5.6-Sol sits at 8%.
- ricardobeat 1mo ago*non-hallucination rate
- josephcooney 1mo agoWhy are hosting providers charging to much to host it, compared to much larger models? https://openrouter.ai/compare/qwen/qwen3.8-27b/deepseek/deepseek-v4-flash-0731 https://openrouter.ai/compare/qwen/qwen3.8-27b/deepseek/deep...
- sleepyeldrazi 1mo ago2 things, 1st: Alibaba's official endpoint pricing. they don't want to undercut too much as there is profit to be made to be close to it but not too low 2nd, and maybe more importantly: KV is not as efficient (vram usage-wise) as something like deepseek v4 flash. for 256k, fp8 kv is 9.3gb (full precision ~17.3gb). deepseek v4 flash is ~2.5b for the same size at full precision (which is fp4/8, if you are interested in it, read the paper, its pretty cool). Doing the math, hosting 27B at NVFP4 (~23gb) with 2.3M total ctx (9 agents) matches the vram usage of ds v4 flash for the same 2.3M ctx (2.3 agents). the break point is 1.5M (6 27B agents) if you use full precision 27B. To be clear, the qwen3.5 architecture (what 3.8 uses) is still considered decent in terms of KV efficiency, its just that dsv4f's architecture is SOTA in that space, and with the lower active params, you get better max kv scaling and higher speed serving that, if you have a lot of gpus.
- josephcooney 1mo agoThanks for the detailed response. I saw some details later in the thread that I had somehow overlooked that touched on this....but anti-procrastination settings prevented me from changing my question.
- Balinares 1mo agoAnd once again, Qwen 3.8 27B beats Opus 4.6, what the hell. It's both funny and a bit terrifying and I still can't quite believe it. It runs decently on a gaming PC! Opus 4.6 came out only 6 months ago and was then broadly considered the new SOTA by a comfortable margin! How in hell did they package capability in the ballpark of a Feb 2026 frontier SOTA into 27B?! More importantly, what's the point of building monster-scale data centers on unprecedented amounts of debt when a more than good enough model runs on a GPU from a couple years ago? The coming months are going to be exciting, that's for sure...
- deleted 1mo ago[deleted]
- fluoridation 1mo agoI'm reminded of that paradox from sci-fi that says that starting an interstellar journey as soon as the technology is capable of it is uneconomical, because the trip will take so long that newer technology will arrive at the destination first, despite departing at a later date.
- hn92726819 1mo agoSpoiler alert if you haven't read it. This happens in The Three Body Problem as well. A second ship is sent after the first and surpasses it. It's a great series.
- boc 1mo ago> More importantly, what's the point of building monster-scale data centers on unprecedented amounts of debt when a more than good enough model runs on a GPU from a couple years ago? Probably because the future "monster" models will be insane. 100T+ param models might be the type of things that can independently run a small business, which means anyone not using them is at a distinct disadvantage to their competitors. The top model from 2025 looks silly compared to the top model of the first half of 2026. Do you feel like progress has stalled?
- 1mo ago
- ComplexSystems 1mo agoChina cleaned house these past few months. Kudos to them. I would really like to see some open source US companies out there.
- chr15m 1mo agoMeta's Glimmer is cracked for local inference!
- jakswa 1mo agoI've been waiting on this model to show up on the Deep SWE benchmark results and treat its absence/delay as an indication of how slow and unusable it is for good results. I bet it thinks to the moon on some of those complex challenges.
- RachelF 1mo agoI have a bad feeling about this. US companies have spent hundreds of billions on their models, and they are not much better than the cheaper open Chinese models. Perhaps they can out-compete them. If not there will be increasing calls to limit access to open models on the grounds of "safety". Basically, if you can't beat 'em, ban 'em.
- chr15m 1mo agoI have a good feeling about this. Unbelievably cheap and remarkable technology is coming. History repeats.
- _ballzdeep_ 1mo agoHow are they going to ban open weight models? Literally a TPB clone for open weight models would be up the next day.
- chr15m 1mo agoThe leveraged US labs are cooked. Debt's coming home. There will be bailouts.
- ranguna 1mo agoWhy? Isn't it all vendor financing with private capital? The loss is private, not public.
- chr15m 1mo agoTrue, but the richest government and military in the world depends on them. Too load-bearing to fail.
- ranguna 1mo ago> richest government and military in the world depends on them How so? stlouisfed said AI companies accounted for around 4% growth of US' GDP in 2025. Not 2008 catastrophical, specially because there's little public debt involved. The actual productivity boost from using AI is there, but its diluted in slop + it's not like AI will disappear if things go south. Chinese models and discounted GPUs will be great for local setups and startups.
- Marvin_RunAI 1mo agoThe number I'd actually want next to every score: run-to-run variance.
- scotty79 1mo agoA model that I can run on a single consumer GPU is as intelligent as gpt-5.6-luna? That's insane.
- cromka 1mo agoWhat's the best price/performance hardware to run it locally today?
- akmarinov 1mo agoA couple of 3090s at about 800-900 euros each, as much RAM as you can afford is king right now.
- linzhangrun 1mo agoUnder the same computing power, the improvement in LLM intelligence and the improvement in the upper limit of LLM intelligence are equally astonishing. At least in coding, the best local models that can run smoothly on a DGX Spark are now less than one year behind the strongest SOTA models in capability (I measured around 60 tok/s).
- JKCalhoun 1mo agoMaybe this is the wrong place to ask… I have been pulling down local models and just playing with them from time to time. Even though I have hoarder tendencies, is there any reason to keep older Qwen3.5 models around taking up hard drive space? (There's so much voodoo to understand—which ones are MoE, which are better at agentic tasks, etc. Versioning isn't as straightforward as it used to be.)
- badsectoracula 1mo agoSupposedly Qwen 3.8 27B has less "world knowledge" (i.e. anything outside coding) than Qwen 3.6 27B or earlier so depending on what you do you may want to keep a copy.
- mortsnort 1mo agoIf Starbucks and Tim Hortons bought the world's coffee supply so you couldn't make cheap coffee at home and had to buy from their stores, people would be outraged. This is what is happening to the memory supply. The only barrier to running AI, powerful enough for most tasks, at home is the memory supply has been bought up by a handful of AI companies.