When the weights are closed I don't believe any benchmark. I just got Qwen3.8-27B to score extra 10% on SWE Pro by adding a proxy in front of it that has few s

When the weights are closed I don't believe any benchmark. I just got Qwen3.8-27B to score extra 10% on SWE Pro by adding a proxy in front of it that has few simple "harness like features": - when the model gets stuck it tells it to "go on" - when it sends no output, malformed json, slips to wrong tool use format, etc it asks it to "try again better" - detects repetition and tells the model. - injects a prompt about "planning tool use" when it seems to be using same tools repeatedly. - injects a reminder it can use tools if there are no tool uses for over X messages. 10% - with just that. I have more to test. My point is, open weights models get tested on naked model quality. "Frontier" models get tested as a model + whatever secret sauce they choose to put in front.