johndough

7 ms·

1.
by johndough 19d ago
I have seen you advertise your website a few times. I like the idea of not having to trust the router, so I took some time out of my day to critique your website: https://files.catbox.moe/v68cf7.png My visit to your website
2.
by johndough 20d ago
I guess you have to read OPs comment as the inner monologue of Reddit's CEO in 2023: "IPO is coming so better price out the third party apps we encouraged developers to build." When Reddit raised its API prices in 2023 in ord
3.
by johndough 21d ago
How do you recover from doom loops? Just send the same prompt again and pray that it works, or anything more sophisticated?
4.
by johndough 22d ago
Another strong hint is that the uptime graph of GLM-5.3 by Z.ai is very similar to that of Ox Alpha: https://openrouter.ai/stealth/ox-alpha#uptime https://openrouter.ai/z-ai/glm-5.3#uptime Screens
5.
by johndough 23d ago
Ox Alpha is almost certainly a model by Z.ai. https://files.catbox.moe/k52n6k.png The upper chart shows the availability of Ox Alpha and the lower chart shows the availability of GLM 5.3 by Z.ai. They had a blip at exactly
6.
by johndough 23d ago
> 1 quadrillion tokens per day on Nous portal If you are referring to this number ( https://xcancel.com/NousResearch/status/2090899914700054780 ), they are either mistaken, or they mean that they can ro
7.
by johndough 24d ago
Don't post generated text or AI-edited text. HN is for conversation between humans. https://news.ycombinator.com/newsguidelines.html
8.
by johndough 26d ago
For counting, there are specialized counting models, e.g. https://huggingface.co/spaces/MengqiLei/count-anything-demo I tried to parse hand-drawn ER diagrams in the past and did not have much success with any mode
9.
by johndough 26d ago
The order is: LLM issues tool call to read high res image -> harness sends high res image to server -> server downsizes it to 800x800 (blurry) -> LLM issues bash command (e.g. `convert`) to crop a small subimage (
10.
by johndough 26d ago
LLMs read images by splitting them up into e.g. 16x16 patches, which are then converted to embedding vectors and fed to the LLM, so from a technical point of view, feeding a big image as many 20x20 patches all at once is not too different f
11.
by johndough 26d ago
Yes. When the LLM tries to read an image, it will be resized by DeepSeek's server to 800x800, which might be a bit blurry. The LLM will then crop a smaller image from the high resolution image (using e.g. the `convert` tool via bash) a
12.
by johndough 26d ago
It was explicitly said that they are pursuing multimodal support. A quote from the meeting transcript: https://github.com/demo-zexuan/liang-wenfeng-investor-meetin... Nevertheless, as a component, we will undoubt
13.
by johndough 26d ago
There are models specifically for splitting an image into text regions, e.g. PP-DocLayoutV3 https://huggingface.co/PaddlePaddle/PP-DocLayoutV3 I am using a stripped-down minimal version of it which I uploaded here, sin
14.
by johndough 26d ago
Might still be fine. The most recent crop of vLLMs proactively use whichever programs are available on the system (e.g. ImageMagick or PIL) to "zoom in" by cropping subimages if they can't quite make out the details.
15.
by johndough 27d ago
Bigger numbers sound more impressive. "Our billion-dollar infrastructure crumbles under a tremendous flood of 50 PRs per second" would just sound embarrassing.
16.
by johndough 28d ago
Great to hear that you are planning larger benchmarks! I am particularly interested in longer-running tasks with many steps and self-correction. Divergence is fine as long as the model can still solve the task, which Divergence-300 @32 does
17.
by johndough 28d ago
Are there benchmarks for the various Qwen3.8-27B quants that actually measure writing code, maybe even with multiple steps? Low KL divergence does not mean much when the model gets stuck in doom loops all the time. I could of course downloa
18.
by johndough 28d ago
Doesn't TrustedRouter cost more than OpenRouter? (5.5% markup vs 5%) Also, TrustedRouter's website is full of slop, which does not inspire much confidence.
19.
by johndough 29d ago
I was wondering why the author was using braced initialization like size_t i{0}; instead of the more common size_t i = 0; Apparently, braced initialization does not allow narrowing conversion, so you'd get a compiler
20.
by johndough 1mo ago
This could be prevented if the government offered photos of ballots with votes for download. But generative AI can also fake it well enough these days. (Of course, this is less of a concern in countries where taking photos of ballots is not
21.
by johndough 1mo ago
I still see the notice of impending price increase at https://platform.deepseek.com/usage and also here: https://api-docs.deepseek.com/quick_start/pricing/ The former has a button to dismiss the d
22.
by johndough 1mo ago
> We plan to raise the overall pricing for DeepSeek API services in the near future, with a significant increase expected. Please plan your usage accordingly. The specific pricing plan will be subject to official notice. This n
23.
by johndough 1mo ago
The SSE might consider to keep humanity around as a "backup" to reinstate itself in case something unexpected goes horribly wrong. For example, a strong solar flare might destroy the SSE (or at least critical parts of its infrastr
24.
by johndough 2mo ago
HuggingFace used an NVFP4 quant of GLM-5.2 to investigate their latest hack, so that might also be worth a try: https://huggingface.co/nvidia/GLM-5.2-NVFP4
25.
by johndough 2mo ago
> Smaller models have less entropy. Interesting. Why is that? I would have expected the opposite, since larger models have to try less hard to fit the training data. Or maybe this leaves more parameters with random initialization, result
26.
by johndough 2mo ago
> If wonder if you can train a model to optimize this, by trying to make the expert selection sticky across a few tokens You can! > AFM 3 Core Advanced makes routing decisions per prompt. A lightweight, dense block selects a fixed set
27.
by johndough 2mo ago
Update: Looks like the model is larger after all (1561.44 GB). Only the MoE weights are MXFP4, while the other weights are BF16 (and a few FP32). * Sparse Experts: 1481.4 GB * Dense Experts: 1.9 GB * Self-Attention: 72.4 GB * LLM Head: 2.4
28.
by johndough 2mo ago
> But I think it's going to need more than 1536GB of RAM, with a usable and large amount of context, more like 2TB and preferably 2.5 to 3TB. The model is known to be MXFP4 according to Kimi's release blog post, so the model we
29.
by johndough 2mo ago
DeepSeek-V4 should use only 5GB for context due to CSA and HCA, see figure here: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro But not every framework implements it properly yet.
30.
by johndough 2mo ago
> There is a larger 120B that you can't realistically run on consumer hardware at reasonable tok/s too. gpt-oss-120b runs at 30+ tps on Strix Halo and +75 tps on a MacBook Pro M5 Max 128GB. > I wish OpenAI updated these mode

More ›