ekojs

5 ms·

1.
by ekojs 13d ago
Well, seems like ECI [0] and the AA index is diverging quite a bit. Benchmarking LLM is tough and I think we are seeing the limitations of current benchmarks and applicability to real tasks. [0]: https://x.com/EpochAIResearc
2.
by ekojs 2mo ago
I copied that from https://platform.kimi.ai/docs/guide/kimi-k3-quickstart but it seems they updated the page to remove the benchmark score now.
3.
by ekojs 2mo ago
Yeah, bad wording it seems. Though a charitable interpretation is that Fable 5 and GPT 5.6 Sol are joint 1st place in the measurement.
4.
by ekojs 2mo ago
> In our evaluations, Kimi K3 delivers frontier-level performance. Among the models tested, its overall intelligence ranks second only to Claude Fable 5 and GPT-5.6 Sol. For the complete benchmark results, see our tech blog. The full mod
5.
by ekojs 4mo ago
Seems like the only good thing about 3.5 Flash is its speed. Not cost-competitive or benchmark-leading by any means.
6.
by ekojs 4mo ago
Not disagreeing with your argument, but: > If you want a good dense model, use qwen3.6 27B instead, speed will be up, and if you don't take my word for it being smarter, take openrouter's prices of it against the bigger, slower
7.
by ekojs 4mo ago
> HTTP is just not a good transport for streaming LLM tokens and for building async agentic applications I don't know if I agree if this is a problem with SSE or HTTP. Something like a Redis Streams-backed SSE would solve most of th
8.
by ekojs 5mo ago
> You cannot run these models at 8-bit on a 32GB card because you need space for context You probably can actually. Not saying that it would be ideal but it can fit entirely in VRAM (if you make sure to quantize the attention layers). KV
9.
by ekojs 5mo ago
Yeah, figure the 'nearly lossless' claim is the most controversial thing. But in my defense, ~97% recovery in benchmarks is what I consider 'nearly lossless'. When quantized with calibration data for a specialized domain
10.
by ekojs 5mo ago
Not at all, I actually run ~30B dense models for production and have tested out 5090/3090 for that. There are gotchas of course, but the speed/quality claims should be roughly there.
11.
by ekojs 5mo ago
As this is a dense model and it's pretty sizable, 4-bit quantization can be nearly lossless. With that, you can run this on a 3090/4090/5090. You can probably even go FP8 with 5090 (though there will be tradeoffs). Probably ~
12.
Show HN: Linux Nvidia GPU V/F Curve Editor for Undervolting/OC(github.com)
4 points by ekojs 6mo ago|1 comments
13.
Gemini API Down(twitter.com)
3 points by ekojs 1y ago|0 comments
14.
Gemini API Billing Bug Causing Erroneous Charge for 'Image Generation'(discuss.ai.google.dev)
4 points by ekojs 1y ago|1 comments
15.
by ekojs 1y ago
Seems pretty widespread. We got mistakenly charged for ~$800 over the weekend. Other Sources: [0]: https://aistudio.google.com/status [1]: https://www.reddit.com/r/GeminiAI/comments/1mycmtk&#x
16.
by ekojs 1y ago
> Btw as an aside, we didn’t announce on Friday because we respected the IMO Board's original request that all AI labs share their results only after the official results had been verified by independent experts & the students h
17.
by ekojs 1y ago
Maybe not a popular sentiment here on HN but I cancelled my Kagi subscription (9+ months) just recently. Increasingly, most of my queries/search have been through LLMs and Google search is just fine (and even better for restaurants, pl
18.
by ekojs 1y ago
https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1S... > Multiple GCP products are experiencing impact due to Identity and Access Management Service Issue IAM issue huh. The post-mortem should be interestin
19.
by ekojs 1y ago
Super duper frustrating having the status page being green. Why can't Google do this properly?
20.
by ekojs 1y ago
I share the sentiment. I think we will only be using Next.js for static sites/prebuilt SPA in the future.
21.
by ekojs 1y ago
I think it's most illustrative to see the sample battles (H2H) that LMArena released [1]. The outputs of Meta's model is too verbose and too 'yappy' IMO. And looking at the verdicts, it's no wonder by people are dis
22.
by ekojs 1y ago
> This will mark the first experimental model with higher rate limits + billing. Excited for this to land and for folks to really put the model through the paces! From https://x.com/OfficialLoganK/status/1904583
23.
by ekojs 1y ago
> The bottleneck then becomes how to self-host the finetuned model in a way that's cost-effective and scalable It's not actually that expensive and hard. For narrow usecases, you can produce 4-bit quantized fine-tunes that perf
24.
by ekojs 2y ago
Normally, yes. But there's a couple rendering modes with these frameworks. In this case, the rendering is most likely 'hybrid'. Some routes are statically pre-rendered, some are served via SSR. You'd need a JS server for
25.
by ekojs 2y ago
Interesting. My hunch is that Next.js is not optimized for the dockerized Node server deployment. I would say that you could get much greater prerendering performance from Next.js by just fronting the assets directly using Caddy/Nginx.
26.
by ekojs 2y ago
> Because of this, we’re evaluating whether to continue serving it in the API long-term as we balance supporting current capabilities with building future models. Seems like it's not going to be deployed for long. $75.00 / 1M t
27.
Multilingual MMLU Dataset from OpenAI (OpenAI/Mmmlu)(huggingface.co)
2 points by ekojs 2y ago|0 comments
28.
by ekojs 2y ago
I do actually wonder if it's more efficient to use something like MessagePack instead of using JSON. It's a lot of strings so it may not matter too much I guess.
29.
by ekojs 2y ago
EventSource is really really limited. However, you can instead use Fetch via something like https://github.com/Azure/fetch-event-source to consume SSEs.
30.
by ekojs 2y ago
Actually really impressive. They went up from 7T tokens to 18T tokens. Curious to see how they perform after finetuning.

More ›