omneity

6 ms·

1.
by omneity 1mo ago
Good old Whisper allows you to enter a prompt with domain specific terms and it will use them for transcription.
2.
by omneity 1mo ago
I'd approach this distinction differently. Prediction from compression is valid within the distribution of the compressed data. Which brings it much closer to LLMs in this case (can an LLM talk about a topic it has never seen in traini
3.
by omneity 1mo ago
I’d argue the opposite. The geometry involved in reconciling photos from two separate lenses with different focal lengths is quite challenging. If you pay close attention you will see some differences in the before/after samples in the
4.
by omneity 2mo ago
I thought the actual 30 papers have never been disclosed. Do you have a source tying the recommendations back to Ilya, or did you come up with this list?
5.
by omneity 3mo ago
Glad it helped! A little credit on the post would go a long way :)
6.
by omneity 3mo ago
The Web MIDI API[0] used by the author has a built-in precise scheduler, that has higher precision and works better than the unreliable setTimeout approach used by OP when coupled with the Performance API[1]. Pass a timestamp as the second
7.
by omneity 3mo ago
I do think it might improve but only marginally. You are however likely to observe better results in smaller models since they're usually more strapped for "cognitive capacity", so two separate calls reduce the load in each r
8.
by omneity 3mo ago
It’s not as simple. I trained an LLM before on exactly this, to scratch the itch of this question. The task was simple, using the MS-MARCO[0] dataset which contains queries, search results, answers, I made a training set that has: 1. Questi
9.
by omneity 4mo ago
The Open in OpenRouter is the same as in OpenSea, as it's the same founder. Make of that what you will.
10.
by omneity 4mo ago
You can increase the context window beyond its max trained context using RoPE scaling[0] which will require more VRAM. But you can increase your context window for the same VRAM by quantizing the KV cache with FP8 (double the context) or Tu
11.
by omneity 4mo ago
Funny, I’ve been cracking[0] at this exact problem with a purpose-built model[1]: 0: https://huggingface.co/posts/omarkamali/593639295164067 1: https://omneitylabs.com/models/sawtone
12.
by omneity 5mo ago
Strong vibes from the novel Manna. https://marshallbrain.com/manna1
13.
by omneity 5mo ago
I'm pretty sure it should be possible to distill HS-TasNet into a version approximate and fast enough for the purpose of animating LEDs. At the end it's "just" chunking streamed audio into windows and predicting which LE
14.
by omneity 6mo ago
Excellent, thank you mandeepj! Curious about the language coverage of your agent and if / how you plan to eval your agent, if you're willing to share more.
15.
by omneity 6mo ago
Hey, this is super cool! I’ve been working on a similar problem, focusing on low-resource and underserved languages including the Mayan family, and have published some research and open resources around that [0, 1]. On the data side, I’ve f
16.
Tokenization Is Killing Our Multilingual LLM Dream(huggingface.co)
1 points by omneity 6mo ago|0 comments
17.
I stopped trusting the official Wikipedia dataset, and what I did about it(omarkamali.com)
4 points by omneity 6mo ago|0 comments
18.
A Wordle for the Worldle(omarkamali.com)
1 points by omneity 7mo ago|0 comments
19.
by omneity 7mo ago
Or your willingness to put up with power banks.
20.
by omneity 7mo ago
This is a great project. FYI all you need is the size of an LLM and the memory amount & bandwidth to know if it fits and the tok/s It’s a simple formula: llm_size = number of params * size_of_param So a 32B model in 4bit needs a mi
21.
Show HN: Wikilangs Games – Wordle-like for 300 Languages(games.wikilangs.org)
2 points by omneity 7mo ago|0 comments
22.
by omneity 7mo ago
It’s a trivial calculation to make (+/- 10%). Number of params == “variables” in memory VRAM footprint ~= number of params * size of a param A 4B model at 8 bits will result in 4GB vram give or take, same as params. At 4 bits ~= 2GB an
23.
by omneity 7mo ago
Attention is calculated during the forward pass of the model, which happens in both inference (forward only) and training (forward & backward).
24.
by omneity 8mo ago
The model being 32B could run in <20GB VRAM with Q4 quantization (minimal loss of quality), or 80GB unquantized at full fidelity. The quoted 160GB is for their recommended evaluation settings. There's a few pre-quantized options[0]
25.
by omneity 8mo ago
RDMA over Thunderbolt is a thing now.
26.
by omneity 8mo ago
I just went through an eerily similar situation where the coding agent was able to muster some pretty advanced math (information geometry) to solve my problem at hand. But while I was able to understand it enough to steer the conversation,
27.
by omneity 8mo ago
Very cool insights, thanks for sharing! Do you have benchmarks for the SGLang vs vLLM latency and throughput question? Not to challenge your point, but I’d like to reproduce these results and fiddle with the configs a bit, also on different
28.
by omneity 8mo ago
Except this is GLM 4.7 Flash which has 32B total params, 3B active. It should fit with a decent context window of 40k or so in 20GB of ram at 4b weights quantization and you can save even more by quantizing the activations and KV cache to 8
29.
by omneity 8mo ago
Why not? Run it with vLLM latest and enable 4bit quantization with bnb, and it will quantize the original safetensors on the fly and fit your vram.
30.
by omneity 8mo ago
Also worth noticing the size of these countries. Mostly on the small/tiny side, besides Norway (an oil exporter) and Ireland (a corporate tax haven) Perhaps making good economic decisions grows exponentially in difficulty with the popu

More ›