nmitchko

7 ms·

1.
by nmitchko 1mo ago
The goal of this auxiliary module is to have the model recurse pre-emit trained on good thinking traces. This includes a 6-to-1 compression of thinking tokens. Therefore output tokens are decodeable, but are trained compressed. So they are
2.
by nmitchko 1mo ago
Fair criticism, i’ll rewrite it… as this is a pet project and post based on another HN comment.
3.
by nmitchko 1mo ago
After vLLM startup I observed this on first request only then all further conversations were fine, new and original. Can’t figure out why but seems to be fine after that.
4.
by nmitchko 1mo ago
My main motivation is to be able to tune the thinking effort without polluting token IO. Inspiration came from the CoLaR paper and other works showing that repeating laters in an LLM got better results.
5.
by nmitchko 1mo ago
I am on my own GPUs so thought it be cool to reduce token output. you can charge per latent loop as a token i guess. I wanted a knob to keep thinking going until it was sure it was done.
6.
by nmitchko 1mo ago
I can do that, main difference seems to be this ablation seems to over think simple prompts, but can do a proper eval (might take 2-3 days)
7.
by nmitchko 1mo ago
Fair critique, ill rewrite (this is just my personal pet project but thought was cool to share here)
8.
by nmitchko 1mo ago
https://news.ycombinator.com/item?id=49230550
9.
Show HN: DeepSeek-V4 Latent Reasoning – moving "thinking" into latent space(blog.n.ichol.ai)
30 points by nmitchko 1mo ago|24 comments
10.
by nmitchko 1mo ago
It’s an adaptation of CoLaR, but my implementation is a little different: - Dedicated stop head to fire when latent thinking hits threshold - MTP support with training taking draft support as first class - different architectural layer 35 -
11.
by nmitchko 1mo ago
Perhaps it might be interesting: a latent thinking version is here https://huggingface.co/nmitchko/DeepSeek-V4-Flash-0731-Laten... Does no thinking emissions for context saving.
12.
by nmitchko 1mo ago
Author of this model - I was curious how large models do adaptive thinking and it got me down the path of implementing a model with latent only thinking. The part I think is most interesting is that the reasoning never touches your context
13.
DeepSeek-V4-Flash-0731-Latent-Reasoning. A model thinking in latent space(blog.n.ichol.ai)
1 points by nmitchko 1mo ago|1 comments
14.
by nmitchko 2mo ago
Poster: The cool part is that the model literally talks itself out of giving the cached wrong answer with a latent only thinking budget.
15.
The Doctor Is Not the Mother: DS4 Latent Reasoning(blog.n.ichol.ai)
3 points by nmitchko 2mo ago|1 comments
16.
by nmitchko 4mo ago
A fantastically simple solution to improving algorithms, I wish I had this years ago in activation engineering: https://blog.n.ichol.ai/llm-activation-engineering-an-easy-f... How do I access AlphaEvolve?
17.
by nmitchko 5mo ago
I used those two in combination to fix pain after 3x surgeries to repair a torn pec + infection. They work and helped me heal from being at a 3/10 constant pain down to baseline. Not something I would do at any point for fun. But anec
18.
by nmitchko 7mo ago
Can someone make a startup that allows me to do this as an individual?
19.
by nmitchko 11mo ago
In case anyone wants to do this themselves, check out the pipeline here: https://github.com/isc-nmitchko/iris-document-search Colnomic and nvidia models are great for embedding images and MUVERA can transform those to
20.
by nmitchko 1y ago
Next steps for AI in general: - additional modalities - Faster FPS (inferences per second) - Reaction time tuning (latency vs quality tradeoff) for visual and audio inputs/outputs - built-in planning modules in the architectu
21.
by nmitchko 2y ago
No other models that are public worth comparing to... Hippocratic advertises good benchmarks but that might be marketing fluff. Have you checked out dataset building with nemotron? The nemotron synthetic data builder is quite powerful. More
22.
by nmitchko 2y ago
Interesting they don't compare to open-bio. Page 7 charts are quite weak. https://huggingface.co/aaditya/Llama3-OpenBioLLM-70B
23.
by nmitchko 2y ago
We're excited to share pitchpilot with the HN community. Our beta users have found the embedded audio particularly useful for enterprise sharing. We're keen to keep improving, and our mission is to make communication easier. In th
24.
Show HN: We built pitchpilot, an AI synthesizer to tailor presentations(pitchpilot.xyz)
1 points by nmitchko 2y ago|1 comments
25.
by nmitchko 2y ago
Given that Generative AI can now read brain scans [1] and this, I wonder how far away we are from "you thought negatively about something, the authorities are on their way". [1] -- https://www.biorxiv.org/content&#
26.
by nmitchko 2y ago
Tin-foil hat time: 1. First, models will predict pollution. The outcomes will help shape urban policy. But these won't solve crime or stop people from driving. 2. Second, models will predict individual behavior and track person level e
27.
by nmitchko 2y ago
How does this compare to ehealthexchange or other qhins that have many years of experience and charge lower costs?
28.
by nmitchko 2y ago
It truly feels like the space race in terms of building LLMs right now. Question is, who lands on the moon first?
29.
by nmitchko 3y ago
This reminds me of the matrix movie scene when they look at the encrypted thoughts of the matrix. https://cdn.swisscows.com/image?url=https%3A%2F%2Fi.pinimg.c...
30.
by nmitchko 3y ago
Great work, will try this tonight. Only question, why do you name variables with the λ symbol?

More ›