k2so

5 ms·

1.
by k2so 1mo ago
Location: Göttingen, Germany Remote: Yes Willing to relocate: Yes, within in Germany Technologies: Python, FastAI, vLLM, PTQ, Inference, Model Training, Transformers, Elastic Search, Vespa, Embeddings, bm25, RLHF, GRPO, trt, prime-rl Résumé
2.
Does a context gate for search agents work?(bluenotebook.io)
2 points by k2so 2mo ago|0 comments
3.
by k2so 4mo ago
Location: Göttingen, Germany Remote: Yes (preferred; hybrid also fine) Willing to relocate: No (remote, or hybrid within Germany) Technologies: Python, Go, search & retrieval, ranking, RAG, agentic LLM systems (MCP, tool use), LLM post-
4.
by k2so 6mo ago
Location: Göttingen, Germany (work authorization available) Remote: Yes (preferred) Willing to relocate: Yes (anywhere in Germany, open to EU) Technologies: Python, Go, PyTorch, vLLM, TensorRT, Kubernetes, ElasticSearch, Vespa, Apache Beam,
5.
by k2so 7mo ago
Yeah, for a single GPU inference, considering the higher VRAM and FP4 support on the RTX 6000, it should fit larger models as well than the H100.
6.
by k2so 7mo ago
Author here. I wanted to train Nanochat d26 to GPT-2 level and had to pick between three H100 variants on Runpod. SXM was the most expensive per hour but cheapest to finish: SXM: 702ms/step - ~$37 (using vast.ai) PCIe: 1,412ms
7.
Which H100 Instance to Train Nanochat – Benchmarking PCIe, SXM, and NVL(bluenotebook.io)
2 points by k2so 7mo ago|3 comments
8.
by k2so 7mo ago
Scroll to visualise steps is such a great idea! Great writeup.
9.
by k2so 8mo ago
Interesting work, in the examples I can see that quite a few of them have the teracotta/warm-cream colour palette, was that an explicit choice to keep them in the prompts? From the official frontend-design skill, on multiple occasions,
10.
Show HN: KARMA – An evaluation framework for Medical AI systems(karma.eka.care)
2 points by k2so 1y ago|0 comments
11.
Why MCP Should Exist(bluenotebook.io)
2 points by k2so 1y ago|1 comments
12.
by k2so 1y ago
I make an argument for Model context protocol, and how it can shape the entire ecosystem
13.
by k2so 2y ago
Beyond the developer, the user massively benefits from MCP. Like you said, using any other SDK to build is a very valid approach but then you are tied down to a single client that you have built on that SDK. If you would like to switch clie
14.
by k2so 2y ago
A neat trick in Vespa (vectors DB among other things) documentation is to use hex representation of vectors after converting them to binary. This trick can be used to reduce your payload sizes. In Vespa, they support this format which is pa
15.
Making LLM workflows human friendly(bluenotebook.io)
2 points by k2so 2y ago|1 comments
16.
by k2so 2y ago
LLMs = Latency? That's how most of us perceive it. When examining the timing breakdown of a request on Claude, you'll notice that the majority of the time is spent in Content Download—essentially, decoding output tokens. In the bl
17.
by k2so 2y ago
In one of my earlier jobs a few years back, we were training deep learning models on VMs with GPUs, back then the tooling was not as extensive (vs-code did not have the remote ssh then) as it is is now. So, we would use SSH into the VM and
18.
by k2so 2y ago
Easier to use libraries over highly complicated (supposedly performant) have a significant advantage in driving more adoption. Recently I was trying to generate text embeddings from a huggingface model. Nvidia triton and text-embedding-infe
19.
by k2so 2y ago
This is awesome, are you contributing this to candle or is it a standalone package?
20.
by k2so 2y ago
I very strongly relate to this, it's been close to 3 months, since I have started working on a blog built on Quarto, and all I have so far is a elaborate design and a half complete blog on an LLM tool I had built. But like the comments
21.
by k2so 2y ago
This was my first thought too, after reading through their blog. This feels like a no-frills website made by an engineer, who makes things that just work. The documentation is great, I really appreciate them putting the roadmap front and ce
22.
ParaLLM: 1300 Tok/S on a MacBook(willcb.com)
1 points by k2so 2y ago|0 comments
23.
by k2so 3y ago
Distill for R Markdown is a great alternative too. I really like the layout and the flexibility you get with the plots. The site can be hosted directly on github pages and you can configure the custom domain on github. Fairly easy to setup
24.
by k2so 3y ago
Tomorrow and Tomorrow and Tomorrow - This was best read of 2023 for me. The narration is very immersive and the author makes you very invested in the story. Shows the messy, complicated human side of things. And another plus was that it was
25.
by k2so 3y ago
Hi, great project! I was wondering if you are planning on extending support for dynamodb