dongecko

8 ms·

1.
by dongecko 6mo ago
What a great read! You got me at the base64 oddity. I also stumbled over this, while trying to dodge some LLM limitation. (was trying to generate images in a time before multimodal was a thing. it only worked to a degree).
2.
by dongecko 7mo ago
Naaa, you have to think of the poster in options. Like, "would you want to have some extra publicity with your poster for an extra 100 bucks?"
3.
by dongecko 8mo ago
I never realized; thanks for pointing that out. Sneakers is one of my favorite movies.
4.
by dongecko 9mo ago
A while back, someone made https://meet.hn . It's a map where people from hackernews could locate themselves. Of course this is by far not comprehensive, but it gives you an idea of where the people are from.
5.
by dongecko 1y ago
Exactly my point of view. For the most part I do not root for my preferred technology, but rather try to inform my powers about the caveats I see. This way at least the right aspects to check have a chance to enter the debates above my payr
6.
by dongecko 2y ago
Yes, they can. Meta described a clever way in their paper on training Llama3 [1] (in section about factuality). The idea is to sample several answers to a question that you know the answer to. Let an LLM decide if the given answers are diff
7.
by dongecko 2y ago
We have a couple of systems at work that incorporate LLMs. There are a bunch of RAG chatbots for large documentation collections and a bunch of extract-info-from-email bots. I would none of these call an agent. The one thing that comes clos
8.
by dongecko 2y ago
Came here for exactly that. Thank you.
9.
by dongecko 2y ago
Bolzman machines were there in the very early days of deep learning. It was a clever hack to train deep nets layer wise and work with limited ressources. Each layer was trained similar to the encoder part of an autoencoder. This way the lay
10.
by dongecko 2y ago
This comment caught me off guard! I literally blew my coffee all over the table.
11.
by dongecko 2y ago
I second that thought. There is a pretty well cited paper from the late eighties called "Multilayer Feedforward Networks are Universal Approximators". It shows that a feedforward network with a single hidden layer containing a fin
12.
by dongecko 2y ago
Motorola used to have a one bit microprocessor, the MC14500B.
13.
by dongecko 2y ago
I simply love this book!
14.
by dongecko 3y ago
At the very beginning of my journey I did some fine tuning with Lora on a (I believe) Falcon model, but I haven't looked at it since. My impression was that injecting knowledge via fine tuning doesn't work, but tweaking behavior d
15.
by dongecko 3y ago
I have not encountered this problem yet. When I was talking about the format of the answer I meant the following: No matter if you're using Langchain, Llamaindex, something self made, or Instructor (just to get a json back); under the
16.
by dongecko 3y ago
I also think of the retrieval part as a bottleneck and I am super excited of what the future holds. In particular, I wonder if RAG systems will soon be a thing of the past, because end to end trained gigantic networks with longer attention
17.
by dongecko 3y ago
That is an interesting observation. I have not gotten to the point of too long cycles and I can think of two reasons for that. Maybe my use case is narrow enough, so that in combination with a rather constraining and strict system message a
18.
by dongecko 3y ago
The company I work for has tons of documentation and regulations for several areas. In some areas the documents are well over a thousand and for the ease of use of these documents we build RAG based chat bots. This is why I have been playin
19.
by dongecko 3y ago
I have been to a couple of hackathons as participant and as organizer. Some sessions where really good and some were really terrible. The biggest takeaway for me in organizing a hackathon is to keep in mind that technical people who partici
20.
by dongecko 3y ago
I read it the same way as you did, or at least it's an approximation. In general that's not really surprising. I remember discussions from some years ago about larger networks leading to smother loss surfaces.
21.
by dongecko 4y ago
There is also the evo64 project which rebuild the c64 with modern parts.
22.
by dongecko 4y ago
I love this thread. The question is so unspecific that it's like a version of "To get the right answer you need to post something wrong on the internet".
23.
by dongecko 4y ago
I think that the display is not used at all. If I understood correctly, the scart connector is hosting a hidden uart interface. This let's you connect a serial terminal via three cables. (You can e.g. buy a serial to usb adapter for 5
24.
by dongecko 4y ago
I use monokai and dracula, depending on the environment. If i can alter something, I usually go for dracula and FiraScript as font.
25.
by dongecko 4y ago
I really love this book. Beautiful data and beautiful visualizations are also nice reads, but beautiful code has a special place in my heart.
26.
by dongecko 4y ago
The Frank-Wolfe algorithm for optimization. Christian Bauckhage has a couple of papers, where he uses it to connect optimization problems to the fix states of a certain type of recurrent neutral networks (echo state networks). The Marchenk
27.
by dongecko 7y ago
I'd second that. Some years ago I had a look at the quake 3 bots code, which I think were also written by John Carmack. The code was amazingly intuitive.