ActivePattern

6 ms·

31.
by ActivePattern 1y ago
It's a OpenAI researcher that's worked on some of their most successful projects, and I think the criticism in his X thread is very clear. Systems that can learn to play Atari efficiently are exploiting the fact that the solutions
32.
by ActivePattern 1y ago
I am quite confident that an LLM will never beat a top chess engine like Stockfish. An LLM is a generalist -- it contains a lot of world knowledge, and nearly all of it is completely irrelevant to chess. Stockfish is a specialist tuned spec
33.
by ActivePattern 1y ago
I don't think that bidding system really works. If one side is strongly favored in the opening, the optimal bid would be essentially 0.4999999999... so that you can pick the color and win the game by a slim margin. Players then increas
34.
by ActivePattern 1y ago
Wouldn't the extra stamina have been rewarded, assuming creatine allowed you to perform extra repetitions? All exercises were done to repetition maximum.
35.
by ActivePattern 1y ago
The study seems to have controlled for training intensity -- all exercises were done to repetition maximum.
36.
by ActivePattern 1y ago
If you read the study, you can see that they controlled for training intensity. All exercises were done to repetition maximum.
37.
by ActivePattern 1y ago
Yes, that appears to be another good solution to the castling trickiness! And probably how you assume castling works in Chess960 if you weren't given the rules.
38.
by ActivePattern 1y ago
The idea of an asymmetric Chess starting position is very interesting, although it does introduce more risk of one side starting with a big advantage (perhaps this has been analyzed). I also like that in this variant, castling works like no
39.
by ActivePattern 1y ago
Who chooses the "correct" use of words? Is it you? Wikipedia disagrees with you: https://en.wikipedia.org/wiki/Gross_margin . Maybe you should make your own encyclopedia.
40.
by ActivePattern 2y ago
The answer will likely match what the reasoning steps bring it to, but that doesn’t mean the computations by the LLM to get that answer are necessarily approximated by the outputted reasoning steps. E.g. you might have an LLM that is traine
41.
by ActivePattern 2y ago
Just want to note that this simple “mimicry” of mistakes seen in the training text can be mitigated to some degree by reinforcement learning (e.g. RLHF), such that the LLM is tuned toward giving responses that are “good” (helpful, honest, h
42.
by ActivePattern 2y ago
Doing a single modulo 9 operation is much faster than summing a d-digit number, which requires d modulo 10s, d divide 10s, and d sums.
43.
by ActivePattern 2y ago
I'm simply pointing out the limitations of LLMs as code writers. Hybrid systems like ChatGPT-o1 that augment LLMs with tools like Python interpreters certainly have the potential to improve their performance. I am in full agreement! I
44.
by ActivePattern 2y ago
Because otherwise we are talking about LLMs augmented with external tools (i.e. Python interpreters). My original comment was pointing to the limitations of LLMs in writing code by themselves.
45.
by ActivePattern 2y ago
I suspect that it would be unlikely to come up with it because it requires execution of a fairly lengthy algorithm (or sophisticated mathematical reasoning) to find the smallest/largest valid numbers in the range. You can verify this
46.
by ActivePattern 2y ago
Correct, this optimization no longer works when you change the problem.
47.
by ActivePattern 2y ago
This is actually a great example of an optimization that would be extremely difficult for an LLM to find. It requires a separate computation to find the smallest /largest numbers in the range with digits summing to 30. Hence, an LLM is
48.
by ActivePattern 2y ago
I've observed this with ChatGPT. It seems to be trained to minimize changes to code earlier in the conversation history. This is helpful in many cases since it's easier to track what it's changed. The downside is that it tend
49.
by ActivePattern 3y ago
The bot has a point...
50.
by ActivePattern 3y ago
Yeah, it does demonstrate that you can use compression to measure similarity of two images. But it also demonstrates that it's a pretty poor similarity measure. Something as simple as counting % of matches between the black and white p
51.
by ActivePattern 3y ago
The goal is to have Mojo be fully compatible with Python, so that you should eventually have access to all of the libraries offered in Python. How many libraries will be written in Mojo remains to be seen, depending on whether it catches on
52.
by ActivePattern 3y ago
Yeah, a lot of folks don't seem to be aware of the people behind this, and their history. Swift was originally closed source too. He's explained that this sort of "incubation period" is helpful for working out the kinks
53.
by ActivePattern 3y ago
I believe that Chris Lattner (Mojo lead) has said that its going to be open-source eventually. He's followed the same process as with Swift, where they did't open it up until major design decisions had been worked out with the cor
54.
by ActivePattern 3y ago
AFAIK, it's a general purpose language. I don't see any reason you wouldn't be able to use it for applications outside of ML.
55.
by ActivePattern 3y ago
I think the reason that they focus on pure Python is because a big use case would be pure Python code bases that can be incrementally ported over to Mojo for better performance, without needing to totally abandon the Python language and e
56.
by ActivePattern 3y ago
If it's training on the same data that it generates, there's no new information being added into the system. You'd be reinforcing everything that it already gets right and wrong, which would lead to zero improvement. That sai
57.
by ActivePattern 3y ago
If we are considering the function to be the neural network with an argmax applied to the output probabilities, it's not overfitting at all. Its classification accuracy over unseen data (validation set) continues to improve. The issue
58.
by ActivePattern 3y ago
I wouldn't have guessed that bamboo has a strong taste. I've used other wooden cutlery before and it doesn't really affect the taste of food at all.
59.
by ActivePattern 3y ago
It's already been reverted.
60.
by ActivePattern 3y ago
I'd be curious how much of the link you read. What they demonstrate is a neural network learning an algorithm that approximates modular addition. The exact workings of this algorithm is explained in the footnotes. The learned algorit

More ›