longdog

5 ms·

1.
by longdog 2y ago
I feel the webpage strongly hints that sparse autoencoders were invented by OpenAI for this project. Very weird that they don't cite this in their webpage and instead bury the source in their paper.
2.
by longdog 2y ago
What are those other areas which display age-based creative decline? Other creative fields I can think of off the top of my head - scientific research, animation, fiction writing, architecture - are overwhelmingly dominated by older people.
3.
by longdog 2y ago
I've been playing with Llama 3 8b instruct but I've found it to be surprisingly low quality compared to some of the better Mistral 7b finetunes (zephyr, dolphin, openorca). Rather surprising because there's no way Mistral or
4.
by longdog 2y ago
Population in Japan has barely fallen (yet). So far it's only a ~2% decline from peak population, but there will be a 20% decline in the next 20 years. There is a long lag between below-replacement fertility and actual population dec
5.
by longdog 2y ago
The author is way overselling how controversial Haidt's claim is. - The effect of social media at a societal level isn't something that can tested experimentally. And like all non-experimental research, results depend heavily on m
6.
by longdog 2y ago
Interesting, but I'm very skeptical. There are over a dozen transformers-based foundation time series model released in the past year and without fail, every one of them claims to be at or near SOTA. For example: - Time-LLM ( https:&#x
7.
by longdog 3y ago
You don't need to be a cutting edge research scientist to train a SOTA LLM. You just need money for scaling. OpenAI's "secret" was just their willingness to spend tens/hundreds of millions without guaranteed returns