ACCount36

9 ms·

1.
by ACCount36 1y ago
LLMs have default personalities - shaped by RLHF and other post-training methods. There is a lot of variance to it, but variance from one LLM to another is much higher than that within the same LLM. If you want an LLM to retain the same def
2.
by ACCount36 1y ago
[flagged]
3.
by ACCount36 1y ago
That's a lie people repeat because they want it to be true. People evaluate dataset quality over time. There's no evidence that datasets from 2022 onwards perform any worse than ones from before 2022. There is some weak evidence o
4.
by ACCount36 1y ago
Plateauing? OpenAI's o1 is revolutionary, less than a year old, and already obsolete. Are you disappointed that there's no sudden breakthrough that yielded an AI that casually beats any human at any task? That human thinking wasn&
5.
by ACCount36 1y ago
We are nowhere near the best learning sample efficiency possible. Unlocking better sample efficiency is algorithmically hard and computationally expensive (with known methods) - but if new high quality data becomes more expensive and comput
6.
by ACCount36 1y ago
It's popular because it's true. By now, the main reason people expect AI progress to halt is cope. People say "AI progress is going to stop, any minute now, just you wait" because the alternative makes them very, very un
7.
by ACCount36 1y ago
That's exactly how it works. Every input of AI performance improves over time, and so do the outcomes. Can you damage existing capabilities by overly specializing an AI in something? Yes. Would you expect that damage to stick around fo
8.
by ACCount36 1y ago
That's about right. And this kind of performance wouldn't be concerning - if only AI performance didn't go up over time. Today's AI systems are the worst they'll ever be. If AI is already capable of doing something,
9.
by ACCount36 1y ago
What makes you look at existing AI systems and then say "oh, this totally isn't capable of describing a problem or figuring out what's actually wrong"? Let alone "this wouldn't EVER be capable of that"?
10.
by ACCount36 1y ago
What? LLMs do benefit from economies of scale. There are a lot of things like MoE sharding or speculative decoding that only begin to make sense to set up and use when you're dealing with a large inference workload targeting a specific
11.
by ACCount36 1y ago
> So for the most part access to AI is way cheaper than it will be in the next 5-10 years. That's a lie people repeat because they want it to be true. AI inference is currently profitable. AI R&D is the money pit. Companies have
12.
by ACCount36 1y ago
If you think that the prospect of "job loss" would, or should, stop progress, you're delusional. There are reasons to slow AI progress down, but "think of all the jobs" certainly isn't one.
13.
by ACCount36 1y ago
If you don't design your compressor to output data that can be compressed further, it's going to trash compressibility. And if you find a way to compress text that isn't insanely computationally expensive, and still makes the
14.
by ACCount36 1y ago
The baseline of "energy consumption pathways in the human body" now is to be severely messed up. Humans did not evolve for an environment where food is overly abundant and physical activity is optional. For almost the entire evolu
15.
by ACCount36 1y ago
Major AI companies are not doing nearly enough to address the sycophancy problem. I get that it's not an easy problem to solve, but how is Anthropic supposed to solve the actual alignment problem if they can't even stop their prod
16.
by ACCount36 1y ago
I'd trust for-profit pharmaceutical companies before I would trust "all chemicals are evil and bad" Facebook moms.
17.
by ACCount36 1y ago
"Dying of old age" is often an agonizing death from multiple organ failure.
18.
by ACCount36 1y ago
Cloudflare is growing more and more vile with each passing year. Half the tools they're building now should never have existed in the first place.
19.
by ACCount36 1y ago
Because using an RTOS for anything complex sucks, and Linux is nice and easy to work with. Same reason why routers run Linux.
20.
by ACCount36 1y ago
Most of their code is MIT, but there's a proprietary streamer engine at the heart of it.
21.
by ACCount36 1y ago
Fun fact: none of the cheap IP camera SoC vendors implement v4l2. They all have their own off-spec kernel drivers, compatible with absolutely nothing. You even have to rewrite camera sensor drivers from scratch for every vendor's middl
22.
by ACCount36 1y ago
No implementation details, no samples from an actual reward model in action, no github repo. Looks like a sales page more than anything. Eww.
23.
by ACCount36 1y ago
Social stigma? Only if you listen to mentally ill Twitter users. It's more that the novelty just wore off. Mainstream image generation in online services is "good enough" for most casual users - and power users are few, and a
24.
by ACCount36 1y ago
Nah, we settled on "all of the above". The modern approach is "fuck over everyone at least twice".
25.
by ACCount36 1y ago
Kessler syndrome is incredibly overrated. It's completely incapable of "permanently blocking access to space". What it's capable of is "shit up specific orbit groups so that you can't loiter in them for years u
26.
by ACCount36 1y ago
It's much, much, much easier. I've been coding for decades already, but if I need to put something together in an unfamiliar language? I can just ask AI about any stupid noob mistake I make. It knows every single stupid noob mista
27.
by ACCount36 1y ago
What the fuck does that have to do with anything at all? The discussion isn't about random movie leaks. It's about creating systems that allow for internet censorship.
28.
by ACCount36 1y ago
Google teams seem to be in love with that Matryoshka tech. I wonder how far that scales.
29.
by ACCount36 1y ago
This is a censorship program. Every time a system that allows for internet content to be blocked is created, it's extended, misused and abused shortly thereafter. "The tools already exist, why don't we use them to fight terro
30.
by ACCount36 1y ago
Of course. But everything adjacent is also deeply flawed, and inevitably leads to discrimination and dehumanisation. Ban non-residental IPs? You blocked all the guys in oppressive countries who route through VPNs to bypass government censor

More ›