bashfulpup

5 ms·

1.
by bashfulpup 1y ago
Other guy said it right. These work and are fine but you lose the legacy stuff. If you know your limits and where the eventual system will end up it's great and probably better. If you are building a expandable long term system and you
2.
by bashfulpup 1y ago
Eh, can you really though? Let's be real if OAI is losing money on a 200$ subscription with hyper advanced effeciency methods are you really going to save money? You should also enjoy the free VC money while it lasts. Just like cheap u
3.
by bashfulpup 1y ago
This space is honestly a mess. I did an in depth survey around 1.5 yrs ago and my eventual conclusion was just to build with airflow. You either get simplicity with the caveate that your systems need to perfectly align. Or you get complexit
4.
by bashfulpup 1y ago
Very typical SV argument that R&D is "complex" and everything else is "simple". Would it blow your mind if I told you 10yrs ago that we'd have AI that can do math/code better than 99% of humans but ordering
5.
by bashfulpup 1y ago
Anything we humans deem private in nature from other humans.
6.
by bashfulpup 1y ago
The entire reason bots are so agressive is because they are cheap to run. If a GPU was required per scrape then >90% simply couldn't afford it at scale.
7.
by bashfulpup 2y ago
We degrade, and I think we are far more valuable than one model.
8.
by bashfulpup 2y ago
Also possible and a fair point. My point is that it's a "tiny" solution that we can scale. I could revise that by saying a kid with a whiteboard. It's an einstein×10 moment so who know when that'll happen.
9.
by bashfulpup 2y ago
He's right but at the same time wrong. Current AI methods are essentially scaled up methods that we learned decades ago. These long horizon (agi) problems have been there since the very beginning. We have never had a solution to them.
10.
by bashfulpup 2y ago
Long explanation. Simple terms, you can't use a fixed box to solve an unbounded problem space. If your problem fits within the box it works, if it doesn't, you need CL. I tried to solve this via expanding the embedding/retrie
11.
by bashfulpup 2y ago
In context learning, learning via training. Both are things we barely understand the mechanism of. RAG is a basically a perfect example to understand the limits of in context learning and AI in general. It's faults are easier to unders
12.
by bashfulpup 2y ago
Continual Learning, it's a barrier that's been there from the very start and we've never had a solution to it. There are no solutions even at the small scale. We fundamentally don't understand what it is or how to do it.
13.
by bashfulpup 2y ago
That implies learning. Solve continual learning and you have agi. Wouldn't it amaze you if you learned 10 years ago that we would have AI that could do math and code better than 99% of all humans. And at the same time they could barely
14.
by bashfulpup 2y ago
Clear your history often. My youtube is actually incredible, massive variety and useful topics. I clear it about once every 2 weeks or month depending on how many of the same topics I see. It works really well in that if you ignore the cont
15.
by bashfulpup 2y ago
Long horizon problems are a completely unsolved problem in AI. See the GAIA benchmark. While this surely will be beat soon enough, the point is that we do exponentially longer horizon tasks than that benchmark every single day. It's ve
16.
by bashfulpup 2y ago
Pythia is stupidly easy to use. Then hookup a simple test harness. - this is like a grand total of 3 commands - git pull, install, point and run a model
17.
by bashfulpup 2y ago
I love this, do tell the direction to be nudged in. I wish to experience this new level of understanding.
18.
by bashfulpup 2y ago
There is little to no research that shows modern AI can perform even the most simple long-running task without training data on that exact problem. To my knowledge, there is no current AI system that can replace a white collar worker in any
19.
by bashfulpup 2y ago
Again, that's why I said it is challenging. I regularly do fine tuning on a model with fine results and little damage to the base functionality. It is possible, but it's too complex for the majority of users. It requires a lot of
20.
by bashfulpup 2y ago
This is true if you don't know what you're doing, so it is good advice for the vast majority. Fine tuning is just training. You can completely change the model if you want make learn anything you want. But there are MANY challenge
21.
by bashfulpup 2y ago
A CL agent is next generation AI. When CL is properly implemented in an LLM agent format, most of these systems vanish.
22.
by bashfulpup 2y ago
Already a big thing. See the constellation architecture used here: https://arxiv.org/html/2403.13313v1
23.
by bashfulpup 2y ago
I looked at the website and have no idea how Arc is supposed to be AGI. Can someone explain?
24.
by bashfulpup 2y ago
The biggest issue the author does not seem aware of is how much compute is required for this. This article is the equivalent of saying that a monkey given time will write Shakespeare. Of course it's correct, but the search space is int