4 ms·

> So you mean AI will be useful for any general job without lots of training for those jobs? How about new tasks? I'm pretty sure we will solve this issue. Eit

by Yopolo 1mo ago

> So you mean AI will be useful for any general job without lots of training for those jobs? How about new tasks? I'm pretty sure we will solve this issue. Either already through World Models or another architecture. It could also be, that we just need a 10 or 100 Trillion Parameter model to match so many generic ways of solving tasks and keeping the concept in the LLMs 'head' to solve it that it will just emerge with parameter size. Like with fable they said that it can chain together exploits which wouldn't work as standalone exploits. What if the only real barrier is the depth of understanding of concepts and this is exactly what is getting solved with parameter count? But look how young this field really is if you start counting it when it became relevant on mass. Its not 'just' an LLM which is changing the world, its machine learning overall. Robotics wouldn't be were it is today if its not for machine learning. Took humans time and energy to take the leap, to start learning what the status quo is and then actually doing more with it. > the paradigm as we know it is a ceiling unless you massively and continuously feed this system with average tasks While I do think its doable to achieve AGI in 5-15 years, even if it doesn't happen and it always means that people train an LLM or whatever, if you need 10 experts to teach this to an LLM OR every single senior has to teach this to their juniors every single time, the LLM will always win. I'm now team lead for 10 years and every single year I teach them the same thing over and over and over again. the craziest thing about this? If i wouldn't tell them what they are doing wrong, they wouldn't even know it. Quality is already a very flexible term for a lot of people. > I can agree that for lots of tasks it does not. But for others it is just not a tool good enough. Yet. > If you want to ship a sub-par thing that will go to the rubbish in a couple of months, then yes, you can do that. But that will fail commercially any way. Now we come to the reality: I have seen so much garbage software its crazy. People using md5 as a password hash in 2024! No clue what coding best practices are, teams without code review, teams without a security expert not even knowing what crazy things they do day in day out. Just a few month ago a team build an API for my team including a Swagger UI. Half of it didn't work. You pressed a button on the Swagger UI and a 500 returned. And do'nt underestimate what it means that a lot of business people don't like software people. You know that fruit basket we get? and water and stuff? they don't do it because they like us they do it because thats what you have to do. if a Product Owner starts vibe coding with AI, he will have leadership convinved in no time, then it goes on production and it will run for waaaaaay longer than anyone would have guest. Besides that there is plenty of software were complexity is less relevant or security is not that big of an issue.

germandiago 1mo ago
> I'm pretty sure we will solve this issue. Either already through World Models or another architecture. Please elaborate. How? With which technique? Currently the only path forward is to feed more data and tweak for specific situations (fitting, basically). How does that help in the general case or in new situations with current tecchnology (LLMs, concretely). Noatter how far you get, this is not a general or reliable solution. It van only simulate more generality or more reliability by training and tweaking. Nothing else. At least, with this paradigm. This does not mean they will not be useful. What I challenge here is the AGI or singularity. We are far from that. > I'm now team lead for 10 years and every single year I teach them the same thing over and over and over again. I have been a lead and an architect also for years at different position. I think you miss how much tacit knowledge and judgement there is inside the brains of each of us that an LLM is not capable of. And if it is, then you have to dumo so much context that it is better to go do it yourself. There is a cost to that also actually. It is not just so "dry and technical" the knowledge. Maybe yes to learn Java patterns or C++ constructors or the like. But not for "given this situation with all these specifics", which solution would you bet on? Probably the LLM will give you a shitty REST API that is not what u need at all.So u tell the AI. It gives u something else generati g 30-50% of "decorated code". Now it seems to workso you use it. Now you do this every day. Come back in 2 months. You generated a lot of fat. Now you have a bug. You do not know even where to start. Thisis theprice to payfor speed, as usual: technical debt. Now you tell me you put three agents to talk and burn 2000 usd in tokens. Great! Is the final solution better than what you would have achieved? Not sure at all. TBH I am not into agents bc I do not trust a tool sniffing all my code and for copyright concerns. But I saw some and use a prompt with limited access and the best I can take out for my speed + control when coding is tech discussions to decide on it, error catching, test generation, one-off scripts... But never "make an app like this or that". If I ever do that (I did it a couple of times) is for scaffolding and later throw away 70%. Namely, to see something that runs on screen quickly. But later you need to spend time yourself as usual. Not a bad thing, just that this is not what you deliver and need the work done. Iterations etc.
- Yopolo 1mo ago
  > Please elaborate. How? With which technique? Reinforcement learning can just solve things even if they are new. It doesn't understand how a tool works? Give it a vm with the tool, a thousand agents and let it discover it automatically. Use the thumbs up/down emoji + chat analysis when a customer is unhappy, feed that to a RL Loop. The AI Researchers though work on World Models, grounding the AI and letting it simulate. It can do the simulation in parallel (unlimited) and choose what is best. > I think you miss how much tacit knowledge and judgement there is inside the brains of each of us that an LLM is not capable of. But thats my problem. Soooo many do not have this even as senior developers. > Now you have a bug. You do not know even where to start. Thisis theprice to payfor speed, as usual: technical debt. Yeah now i just ask the LLM to describe to me the bug. Works very well. > Now you tell me you put three agents to talk and burn 2000 usd in tokens. Great! Is the final solution better than what you would have achieved? Not sure at all. This is the thing. It only needs to make the team 10-30% better to compensate token budget with one work collegue. We have reached this level in my opinion already. Choosing a head count vs. choosing tokens. But it becomes cheaper and easier and better. So you will not just be able to do ith with 3 agents but with 20, 50 or 100. It will be better if your team is an avg team. It will be worse if you have a high profile team, for now. But man our industry has such a weird broad quality spectrum. > TBH I am not into agents bc I do not trust a tool sniffing all my code and for copyright concerns In worst case, my team always do code reviews, I do a code review on an ai instead of a human and adjust the harness or the infos the ai can access. I can actually work on making this workflow better and then i can clone it or spin it up for every single PR. For a human? I have to train them and they might leave. But there are plenty of cases were code doesn't matter. Researchers write a lot of random shitty uggly code as long as it does what it does, it doesn't matter. I have scripts for small tasks, we have microservices which do one thing because it is a tech stack we only need for one use case.

by Yopolo 1mo ago

germandiago 1mo ago

> I'm pretty sure we will solve this issue. Either already through World Models or another architecture. Please elaborate. How? With which technique? Currently the only path forward is to feed more data and tweak for specific situations (fitting, basically). How does that help in the general case or in new situations with current tecchnology (LLMs, concretely). Noatter how far you get, this is not a general or reliable solution. It van only simulate more generality or more reliability by training and tweaking. Nothing else. At least, with this paradigm. This does not mean they will not be useful. What I challenge here is the AGI or singularity. We are far from that. > I'm now team lead for 10 years and every single year I teach them the same thing over and over and over again. I have been a lead and an architect also for years at different position. I think you miss how much tacit knowledge and judgement there is inside the brains of each of us that an LLM is not capable of. And if it is, then you have to dumo so much context that it is better to go do it yourself. There is a cost to that also actually. It is not just so "dry and technical" the knowledge. Maybe yes to learn Java patterns or C++ constructors or the like. But not for "given this situation with all these specifics", which solution would you bet on? Probably the LLM will give you a shitty REST API that is not what u need at all.So u tell the AI. It gives u something else generati g 30-50% of "decorated code". Now it seems to workso you use it. Now you do this every day. Come back in 2 months. You generated a lot of fat. Now you have a bug. You do not know even where to start. Thisis theprice to payfor speed, as usual: technical debt. Now you tell me you put three agents to talk and burn 2000 usd in tokens. Great! Is the final solution better than what you would have achieved? Not sure at all. TBH I am not into agents bc I do not trust a tool sniffing all my code and for copyright concerns. But I saw some and use a prompt with limited access and the best I can take out for my speed + control when coding is tech discussions to decide on it, error catching, test generation, one-off scripts... But never "make an app like this or that". If I ever do that (I did it a couple of times) is for scaffolding and later throw away 70%. Namely, to see something that runs on screen quickly. But later you need to spend time yourself as usual. Not a bad thing, just that this is not what you deliver and need the work done. Iterations etc.

Yopolo 1mo ago
> Please elaborate. How? With which technique? Reinforcement learning can just solve things even if they are new. It doesn't understand how a tool works? Give it a vm with the tool, a thousand agents and let it discover it automatically. Use the thumbs up/down emoji + chat analysis when a customer is unhappy, feed that to a RL Loop. The AI Researchers though work on World Models, grounding the AI and letting it simulate. It can do the simulation in parallel (unlimited) and choose what is best. > I think you miss how much tacit knowledge and judgement there is inside the brains of each of us that an LLM is not capable of. But thats my problem. Soooo many do not have this even as senior developers. > Now you have a bug. You do not know even where to start. Thisis theprice to payfor speed, as usual: technical debt. Yeah now i just ask the LLM to describe to me the bug. Works very well. > Now you tell me you put three agents to talk and burn 2000 usd in tokens. Great! Is the final solution better than what you would have achieved? Not sure at all. This is the thing. It only needs to make the team 10-30% better to compensate token budget with one work collegue. We have reached this level in my opinion already. Choosing a head count vs. choosing tokens. But it becomes cheaper and easier and better. So you will not just be able to do ith with 3 agents but with 20, 50 or 100. It will be better if your team is an avg team. It will be worse if you have a high profile team, for now. But man our industry has such a weird broad quality spectrum. > TBH I am not into agents bc I do not trust a tool sniffing all my code and for copyright concerns In worst case, my team always do code reviews, I do a code review on an ai instead of a human and adjust the harness or the infos the ai can access. I can actually work on making this workflow better and then i can clone it or spin it up for every single PR. For a human? I have to train them and they might leave. But there are plenty of cases were code doesn't matter. Researchers write a lot of random shitty uggly code as long as it does what it does, it doesn't matter. I have scripts for small tasks, we have microservices which do one thing because it is a tech stack we only need for one use case.