6 ms·

This over reliance on llms is crazy. People are going to forget how to code. Sometimes the llm makes up shit or uses the wrong version of the API. Sometimes it'

This over reliance on llms is crazy. People are going to forget how to code. Sometimes the llm makes up shit or uses the wrong version of the API. Sometimes it's easier to look up the documentation and write some code.

xp84 1y ago
I mean, it’s just like having an army of interns that works for (near) free. It’s a huge positive for productivity, and I don’t think we will forget how. I’m more concerned with how we make new senior/staff engineers from now on, since the old “do grunt work for a couple years, then do simple well defined work for a few years” is 100% not a career path that exists even now.
- LTL_FTC 1y ago
  This is my question as well. I am already wondering how prepared college grads will be. Getting help with programming assignments meant going to the dungeon and collaborating with fellow students while the TA made their rounds and overall just figuring it out. Today, an LLM knocks out the programming assignments in once shot, probably. And industry seems hellbent on hiring seniors mostly so where are the juniors to become seniors going to come from? I think the talent pipeline has contracted and/or will and overcorrect. But maybe the industry’s carrying capacity of devs has shrunk.
  - xp84 1y ago
    The ironic part is that for about 10 years from 2012-2022 I used to tell people that I was very bullish on programming as a career, since I couldn't imagine any possible future where we needed less software to be written in year N+1 than we did in year N. I just didn't think of any world where we could have more software written with fewer engineers. Surprise!
dawnerd 1y ago
The future is going to be great for us that have been resisting going all in. Unfortunately I feel a lot of work will be detangling the mess these llms make in larger repos.
- skydhash 1y ago
  The devils is in the details, as they say. And software engineering used to be exorcism, now they want it to be summoning. Now I'm just hopping for the majority to realize that hell is not a great environment for business.
  - Bluestein 1y ago
    > software engineering used to be exorcism, now they want it to be summoning. Now I'm just hopping for the majority to realize that hell is not a great environment for business With your leave, this is going up on my wall :)
amelius 1y ago
> People are going to forget how to code. Which is a problem when exactly? When civilization collapses?
cft 1y ago
It will probably be like coding in assembly after the advent of the compilers. There are some people who still code in assembly, but it's rare.
- hooverd 1y ago
  so software will get even worse because nobody understands anything about how computers work?
  - missedthecue 1y ago
    Your average professional python programmer knows a lot less about how computers work than the assembly machine level programmers of yesteryear. Software today is both worse and better. Slack uses 2gb of RAM, but is there anyone who wants to go back? Things will probably continue in that general direction. And just like today, a small number of people who really know what they're doing keep everything humming so people can build on top of it. By importing 13 python libraries they don't completely understand, or having an AI build 75% of their CRUD app.
    - hooverd 1y ago
      I think it's a problem in that each 1% of slop here and there massive compounds overall.
  - crazylogger 1y ago
    Me and my LLM buddy together understand exactly how computers work!
- Bluestein 1y ago
  Wait until only LLMs know how to build compilers. That's going to be a riot ...
fassssst 1y ago
Just like how people forgot how to patch phone lines and punch cards.
aquariusDue 1y ago
It's the calculator all over again! Jokes aside, while I'm almost sure that the ability to code can be lost and regained just like training a muscle what I'm more worried is the rug pull and squeeze that is bound to happen sometime in the next 5 to 10 years unless LLMs go the way of Free Software GNU style. If the latter happens then LLMs for coding will be like calculators and such more or less and personally I don't know how more harmful that would be compared to the boost in productivity. That said if the former becomes reality (and I hope not!) then we're in for some huge existential crises when people realize they can barely materialize the labour part of their jobs after doing the thinky part and the meetings part.
- timschmidt 1y ago
  I don't think the rug pull and squeeze are possible. Because I've had the same worry. But using an existing LLM to train or fine tune a new one seems to be standard practice, and to work quite well. So any LLM with an API will end up training all the others - even open source LLMs - and all will benefit. And every day that passes, Moore makes it less and less costly for amateurs to commit the compute necessary for fine tuning, and eventually training from scratch. In time, even video and embodied training may be possible for amateurs, though that's difficult to contemplate today.
- alwillis 1y ago
  There are already lots of open source/open weight models than can run locally on a laptop. People into homelabs have been running AI tools on home servers for years.
  - bakugo 1y ago
    > There are already lots of open source/open weight models than can run locally on a laptop. And they're all too small and dumb to be useful for anything but the most basic tasks.
KronisLV 1y ago
> People are going to forget how to code. Pretty much me with some IDEs and their code inspections and refactoring capabilities and run profile configurations (especially in your average enterprise Java codebase). Oh well.
alwillis 1y ago
Just a reminder you can tell the LLM the version of an API to use. Your code should have tests the AI can use to test the code it wrote. And thanks to MCP, you can literally point your LLM to the documentation of your preferred tool [1]. [1]: https://context7.com/about https://context7.com/about
grogenaut 1y ago
I think it's more like every engineer will either become like a lead or a principal or have problems. I'm a principal. I have for years had multiple teams building things that I prototyped designed or worked with them on the specs for. There's a level of touch and letting go that you have to employ to not over burden them or you getting bogged down in details that don't matter and missing those that do. One of the skills I've developed is spinning (back) up on problems quickly while holding the overall in my head. I find with AI I'm just doing that even more often and I now have 5 direct reports (AI) added to the 6 teams of 8 I work with through managers and roadmaps.
- eikenberry 1y ago
  I think there is one big difference that will differentiate between principal/lead devs and euqally experienced senior devs working with AI.. AIs are not people. Lead/principal developers are good at delegating work to, and managing, people. People and AIs have very little in common and I don't think the skills will really translate that well. I think the people who will really shine with AI are those at the principal level of skill but who are better with computers than people. They will be able to learn the AI system interaction skills without first having to unlearn all the people interaction skills and I'm not sure if the "leadership skills" that are prized in principal devs can even be unlearned they seem to be more a natural affinity than a skill.
IgorPartola 1y ago
All you need is a magnetized needle and a steady hand. Years ago I interviewed at Rackspace. They did a data structures and algorithms type interview. One of the main questions they had was about designing a data structure for a distributed hash table, using C or equivalent, to be used as a cache and specifically addressing cache invalidation. After outlining the basic approach I stopped and said that I have used a system like that in several projects at my current and former jobs and I would use something like Redis, memcache, or even Postgres in one instance, and do a push to cache on write system rather than a cache server pulling values from the source of truth if it suspected it had stale data. They did not like that answer. I asked why and they said it’s because I’m not designing a data structure from scratch. I asked them if the job I am applying for involved creating cache servers from scratch and they said “of course not. We use Redis.” (It might have been memcache, I honestly don’t remember which datadore they liked). Needless to say, this wasn’t a fit for either of us. While I am perfectly capable of creating toy versions of these kinds of services, I still stand by using existing battle tested software over rolling your own. If you worry about forgetting how to code, then code. You already don’t know how to code 99% of the system you are using to post this comment (Verilog, CPU microcode, GPU equivalents, probably MMU programming, CPU-specific assembly, and so on). You can get ahead of the competition by learning some of that tech. Or not. But technically all you need is a magnetized needle and a steady hand.
- lesuorac 1y ago
  Heh, I remember an interview once and they wanted me to figure out if a word contained double letters (i.e. there's 2 L's in letters). I was like well I'd probably just make a Set in Java and insert letters until it has a duplicate. They didn't like that. So I was like well I guess I can make a double for-loop and check that way and they liked that ... It is weird how people like you to re-invent the wheel as opposed to just import one.
  - esperent 1y ago
    Not that I'm a fan of this kind of interview, but these answers illustrate different kinds of skill/intelligence. One is domain knowledge which is less important in the age Google search and StackOverflow (and even less so in the age of LLMs but I guess interview techniques haven't caught up yet). The second is the ability to understand a nested for loop, and if a coder can't do that by the point they reach an interview, it can probably never be taught. It could be argued that being able to think up using a set in this instance is also an important skill, and I agree. But nested for loops are foundational skills, if the interviewee has problems there it's a good thing to know about early. It could also be argued that they should just say directly "solve this using loops" if that's what they want, and well, yeah.
Arubis 1y ago
This rhymes with the discussion we had when higher level languages became popularized. And many of us did forget how to write assembly! What might the world have looked like otherwise?
- mystified5016 1y ago
  Maybe computers wouldn't have gotten slower as time goes on. We definitely would not have Electron and that's a world I want to live in.
_pdp_ 1y ago
This is a problem for new generations that should be mindful of but then again how many people can whip some assembly? Certainly not the majority of developers and it is certainly not required for most programming tasks. We might end up in the same situation - most of the plumbing will be done by high-level coding with the help of AI agents.
danjl 1y ago
Another comment that has nothing to do with the tool in question. You should also eat more vegetables.

xp84 1y ago

I mean, it’s just like having an army of interns that works for (near) free. It’s a huge positive for productivity, and I don’t think we will forget how. I’m more concerned with how we make new senior/staff engineers from now on, since the old “do grunt work for a couple years, then do simple well defined work for a few years” is 100% not a career path that exists even now.

LTL_FTC 1y ago
This is my question as well. I am already wondering how prepared college grads will be. Getting help with programming assignments meant going to the dungeon and collaborating with fellow students while the TA made their rounds and overall just figuring it out. Today, an LLM knocks out the programming assignments in once shot, probably. And industry seems hellbent on hiring seniors mostly so where are the juniors to become seniors going to come from? I think the talent pipeline has contracted and/or will and overcorrect. But maybe the industry’s carrying capacity of devs has shrunk.
- xp84 1y ago
  The ironic part is that for about 10 years from 2012-2022 I used to tell people that I was very bullish on programming as a career, since I couldn't imagine any possible future where we needed less software to be written in year N+1 than we did in year N. I just didn't think of any world where we could have more software written with fewer engineers. Surprise!

dawnerd 1y ago

The future is going to be great for us that have been resisting going all in. Unfortunately I feel a lot of work will be detangling the mess these llms make in larger repos.

skydhash 1y ago
The devils is in the details, as they say. And software engineering used to be exorcism, now they want it to be summoning. Now I'm just hopping for the majority to realize that hell is not a great environment for business.
- Bluestein 1y ago
  > software engineering used to be exorcism, now they want it to be summoning. Now I'm just hopping for the majority to realize that hell is not a great environment for business With your leave, this is going up on my wall :)

amelius 1y ago

> People are going to forget how to code. Which is a problem when exactly? When civilization collapses?

cft 1y ago

It will probably be like coding in assembly after the advent of the compilers. There are some people who still code in assembly, but it's rare.

hooverd 1y ago
so software will get even worse because nobody understands anything about how computers work?
- missedthecue 1y ago
  Your average professional python programmer knows a lot less about how computers work than the assembly machine level programmers of yesteryear. Software today is both worse and better. Slack uses 2gb of RAM, but is there anyone who wants to go back? Things will probably continue in that general direction. And just like today, a small number of people who really know what they're doing keep everything humming so people can build on top of it. By importing 13 python libraries they don't completely understand, or having an AI build 75% of their CRUD app.
  - hooverd 1y ago
    I think it's a problem in that each 1% of slop here and there massive compounds overall.
- crazylogger 1y ago
  Me and my LLM buddy together understand exactly how computers work!
Bluestein 1y ago
Wait until only LLMs know how to build compilers. That's going to be a riot ...

fassssst 1y ago

Just like how people forgot how to patch phone lines and punch cards.

aquariusDue 1y ago

It's the calculator all over again! Jokes aside, while I'm almost sure that the ability to code can be lost and regained just like training a muscle what I'm more worried is the rug pull and squeeze that is bound to happen sometime in the next 5 to 10 years unless LLMs go the way of Free Software GNU style. If the latter happens then LLMs for coding will be like calculators and such more or less and personally I don't know how more harmful that would be compared to the boost in productivity. That said if the former becomes reality (and I hope not!) then we're in for some huge existential crises when people realize they can barely materialize the labour part of their jobs after doing the thinky part and the meetings part.

timschmidt 1y ago
I don't think the rug pull and squeeze are possible. Because I've had the same worry. But using an existing LLM to train or fine tune a new one seems to be standard practice, and to work quite well. So any LLM with an API will end up training all the others - even open source LLMs - and all will benefit. And every day that passes, Moore makes it less and less costly for amateurs to commit the compute necessary for fine tuning, and eventually training from scratch. In time, even video and embodied training may be possible for amateurs, though that's difficult to contemplate today.
alwillis 1y ago
There are already lots of open source/open weight models than can run locally on a laptop. People into homelabs have been running AI tools on home servers for years.
- bakugo 1y ago
  > There are already lots of open source/open weight models than can run locally on a laptop. And they're all too small and dumb to be useful for anything but the most basic tasks.

KronisLV 1y ago

> People are going to forget how to code. Pretty much me with some IDEs and their code inspections and refactoring capabilities and run profile configurations (especially in your average enterprise Java codebase). Oh well.

alwillis 1y ago

Just a reminder you can tell the LLM the version of an API to use. Your code should have tests the AI can use to test the code it wrote. And thanks to MCP, you can literally point your LLM to the documentation of your preferred tool [1]. [1]: https://context7.com/about https://context7.com/about

grogenaut 1y ago

I think it's more like every engineer will either become like a lead or a principal or have problems. I'm a principal. I have for years had multiple teams building things that I prototyped designed or worked with them on the specs for. There's a level of touch and letting go that you have to employ to not over burden them or you getting bogged down in details that don't matter and missing those that do. One of the skills I've developed is spinning (back) up on problems quickly while holding the overall in my head. I find with AI I'm just doing that even more often and I now have 5 direct reports (AI) added to the 6 teams of 8 I work with through managers and roadmaps.

eikenberry 1y ago
I think there is one big difference that will differentiate between principal/lead devs and euqally experienced senior devs working with AI.. AIs are not people. Lead/principal developers are good at delegating work to, and managing, people. People and AIs have very little in common and I don't think the skills will really translate that well. I think the people who will really shine with AI are those at the principal level of skill but who are better with computers than people. They will be able to learn the AI system interaction skills without first having to unlearn all the people interaction skills and I'm not sure if the "leadership skills" that are prized in principal devs can even be unlearned they seem to be more a natural affinity than a skill.

IgorPartola 1y ago

All you need is a magnetized needle and a steady hand. Years ago I interviewed at Rackspace. They did a data structures and algorithms type interview. One of the main questions they had was about designing a data structure for a distributed hash table, using C or equivalent, to be used as a cache and specifically addressing cache invalidation. After outlining the basic approach I stopped and said that I have used a system like that in several projects at my current and former jobs and I would use something like Redis, memcache, or even Postgres in one instance, and do a push to cache on write system rather than a cache server pulling values from the source of truth if it suspected it had stale data. They did not like that answer. I asked why and they said it’s because I’m not designing a data structure from scratch. I asked them if the job I am applying for involved creating cache servers from scratch and they said “of course not. We use Redis.” (It might have been memcache, I honestly don’t remember which datadore they liked). Needless to say, this wasn’t a fit for either of us. While I am perfectly capable of creating toy versions of these kinds of services, I still stand by using existing battle tested software over rolling your own. If you worry about forgetting how to code, then code. You already don’t know how to code 99% of the system you are using to post this comment (Verilog, CPU microcode, GPU equivalents, probably MMU programming, CPU-specific assembly, and so on). You can get ahead of the competition by learning some of that tech. Or not. But technically all you need is a magnetized needle and a steady hand.

lesuorac 1y ago
Heh, I remember an interview once and they wanted me to figure out if a word contained double letters (i.e. there's 2 L's in letters). I was like well I'd probably just make a Set in Java and insert letters until it has a duplicate. They didn't like that. So I was like well I guess I can make a double for-loop and check that way and they liked that ... It is weird how people like you to re-invent the wheel as opposed to just import one.
- esperent 1y ago
  Not that I'm a fan of this kind of interview, but these answers illustrate different kinds of skill/intelligence. One is domain knowledge which is less important in the age Google search and StackOverflow (and even less so in the age of LLMs but I guess interview techniques haven't caught up yet). The second is the ability to understand a nested for loop, and if a coder can't do that by the point they reach an interview, it can probably never be taught. It could be argued that being able to think up using a set in this instance is also an important skill, and I agree. But nested for loops are foundational skills, if the interviewee has problems there it's a good thing to know about early. It could also be argued that they should just say directly "solve this using loops" if that's what they want, and well, yeah.

Arubis 1y ago

This rhymes with the discussion we had when higher level languages became popularized. And many of us did forget how to write assembly! What might the world have looked like otherwise?

mystified5016 1y ago
Maybe computers wouldn't have gotten slower as time goes on. We definitely would not have Electron and that's a world I want to live in.

_pdp_ 1y ago

This is a problem for new generations that should be mindful of but then again how many people can whip some assembly? Certainly not the majority of developers and it is certainly not required for most programming tasks. We might end up in the same situation - most of the plumbing will be done by high-level coding with the help of AI agents.

danjl 1y ago

Another comment that has nothing to do with the tool in question. You should also eat more vegetables.