13 ms·

> This all sounds good on paper but I have an honest question: How do you develop judgement as a junior or mid level engineer to critique AI if you yourself nev

> This all sounds good on paper but I have an honest question: How do you develop judgement as a junior or mid level engineer to critique AI if you yourself never learned that skill by making mistakes in the first place? Shouldn't this be a pretty fundamental part of a degree? If you're in school right now for a CS or SWE related degree, I would imagine you're learning how code actually works, how the math actually works, etc. If an LLM makes a mistake, you should be able to call that out. You might be arguing that a junior engineer may not have the RWE to make those judgements, but by the time you've completed a 4-5 year degree, you should have done the following: - Completed coursework where you've learned the fundamentals of programming and software engineering - Have done dozens of projects (building everything from basic web apps to more advanced pieces of software) where you've seen what works, what doesn't work, etc. This also gives you real world exposure to the latest and greatest frameworks, tools, etc. - Have done several (at least 2) internships where you've worked at a real company writing real code, and have seen/been mentored into what AI is good at, where it fails, etc.

sarmadgulzar 1mo ago
> Shouldn't this be a pretty fundamental part of a degree? If you're in school right now for a CS or SWE related degree, I would imagine you're learning how code actually works, how the math actually works, etc. I’m sorry to say but colleges don’t teach the kinda stuff you’re gonna use day-to-day. They’ll focus on architecture, data structures, algorithms, databases, OS internals theory that only a minuscule number of systems engineers would get to work on. They won’t teach you that in your /login endpoint, if a user is not found, you should verify the password against a pre-computed dummy hash so the response delay matches a real user account workflow to avoid timing attacks. You only learn this on the job under the supervision of a senior mentor.
- ryanSrich 1mo ago
  > I’m sorry to say but colleges don’t teach the kinda stuff you’re gonna use day-to-day. This was not my experience in college. A lot of it was VERY applied. Granted, that was 13 years ago. But even so, if you're at a college where you feel that you're not getting enough exposure, that's why I also called out "Have done dozens of projects" and "Have done several (at least 2) internships".
- bonoboTP 1mo ago
  Depends a lot on the college. I had electives like Java phone app development (J2ME, dating myself), DirectX game engine development, web development, alongside the more theoretical math classes. > They won’t teach you that in your /login endpoint, if a user is not found, you should verify the password against a pre-computed dummy hash so the response delay matches a real user account workflow to avoid timing attacks. You only learn this on the job under the supervision of a senior mentor. Actually this kind of timing attacks were taught to me in information security and cryptography class, alongside other side channel attacks.
  - sarmadgulzar 1mo ago
    Agreed. I’m not insinuating that colleges should teach all of these extremely domain-specific things. Most of the stuff would indeed be unnecessary for most students depending on what they do in the future. But that is my original point that you learn this on the job while making a mistake and being corrected by a senior mentor. AI wouldn’t call this out by itself unless you give it a very specific prompt which you also can’t write if you don’t know that these kinds of problems happen in the first place.
    - bonoboTP 1mo ago
      Not so sure. AI can flag non-best-practices like this quite easily from generic prompts. You just have to remember to ask it. I mean general review / audit prompts, like "are there any correctness bugs, or things that don't match best practices for efficiency, maintainability, security, or typical UX/API expectations?" And frontier models often flag such things. And it's just getting better. I would not confidently state "AI will never be able to X".
      - sarmadgulzar 1mo ago
        Sure you can do it but I think you’ll agree with what I said about ownership. Whenever I ask AI to audit my codebase, it always plays it safe and gives me a few options to choose from. It just won’t give me a concrete answer unless I insist it to give only one but then I wouldn’t trust that answer that I basically forced out of it.
layer8 1mo ago
> internships where you've worked at a real company writing real code Though why would companies make that nontrivial investment if they can use AI instead? Furthermore, I’d argue that learning good judgement takes around a decade of full-time dev experience. In particular, experiencing the long-term consequences of one’s design and implementation decisions. You don’t get that just by college education and a few internships.
- bonoboTP 1mo ago
  Do you get it from the job hopping every 1-3 years as typical in the US?
  - layer8 1mo ago
    Less effectively, I presume. I’m not in the US.
wild_egg 1mo ago
> Shouldn't this be a pretty fundamental part of a degree? If you're in school right now for a CS or SWE related degree, I would imagine you're learning how code actually works, how the math actually works, etc. CS and SWE are radically different subjects. Science vs Engineering. I'm sure it varies by school but many CS grads will have approximately zero exposure to engineering concepts or any of the latest and greatest frameworks.
- ryanSrich 1mo ago
  SWE was a sub degree within the school of CS where I went to school. You could do CS with a minor in SWE and get the best of both. I suppose if you're just doing a straight ticket CS degree you'll have little exposure to engineering (I believe when I went to school 13 years ago there were 3 courses over 4 years on engineering within a pure CS degree).
- ghaff 1mo ago
  That's sort of broadly true of engineering (including CS if they call it that) at major research universities. They don't really teach practical basics but mostly expect you to pick it up on your own if you don't already know it. They may use Python as a relatively accessible tool in an algorithms class but they mostly don't teach it.
- bonoboTP 1mo ago
  In Europe, informatics degrees, or more specifically technical informatics or engineer-informatics is often the right balance if you want math but also practice. In Hungary they call it engineer-informatics, but officially translate it as Computer Engineering, but it's not really heavily about how to engineer a computer as a machine, though there is some architecture and flipflops etc. of course in the curriculum. It's housed under the electrical engineering faculty, and not under the math and natural sciences faculty, but they still have rigorous discrete math, analysis, graph theory, optimization theory etc. classes. It always baffled me that US colleges seem not to offer such programs. Or maybe they do, they are just not prestigious enough.
Levitz 1mo ago
These are two completely different types of mistakes. LLMs are, by now, pretty good at not making "code doesn't work" and "math doesn't work" kind of mistakes when writing code. These are also "easy" things to get good at, you learn what each part does, understand the abstractions, ensure it makes sense, and go on with your day. Unit testing helps here. LLMs are not that good at not making "this works wrong" kind of mistakes. Maybe the code compiles and does what it has to do, but maybe it's 2 lines of code with 6 lines of comments (looking at you Claude), maybe it defines three helper functions it doesn't really need, maybe it does something "here" when it should be doing that something "there" instead, maybe it finds itself in a framework and completely disregards how the framework is supposed to do things, etc etc. These are harder things to get good at and you WILL end up with an unreadable mess if you disregard caution and let the LLM go at it.
- ryanSrich 1mo ago
  > but maybe it's 2 lines of code with 6 lines of comments (looking at you Claude), maybe it defines three helper functions it doesn't really need, maybe it does something "here" when it should be doing that something "there" instead, maybe it finds itself in a framework and completely disregards how the framework is supposed to do things, etc. I would fully expect a brand new college grad to call these things out. There's nothing in here that requires extensive experience to understand. These are basic principles any SWE should know. Nonsense comments are common sense to pull, if the LLM is pulling in a framework you should look into that framework and understand how it works. Don't know the framework during the interview? Say that. Tell the person interviewing you "looks like it's pulling in XYZ. I'm not entirely familiar with that. I understand at a high level what it's doing, but I'd want to dig deeper and understand if the LLM is doing this part right"

sarmadgulzar 1mo ago

> Shouldn't this be a pretty fundamental part of a degree? If you're in school right now for a CS or SWE related degree, I would imagine you're learning how code actually works, how the math actually works, etc. I’m sorry to say but colleges don’t teach the kinda stuff you’re gonna use day-to-day. They’ll focus on architecture, data structures, algorithms, databases, OS internals theory that only a minuscule number of systems engineers would get to work on. They won’t teach you that in your /login endpoint, if a user is not found, you should verify the password against a pre-computed dummy hash so the response delay matches a real user account workflow to avoid timing attacks. You only learn this on the job under the supervision of a senior mentor.

ryanSrich 1mo ago
> I’m sorry to say but colleges don’t teach the kinda stuff you’re gonna use day-to-day. This was not my experience in college. A lot of it was VERY applied. Granted, that was 13 years ago. But even so, if you're at a college where you feel that you're not getting enough exposure, that's why I also called out "Have done dozens of projects" and "Have done several (at least 2) internships".
bonoboTP 1mo ago
Depends a lot on the college. I had electives like Java phone app development (J2ME, dating myself), DirectX game engine development, web development, alongside the more theoretical math classes. > They won’t teach you that in your /login endpoint, if a user is not found, you should verify the password against a pre-computed dummy hash so the response delay matches a real user account workflow to avoid timing attacks. You only learn this on the job under the supervision of a senior mentor. Actually this kind of timing attacks were taught to me in information security and cryptography class, alongside other side channel attacks.
- sarmadgulzar 1mo ago
  Agreed. I’m not insinuating that colleges should teach all of these extremely domain-specific things. Most of the stuff would indeed be unnecessary for most students depending on what they do in the future. But that is my original point that you learn this on the job while making a mistake and being corrected by a senior mentor. AI wouldn’t call this out by itself unless you give it a very specific prompt which you also can’t write if you don’t know that these kinds of problems happen in the first place.
  - bonoboTP 1mo ago
    Not so sure. AI can flag non-best-practices like this quite easily from generic prompts. You just have to remember to ask it. I mean general review / audit prompts, like "are there any correctness bugs, or things that don't match best practices for efficiency, maintainability, security, or typical UX/API expectations?" And frontier models often flag such things. And it's just getting better. I would not confidently state "AI will never be able to X".
    - sarmadgulzar 1mo ago
      Sure you can do it but I think you’ll agree with what I said about ownership. Whenever I ask AI to audit my codebase, it always plays it safe and gives me a few options to choose from. It just won’t give me a concrete answer unless I insist it to give only one but then I wouldn’t trust that answer that I basically forced out of it.

layer8 1mo ago

> internships where you've worked at a real company writing real code Though why would companies make that nontrivial investment if they can use AI instead? Furthermore, I’d argue that learning good judgement takes around a decade of full-time dev experience. In particular, experiencing the long-term consequences of one’s design and implementation decisions. You don’t get that just by college education and a few internships.

bonoboTP 1mo ago
Do you get it from the job hopping every 1-3 years as typical in the US?
- layer8 1mo ago
  Less effectively, I presume. I’m not in the US.

wild_egg 1mo ago

> Shouldn't this be a pretty fundamental part of a degree? If you're in school right now for a CS or SWE related degree, I would imagine you're learning how code actually works, how the math actually works, etc. CS and SWE are radically different subjects. Science vs Engineering. I'm sure it varies by school but many CS grads will have approximately zero exposure to engineering concepts or any of the latest and greatest frameworks.

ryanSrich 1mo ago
SWE was a sub degree within the school of CS where I went to school. You could do CS with a minor in SWE and get the best of both. I suppose if you're just doing a straight ticket CS degree you'll have little exposure to engineering (I believe when I went to school 13 years ago there were 3 courses over 4 years on engineering within a pure CS degree).
ghaff 1mo ago
That's sort of broadly true of engineering (including CS if they call it that) at major research universities. They don't really teach practical basics but mostly expect you to pick it up on your own if you don't already know it. They may use Python as a relatively accessible tool in an algorithms class but they mostly don't teach it.
bonoboTP 1mo ago
In Europe, informatics degrees, or more specifically technical informatics or engineer-informatics is often the right balance if you want math but also practice. In Hungary they call it engineer-informatics, but officially translate it as Computer Engineering, but it's not really heavily about how to engineer a computer as a machine, though there is some architecture and flipflops etc. of course in the curriculum. It's housed under the electrical engineering faculty, and not under the math and natural sciences faculty, but they still have rigorous discrete math, analysis, graph theory, optimization theory etc. classes. It always baffled me that US colleges seem not to offer such programs. Or maybe they do, they are just not prestigious enough.

Levitz 1mo ago

These are two completely different types of mistakes. LLMs are, by now, pretty good at not making "code doesn't work" and "math doesn't work" kind of mistakes when writing code. These are also "easy" things to get good at, you learn what each part does, understand the abstractions, ensure it makes sense, and go on with your day. Unit testing helps here. LLMs are not that good at not making "this works wrong" kind of mistakes. Maybe the code compiles and does what it has to do, but maybe it's 2 lines of code with 6 lines of comments (looking at you Claude), maybe it defines three helper functions it doesn't really need, maybe it does something "here" when it should be doing that something "there" instead, maybe it finds itself in a framework and completely disregards how the framework is supposed to do things, etc etc. These are harder things to get good at and you WILL end up with an unreadable mess if you disregard caution and let the LLM go at it.

ryanSrich 1mo ago
> but maybe it's 2 lines of code with 6 lines of comments (looking at you Claude), maybe it defines three helper functions it doesn't really need, maybe it does something "here" when it should be doing that something "there" instead, maybe it finds itself in a framework and completely disregards how the framework is supposed to do things, etc. I would fully expect a brand new college grad to call these things out. There's nothing in here that requires extensive experience to understand. These are basic principles any SWE should know. Nonsense comments are common sense to pull, if the LLM is pulling in a framework you should look into that framework and understand how it works. Don't know the framework during the interview? Say that. Tell the person interviewing you "looks like it's pulling in XYZ. I'm not entirely familiar with that. I understand at a high level what it's doing, but I'd want to dig deeper and understand if the LLM is doing this part right"