5 ms·

> I'm not sure that having the patience to work with something with a very inconsistent performance and that frequently lies is an extension of existing develop

by unoti 11mo ago

> I'm not sure that having the patience to work with something with a very inconsistent performance and that frequently lies is an extension of existing development skills. If you’ve be been tasked with leadership of an engineering effort involving multiple engineers and stakeholders you know that this is in fact a crucial part of the role the more senior you get. It is much the same with people: know their limitations, show them a path to success, help them overcome their limitations by laying down the right abstractions and giving them the right coaching, make it easier to do the right thing. Most of the same approaches apply. When we do these things with people it’s called leadership or management. With agents, it’s context engineering.

pron 11mo ago
Because I reached that position 15 years ago, I can tell you that this is untrue (in the sense that the experience is completely different from an LLM). Training is one thing, but training doesn't increase the productivity of the trainer; it's meant to improve the capability of the trainee. At any level of capability, though - whether we're talking about an intern after one year of university or a senior developer with 20 years of experience - effective management requires that you're able to trust that the person tells you when they've hit a snag or anything else you may need to know. We may not be talking 100% of trust, but not too far from that, either. You can't continue working with someone that doesn't tell you what you need to know even 10% of the time, regardless of their level. LLMs are not at that acceptable level yet, so the experience is not similar to technical leadership. If you've ever been tasked with leading one or more significant projects you'd know that if you feel you have to review every line of code anyone on the team writes, at every step of the process, that's not the path to success (if you did that, not only would progress be slow, but your team wouldn't like you very much). Code review is a very important part of the process, but it's not an efficient mechanism for day-to-day communication.
- unoti 11mo ago
  > effective management requires that you're able to trust that the person tells you when they've hit a snag or anything else you may need to know This is what we shoot for, yes, but many of the most interesting war stories involve times when people should have been telling you about snags but weren't-- either because they didn't realize they were spinning their wheels, or because they were hoping they'd somehow magically pull off the win before the due date, or innumerable other variations on the theme. People are most definitely not reliable about telling you things they should have told you. > if you feel you have to review every line of code anyone on the team writes... Somebody has to review the code, and step back and think about it. Not necessarily the manager, but someone does.
  - discreteevent 11mo ago
    > the most interesting war stories involve times when people should have been telling you about snags but weren't- This comes up a lot. A person sometimes does an undesirable thing that an AI also does. So you might as well use the AI. But we don't apply this thinking to people. If a person does something undesirable sometimes then we accept that because they are human. If they do it very frequently then at some point, given a choice, you will stop working with that person.
- herval 11mo ago
  > effective management requires that you're able to trust that the person tells you when they've hit a snag or anything else you may need to know Nope, effective management is on YOU, not them. If everyone you’re managing is completely transparent and immediately tells you stuff, you’re playing in easy mode
  - unoti 11mo ago
    > If everyone you’re managing is completely transparent and immediately tells you stuff, you’re playing in easy mode So much this. There are many managers who are effective at managing people who do not need management.
    - herval 11mo ago
      The vast majority of managers, much like most engineers, only has to deal with “maintenance mode” throughout most of their career. Particularly common in people whose experience has been in large corporations - you simply don’t realize how much was built for you and “works” (even if badly)
  - pron 11mo ago
    So the role of a coding agent is to challenge me to play in hard mode? And suppose getting developers to not lie or hide important information is on me, what should I do to get an LLM to not do that?
    - herval 11mo ago
      no, the point is LLMs will behave the same way humans you have to manage do (there's obviously differences - eg LLMs tend to forget context more often than most humans, but also they tend to know a lot more than the average human). So some of the same skills that'll help you manage humans will also help you get more consistency out of LLMs.
      - pron 11mo ago
        I don't know of anyone who would like to work with someone who lies to them over and over, and will never stop. LLMs do certain things better than people, but my point is that there's nothing you can trust them to do. That's fine for research (we don't trust, and don't need to trust, any human or tool to do a fully exhaustive research, anyway), but not for most other work tasks. That's not to say that LLMs can't be utilised usefully, but something that can never be trusted behaves like neither person nor tool.
boleary-gl 11mo ago
1000% this. Today LLMs are like enthusiastic, energetic, over-confident, well-read junior engineers. Does it take effort to work with them and get them to be effective in your code base? Yes. But is there a way to lead them in such a way that your "team" (you in this case) gets more done? Yes. But it does take effort. That's why I love "vibe engineering" as a term because the engineering (or "senior" or "lead" engineering) is STILL what we are doing.
dandellion 11mo ago
Inconsistent performance and frequent lies are a crucial part of the role, really? I've only met a couple of people like that on my career. Interviews go both ways: if I can't establish that the team I'll be working with is composed and managed by honest and competent people, I don't accept their offer. Sometimes it has meant missing out on the highest compensation, but at least I don't deal with lies and inconsistent performance.

by unoti 11mo ago

pron 11mo ago

Because I reached that position 15 years ago, I can tell you that this is untrue (in the sense that the experience is completely different from an LLM). Training is one thing, but training doesn't increase the productivity of the trainer; it's meant to improve the capability of the trainee. At any level of capability, though - whether we're talking about an intern after one year of university or a senior developer with 20 years of experience - effective management requires that you're able to trust that the person tells you when they've hit a snag or anything else you may need to know. We may not be talking 100% of trust, but not too far from that, either. You can't continue working with someone that doesn't tell you what you need to know even 10% of the time, regardless of their level. LLMs are not at that acceptable level yet, so the experience is not similar to technical leadership. If you've ever been tasked with leading one or more significant projects you'd know that if you feel you have to review every line of code anyone on the team writes, at every step of the process, that's not the path to success (if you did that, not only would progress be slow, but your team wouldn't like you very much). Code review is a very important part of the process, but it's not an efficient mechanism for day-to-day communication.

unoti 11mo ago
> effective management requires that you're able to trust that the person tells you when they've hit a snag or anything else you may need to know This is what we shoot for, yes, but many of the most interesting war stories involve times when people should have been telling you about snags but weren't-- either because they didn't realize they were spinning their wheels, or because they were hoping they'd somehow magically pull off the win before the due date, or innumerable other variations on the theme. People are most definitely not reliable about telling you things they should have told you. > if you feel you have to review every line of code anyone on the team writes... Somebody has to review the code, and step back and think about it. Not necessarily the manager, but someone does.
- discreteevent 11mo ago
  > the most interesting war stories involve times when people should have been telling you about snags but weren't- This comes up a lot. A person sometimes does an undesirable thing that an AI also does. So you might as well use the AI. But we don't apply this thinking to people. If a person does something undesirable sometimes then we accept that because they are human. If they do it very frequently then at some point, given a choice, you will stop working with that person.
herval 11mo ago
> effective management requires that you're able to trust that the person tells you when they've hit a snag or anything else you may need to know Nope, effective management is on YOU, not them. If everyone you’re managing is completely transparent and immediately tells you stuff, you’re playing in easy mode
- unoti 11mo ago
  > If everyone you’re managing is completely transparent and immediately tells you stuff, you’re playing in easy mode So much this. There are many managers who are effective at managing people who do not need management.
  - herval 11mo ago
    The vast majority of managers, much like most engineers, only has to deal with “maintenance mode” throughout most of their career. Particularly common in people whose experience has been in large corporations - you simply don’t realize how much was built for you and “works” (even if badly)
- pron 11mo ago
  So the role of a coding agent is to challenge me to play in hard mode? And suppose getting developers to not lie or hide important information is on me, what should I do to get an LLM to not do that?
  - herval 11mo ago
    no, the point is LLMs will behave the same way humans you have to manage do (there's obviously differences - eg LLMs tend to forget context more often than most humans, but also they tend to know a lot more than the average human). So some of the same skills that'll help you manage humans will also help you get more consistency out of LLMs.
    - pron 11mo ago
      I don't know of anyone who would like to work with someone who lies to them over and over, and will never stop. LLMs do certain things better than people, but my point is that there's nothing you can trust them to do. That's fine for research (we don't trust, and don't need to trust, any human or tool to do a fully exhaustive research, anyway), but not for most other work tasks. That's not to say that LLMs can't be utilised usefully, but something that can never be trusted behaves like neither person nor tool.

boleary-gl 11mo ago

1000% this. Today LLMs are like enthusiastic, energetic, over-confident, well-read junior engineers. Does it take effort to work with them and get them to be effective in your code base? Yes. But is there a way to lead them in such a way that your "team" (you in this case) gets more done? Yes. But it does take effort. That's why I love "vibe engineering" as a term because the engineering (or "senior" or "lead" engineering) is STILL what we are doing.

dandellion 11mo ago

Inconsistent performance and frequent lies are a crucial part of the role, really? I've only met a couple of people like that on my career. Interviews go both ways: if I can't establish that the team I'll be working with is composed and managed by honest and competent people, I don't accept their offer. Sometimes it has meant missing out on the highest compensation, but at least I don't deal with lies and inconsistent performance.