17 ms·

My department at the place I work is actively hiring Software Engineers. We have nine open requisitions for any seniority level and are regularly conducting int

My department at the place I work is actively hiring Software Engineers. We have nine open requisitions for any seniority level and are regularly conducting interviews, but the new-grad candidates this year have been... disappointing. I've conducted two phone screens this month and asked each candidate to implement FizzBuzz in their language of choice after giving them an explanation of the problem. Both took more than ten minutes to write out a solution and we don't even require them to run it; I'll excuse trivial syntax errors in an interview setting if I can tell what you meant. When CS students can't write a basic for loop and use the modulo operator without relying on AI, I weep for their generation.

ikiris 1y ago
Are you offering enough pay that competent people would want to work there?
- coderjames 1y ago
  We're in the greater Seattle area and I make north of $200k, so I feel like yes :shrug:
testing1235 1y ago
I also tutor students in the entry level C++ and Python courses (which are taken during your first two semesters as a CS student), and I must agree that a large cohort of my class is only able to program if they have ChatGPT/Claude open on one half of their screen. I'm not sure how to solve this either, unless we want to start doing in person "interview" styled questions as an exam on a locked down computer. I honestly think that doing an in person fake technical interview with a few easy Leetcode questions at the end of your education would be a good way to weed out those that have failed to even learn the basics of the trade.
- linguae 1y ago
  I teach computer science at a community college in Silicon Valley. Even before generative AI became available to the general public, cheating has been an issue with CS programming assignments. One way I try to disincentivize cheating on projects is by having in-class paper exams, including weekly quizzes, as well as in-class paper assignments, and making sure that these in-class assessments are weighted significantly (roughly 60% of the overall grade). No electronic devices are allowed for these assignments. This forces my students to be able to write code without being able look up things online or consult an AI tool. I still assign take-home programming projects that take 1-2 weeks to complete; students submit compilable source code. Practical hands-on programming experience is still vital, and even though cheating is possible, the vast majority of my students want to learn and are honest. Still, for in-person assessments, if I had the budget, I’d prefer to hand out laptops with no Internet connection and a spartan selection of software, just a text editor and the relevant compiler/interpreter. It would making grading in-class submissions easier. But since we don’t have this budget, in-class exams and exercises are the next best solution I could think of.
  - bruce511 1y ago
    This reply will likely sound disrespectful, but I post it not to be so, but rather to perhaps spark an alternate path. As the world changes, education can be slowest to adapt. My father did his math on a slide rule. I was in high school as we transitioned to using calculators. My personal take on your approach is that you're seeing this from the wrong side. Creating an artificial environment for testing suggests to me you're testing the wrong thing. Of course most school, and college, classes devolve to testing memory. "Here's the stuff to learn, remember it enough to pass the exam." And I get it, this is the way it's always been, regardless of the uselessness of the information. Who can remember when Charles 1st was beheaded? Who can't Google it in an instant? Programing on paper without online reference tools isn't a measure of anything, because in the real world those tools exist. Indeed, the very notion that we should even be testing "ability to write code" is outdated. That the student can create code should be a given. Rather an exam should test understanding, not facts. Here's 2 blocks of code, which is better and why? Here's some code, what are the things about it that concern you? Instead of treating the use of AI (or Google, or online help, or that giant C reference book I had) as "cheating", perhaps teach and assess in a world where AI exists. I truly do get it. Testing comprehension is hard. Testing understanding is hard. Testing to sift wheat from chaff is hard. But, and I'm being harsh here i know, testing memory as a proxy for intelligence or testing hand-code-output as a proxy for understanding code is borderline meaningless. Perhaps in the age of AI the focus switches from 'writing code' to 'reading code'. From the ability to write to the ability to prompt, review, evaluate and so on. Perhaps the skill that needs to be taught (to the degree that community college seeks to teach skills) needs to be programing with AI, not against it. I say all this with respect for how hard your job is, and with my thanks that you do it at all. I also say it understanding that it's a huge burden on you that you didn't necessarily sign up for.
    - const_cast 1y ago
      The problem is that tools like AI are useful if and only if you have the prerequisite knowledge, otherwise they are self-destructive. It's similar to a calculator. We give student graphing calculators, but ONLY after they have already graphed by-hand hundreds of times. Why? Because education does not work like other things. Efficiency, in education, is bad. We don't want to solve problems as fast as possible, we want to form the best understanding of problems possible. When I, say, want to book an airplane ticket, I want to do that in the fastest way possible. The most efficient manner. I care not about how an airport works, or how flight numbers are decided, or how planes work. But efficient education is bad education. We can skip 99% of education, if we wanted. We can have, say, the SAT - and spend 1 year studying only for the SAT. Don't bother with the other 12 years of schooling. Will you get an acceptable score on the SAT this way? Maybe. Will you be intelligent? No, you will be functionally illiterate. If we use AI for programming before we can program, then we will be bad programmers. Yes, we can pass a test. Yes, we can pass a quiz. But we don't know what we're doing, because education is cumulative. If we skip steps, we lose. If we cut corners, we lose. It's like trying to put a roof on a house when the foundation isn't even poured.
- bruce511 1y ago
  I'm so old I remember when calculators started appearing in general people's hands. Schools first banned them (what da ya mean you can't add a column of numbers by eye?) But gradually we switched over. We had a flirting interaction with log tables, and never did get to use a slide rule. I've no doubt old-school businesses were aghast at our ineptitude. I'm so old we learned to program with giant C reference books. There was no internet, much less Google. We didn't have no fancy auto-complete, crumbs a text editor was considered advanced. Them youngsters coming to us couldn't program without Googling syntax, or using an IDE. So yeah, sure, AI is changing the game. It's hard to evaluate students because the tools they are using are different to our experience. For decades we "make them code" as a measure of ability. In 3 years (their college experience) the toolset has changed. Good students, good employees, are those who understand the problem and can adapt to a solution. AI is a tool that can be wielded well, or badly. Our approach to hiring will need to adapt as well. But good people are still out there, and good people make good workers. To be honest I never was much in love with the leet code measure of hiring. Past a certain coding skill level I was more interested in the person than their ability to memorize an algorithm. Today that necessary skill level is lower, or at least harder to evaluate, but the problem-solving-mind is still the thing we're looking for. So be careful of seeing the use of new tools as a weakness. The history of the world is littered with obsolete technologies. (Here's a sextant, where are we?) Rather see people who use tools for what they are, tools. Look for people who are curious, who see patterns, who get things done. And to students I say, mastery of tools is a necessary step, but ultimately an uninteresting one. See beyond them. Be curious. Look under the hood. Ask questions like "is this code good enough to be running 30 years from now?" Because a huge amount of what you see now has foundations in code written a long time ago, and written well enough to stand for decades. College is not "learning to program". College is learning how to adapt to an ever changing world, that will require your adapting many times over your career.
  - bluefirebrand 1y ago
    > College is not "learning to program". College is learning how to adapt to an ever changing world, that will require your adapting many times over your career. You're gonna have to do a lot of work to convince me that people who only know how to drive an LLM are learning how to adapt to sweet fuck all At least with a calculator, people still had to know the difference between addition and multiplication, in order to use the calculator correctly
- CamperBob2 1y ago
  I'm not sure how to solve this either, unless we want to start doing in person "interview" styled questions as an exam on a locked down computer. Don't lock down the computer unless you are hiring people to work in a SCIF. Instead, give candidates a brutally hard/weird problem and tell them to use any resources they can get their hands on, by fair means or foul. (They will do that anyway if you hire them.) Then watch how they deal with it. Do they just give up and stalk off in a huff? If they Google for answers, do they use sensible queries? If they use AI, do their prompts show skill at getting ideas, avoiding blind alleys, and creating effective tests? If they call their friends, see how effective they are at communicating the requirements and turning the answers into a solution. Might be management material.
lispisok 1y ago
When I was in school in the early 2010s I was working in a professor's lab and overheard conversations that the administration was telling profs/TAs to pass kids who profs/TA's thought should have failed. I've since seen the required coursework to graduate become less rigorous. There were students I worked with personally I graduated with who were very bad. I'm sure there still great students who care about learning but I cannot imagine how bad the average student is with ChatGPT being able to do student's assignments.
klipklop 1y ago
FizzBuzz was always a great filter. Even in the pre-LLM days. Many people can code for years and never once use the modulo operator. Solving the problem gets a lot more clunky without it and they get rejected.
- SamuelAdams 1y ago
  Yes but also it is one of the most common programming questions for non-FAANG companies. Are grads not preparing for interviews? It is one google search to Jeff Atwood’s blog.
orzig 1y ago
I’ll second this, and we had enough resumes to only interview those with a relevant Master’s degree. I was shocked and I still don’t have a full explanation. I don’t doubt that it’s also hard out there, but on the hiring side we also did far more interviews than we wanted. (And yes the salary is >>100k, full remote, benefits etc)
deleted 1y ago
[deleted]
calderwoodra 1y ago
2 data points and you're drawing a conclusion about an entire graduating class? For all we know, you might be experiencing a reality that you're company isn't able to attract great young talent.
xboxnolifes 1y ago
> When CS students can't write a basic for loop and use the modulo operator without relying on AI, I weep for their generation. I feel like this doesn't get said enough, but I'm almost certain your issue is happening during filtering prior to even getting to the interview stage. Companies are straight up choosing (the wrong) applicants to interview, the applicant fails the interview, the company does not move forward with them, and then the companies does not go back and and consider the people they originally filtered out. I know companies get swamped with tons of applications, and filtering is basically an impossible problem since anyone can make their resume look good, but every applicant that applied can't be that bad. Bad applicant filtering at the first step is hurting both companies and applicants.
NoGravitas 1y ago
My experience is that this problem significantly predates AI. Not that AI won't make it worse, but pre-2020 the majority of entry level developer applicants I interviewed could not write a basic for loop in their choice of language, never mind the modulo operator.

ikiris 1y ago

Are you offering enough pay that competent people would want to work there?

coderjames 1y ago
We're in the greater Seattle area and I make north of $200k, so I feel like yes :shrug:

testing1235 1y ago

I also tutor students in the entry level C++ and Python courses (which are taken during your first two semesters as a CS student), and I must agree that a large cohort of my class is only able to program if they have ChatGPT/Claude open on one half of their screen. I'm not sure how to solve this either, unless we want to start doing in person "interview" styled questions as an exam on a locked down computer. I honestly think that doing an in person fake technical interview with a few easy Leetcode questions at the end of your education would be a good way to weed out those that have failed to even learn the basics of the trade.

linguae 1y ago
I teach computer science at a community college in Silicon Valley. Even before generative AI became available to the general public, cheating has been an issue with CS programming assignments. One way I try to disincentivize cheating on projects is by having in-class paper exams, including weekly quizzes, as well as in-class paper assignments, and making sure that these in-class assessments are weighted significantly (roughly 60% of the overall grade). No electronic devices are allowed for these assignments. This forces my students to be able to write code without being able look up things online or consult an AI tool. I still assign take-home programming projects that take 1-2 weeks to complete; students submit compilable source code. Practical hands-on programming experience is still vital, and even though cheating is possible, the vast majority of my students want to learn and are honest. Still, for in-person assessments, if I had the budget, I’d prefer to hand out laptops with no Internet connection and a spartan selection of software, just a text editor and the relevant compiler/interpreter. It would making grading in-class submissions easier. But since we don’t have this budget, in-class exams and exercises are the next best solution I could think of.
- bruce511 1y ago
  This reply will likely sound disrespectful, but I post it not to be so, but rather to perhaps spark an alternate path. As the world changes, education can be slowest to adapt. My father did his math on a slide rule. I was in high school as we transitioned to using calculators. My personal take on your approach is that you're seeing this from the wrong side. Creating an artificial environment for testing suggests to me you're testing the wrong thing. Of course most school, and college, classes devolve to testing memory. "Here's the stuff to learn, remember it enough to pass the exam." And I get it, this is the way it's always been, regardless of the uselessness of the information. Who can remember when Charles 1st was beheaded? Who can't Google it in an instant? Programing on paper without online reference tools isn't a measure of anything, because in the real world those tools exist. Indeed, the very notion that we should even be testing "ability to write code" is outdated. That the student can create code should be a given. Rather an exam should test understanding, not facts. Here's 2 blocks of code, which is better and why? Here's some code, what are the things about it that concern you? Instead of treating the use of AI (or Google, or online help, or that giant C reference book I had) as "cheating", perhaps teach and assess in a world where AI exists. I truly do get it. Testing comprehension is hard. Testing understanding is hard. Testing to sift wheat from chaff is hard. But, and I'm being harsh here i know, testing memory as a proxy for intelligence or testing hand-code-output as a proxy for understanding code is borderline meaningless. Perhaps in the age of AI the focus switches from 'writing code' to 'reading code'. From the ability to write to the ability to prompt, review, evaluate and so on. Perhaps the skill that needs to be taught (to the degree that community college seeks to teach skills) needs to be programing with AI, not against it. I say all this with respect for how hard your job is, and with my thanks that you do it at all. I also say it understanding that it's a huge burden on you that you didn't necessarily sign up for.
  - const_cast 1y ago
    The problem is that tools like AI are useful if and only if you have the prerequisite knowledge, otherwise they are self-destructive. It's similar to a calculator. We give student graphing calculators, but ONLY after they have already graphed by-hand hundreds of times. Why? Because education does not work like other things. Efficiency, in education, is bad. We don't want to solve problems as fast as possible, we want to form the best understanding of problems possible. When I, say, want to book an airplane ticket, I want to do that in the fastest way possible. The most efficient manner. I care not about how an airport works, or how flight numbers are decided, or how planes work. But efficient education is bad education. We can skip 99% of education, if we wanted. We can have, say, the SAT - and spend 1 year studying only for the SAT. Don't bother with the other 12 years of schooling. Will you get an acceptable score on the SAT this way? Maybe. Will you be intelligent? No, you will be functionally illiterate. If we use AI for programming before we can program, then we will be bad programmers. Yes, we can pass a test. Yes, we can pass a quiz. But we don't know what we're doing, because education is cumulative. If we skip steps, we lose. If we cut corners, we lose. It's like trying to put a roof on a house when the foundation isn't even poured.
bruce511 1y ago
I'm so old I remember when calculators started appearing in general people's hands. Schools first banned them (what da ya mean you can't add a column of numbers by eye?) But gradually we switched over. We had a flirting interaction with log tables, and never did get to use a slide rule. I've no doubt old-school businesses were aghast at our ineptitude. I'm so old we learned to program with giant C reference books. There was no internet, much less Google. We didn't have no fancy auto-complete, crumbs a text editor was considered advanced. Them youngsters coming to us couldn't program without Googling syntax, or using an IDE. So yeah, sure, AI is changing the game. It's hard to evaluate students because the tools they are using are different to our experience. For decades we "make them code" as a measure of ability. In 3 years (their college experience) the toolset has changed. Good students, good employees, are those who understand the problem and can adapt to a solution. AI is a tool that can be wielded well, or badly. Our approach to hiring will need to adapt as well. But good people are still out there, and good people make good workers. To be honest I never was much in love with the leet code measure of hiring. Past a certain coding skill level I was more interested in the person than their ability to memorize an algorithm. Today that necessary skill level is lower, or at least harder to evaluate, but the problem-solving-mind is still the thing we're looking for. So be careful of seeing the use of new tools as a weakness. The history of the world is littered with obsolete technologies. (Here's a sextant, where are we?) Rather see people who use tools for what they are, tools. Look for people who are curious, who see patterns, who get things done. And to students I say, mastery of tools is a necessary step, but ultimately an uninteresting one. See beyond them. Be curious. Look under the hood. Ask questions like "is this code good enough to be running 30 years from now?" Because a huge amount of what you see now has foundations in code written a long time ago, and written well enough to stand for decades. College is not "learning to program". College is learning how to adapt to an ever changing world, that will require your adapting many times over your career.
- bluefirebrand 1y ago
  > College is not "learning to program". College is learning how to adapt to an ever changing world, that will require your adapting many times over your career. You're gonna have to do a lot of work to convince me that people who only know how to drive an LLM are learning how to adapt to sweet fuck all At least with a calculator, people still had to know the difference between addition and multiplication, in order to use the calculator correctly
CamperBob2 1y ago
I'm not sure how to solve this either, unless we want to start doing in person "interview" styled questions as an exam on a locked down computer. Don't lock down the computer unless you are hiring people to work in a SCIF. Instead, give candidates a brutally hard/weird problem and tell them to use any resources they can get their hands on, by fair means or foul. (They will do that anyway if you hire them.) Then watch how they deal with it. Do they just give up and stalk off in a huff? If they Google for answers, do they use sensible queries? If they use AI, do their prompts show skill at getting ideas, avoiding blind alleys, and creating effective tests? If they call their friends, see how effective they are at communicating the requirements and turning the answers into a solution. Might be management material.

lispisok 1y ago

When I was in school in the early 2010s I was working in a professor's lab and overheard conversations that the administration was telling profs/TAs to pass kids who profs/TA's thought should have failed. I've since seen the required coursework to graduate become less rigorous. There were students I worked with personally I graduated with who were very bad. I'm sure there still great students who care about learning but I cannot imagine how bad the average student is with ChatGPT being able to do student's assignments.

klipklop 1y ago

FizzBuzz was always a great filter. Even in the pre-LLM days. Many people can code for years and never once use the modulo operator. Solving the problem gets a lot more clunky without it and they get rejected.

SamuelAdams 1y ago
Yes but also it is one of the most common programming questions for non-FAANG companies. Are grads not preparing for interviews? It is one google search to Jeff Atwood’s blog.

orzig 1y ago

I’ll second this, and we had enough resumes to only interview those with a relevant Master’s degree. I was shocked and I still don’t have a full explanation. I don’t doubt that it’s also hard out there, but on the hiring side we also did far more interviews than we wanted. (And yes the salary is >>100k, full remote, benefits etc)

deleted 1y ago

[deleted]

calderwoodra 1y ago

2 data points and you're drawing a conclusion about an entire graduating class? For all we know, you might be experiencing a reality that you're company isn't able to attract great young talent.

xboxnolifes 1y ago

> When CS students can't write a basic for loop and use the modulo operator without relying on AI, I weep for their generation. I feel like this doesn't get said enough, but I'm almost certain your issue is happening during filtering prior to even getting to the interview stage. Companies are straight up choosing (the wrong) applicants to interview, the applicant fails the interview, the company does not move forward with them, and then the companies does not go back and and consider the people they originally filtered out. I know companies get swamped with tons of applications, and filtering is basically an impossible problem since anyone can make their resume look good, but every applicant that applied can't be that bad. Bad applicant filtering at the first step is hurting both companies and applicants.

NoGravitas 1y ago

My experience is that this problem significantly predates AI. Not that AI won't make it worse, but pre-2020 the majority of entry level developer applicants I interviewed could not write a basic for loop in their choice of language, never mind the modulo operator.