7 ms·

I became a software engineer after 5 years in data science

47 points by thomzi12 4y ago

system2 4y ago
Unless studied software engineering, you can switch to become an "engineer"? Wouldn't be software developer term more fitting?
- mythhouse 4y ago
  oh no not this tired tangent again.
  - thinking4real 4y ago
    The ramblings of someone who’s low skill and low ambition usually
    - gjvc 4y ago
      it's not clear to whom you are referring
      - melony 4y ago
        People obsessed with protecting the term “engineer” are usually those who want to gatekeep their jobs and salary. In other words, they are worried about competition and want to keep the career “prestigious” and kick the ladder away.
        bobleeswagger 4y ago
        It is a bit facetious to say that all analysts are engineers, or that it is "easy" for any analyst to become an engineer. There is overlap, but they are not identical skillsets. Analysts are certainly developers, but engineering involves more of a sense of scalability, analysis can do this, but it is generally more granular.
        bee_rider 4y ago
        I’m under the impression that programmers’ salary progression typically compares quite favorably to classical engineers’.
- thomzi12 4y ago
  Sure! My company calls the role "software engineer" so that's how I describe the role but I think developer is also a good word for it
- myth_drannon 4y ago
  We also have support engineers with only highschool diploma! And account managers...
- Bingaloopa 4y ago
  Software Engineer and Software Developer are used interchangeably for the most part and I don't think you can really get any meaningful distinction just by the name (in most countries at least). Some countries (Canada I think being one?) may require you be licensed/registered with an engineering body before you're allowed to call yourself an engineer, but thats not the case in most countries AFAIK
  - Waterluvian 4y ago
    Indeed. I worked at a Canadian company that got a legal nastygram from the professional association saying to stop calling us engineers. Then I started working for an American company that was like shrug no laws against that. I just can’t say professional engineer. The whole thing makes a ton of sense when we’re building bridges or airplane safety rated computer systems, but not so much when building websites or whatnot.
    - eddsh1994 4y ago
      I work on critical software and have the 'engineer' label, but do have a nagging feeling it's less structured than 'normal' engineering disciplines. I'm taking a part-time BEng to see if that's the case though!
    - Sephr 4y ago
      You can definitely call yourself a professional software engineer in the US.
- halfmatthalfcat 4y ago
  The US doesn't have software "engineering" degrees, so how does one exactly study software engineering? Computer science, too, is not "engineering".
  - eddsh1994 4y ago
    The UK does, you can get chartered engineer status with a BSc & MSc in Software Engineering or Computer Science if it's applied enough with the IET. When I looked at the US i'm 99% sure I found similar degrees in Software Engineering too.
    - nkingsy 4y ago
      I’ve never seen a “software engineering” degree on a resume as an interviewer in the us.
      - eddsh1994 4y ago
        Fwiw, I have. They're becoming more common - and probably a good thing too, covering important non-CS skills like requirements engineering that are mostly made up in small tech companies without that experience.
  - uxp100 4y ago
    It does…
    - geethree 4y ago
      NCEES even offered a Professional Engineering license in it for awhile. Since discontinued due to lack of interest. https://ncees.org/ncees-discontinuing-pe-software-engineering-exam/ https://ncees.org/ncees-discontinuing-pe-software-engineerin...
  - DavidMiserak 4y ago
    There are colleges and universities in the USA that offer SE degrees, CMU is of note[0] [0] https://www.ece.cmu.edu/academics/ms-se/index.html https://www.ece.cmu.edu/academics/ms-se/index.html
- echelon 4y ago
  If you do any sort of measurement, analysis, or design before implementing a system, you're leveraging engineering principles. Examples: measuring data scale for growth rate and future needs, calculating incremental costs of cloud instances and data stores, designing and proving a state machine, instrumenting systems, measuring and stress testing a system against load, understanding back pressure and dynamical behaviors of distributed systems, designing threaded or active/active systems, vector clocks, consensus algoritms, test suites, etc. etc. I've had to write short inductive proofs in several of the systems I've built. It's engineering. The sooner we get over the imposter syndrome debate of what we can and cannot call ourselves and embrace the full scope and possibilty of what we can achieve, the sooner we can become better and more capable software engineers.
  - angrais 4y ago
    >> I've had to write short inductive proofs in several of the systems I've built. In what context was this necessary?
    - recursive 4y ago
      To provide evidence that a recursive algorithm actually works maybe?
    - eddsh1994 4y ago
      This is pretty common in the Blockchain space to verify transactions behave correctly when arriving out of sync with tools like Agda, Coq, etc... I assume it's the same on databases & I heard Leslie Lamport give a talk at work where he mentioned AWS used TLA+ to prove some of its properties.
- cellis 4y ago
  Upvoted not because I agree, but because I'm genuinely interested in the discussion. What is the difference in your opinion?
  - thundergolfer 4y ago
    This only opinion piece on the issue worth reading, because it's the only one that bothered to seek the opinions of the people that would know best: those people that have been software engineers _and_ some other kind of engineer. https://www.hillelwayne.com/post/are-we-really-engineers/ https://www.hillelwayne.com/post/are-we-really-engineers/
    - wheelinsupial 4y ago
      I think I'm missing something, but why is an opinion piece needed on this issue? Software engineering is a licensed engineering profession in many places. It used to be possible to be licensed as a software engineering in the United States [1, 2](little demand caused the licensing exams to be discontinued, possibly because regulation wasn't enforced, according to [2]). So, I think it's fair to say enough of the engineering profession has spoken to say software is a branch of engineering. What I think are the relevant extracts from the article: > “Do you consider software engineering actually engineering?” > Of the 17 crossovers I talked to, 15 said yes. > That said, many of the crossovers [3] also added an additional qualification: software engineering is real engineering, but a lot of people who write software aren’t doing software engineering. This is not a problem with them, rather a problem with our field: we don’t have a rich enough vocabulary to talk about what these developers do. The engineering field has developed a vocabulary for different professionals in the field. Why can't this be applied to the software field since it is a form of engineering? There are engineers, technologists, technicians, and trades (at least where I am). The issue is there are education and work experience requirements that lead to licensing, which people will say are inequitable and gatekeeping [4]. From 2, there was an experience-only path available to get software engineering licensure, so if the field really wants it, there seems to be precedent to allow for alternative routes into those titles. [1] https://ncees.org/engineering/pe/software/ https://ncees.org/engineering/pe/software/ [2] https://www.nspe.org/resources/pe-magazine/may-2018/ncees-ends-software-engineering-pe-exam https://www.nspe.org/resources/pe-magazine/may-2018/ncees-en... [3] The definition of "crossovers:" "people who used to be professional engineers and then became professional software developers. I call these people crossovers, hybrids between the two worlds." [4] I'm sympathetic to these concerns. I studied engineering technology and worked as a technologist at the beginning of my career. I wanted to become a mechanical engineer, but I couldn't afford to return to school full time to complete the remaining 2 years for the BEng. (My school offered a 2 year diploma for engineering technologists, which led directly to years 3 and 4 year of the BEng.) When I wanted to complete the degree, a full time course load was required (7 courses per term at my school) for the engineering program to maintain accreditation.
- switchbak 4y ago
  I've worked with a bunch of engineers, and while I respect and admire them tremendously, I'd not want to be lumped in with the kind of code they generally write. And don't get me started on the typical code a scientist writes! Let the downvotes commence. Edit: This is a gentle ribbing, but there really does seem to be a different approach in the philosophy of how to solve a problem. Engineers seem to eschew abstraction while software folks embrace is (sometimes to a fault). It's actually pretty fascinating.
  - nicce 4y ago
    Very odd. Usually engineers love abstraction where I am coming from. The simpler the solution, better it usually is. Helps understanding the whole system, making it less complex. Simplicity causes less bugs. Of course, you can do too much abstraction.
    - cscurmudgeon 4y ago
      Have you never run into Matlab code with a few hundred global variables mutating frequently? :)
      - nicce 4y ago
        Okay. I was referreing into software engineers, not data engineers :-)
        cscurmudgeon 4y ago
        Ah, I was referring to general engineers, like chemical or mech engineers :)
  - bee_rider 4y ago
    If they wanted to built their own abstractions they’d be scientists. Actually this is one thing that has always confused me about “software engineers,” at least as someone with an engineering education who isn’t doing engineering work really: we learned how to do particular types of problems very well and reliably, and generally learned a bunch of math tricks. But at a fundamental level the material that the physics students were learning was basically more complicated. Scientist has always felt like a more prestigious title to me. Since most programmers have computer science degrees, why don’t we call ourselves computer scientists?
- dang 4y ago
  People use 'engineer' differently in different parts of the world.
thomzi12 4y ago
Hey everyone, OP here -- thanks for reading or discussing. Curious if anyone else has switched from data science to software engineering, or has thoughts on comparing the two as a career choice. I'm all ears!
- kfk 4y ago
  I run a small data consulting company and whenever I find an open minded data scientist or analyst I tell them to consider data and software engineering or business analytics (learning about real business problems). DA, BA and SE roles are harder to find, manage more complexity, but most importantly are closer to having an impact as they are the ones close to prod. It’s a hard pitch, especially for juniors, AI and DS are very hyped but it’s hard to guess how much they will grow (or not). My current hunch is that AI and DS will be productized in libraries and services, so the majority of companies will not need people fully dedicated to build models.
  - thomzi12 4y ago
    Yep. In any case, building a model is a just part of the puzzle -- serving it in production is a whole other skillset that is very valuable. imho
- bobleeswagger 4y ago
  I have always been an engineer, but when I started as an intern in 2015, most of my work centered around helping analyses: cleaning up CSVs, data mining, minor processing and transformations, etc. None of it was connected to production but these were ad-hoc requests. I think this start was invaluable to my general philosophy today (devops or die), and believe there are a lot of missed opportunities identifying the right analysts that should take the next step into complexity and help craft better devops processes for analysts. They're developers too. One thing I don't think gets talked about enough: How much time analysts spend doing analysis vs. time spent dealing with data quality/architecture/process. So much analyst time is lost in the latter, which I think contributes greatly to burnout. But the growing intersection with engineering has and will continue to address this, albeit slowly.
- 4y ago
darth_avocado 4y ago
Sorry for the tangent, but medium is so bad. Can’t read content without installing the app or logging in. Part of the reason I stopped writing on medium and jumped to Substack.
- thomzi12 4y ago
  Oh, I'm sorry about that! Here's a friend link that should allow you to bypass any paywall (if you've already read a couple paywalled posts): https://zachary-thomas.medium.com/how-i-became-a-software-engineer-after-5-years-in-data-science-e0c56b872257?sk=67f653a7329e8894f2ea2fc22d032d0d https://zachary-thomas.medium.com/how-i-became-a-software-en... Here's an earlier draft of the post on Quip: https://gstudent.quip.com/ILF1ABpvjsSh/Can-a-Data-Scientist-become-a-Software-Engineer https://gstudent.quip.com/ILF1ABpvjsSh/Can-a-Data-Scientist-...
  - darth_avocado 4y ago
    Thank you for sharing!
cletus 4y ago
I will add that certain SWE fields rely upon and interact heavily with DS and for anyone wanting to make the switch, this is a natural "in". Any company with a rigorous approach to product development or even SRE will rely heavily on metrics. The ability to create, maintain, monitor, transform, understand and reason about those metrics is invaluable. To decide what to work on and the quantify your contribution, the ability to effectively run experiments is invaluable. This will naturally fit into a structured product management cycle. This will be a way more natural fit than, say, "purer" software development (eg writing device drivers). So much of successful software engineering is not just how to do something but deciding what to do (and not do).
- thomzi12 4y ago
  Yes -- one thing I struggle with is choosing whether to work on data engineering projects (which I know I'm good at and can provide immediate value) or challenging myself with more "traditional" full-stack engineering work where I can learn a lot. Maybe a 60/40 mix is optimal
b33j0r 4y ago
I came of age in the industry when there was little distinction (python guy since 2004 or so). That provides many pros and cons. The more you know broadly, the smarter you’ll sound. And generally, you will tend to be more useful and fulfilled. If you want a career that is focused, you have to choose a focus. I’ve struggled with that. You need a broad base of knowledge to be good at understanding all things, and how they connect. But eventually someone needs a lightbulb changed, and I’m testing the grounding of a nearby lightswitch while on the phone with a contractor about shielding noise from the antiquated home stereo system. Edit: I should probably mention explicitly. Investing in TensorFlow or CUDA is one thing. Investing in certain ML techniques or modeling domains another—and python yet another— Systems and general engineering is a completely messier enchilada. You still get paid! So. I guess. Do better than I. Choose a scale. It’s impossible to focus on everything unless that’s just an impulse you’ve got :)
- thomzi12 4y ago
  Yeah that's fair -- maybe just focusing on becoming an expert in data and ML engineering in the long-run is a more prudent career choice than totally dropping those for another corner of engineering. There's some wisdom here.
  - b33j0r 4y ago
    Don’t take it as discouragement from being a generalist, in er… general. But strike where the iron is hot. Right now the future is ML, not learning the intricacies of the 2022 tech stack. Heck, I’m an automation enthusiast, but if that’s my only trick and I’m good at it, I’m out of work like next week ;)
OhNoNotAgain_99 4y ago
[dead]
deleted 4y ago
[deleted]
carabiner 4y ago
I was hoping for more advice for those pursuing this transition, but it was more just an account of Google's incredible support for his wish to change careers. He never mentions his employer by name, but I think it'd be helpful to point this out. Those of us on the outside (I and a few other DS's I know are doing this) are spending 6 months or more grinding leetcode and studying system design, doing mock interviews, and applying to hundreds of jobs. So it can be much more painful if you don't have an employer that holds your hand (and pays you) through this entire transition.
- legerdemain 4y ago
  ME (at not-Google): I wanna do a 12-week dev bootcamp! MY LEAD: Uhhhh... What if it was two weeks long, tops? Or even less? MY ASSIGNED HR PERSON: Great, let's process your leave of absence! We'll cut your benefits and convert your stock options so you'll have to buy them out a lot sooner than you were prepared to!
  - thomzi12 4y ago
    Taking that much time off of any job is pretty hard to do! The real goal of a bootcamp though (besides networking) is adding a project to your portfolio. You can then show off this project to hiring managers to demonstrate your competence. You don't need to do a full-time bootcamp to make this happen! I just checked and Flatiron school and Hack Reactor both have online part-time full-stack bootcamps. Also, while not as big of an investment, online platforms like Codecademy have career paths with a capstone project you can complete and add to your portfolio. So, overall there are a variety of pathways to do this. Totally fair to point out though a bootcamp of some kind is a nice option in the off-chance it's a possibility.
    - legerdemain 4y ago
      I didn't end up doing a bootcamp in the end. I sort of wish I did: friends of mine who did ended up with some strong connections as a result. Then again, someone else I know had an extremely negative experience with Hack Reactor and lost thousands of dollars in it, so maybe it's for the best that I didn't go.

47 points by thomzi12 4y ago

Unless studied software engineering, you can switch to become an "engineer"? Wouldn't be software developer term more fitting?

mythhouse 4y ago
oh no not this tired tangent again.
- thinking4real 4y ago
  The ramblings of someone who’s low skill and low ambition usually
  - gjvc 4y ago
    it's not clear to whom you are referring
    - melony 4y ago
      People obsessed with protecting the term “engineer” are usually those who want to gatekeep their jobs and salary. In other words, they are worried about competition and want to keep the career “prestigious” and kick the ladder away.
      - bobleeswagger 4y ago
        It is a bit facetious to say that all analysts are engineers, or that it is "easy" for any analyst to become an engineer. There is overlap, but they are not identical skillsets. Analysts are certainly developers, but engineering involves more of a sense of scalability, analysis can do this, but it is generally more granular.
      - bee_rider 4y ago
        I’m under the impression that programmers’ salary progression typically compares quite favorably to classical engineers’.
thomzi12 4y ago
Sure! My company calls the role "software engineer" so that's how I describe the role but I think developer is also a good word for it
myth_drannon 4y ago
We also have support engineers with only highschool diploma! And account managers...
Bingaloopa 4y ago
Software Engineer and Software Developer are used interchangeably for the most part and I don't think you can really get any meaningful distinction just by the name (in most countries at least). Some countries (Canada I think being one?) may require you be licensed/registered with an engineering body before you're allowed to call yourself an engineer, but thats not the case in most countries AFAIK
- Waterluvian 4y ago
  Indeed. I worked at a Canadian company that got a legal nastygram from the professional association saying to stop calling us engineers. Then I started working for an American company that was like shrug no laws against that. I just can’t say professional engineer. The whole thing makes a ton of sense when we’re building bridges or airplane safety rated computer systems, but not so much when building websites or whatnot.
  - eddsh1994 4y ago
    I work on critical software and have the 'engineer' label, but do have a nagging feeling it's less structured than 'normal' engineering disciplines. I'm taking a part-time BEng to see if that's the case though!
  - Sephr 4y ago
    You can definitely call yourself a professional software engineer in the US.
halfmatthalfcat 4y ago
The US doesn't have software "engineering" degrees, so how does one exactly study software engineering? Computer science, too, is not "engineering".
- eddsh1994 4y ago
  The UK does, you can get chartered engineer status with a BSc & MSc in Software Engineering or Computer Science if it's applied enough with the IET. When I looked at the US i'm 99% sure I found similar degrees in Software Engineering too.
  - nkingsy 4y ago
    I’ve never seen a “software engineering” degree on a resume as an interviewer in the us.
    - eddsh1994 4y ago
      Fwiw, I have. They're becoming more common - and probably a good thing too, covering important non-CS skills like requirements engineering that are mostly made up in small tech companies without that experience.
- uxp100 4y ago
  It does…
  - geethree 4y ago
    NCEES even offered a Professional Engineering license in it for awhile. Since discontinued due to lack of interest. https://ncees.org/ncees-discontinuing-pe-software-engineering-exam/ https://ncees.org/ncees-discontinuing-pe-software-engineerin...
- DavidMiserak 4y ago
  There are colleges and universities in the USA that offer SE degrees, CMU is of note[0] [0] https://www.ece.cmu.edu/academics/ms-se/index.html https://www.ece.cmu.edu/academics/ms-se/index.html
echelon 4y ago
If you do any sort of measurement, analysis, or design before implementing a system, you're leveraging engineering principles. Examples: measuring data scale for growth rate and future needs, calculating incremental costs of cloud instances and data stores, designing and proving a state machine, instrumenting systems, measuring and stress testing a system against load, understanding back pressure and dynamical behaviors of distributed systems, designing threaded or active/active systems, vector clocks, consensus algoritms, test suites, etc. etc. I've had to write short inductive proofs in several of the systems I've built. It's engineering. The sooner we get over the imposter syndrome debate of what we can and cannot call ourselves and embrace the full scope and possibilty of what we can achieve, the sooner we can become better and more capable software engineers.
- angrais 4y ago
  >> I've had to write short inductive proofs in several of the systems I've built. In what context was this necessary?
  - recursive 4y ago
    To provide evidence that a recursive algorithm actually works maybe?
  - eddsh1994 4y ago
    This is pretty common in the Blockchain space to verify transactions behave correctly when arriving out of sync with tools like Agda, Coq, etc... I assume it's the same on databases & I heard Leslie Lamport give a talk at work where he mentioned AWS used TLA+ to prove some of its properties.
cellis 4y ago
Upvoted not because I agree, but because I'm genuinely interested in the discussion. What is the difference in your opinion?
- thundergolfer 4y ago
  This only opinion piece on the issue worth reading, because it's the only one that bothered to seek the opinions of the people that would know best: those people that have been software engineers _and_ some other kind of engineer. https://www.hillelwayne.com/post/are-we-really-engineers/ https://www.hillelwayne.com/post/are-we-really-engineers/
  - wheelinsupial 4y ago
    I think I'm missing something, but why is an opinion piece needed on this issue? Software engineering is a licensed engineering profession in many places. It used to be possible to be licensed as a software engineering in the United States [1, 2](little demand caused the licensing exams to be discontinued, possibly because regulation wasn't enforced, according to [2]). So, I think it's fair to say enough of the engineering profession has spoken to say software is a branch of engineering. What I think are the relevant extracts from the article: > “Do you consider software engineering actually engineering?” > Of the 17 crossovers I talked to, 15 said yes. > That said, many of the crossovers [3] also added an additional qualification: software engineering is real engineering, but a lot of people who write software aren’t doing software engineering. This is not a problem with them, rather a problem with our field: we don’t have a rich enough vocabulary to talk about what these developers do. The engineering field has developed a vocabulary for different professionals in the field. Why can't this be applied to the software field since it is a form of engineering? There are engineers, technologists, technicians, and trades (at least where I am). The issue is there are education and work experience requirements that lead to licensing, which people will say are inequitable and gatekeeping [4]. From 2, there was an experience-only path available to get software engineering licensure, so if the field really wants it, there seems to be precedent to allow for alternative routes into those titles. [1] https://ncees.org/engineering/pe/software/ https://ncees.org/engineering/pe/software/ [2] https://www.nspe.org/resources/pe-magazine/may-2018/ncees-ends-software-engineering-pe-exam https://www.nspe.org/resources/pe-magazine/may-2018/ncees-en... [3] The definition of "crossovers:" "people who used to be professional engineers and then became professional software developers. I call these people crossovers, hybrids between the two worlds." [4] I'm sympathetic to these concerns. I studied engineering technology and worked as a technologist at the beginning of my career. I wanted to become a mechanical engineer, but I couldn't afford to return to school full time to complete the remaining 2 years for the BEng. (My school offered a 2 year diploma for engineering technologists, which led directly to years 3 and 4 year of the BEng.) When I wanted to complete the degree, a full time course load was required (7 courses per term at my school) for the engineering program to maintain accreditation.
switchbak 4y ago
I've worked with a bunch of engineers, and while I respect and admire them tremendously, I'd not want to be lumped in with the kind of code they generally write. And don't get me started on the typical code a scientist writes! Let the downvotes commence. Edit: This is a gentle ribbing, but there really does seem to be a different approach in the philosophy of how to solve a problem. Engineers seem to eschew abstraction while software folks embrace is (sometimes to a fault). It's actually pretty fascinating.
- nicce 4y ago
  Very odd. Usually engineers love abstraction where I am coming from. The simpler the solution, better it usually is. Helps understanding the whole system, making it less complex. Simplicity causes less bugs. Of course, you can do too much abstraction.
  - cscurmudgeon 4y ago
    Have you never run into Matlab code with a few hundred global variables mutating frequently? :)
    - nicce 4y ago
      Okay. I was referreing into software engineers, not data engineers :-)
      - cscurmudgeon 4y ago
        Ah, I was referring to general engineers, like chemical or mech engineers :)
- bee_rider 4y ago
  If they wanted to built their own abstractions they’d be scientists. Actually this is one thing that has always confused me about “software engineers,” at least as someone with an engineering education who isn’t doing engineering work really: we learned how to do particular types of problems very well and reliably, and generally learned a bunch of math tricks. But at a fundamental level the material that the physics students were learning was basically more complicated. Scientist has always felt like a more prestigious title to me. Since most programmers have computer science degrees, why don’t we call ourselves computer scientists?
dang 4y ago
People use 'engineer' differently in different parts of the world.

thomzi12 4y ago

Hey everyone, OP here -- thanks for reading or discussing. Curious if anyone else has switched from data science to software engineering, or has thoughts on comparing the two as a career choice. I'm all ears!

kfk 4y ago
I run a small data consulting company and whenever I find an open minded data scientist or analyst I tell them to consider data and software engineering or business analytics (learning about real business problems). DA, BA and SE roles are harder to find, manage more complexity, but most importantly are closer to having an impact as they are the ones close to prod. It’s a hard pitch, especially for juniors, AI and DS are very hyped but it’s hard to guess how much they will grow (or not). My current hunch is that AI and DS will be productized in libraries and services, so the majority of companies will not need people fully dedicated to build models.
- thomzi12 4y ago
  Yep. In any case, building a model is a just part of the puzzle -- serving it in production is a whole other skillset that is very valuable. imho
bobleeswagger 4y ago
I have always been an engineer, but when I started as an intern in 2015, most of my work centered around helping analyses: cleaning up CSVs, data mining, minor processing and transformations, etc. None of it was connected to production but these were ad-hoc requests. I think this start was invaluable to my general philosophy today (devops or die), and believe there are a lot of missed opportunities identifying the right analysts that should take the next step into complexity and help craft better devops processes for analysts. They're developers too. One thing I don't think gets talked about enough: How much time analysts spend doing analysis vs. time spent dealing with data quality/architecture/process. So much analyst time is lost in the latter, which I think contributes greatly to burnout. But the growing intersection with engineering has and will continue to address this, albeit slowly.
4y ago

darth_avocado 4y ago

Sorry for the tangent, but medium is so bad. Can’t read content without installing the app or logging in. Part of the reason I stopped writing on medium and jumped to Substack.

thomzi12 4y ago
Oh, I'm sorry about that! Here's a friend link that should allow you to bypass any paywall (if you've already read a couple paywalled posts): https://zachary-thomas.medium.com/how-i-became-a-software-engineer-after-5-years-in-data-science-e0c56b872257?sk=67f653a7329e8894f2ea2fc22d032d0d https://zachary-thomas.medium.com/how-i-became-a-software-en... Here's an earlier draft of the post on Quip: https://gstudent.quip.com/ILF1ABpvjsSh/Can-a-Data-Scientist-become-a-Software-Engineer https://gstudent.quip.com/ILF1ABpvjsSh/Can-a-Data-Scientist-...
- darth_avocado 4y ago
  Thank you for sharing!

cletus 4y ago

I will add that certain SWE fields rely upon and interact heavily with DS and for anyone wanting to make the switch, this is a natural "in". Any company with a rigorous approach to product development or even SRE will rely heavily on metrics. The ability to create, maintain, monitor, transform, understand and reason about those metrics is invaluable. To decide what to work on and the quantify your contribution, the ability to effectively run experiments is invaluable. This will naturally fit into a structured product management cycle. This will be a way more natural fit than, say, "purer" software development (eg writing device drivers). So much of successful software engineering is not just how to do something but deciding what to do (and not do).

thomzi12 4y ago
Yes -- one thing I struggle with is choosing whether to work on data engineering projects (which I know I'm good at and can provide immediate value) or challenging myself with more "traditional" full-stack engineering work where I can learn a lot. Maybe a 60/40 mix is optimal

b33j0r 4y ago

I came of age in the industry when there was little distinction (python guy since 2004 or so). That provides many pros and cons. The more you know broadly, the smarter you’ll sound. And generally, you will tend to be more useful and fulfilled. If you want a career that is focused, you have to choose a focus. I’ve struggled with that. You need a broad base of knowledge to be good at understanding all things, and how they connect. But eventually someone needs a lightbulb changed, and I’m testing the grounding of a nearby lightswitch while on the phone with a contractor about shielding noise from the antiquated home stereo system. Edit: I should probably mention explicitly. Investing in TensorFlow or CUDA is one thing. Investing in certain ML techniques or modeling domains another—and python yet another— Systems and general engineering is a completely messier enchilada. You still get paid! So. I guess. Do better than I. Choose a scale. It’s impossible to focus on everything unless that’s just an impulse you’ve got :)

thomzi12 4y ago
Yeah that's fair -- maybe just focusing on becoming an expert in data and ML engineering in the long-run is a more prudent career choice than totally dropping those for another corner of engineering. There's some wisdom here.
- b33j0r 4y ago
  Don’t take it as discouragement from being a generalist, in er… general. But strike where the iron is hot. Right now the future is ML, not learning the intricacies of the 2022 tech stack. Heck, I’m an automation enthusiast, but if that’s my only trick and I’m good at it, I’m out of work like next week ;)

OhNoNotAgain_99 4y ago

[dead]

deleted 4y ago

[deleted]

carabiner 4y ago

I was hoping for more advice for those pursuing this transition, but it was more just an account of Google's incredible support for his wish to change careers. He never mentions his employer by name, but I think it'd be helpful to point this out. Those of us on the outside (I and a few other DS's I know are doing this) are spending 6 months or more grinding leetcode and studying system design, doing mock interviews, and applying to hundreds of jobs. So it can be much more painful if you don't have an employer that holds your hand (and pays you) through this entire transition.

legerdemain 4y ago
ME (at not-Google): I wanna do a 12-week dev bootcamp! MY LEAD: Uhhhh... What if it was two weeks long, tops? Or even less? MY ASSIGNED HR PERSON: Great, let's process your leave of absence! We'll cut your benefits and convert your stock options so you'll have to buy them out a lot sooner than you were prepared to!
- thomzi12 4y ago
  Taking that much time off of any job is pretty hard to do! The real goal of a bootcamp though (besides networking) is adding a project to your portfolio. You can then show off this project to hiring managers to demonstrate your competence. You don't need to do a full-time bootcamp to make this happen! I just checked and Flatiron school and Hack Reactor both have online part-time full-stack bootcamps. Also, while not as big of an investment, online platforms like Codecademy have career paths with a capstone project you can complete and add to your portfolio. So, overall there are a variety of pathways to do this. Totally fair to point out though a bootcamp of some kind is a nice option in the off-chance it's a possibility.
  - legerdemain 4y ago
    I didn't end up doing a bootcamp in the end. I sort of wish I did: friends of mine who did ended up with some strong connections as a result. Then again, someone else I know had an extremely negative experience with Hack Reactor and lost thousands of dollars in it, so maybe it's for the best that I didn't go.