6 ms·

Why is "hard tech" == "low-level"? There are plenty of interesting and challenging technical problems that are "higher level" but more difficult due to scale/la

Why is "hard tech" == "low-level"? There are plenty of interesting and challenging technical problems that are "higher level" but more difficult due to scale/latency/etc demands.

Cyph0n 8y ago
Besides, DMA isn’t really a “hard” problem as far as I am aware...
- deanmoriarty 8y ago
  https://baus.net/you-cant-impress-developers/ https://baus.net/you-cant-impress-developers/
  - Cyph0n 8y ago
    Oh, so now we need to feign being impressed so no one gets their feelings hurt? The fact of the matter is, were DMA indeed a hard problem, I would not have posted the comment. So instead of posting a link to a blog post, perhaps you could try to disprove my claim?
    - deanmoriarty 8y ago
      In my experience, if you think that implementing from scratch a DMA solution is not hard, then you are an extraordinary talented engineer, far above the level you find at FAANG, so your opinion on this matter is not terribly relevant. It would be as if Jeff Dean came and said writing MapReduce is not a hard task. Nice, but not relevant for the 99.9999% of the talent pool. I got a couple offers over the years for L5-L6 positions at FAANG (specifically Google and Facebook) to work on low level stuff, including system programming and kernel development, since I have a few dozens upstream Linux kernel contributions in some subsystems I dabbled with. I also had to deal with writing a video4linux custom driver for an embedded industrial machine and a very custom acquisition card using DMA in the past, and it was significantly difficult even to just make it work playing with the well-defined kernel DMA APIs. I can easily see how the complexity required to write the entire kernel plumbing framework that makes DMA possible to leverage for normal developers like me would be a hard engineering task, definitely one that deserves an L7 architect/engineer capable of coming up with the right level of abstractions, incredibly important for a C kernel API. Just because one can explain on a whiteboard what DMA is and how it is supposed to work by remembering the OS college class (I'm not saying that's you), it doesn't make it an easy engineering task, engineering it's about making things work. I'm sure if you go back to the LKML mailing list and look at the time period where DMA support was initially introduced in the kernel, you'll be surprised at the sheer engineering complexity that was discussed when introducing such core kernel feature. In conclusion, the fact that FB and GOOG wanted to hire me to work on stuff relatively close to what we're talking about, at a level that is considered senior, gives me some legitimacy to say that me thinking it's a hard problem isn't probably too far from the truth, otherwise we are somehow assuming that FAANG extends offers to incompetent people (multiple times), mistakenly considering them senior engineers. I take your original comment as naive as the people who say "I could have built Instagram/Whatsapp tech in a week!", because you either are a super expert, or you simply don’t know what you’re talking about. Perhaps you should consider getting an offer for $1M/y (or more! Since L7 tasks are easy for you maybe you can shoot for an L9 and ask $2M/y, I have a close friend at GOOG at that level who gets paid that much) there then, if you're not already in that elite band of compensation?
      - Cyph0n 8y ago
        Perhaps we’ve understood the original comment differently. I took it to mean writing a driver that uses an existing DMA API to talk to a controller. But yes, I agree: implementing such a DMA interface from scratch — even with an existing controller — is not a trivial problem. Testing and verifying said implementation would also be difficult. Well, I hope that one day I can make 1M/yr! For now, 3 months into my first job, I’m fairly satisfied with where I’m at :) I’m considering shifting into kernel development next. Given your experience, would you advise going down that career route? Any tips for a career in low-level development?
        deanmoriarty 8y ago
        In my opinion (which is not shared by many these days) understanding things at a reasonably low level is critical, I firmly believe that makes you a better programmer, even if you end up doing frontend development. My experience with the kernel has always been mostly tangential, typically a byproduct of what I do day to day, which is working at a SaaS startup at scale and wearing many hats, from backend development (Java/C++) to infrastructure management. As I find performance problems, many times I find myself digging into the kernel code and exploring parts of the networking stack (e.g. finding out the specific behavior of a socket option), virtual memory management (e.g. finding out some quirks about the page fault management system), ... Sometimes that leads to some small kernel patches and I've done several during the years, so that's typically my limit. The v4l driver I mentioned above was 6-7 years ago when I was in college and took a part time job working on that with a very talented team. Apparently FAANG likes those kind of background, especially if you can get into a more SRE-type role where they are actually not looking to quiz you only on dynamic programming algorithms, but also on your ability to reason with system design, low level systems and performance, at least that's my experience.
        josh2600 8y ago
        I am talking about doing a DMA interface from scratch.

Why is "hard tech" == "low-level"? There are plenty of interesting and challenging technical problems that are "higher level" but more difficult due to scale/latency/etc demands.

Cyph0n 8y ago

Besides, DMA isn’t really a “hard” problem as far as I am aware...

deanmoriarty 8y ago
https://baus.net/you-cant-impress-developers/ https://baus.net/you-cant-impress-developers/
- Cyph0n 8y ago
  Oh, so now we need to feign being impressed so no one gets their feelings hurt? The fact of the matter is, were DMA indeed a hard problem, I would not have posted the comment. So instead of posting a link to a blog post, perhaps you could try to disprove my claim?
  - deanmoriarty 8y ago
    In my experience, if you think that implementing from scratch a DMA solution is not hard, then you are an extraordinary talented engineer, far above the level you find at FAANG, so your opinion on this matter is not terribly relevant. It would be as if Jeff Dean came and said writing MapReduce is not a hard task. Nice, but not relevant for the 99.9999% of the talent pool. I got a couple offers over the years for L5-L6 positions at FAANG (specifically Google and Facebook) to work on low level stuff, including system programming and kernel development, since I have a few dozens upstream Linux kernel contributions in some subsystems I dabbled with. I also had to deal with writing a video4linux custom driver for an embedded industrial machine and a very custom acquisition card using DMA in the past, and it was significantly difficult even to just make it work playing with the well-defined kernel DMA APIs. I can easily see how the complexity required to write the entire kernel plumbing framework that makes DMA possible to leverage for normal developers like me would be a hard engineering task, definitely one that deserves an L7 architect/engineer capable of coming up with the right level of abstractions, incredibly important for a C kernel API. Just because one can explain on a whiteboard what DMA is and how it is supposed to work by remembering the OS college class (I'm not saying that's you), it doesn't make it an easy engineering task, engineering it's about making things work. I'm sure if you go back to the LKML mailing list and look at the time period where DMA support was initially introduced in the kernel, you'll be surprised at the sheer engineering complexity that was discussed when introducing such core kernel feature. In conclusion, the fact that FB and GOOG wanted to hire me to work on stuff relatively close to what we're talking about, at a level that is considered senior, gives me some legitimacy to say that me thinking it's a hard problem isn't probably too far from the truth, otherwise we are somehow assuming that FAANG extends offers to incompetent people (multiple times), mistakenly considering them senior engineers. I take your original comment as naive as the people who say "I could have built Instagram/Whatsapp tech in a week!", because you either are a super expert, or you simply don’t know what you’re talking about. Perhaps you should consider getting an offer for $1M/y (or more! Since L7 tasks are easy for you maybe you can shoot for an L9 and ask $2M/y, I have a close friend at GOOG at that level who gets paid that much) there then, if you're not already in that elite band of compensation?
    - Cyph0n 8y ago
      Perhaps we’ve understood the original comment differently. I took it to mean writing a driver that uses an existing DMA API to talk to a controller. But yes, I agree: implementing such a DMA interface from scratch — even with an existing controller — is not a trivial problem. Testing and verifying said implementation would also be difficult. Well, I hope that one day I can make 1M/yr! For now, 3 months into my first job, I’m fairly satisfied with where I’m at :) I’m considering shifting into kernel development next. Given your experience, would you advise going down that career route? Any tips for a career in low-level development?
      - deanmoriarty 8y ago
        In my opinion (which is not shared by many these days) understanding things at a reasonably low level is critical, I firmly believe that makes you a better programmer, even if you end up doing frontend development. My experience with the kernel has always been mostly tangential, typically a byproduct of what I do day to day, which is working at a SaaS startup at scale and wearing many hats, from backend development (Java/C++) to infrastructure management. As I find performance problems, many times I find myself digging into the kernel code and exploring parts of the networking stack (e.g. finding out the specific behavior of a socket option), virtual memory management (e.g. finding out some quirks about the page fault management system), ... Sometimes that leads to some small kernel patches and I've done several during the years, so that's typically my limit. The v4l driver I mentioned above was 6-7 years ago when I was in college and took a part time job working on that with a very talented team. Apparently FAANG likes those kind of background, especially if you can get into a more SRE-type role where they are actually not looking to quiz you only on dynamic programming algorithms, but also on your ability to reason with system design, low level systems and performance, at least that's my experience.
      - josh2600 8y ago
        I am talking about doing a DMA interface from scratch.