9 ms·
Relevant HN comment: https://news.ycombinator.com/item?id=18463181 https://news.ycombinator.com/item?id=18463181 Apparently their software development process
by nn3 3y ago
Relevant HN comment:
https://news.ycombinator.com/item?id=18463181 https://news.ycombinator.com/item?id=18463181
Apparently their software development processes are terrible.
- 1123581321 3y agoFrom that comment I concluded that software patching speed isn’t a bottleneck there. It sounds frustrating but to be terrible it needs to significantly impede the product or business relationships, which isn’t at all happening.
- away271828 3y agoStability and long product lifecycles are incredibly important to them. They're pretty much the opposite of move fast and break things or deploy early/deploy often.
- hcfman 3y agoTry searching how much money is lost by clients if the fab is down. You have to be very careful with software for fabs.
- ghaff 3y agoAbsolutely. What makes a lot of sense for one organization would be absolutely insane for others. ASML has incredible innovation in some areas but I'd expect them to prioritize stability in others.
- awesomeMilou 3y agoHonestly, if you think about the availability guarantees that these FAB's have to deliver, I'm kinda surprised they don't run off of IBM mainframes. Can't believe it but it would be a perfect use case. High bandwidth I/O, paired with ungodly, over-the-horizon-radar like availability.
- konschubert 3y agoI find it strange that they are making so few machines. The price would probably drop slightly if they made 10x more machines, but they would still end up earning more. And the world would be a better place, too. So I am left wondering ... what is capping their output? What is the bottleneck?
- 1123581321 3y agoMy understanding is that production and installation do not scale since the machines are essentially replications of a successful physics experiment environment. Manufacture is a long, multi-step process requiring technical expertise (which is limited). Then, machines are re-assembled at the customer site--again, carefully and with a large technical staff. To commit engineering to achieving 10x manufacture rate would probably require giving up the lead on next generation processes. Personally, I think there must be an end to this. There are steps we are going to learn to more reliably replicate, and reliably reassemble. Machines won't become smaller, but other heavy industries have learned to do faster, de-skilled installations so semiconductors should eventually as well. However, I don't think we'll get the 'good for the world' benefit because by the time every country can make leading edge semis, some other critical, supply-restricted part will become what's contested.
- konschubert 3y agoYou might be right. On the absolute forefront, it has to be artisanal. But we can learn to scale making the machines that were cutting edge two years ago. Maybe ASML just doesn’t do this because they leave it to other companies? Either way, somebody should be the one that’s always two years behind but makes 2000 machines a year. And of course, when you remove a bottleneck, you find another. But that other bottleneck will be wider.
- brnt 3y agoAsml does this, learning how to scale the cutting edge of two years ago. This is what scaling cutting edge physics looks like. Academic groups sometimes take the better part of a decade (or more) to replicate a setup by another group. There are few components you can buy off the shelf in this space: you'll have to understand, design and make them all yourself. Market's way too small to suppory competing suppliers. Takes longer than two years to farm all of that out and create a market of sufficient size. Plus, buyers tend to not want (or pay) for one or two decade old stuff, so that market (for part suppliers) may never quite materialise. And then, these machine are indeed more like physics experiments than a device. If you compare it with that, ASML is actually quite fast. And the market knows all of this, hence their valuation.
- olejorgenb 3y agoLinked comment is from 2018. Not really that old on this context though.
- hcfman 3y agoExcept that link is to a story with a lot of garbage in it.
- ziddoap 3y agoDo you have other insight? A counterpoint? Anecdote that supports a different narrative? Anything, at all, that would contribute to the conversation and support your "garbage" comment?
- seper8 3y agoIt's not accurate as I said I other comment. Software is tested not just on the machine nowadays. I think you'll find most asml employees hesitant to share more, I'm not sure what I can say either.
- skrebbel 3y agoHcfman works at ASML and I (who wrote the comment back in 2018) don’t. Fair chance he knows what he’s talking about.
- hcfman 3y agoOkay I'll bite. Usually with large and important companies like ASML one has to be very careful with what you say. Usually it's better to say nothing at all. This time I felt I had to jump in and defend ASML as it was painting rather a misrepresentational picture. First, I'd like to say that skrebbel has reacted very gentlemanly to my comment considering how hard hitting it was, my complements to his balance in this. I can only assume he may possibly have been burnt on something he put a lot of effort into, I can relate to that. I've been there too. But ASML is a very large company and parts of it that I have worked with have very impressive testing capability with a large farm of servers testing all code against regression and progression before any integration with release code. I'm happy to say that I've personally improved parts of this farm in both quality and performance of the testing environment (even though I wasn't part of that group). The challenge is to be able to maintain and improve upon that when the software is so huge and growing. I poke and push at every opportunity I can. I'm sure that people in the company will recognise this :) On a personal note and I think a lot of people here can relate to this, you can do a lot for quality simply by recognising the efforts of people correctly, the improvement in motivation translates to improvements in quality. In that area I think things are indeed changing since 2018. Just last week I was very pleasantly surprised to see in a group close to mine the announcement of a new program to do just that. To recognise the efforts of people that go the extra mile in both effort and innovation. Very nice to see. Now it's best for me to shutup.
- newprint 3y agoLol, thank you for link. I was expecting the opposite from this company, but for some reason, after reading this, I'm not surprised at all.
- bee_rider 3y agoThey are physicists and hardware folks, right? I would expect messy code written by convoluted processes for nightmare applications, that somehow works.
- ethanbond 3y agoBro if they're not using Linear and having AI write their PRDs then they are Bad At Software.
- FirmwareBurner 3y ago>I was expecting the opposite from this company Why? Because they're hot on the stock market since the pandemic? Nokia's SW was the same kind of shitshow when it was dominant. IIRC a long time ago someone on HN wrote here that compiling Symbian OS at Nokia took 2 whole days and Nokia management saw no problem with that. To me, it's exactly what I expect from a HW company, from personal experience. SW is seen just as a necessary evil, another item on the BoM. Oh, and there's a bunch of useless processes designed as jobs programs to keep some useless managers employed, where each of them needs to review your change and give their green light despite them not being up to date on the technical side for >10 years. I know this because it's exactly the same at another major Dutch semiconductor spin-off from Philipps I was at in a past life. Just because ASML is hot right now, doesn't mean they value and employ top SW talent, because they don't sell SW, they sell HW and that's what their customers value, and so ASML values physicists and traditional engineers, not SW devs.
- hef19898 3y agoSo, they do everything right? Because, if your product is HW, the SW needed to run it is just another BoM item.
- 3y ago
- svilen_dobrev 3y agoi advised one of their software-making vendors on software-organisation+culture and similars, about 10y ago. It went pretty well, but the funny thing was, The tools and environment they had to use (because of ASML) to check/test stuff, was same as it was in Motorola 25 years ago (Sun CDE /Common Desktop Env? something like that). I felt like dropped in a time machine :/
- jessriedel 3y agoTbc, based on the comment that looks to be driven almost completely by the fact that they are making a tiny number of incredibly expensive bespoke machines, so they can't just keep extra machines lying around to do unit tests with (both because each machine is expensive, and because the results wouldn't necessarily generalize to other machines). So it sounds like a constraint of the field, not bad choices.
- nextaccountic 3y agoThe build taking 8 hours doesn't seem to be consequence of the field they are operating
- jessriedel 3y agoThat sounds right, although I'd want to know a lot more before judging
- FirmwareBurner 3y agoWhy do you need the actual production machine for unit tests? Why can't you build a SW simulator for them and run unit tests on them? I used to work in automotive and all unit tests didn't require the finished car, just the SW. Nasa doesn't hava another Voyager probe in their lab floating around in zero gravity to run unit tests on before sending the SW patches, they use simulators. For calibration you do need the final production HW, but unit tests shouldn't, so maybe there's a confusion here about the type of tests ran.
- gowings97 3y agoto do finite element analysis on the (nanometer) scale of processes/manufacturing they do - heh.
- jessriedel 3y ago> Nasa doesn't hava another Voyager probe in their lab floating around in zero gravity to run unit tests on before sending the SW patches, they use simulators. JPL keeps a working duplicate of their Mars rovers on Earth and tests software updates there. So if the Voyager zero-gee environment prevents a useful duplicate, that means testing software in simulation is considered inferior to using a (massively expensive) physical duplicate like for the Mars robers. https://mars.nasa.gov/news/8749/nasa-readies-perseverance-mars-rovers-earthly-twin/ https://mars.nasa.gov/news/8749/nasa-readies-perseverance-ma...
- karmasimida 3y agoI don't think so. Any update that is as complex, expensive and critical, to equipments such as ASML's, should be carefully examined to the teeth.
- lnxg33k1 3y agoI've lived in NL for a while, few years, changed few companies, it's a place where processes are more important than actual code quality, where people stay in the same job for 10-15 years and everything below that is seen as a bad thing, people is hired for the culture primarily, having awareness of software principles is really worthless. I ran away as the last company I interviewed with, after solving a coding puzzle in 30 seconds, finishing in 15 mins something which had to take 45 mins, asked "Are you available to come at the office on Friday for company parties?". So I can only have nightmares about software practices in NL companies I've seen files with 13k lines of if/else/switch, how do you test that shit .-.
- konschubert 3y agoI thought that the dutch would be more ... modern?
- lnxg33k1 3y agoI don't know, I've worked in Germany, but only for US companies, so I can't speak for proper German company software development culture :(
- Tarq0n 3y agoIt's a pretty large gap between enterprise organizations that have lost the capability to do anything but buy mostly terrible software in the open market and organizations that maintain some amount of ability to make things. If you're at one of the former, expect to be locked into a digital landscape that's decades out of date.
- DJHenk 3y ago> I thought that the dutch would be more ... modern? As a Dutchman: no, we are not when it comes to businesses software. Dutch companies are very conservative and always choose the safe option. They are modern in that they updated the slogan 'Nobody ever got fired for choosing IBM' to the current century. It is the main reason that I have only worked at companies with a healthy mix of different nationalities for the last ten years. Doing software in a Dutch monoculture is suffocating.
- seper8 3y agoThey don't just test software on the machines. Inaccurate comment at least in 2024.
- hcfman 3y agoThe comments in that post are full of untruths. ASML is a huge company so no one person can talk about all of it. As with all large companies there will be great bits and bits that can be done better and the larger the company gets and the more critical the software is the slower some parts will be done. There's always room for improvement. And ASML has improved immensely over the last 10 years. In the teams that I've worked in the caliber of the people has been extremely high. The code reviews are rigorous and that's a good thing. There are a lot of extremely smart people working for ASML. To call it Dutch is interesting, most of the teams I've worked in are international with less Dutch than foreign people in them. ASML has changed from a much less interesting company when I joined to a very interesting company with respect to the software stack. Yes, I believe there's still lots of room for improvement but that's the case with all big companies. But heck, I don't know about everything, I've only worked there 12 years.
- seper8 3y agoYeah there are a lot of silos. Partly that's by design, partly that's a an arguably bad part of it's culture. I'm in a position where I can see a lot of them work in isolation, all choosing to use their own tooling: programming languages, frameworks ides, version control systems... Indeed no one team works quite like the other.
- themerone 3y agoI expect that if their costumers had to choose between ASML keeping machines around for software testing or shipping every machine the can build, they would want the hardware.
- yau8edq12i 3y agoI love these threads. Everyone will chime in about how they know better how to run the company, with suggestions on the processes that they don't know and so on. Meanwhile the story is "business is #1 and booming". Elegant software stacks aren't always the most important part of the business.
- leokennis 3y agoExactly. ASML made a profit of about $2,050,000,000,- in 2023. I doubt they are very interested in complaints about how some random dude expected more time to run his tests.
- deleted 3y ago[deleted]
- random_walker 3y agoI think they are pretty serious about high quality software development. There are often questions and concerns from (tech/non-tech) management about build times, compute costs, software product’s performance etc. PS:I work at ASML.
- random_walker 3y agoExactly, I guess that's because HN has quite software-heavy crowd. Most of them like to see software changing the world.
- krapp 3y agoNo, most of them want to make a ton of money. They're hanging out virtue signaling their cleverness on a forum owned by a Silicon Valley startup incubator for a reason.
- random_walker 3y agoYou said it better than I could have :)
- midasz 3y agoThere's not going to much meaningful pushback on this comment because any (ex-)ASML employee is going to be super hesitant to share anything. So feel free to think what you want.
- gostsamo 3y agoFormer employee of an ASML contractor and the software is terrible. Here, you have it.
- skrebbel 3y agoAre you sure? Every ASML person I know loves sharing these stories at parties. It’s not a secret or anything. Stock wont go down if people find out that the code is a mess, everybody who cares about this already knows it. ASML is impressive because they managed to ship despite the code having been a mess. It’s a mad accomplishment of both engineering and organization (they solve the messy code problem by simply employing 10x the programmers than they might need if the code was better, and somehow that actually works! That’s impressive organization, cause conventional wisdom eg Mythical Man Month suggests you can’t do that) I think it’s more that not many ASML people are on HN.
- midasz 3y agoNo I'm not sure and maybe it's just me. Also but of a difference between a party and posts on the internet imo.
- 3y ago
- skrebbel 3y agoNote that I wrote that in 2018 based on hearsay from well before that (I live in the area and have friends who worked at ASML at the time). Things might’ve changed in the ~10 years since then (for better or worse!). EDIT: there’s a commenter named hcfman here in the thread who works at ASML and says that my comment is garbage (which I take to mean that it’s way outdated). Consider updating your impression accordingly, I see no reason to doubt what they’re saying. Last I checked ASML really did want to improve the software situation, looks like they did.