6 ms·

The best thing they could do to new PLC's would be to have the src code live alongside the firmware inside. So any new person would be able to open it and reaso

The best thing they could do to new PLC's would be to have the src code live alongside the firmware inside. So any new person would be able to open it and reason about it.

deleted 1mo ago
[deleted]
ssl-3 1mo ago
You mean like, just hang a thumb drive on a lanyard inside of the cabinet -- with source code and such? That will probably just disappear. Maybe with good intention or maybe with bad, but it's probably gone either way. If it doesn't disappear, then it's hanging right there for any of the competitors to use. That's a problem for the original installer's ongoing employment. And if it includes the programming software, then that lets Joe (from over in shipping) have a go at rejiggering the packaging machine. That's a problem for whoever has to pay someone with a clue to show up and fix it.
- skinfaxi 1mo ago
  > If it doesn't disappear, then it's hanging right there for any of the competitors to use. That's a problem for the original installer's ongoing employment. Is that really an issue? Do PLC vendors have concerns about other vendors turning up and yanking USB drives out of critical infrastructure with reckless abandon? Do the facilities themselves see no issue with potential vendors yanking operational utilities out of their systems while they are running?
  - ssl-3 1mo ago
    Eh? It's just a thumb drive, on a lanyard, hanging in a cabinet. It's not like it has guards posted. :) Maybe Joe (from shipping) took it back to his desk to have a squiz at it and try to passively learn something new during some otherwise-downtime and never returned it -- years ago. That cabinet is in his area, so he has the key. Maybe Tim (the plant electrician) borrowed it to get something else done, and it unintentionally disappeared somewhere along the way. Tim has all of the keys for all of the things. Maybe Emily (the ubergeek who prides herself on being able to code her way out of a mess on any system) had it in her office before she got hit by a car and lost the ability to communicate with polysyllabic words. She borrowed Joe's cabinet key after he mentioned an issue. Either way, it was just a thumb drive that had been inside of the cabinet, and now it is gone.
    - skinfaxi 1mo ago
      Things like thumb drives physically connected to cabinets can just disappear in your environment with no followup whatsoever? And that's regular, expected, and desired?
      - ssl-3 1mo ago
        So they conduct a formal investigation. Top Men descend upon the facility to figure out whether we blame Joe, or Tim, or Emily. Heads roll. And the thumb drive is still gone, isn't it? --- No, it's not regular. It's not desired. Does shit never go wrong in your world? If not, then: Perhaps you should start expecting it to. :)
      - lostlogin 1mo ago
        > Things like thumb drives physically connected to cabinets can just disappear in your environment with no followup whatsoever? Yes. I follow up. No one ever knows. > And that's regular, Yes > expected Yes > and desired? No.
  - kotaKat 1mo ago
    Hospital IT got a call asking where the little binder with the DVDs for the CT scanner were from someone in Radiology 'cause the Siemens tech showed up. No, we didn't have them, and they told us they last saw them in the back of a cabinet in their own work era. Turns out those were the unit-specific reload DVDs for half the reconstruction computers and some serial-specific license stuff. Whoopsie...
- procarch2019 1mo ago
  Even more so, if it’s just there you’re making it easy for the customer to find someone bidding less to execute new projects. Guess who they’re calling if they don’t have the plc project (or don’t understand they need it). Job security. Plus, yea, someone could really make a mess even if they have the best intention. A guy from my company almost uploaded the wrong program to a large bioreactor. At best it would have made the bioreactor just unusable until corrected. At worst it could have broken the equipment or caused a safety issue. This guy had been around the block a few times too. Luckily someone noticed before it was too late.
  - ssl-3 1mo ago
    I don't get into PLCs, per se, but I've done some stuff with other building-scale gear. Like: We took care of the centralized controls for the inmate portion of a jail that was built in the 1980s. We rejiggered the controls for that whole jail back around 2010 or so with new kit and had been taking care of it since then. All the wiring for all of the individual doors and intercoms was documented in one old 3-ring binder. (There were probably 3 copies originally -- for the architect, the maintenance room, and the equipment room, but just 1 remained and it had hand-written notes.) One day, around 2020-ish, they called us because the book was missing. I didn't have it (I never once left the equipment room with it), and the co-workers who had been there insisted that they didn't have it either. As time passed, we'd get occasional phone calls of "Hey, we really could use the book for our building back if you guys ever find it." They weren't angry calls, but sheesh: That book had real value and getting stuff done without it was difficult. So every time when they'd call again, I'd ask around our shop -- again -- if it had turned up anywhere. And then, finally: One of our guys cleaned out his truck and found it. He must have taken the book out to look at and have a smoke about some problem or other, and then it just stayed in his truck. For years. This shit happens. People are imperfect and sometimes we screw stuff up.
- palmotea 1mo ago
  > You mean like, just hang a thumb drive on a lanyard inside of the cabinet -- with source code and such? This guy is very happy when someone literally does that: https://www.youtube.com/@Cursed_Controls https://www.youtube.com/@Cursed_Controls (great channel btw), because usually he has to deal with worse. > And if it includes the programming software, then that lets Joe (from over in shipping) have a go at rejiggering the packaging machine. That's a problem for whoever has to pay someone with a clue to show up and fix it. You'd think Joe from shipping wouldn't be given a key to the cabinet with the PLC.
  - thyristan 1mo ago
    The PLC is usually in a control cabinet that can be opened by a control cabinet key: https://www.knipex.de/schaltschrankschluessel?v=2921 https://www.knipex.de/schaltschrankschluessel?v=2921 Real locks in such systems are rare, sometimes even disallowed. Usually, a lock with a real key is only used to temporarily prevent switching on a machine during maintenance or such.
    - palmotea 1mo ago
      > The PLC is usually in a control cabinet that can be opened by a control cabinet key: As far as I can tell, that thing's $50. Why would Joe from shipping have one? > Usually, a lock with a real key is only used to temporarily prevent switching on a machine during maintenance or such. Most locks aren't really meant to resist and adversary. They're meant to communicate who is allowed and who is not, and back that up by making violations a little non-trivial.
      - thyristan 1mo ago
        That's the first search engine hit I've pasted. The cheap ones that work just as well are single-digit prices. And usually there is one in every toolbox on the shop floor. Edit: if you need proof, 2 for 6.50€, shipping included: https://www.ebay.de/itm/277671545941 https://www.ebay.de/itm/277671545941
- b3orn 1mo ago
  If it's a custom industrial installation then the source code is part of the documentation just like the electrical schematics, specifically so that any trained electrician can maintain it. On the other hand Joe from shipping wouldn't even know what a PLC is and if you let untrained staff maintain electrical systems I think you have some more serious problems.
  - ssl-3 1mo ago
    > On the other hand Joe from shipping wouldn't even know what a PLC is Joe from shipping is undefined. It is possible that Joe knows more about PLCs than you think you know about shipping, and that he has a god complex worse than that of a grizzled Haskell programmer. People all over have jobs that don't align with their interests, education, and prior work experiences. That's pretty normal; the bills still need paid and people find work where they can. > if you let untrained staff maintain electrical systems I think you have some more serious problems. Or maybe Joe from shipping is a departmental director who has this role because he is a strong leader who takes the initiative to embrace and overcome new challenges. We don't know.
coryrc 1mo ago
Relatedly, I made an embedded product. I built a live CD (yes this was a while ago) for my employer with the entire development environment. As long as you can find an x86 machine with CD-ROM you could build, compile, test, and program using the exact environment I did twenty years ago. Maybe today I'd do micropython on ESP32. Download text file from device, edit, upload back on.
- aliasxneo 1mo ago
  There's at least one customer out there who has an RPi with NixOS on it from me. Probably still works to this day. It was the best use of NixOS I think I've ever found.
  - Kim_Bruning 1mo ago
    I've switched up to NUC-ish hardware which WAS cheap until recently. All the components just a bit higher quality than the lowest-bidder you get on the Pi. Run nixos on that and you've got a really solid platform.
  - rurban 1mo ago
    More probably worst use of NixOS. In embedded we care for every single kb, and don't have tolerance for this kind of space waste management system
    - voakbasda 1mo ago
      Embedded means different things. Personally I have a hard time calling anything that runs Linux as embedded, having cut my teeth on microcontrollers.
      - hiAndrewQuinn 1mo ago
        If it's a custom Yocto thing or it involved buildroot at any state of provisioning it gets the moniker "quasi-embedded" to me. Doubly so if it never actually connects to the internet, unlike these water controllers, apparently.
hommelix 1mo ago
> The best thing they could do to new PLC's would be to have the src code live alongside the firmware inside. So any new person would be able to open it and reason about it. This is already the case. Some PLC allows you to store and retrieve the source code of the programmed directly in the flash of the PLC. Sometimes this feature is behind a higher access level or password.
- NotMichaelBay 1mo ago
  Is it version controlled too?
KaiserPro 1mo ago
Nooooope nope nope. An entire class of controller with really shit security now (as in scanning a network with PLC controllers has a non-trivial chance of rebooting or freezing any number of devices on the network) trained to just plug USB sticks in? I understand the idea, and its laudable. But the entire model of PLC security needs root and branches overhaul.

The best thing they could do to new PLC's would be to have the src code live alongside the firmware inside. So any new person would be able to open it and reason about it.

deleted 1mo ago

[deleted]

ssl-3 1mo ago

You mean like, just hang a thumb drive on a lanyard inside of the cabinet -- with source code and such? That will probably just disappear. Maybe with good intention or maybe with bad, but it's probably gone either way. If it doesn't disappear, then it's hanging right there for any of the competitors to use. That's a problem for the original installer's ongoing employment. And if it includes the programming software, then that lets Joe (from over in shipping) have a go at rejiggering the packaging machine. That's a problem for whoever has to pay someone with a clue to show up and fix it.

skinfaxi 1mo ago
> If it doesn't disappear, then it's hanging right there for any of the competitors to use. That's a problem for the original installer's ongoing employment. Is that really an issue? Do PLC vendors have concerns about other vendors turning up and yanking USB drives out of critical infrastructure with reckless abandon? Do the facilities themselves see no issue with potential vendors yanking operational utilities out of their systems while they are running?
- ssl-3 1mo ago
  Eh? It's just a thumb drive, on a lanyard, hanging in a cabinet. It's not like it has guards posted. :) Maybe Joe (from shipping) took it back to his desk to have a squiz at it and try to passively learn something new during some otherwise-downtime and never returned it -- years ago. That cabinet is in his area, so he has the key. Maybe Tim (the plant electrician) borrowed it to get something else done, and it unintentionally disappeared somewhere along the way. Tim has all of the keys for all of the things. Maybe Emily (the ubergeek who prides herself on being able to code her way out of a mess on any system) had it in her office before she got hit by a car and lost the ability to communicate with polysyllabic words. She borrowed Joe's cabinet key after he mentioned an issue. Either way, it was just a thumb drive that had been inside of the cabinet, and now it is gone.
  - skinfaxi 1mo ago
    Things like thumb drives physically connected to cabinets can just disappear in your environment with no followup whatsoever? And that's regular, expected, and desired?
    - ssl-3 1mo ago
      So they conduct a formal investigation. Top Men descend upon the facility to figure out whether we blame Joe, or Tim, or Emily. Heads roll. And the thumb drive is still gone, isn't it? --- No, it's not regular. It's not desired. Does shit never go wrong in your world? If not, then: Perhaps you should start expecting it to. :)
    - lostlogin 1mo ago
      > Things like thumb drives physically connected to cabinets can just disappear in your environment with no followup whatsoever? Yes. I follow up. No one ever knows. > And that's regular, Yes > expected Yes > and desired? No.
- kotaKat 1mo ago
  Hospital IT got a call asking where the little binder with the DVDs for the CT scanner were from someone in Radiology 'cause the Siemens tech showed up. No, we didn't have them, and they told us they last saw them in the back of a cabinet in their own work era. Turns out those were the unit-specific reload DVDs for half the reconstruction computers and some serial-specific license stuff. Whoopsie...
procarch2019 1mo ago
Even more so, if it’s just there you’re making it easy for the customer to find someone bidding less to execute new projects. Guess who they’re calling if they don’t have the plc project (or don’t understand they need it). Job security. Plus, yea, someone could really make a mess even if they have the best intention. A guy from my company almost uploaded the wrong program to a large bioreactor. At best it would have made the bioreactor just unusable until corrected. At worst it could have broken the equipment or caused a safety issue. This guy had been around the block a few times too. Luckily someone noticed before it was too late.
- ssl-3 1mo ago
  I don't get into PLCs, per se, but I've done some stuff with other building-scale gear. Like: We took care of the centralized controls for the inmate portion of a jail that was built in the 1980s. We rejiggered the controls for that whole jail back around 2010 or so with new kit and had been taking care of it since then. All the wiring for all of the individual doors and intercoms was documented in one old 3-ring binder. (There were probably 3 copies originally -- for the architect, the maintenance room, and the equipment room, but just 1 remained and it had hand-written notes.) One day, around 2020-ish, they called us because the book was missing. I didn't have it (I never once left the equipment room with it), and the co-workers who had been there insisted that they didn't have it either. As time passed, we'd get occasional phone calls of "Hey, we really could use the book for our building back if you guys ever find it." They weren't angry calls, but sheesh: That book had real value and getting stuff done without it was difficult. So every time when they'd call again, I'd ask around our shop -- again -- if it had turned up anywhere. And then, finally: One of our guys cleaned out his truck and found it. He must have taken the book out to look at and have a smoke about some problem or other, and then it just stayed in his truck. For years. This shit happens. People are imperfect and sometimes we screw stuff up.
palmotea 1mo ago
> You mean like, just hang a thumb drive on a lanyard inside of the cabinet -- with source code and such? This guy is very happy when someone literally does that: https://www.youtube.com/@Cursed_Controls https://www.youtube.com/@Cursed_Controls (great channel btw), because usually he has to deal with worse. > And if it includes the programming software, then that lets Joe (from over in shipping) have a go at rejiggering the packaging machine. That's a problem for whoever has to pay someone with a clue to show up and fix it. You'd think Joe from shipping wouldn't be given a key to the cabinet with the PLC.
- thyristan 1mo ago
  The PLC is usually in a control cabinet that can be opened by a control cabinet key: https://www.knipex.de/schaltschrankschluessel?v=2921 https://www.knipex.de/schaltschrankschluessel?v=2921 Real locks in such systems are rare, sometimes even disallowed. Usually, a lock with a real key is only used to temporarily prevent switching on a machine during maintenance or such.
  - palmotea 1mo ago
    > The PLC is usually in a control cabinet that can be opened by a control cabinet key: As far as I can tell, that thing's $50. Why would Joe from shipping have one? > Usually, a lock with a real key is only used to temporarily prevent switching on a machine during maintenance or such. Most locks aren't really meant to resist and adversary. They're meant to communicate who is allowed and who is not, and back that up by making violations a little non-trivial.
    - thyristan 1mo ago
      That's the first search engine hit I've pasted. The cheap ones that work just as well are single-digit prices. And usually there is one in every toolbox on the shop floor. Edit: if you need proof, 2 for 6.50€, shipping included: https://www.ebay.de/itm/277671545941 https://www.ebay.de/itm/277671545941
b3orn 1mo ago
If it's a custom industrial installation then the source code is part of the documentation just like the electrical schematics, specifically so that any trained electrician can maintain it. On the other hand Joe from shipping wouldn't even know what a PLC is and if you let untrained staff maintain electrical systems I think you have some more serious problems.
- ssl-3 1mo ago
  > On the other hand Joe from shipping wouldn't even know what a PLC is Joe from shipping is undefined. It is possible that Joe knows more about PLCs than you think you know about shipping, and that he has a god complex worse than that of a grizzled Haskell programmer. People all over have jobs that don't align with their interests, education, and prior work experiences. That's pretty normal; the bills still need paid and people find work where they can. > if you let untrained staff maintain electrical systems I think you have some more serious problems. Or maybe Joe from shipping is a departmental director who has this role because he is a strong leader who takes the initiative to embrace and overcome new challenges. We don't know.

coryrc 1mo ago

Relatedly, I made an embedded product. I built a live CD (yes this was a while ago) for my employer with the entire development environment. As long as you can find an x86 machine with CD-ROM you could build, compile, test, and program using the exact environment I did twenty years ago. Maybe today I'd do micropython on ESP32. Download text file from device, edit, upload back on.

aliasxneo 1mo ago
There's at least one customer out there who has an RPi with NixOS on it from me. Probably still works to this day. It was the best use of NixOS I think I've ever found.
- Kim_Bruning 1mo ago
  I've switched up to NUC-ish hardware which WAS cheap until recently. All the components just a bit higher quality than the lowest-bidder you get on the Pi. Run nixos on that and you've got a really solid platform.
- rurban 1mo ago
  More probably worst use of NixOS. In embedded we care for every single kb, and don't have tolerance for this kind of space waste management system
  - voakbasda 1mo ago
    Embedded means different things. Personally I have a hard time calling anything that runs Linux as embedded, having cut my teeth on microcontrollers.
    - hiAndrewQuinn 1mo ago
      If it's a custom Yocto thing or it involved buildroot at any state of provisioning it gets the moniker "quasi-embedded" to me. Doubly so if it never actually connects to the internet, unlike these water controllers, apparently.

hommelix 1mo ago

> The best thing they could do to new PLC's would be to have the src code live alongside the firmware inside. So any new person would be able to open it and reason about it. This is already the case. Some PLC allows you to store and retrieve the source code of the programmed directly in the flash of the PLC. Sometimes this feature is behind a higher access level or password.

NotMichaelBay 1mo ago
Is it version controlled too?

KaiserPro 1mo ago

Nooooope nope nope. An entire class of controller with really shit security now (as in scanning a network with PLC controllers has a non-trivial chance of rebooting or freezing any number of devices on the network) trained to just plug USB sticks in? I understand the idea, and its laudable. But the entire model of PLC security needs root and branches overhaul.