11 ms·

SFMTA's train system running on floppy disks; city fears 'catastrophic failure'

91 points by sidlls 2y ago

abeppu 2y ago
Frustratingly, though the article goes into how replacing/upgrading the whole control system will be an big, expensive project, they don't attempt to say why they can't just update the "read from floppy" part of the system to read the same info from a modern component.
- inferiorhuman 2y ago
  Floppies aren't the problem, the main problem is lack of support. The MTA moved off of OS/2 back in 2008. Yeah, halfway into its intended lifespan Alcatel/Thales basically stopped supporting the whole mess. If I had to guess they're using an equally unsupported version of Windows now.
sevenseventen 2y ago
Can someone just ask Foone or lcamtuf to help them get it running off an internet-connected toothbrush or something? It seems like there are so many people doing complex reverse-engineering of ancient stuff _just for kicks_ that this floppy-disk issue just shouldn't require a massive project. yes, I know safety-critical systems are different. I also expect that the floppy-disk issue is just the easiest problem to explain of a long chain of terrible legacy lock-ins. However, if they're literally holding their breath every morning when it's time to IPL the system off a floppy...that part sounds solvable.
- gojomo 2y ago
  The hypercompetence of tech enthusiasts and the mind-numbing idiocy & dishonesty of politial governance don't mix, so cities have to spend tens of millions with parasitic contractors for what a few clever hackers could do as a side-project.
WaitWaitWha 2y ago
>Turns out that in 1998, SFMTA had the latest cutting edge technology when they installed their automatic train control system. > "We were the first agency in the U.S. to adopt this particular technology but it was from an era that computers didn't have a hard drive so you have to load the software from floppy disks on to the computer," In 1998, most personal computers already had hard drives [0]. From Wikipedia "The IBM PC/XT in 1983 included an internal 10 MB HDD, and soon thereafter, internal HDDs proliferated on personal computers." The 3.5" floppy is from the mid 80's, again from Wiki [1] "In the early 1980s, many manufacturers introduced smaller floppy drives and media in various formats. A consortium of 21 companies eventually settled on a 3½-inch design..." [0] https://en.wikipedia.org/wiki/Hard_disk_drive https://en.wikipedia.org/wiki/Hard_disk_drive [1] https://en.wikipedia.org/wiki/Floppy_disk https://en.wikipedia.org/wiki/Floppy_disk Why do I have to do this research instead of the "journalist"?
- abeppu 2y ago
  Given that the agency seems not to be able to cope with change at a reasonable pace, I wonder if 1998 was just when the control system project finished, having been planned and started several years prior.
  - HumblyTossed 2y ago
    > I wonder if 1998 was just when the control system project finished, having been planned and started several years prior. Probably this. Probably they got the plans worked up in 1990, but didn't finish everything until 1998.
    - rootusrootus 2y ago
      I'd say they must have started closer to 1980. By the time we had the 286, most PCs came with a hard drive, and 1990 was the era of the 486.
- compuguy 2y ago
  I have this feeling that they meant 1988 instead of 1998....but I could be wrong? Edit: Even the news article hints at 5 inch floppy disks...which makes 1998 make absolutely no sense to me at least. https://abc7news.com/san-francisco-train-system-has-been-running-on-floppy-disks-but-city-fears-catastrophic-failure-before-upgrade/14624828/#:~:text=SFMTA%27s%20train%20control%20system%20relies%20every%20morning%20on%205%20inch%20floppy%20disks https://abc7news.com/san-francisco-train-system-has-been-run....
t1c 2y ago
"If it ain't broke, don't fix it" definitely has its practical limits
nvahalik 2y ago
I don't understand. The retro community has proven reliably that a simple Raspberry PI can easily bit-bang floppy controllers. We have myriad floppy-to-SD card adapters. Surely a plug-and-play solution that removes the area of most concern (reliance on the media itself) should be easily achievable in a few months?
- jwells89 2y ago
  Speaking naively (I have no insight into the situation), I would guess that this is probably the result of the officials in charge not having technical understanding and contracting it all out. Contractors aren’t going to be choosing the quick or cost-effective option, because that’d undercut their profits.
  - deweller 2y ago
    Also, it is difficult to sell a solution of "well - we aren't really modernizing the system, we are just putting a fix in place that pretends like it is a floppy drive." I would guess higher ups don't want to spend money on a band-aid without really fixing the problem.
    - krallja 2y ago
      > higher ups don't want to spend money on a band-aid without really fixing the problem I have literally never had this problem in my career. Higher-ups LOVE to push the problem off to after the next budget cycle.
- jdblair 2y ago
  You are correct that this would be much simpler if the only goal was to replace the floppy disk part of the system. The floppy disk angle is there to make a good headline. The article makes it clear this is a much bigger project than just replacing the floppy disks. "The detail[ed] project schedule will be finalized once we have a contractor onboard. This is effectively a multi-phase decade long project that starts with pieces of market street subway and pieces in the surface. Ultimately our goal is to have a single train control system for the entire rail system," said Tumlin.
sourcecodeplz 2y ago
Didn't it used to be that they broke often when you would keep them in the sun, even for a little?
nerdjon 2y ago
Well... I feel slightly better about Boston constantly pushing back being able to use our phones as tickets after seeing the timeline for the transition and how just that they are still doing this. What is it about public transit in the US that it is so... bad? Inadequate funding seems to be the easy one, but the MBTA (Boston) doesn't even handle the funds it has well. Yeah it needs more funding but there is also just a core issue to how it's run. It is sad to see the state of public transit in this country, particularly in dense urban areas where we should be discouraging Car use as much as possible. I am very curious what other countries are doing that we are not.
- asdff 2y ago
  I think about this all the time every time I see an lcd screen in my local train stations. If people who designed them actually used transit, maybe they'd have the signs you can only see as you walk out of the station say something about the bus lines you'd transfer to, instead of when the train behind you that just departed from is set to arrive. At some stations the screens they installed don't display anything useful at all, not when the next train will come, just the date and time as if everyone doesn't have that in their pocket, and this needs to be displayed every 25 feet on the platform. A lot of transit could be fixed by just taking a regular routine user, empowering them to become a dictator for a week and point out all the friction points they hit actually using the system. But then that would make the entire bureaucratic system that is the transit agency look like idiots who don't understand their own jobs, so it will unfortunately never be done.
  - xcrunner529 2y ago
    Well and lots that don’t even take the transit they manage. It’s just so infuriating.
    - asdff 2y ago
      Metros would rather build a sprawling parking lot for their bus and train operators to park their cars in, than to ask why the local metro system is such a nonstarter to so many of the metros own employees.
therealmarv 2y ago
Upgrade to 3.5 inch ones!
- yonran 2y ago
  They do use 3.5 inch disks; this is a misstatement: “SFMTA's train control system relies every morning on 5 inch floppy disks.” The SFMTA Board presentation justifying the >$600M upgrade shows a photo of the real 3.5 inch floppy disks in use that were written in 2021. https://www.sfmta.com/sites/default/files/reports-and-documents/2023/11/11-7-23_mtab_item_11_train_control_upgrade_project_consultant_rfp_-_slide_presentation.pdf#page=3 https://www.sfmta.com/sites/default/files/reports-and-docume...
anigbrowl 2y ago
They haven't even settled on a contractor yet? Maybe the problem here is that they're trying to write one check to fix all their problems at once instead of taking an incremental approach.
- wongarsu 2y ago
  That seems to be common in US infrastructure projects. If it isn't a megaproject, is it even worth attempting? /s
  - rtkwe 2y ago
    I get it though, it's a nightmare getting a project through approvals so by the time you have an issue big enough to push an overhaul through it's a major project and you want to pile as many things onto it as possible because it might be a decade before you get another decent funding injection to do what should be regular maintenance.
- maxvt 2y ago
  This takes deep expertise and a system oriented, long term view with a very precise eye towards the details of what's technically possible at every individual step. This kind of knowledge and experience doesn't come cheap, and we all know how much US city governments pay. I was at one point, very briefly, motivated to apply to a city job before I learned the pay is approximately 1/3 to 1/4 of what the private sector pays. The USDS routinely posts on "who wants to get hired" and the comments on that and other thread also mention a 66% to 75% salary reduction from baseline. This is even before we get to legal liability and political risk shifting that can happen when there is a fully responsible contracted company involved.
YawnNews 2y ago
[flagged]
_trampeltier 2y ago
Floppy disks seems common in the us. Just since 2019, they don't use 8" floppys for the nuclear rockets. https://www.theverge.com/2019/10/25/20931800/usa-nuclear-8-inch-floppy-disk-solid-state-transition https://www.theverge.com/2019/10/25/20931800/usa-nuclear-8-i...
- rootusrootus 2y ago
  Hardly specific to the US. Lots of stodgy old institutions across the world still run on elderly hardware. If it works, it works.
m3kw9 2y ago
If it works don’t fix it
FpUser 2y ago
>""The system is currently working just fine, but we know that with each increasing year, risk of data degradation on the floppy disks increases and that at some point there will be a catastrophic failure," Tumlin told ABC7." Do they know that floppy can be backed up?
- DerekL 2y ago
  Yes, but where are they going to get more blank disks? No one makes them anymore, they'll have to search for new old stock.
zitterbewegung 2y ago
Japan is doing away with 3 1/2 floppies . https://arstechnica.com/gadgets/2024/01/floppy-disk-requirements-finally-axed-from-japan-government-regulations/ https://arstechnica.com/gadgets/2024/01/floppy-disk-requirem...
- wongarsu 2y ago
  German high speed trains used 3.5" floppies to update seat reservations until ~2016.
- cortesoft 2y ago
  The article picture is of a 3.5" floppy, but the article says they are using 5.25"
HumblyTossed 2y ago
> This system was designed to last 20 to 25 years. SFMTA's director Jeffrey Tumlin said upgrading the system will take another decade and cost hundreds of millions of dollars. This is the problem, not that they're using a floppy. This isn't web dev where you get to rewrite everything every 6 mos. Systems have to have decades long life cycles BUT THEY EVENTUALLY NEED TO BE REPLACED and that's not happening quickly enough here. Edit: It was last updated in 1998, so it's due now not a decade from now.
- markus_zhang 2y ago
  I'd like to see a breakdown of that "hundreds of millions of dollars" and which companies provided the quotes, plus who owns the companies and who are their husbands/wives/uncles/nephews/nieces/etc.
  - kion 2y ago
    I looked into this last year, a significant part of the cost is in new hardware, both on the vehicles, but also sensors spread through the system. They all need commercial grade RF systems that allow for 2 way signaling communication and integration into the vehicles control systems.
    - bobthepanda 2y ago
      also they need to be rated for the operating environment. you can't stick any random part from newegg in there. Tesla used non-automotive grade touchscreens in some cars and so they died early: https://www.thedrive.com/tech/27989/teslas-screen-saga-shows-why-automotive-grade-matters https://www.thedrive.com/tech/27989/teslas-screen-saga-shows... Automotive grade has very high requirements: > Grade 0: High Temperature Operating Life (HTOL): +175C for 408 hours. High Temperature Storage Life (HTSL): +175C for 1,000 hours. Temperature Cycling: -65C tp +175C for 500 cycles And generally speaking transit agencies aspire to buy higher-reliability, longer-lasting equipment than your average consumer car.
      - 2y ago
Aloha 2y ago
I dont really see the issue? The technology works, there is a replacement outlined, there is no shortage of floppy disks - even 5 1/4 ones.
- perihelions 2y ago
  Yeah, the article reads as if the real problem is a lack of a tested system of backups. Or I don't know how else to parse this line: - "It's a question of risk. The system is currently working just fine but we know that with each increasing year risk of data degradation on the floppy disks increases and that at some point there will be a catastrophic failure." "Data degradation on the floppies" should not, by itself, cause any sort of "catastrophic failure" in a sound system. If one software disk fails, you should have five identical copies in a drawer, five more in a different room, ten more off-site, plus a disk image on cloud storage that you can write fresh floppy disks from. I mean, it's probably a good idea to change the storage medium too—but that's not the root problem.
  - jcgrillo 2y ago
    checksums are a whole thing
shrubble 2y ago
Does no one at the organization know what a GoTek FlashFloppy is? Sounds like they are using the floppy as an excuse to push for an upgrade that has nothing to do with the floppy drives.
- rtkwe 2y ago
  The entire system has reached the end of it's design life and the floppy is just the shocking bite to get people to click through. Really this is the regular replacement of an old system with a newer control system that will make the trains run better and more reliably.
  - nelsondev 2y ago
    Why will a new system make the trains run better? It could also make it run worse, if they don’t cover all the current cases. I’ve ridden Muni in SF for years, drivers control most of the decision making. There’s a handful of single track tunnels/sensors/necessary software-based coordination, but as another commenter pointed out, the doors cause more issues than signal problems.
    - inferiorhuman 2y ago
      I’ve ridden Muni in SF for years, drivers control most of the decision making. All of the tunnels are under automatic operation, and the majority of the switches (street or tunnel) are computer controlled. the doors cause more issues than signal problems. While doors are a real common failure/abuse point, I'd want to see some actual numbers on that. Train control failure may not always be self-evident to a rider, especially if the remedy is to run things manually (such that things aren't ever stopped completely). Historically the cab signaling system would glitch in wet or damp weather. Transponders for the switches had a near 100% failure rate. Trains chewed up the inductive loops, forcing emergency braking. That was common enough that they had to dial back the emergency deceleration. Twenty years on they've mostly worked out the kinks but it hasn't always been that way.
VyseofArcadia 2y ago
> "It's like if you lose your memory overnight, and every morning, somebody has to tell you hey 'this is who you are and what your purpose is what you have to do today,'" said Maguire. Yeah, that's called a cold boot. Moving to not-floppies doesn't mean you can avoid this. Clearly it's off of floppies instead of ROM so you can more easily update the software, but I am wondering how often that ended up happening. Maybe EEPROMs would have been better. > Luz Pena: "How dire is it to change the system to upgrade it from a floppy disk to a wireless system?" I agree that floppies aren't the peak of reliability, but "a wireless system" also sounds like a disaster. I don't want critical urban infrastructure running on extremely hackable OTA updates. For the love of god, SF, you can avoid pretty much all potential cybersecurity problems by just not putting your trains online. I feel like neither the interviewer nor the interviewee really had the technical expertise to speak to this. This entire piece is just, "oooooo, floppies are old. Old bad! Why not new yet? New good."
- JTbane 2y ago
  Couldn't the city set up a private WAN for the trains, avoiding the security issues of being connected to the internet?
  - VyseofArcadia 2y ago
    That doesn't avoid security issues inherent in wireless, but at least you have to be somewhat near the train instead of the other side of the planet.
ChrisArchitect 2y ago
The reporter waving around a 3.5" floppy because most viewers wouldn't even know what the 5.25" was
- rainbowzootsuit 2y ago
  The 8" are reserved for the ICBM targeting systems and financial institutions.
sitkack 2y ago
So, replace the floppies with a high reliability flash device that emulates a floppy drive, should have happened years ago.
jcgrillo 2y ago
> "Wow. I mean I thought we were moving on to AI. So why are we doing floppy disk," said Katie Guillen, SFMTA passenger. This naivety is not Katie's fault. We who work in tech are to blame for constantly pushing our half-baked experimental garbage as if it was "engineering" on par with civil or aeronautical systems. We can't blame people for occasionally believing the lies. The tech-washed version of this quote might go something like "wow I thought everything was moving to the multi-cloud serverless kooberneetus now, why is it still running on a computer?" > It is easy to run a secure computer system. You merely have to disconnect all dial-up connections and permit only direct-wired terminals, put the machine and its terminals in a shielded room, and post a guard at the door[1]. This is the kind of thing I want running the trains. Give it ECC RAM too, please. [1] https://en.m.wikipedia.org/wiki/Robert_Morris_(cryptographer)#cite_note-17 https://en.m.wikipedia.org/wiki/Robert_Morris_(cryptographer...
iancmceachern 2y ago
I emailed them. I live I'm SF. This is the sake, solved, problem we have in the machine tool world (milling machines, lathes, etc). Similar to old keyboards, you can just buy and use a floppy emulator. They emailed me back, they said that the floppy thing makes a good headline but is really just the tip of the iceberg. It's really the whole system that's like this at every layer, it needs replacing they say.
- boringg 2y ago
  Protected from hackers though!
  - jmb99 2y ago
    Unless said hacker has a big magnet.
- Hrnrurj 2y ago
  Here is 2017 article from NY to get an idea. https://www.businessinsider.com/nyc-mta-subway-delay-2017-6 https://www.businessinsider.com/nyc-mta-subway-delay-2017-6
- akira2501 2y ago
  > it needs replacing they say. It sounds like they let this problem fester until it has reached this existential end. They clamor for new technology, which they get, then they use the most risk adverse management strategy and never upgrade or change it, until it reaches this problem state. They either need significant third party help deploying and managing this system, or they should go back a few generations of technology and use the simplest possible system that meets their needs. Pen and paper should be considered if it can be made more efficient. I guess the people who work on public transit aren't interested in having the best public transit system available. They're only interested in keeping it running for as long as possible with zero changes or responsibility. This is why I'm strongly doubtful on public transportation in the US. Our bureaucracy can't handle it.
  - 2y ago
gojomo 2y ago
Remember when, less than 3 years ago, SF projected a $108M surplus for the next two years? Woulda been a nice time to clean up some of this technical debt! Or how about the SF Emergency Sirens, taken offline in late 2019 for a "2 year" upgrade plan that officeholders implied was already in place? In August 2023, with no progress whatsover, with the Maui fire disaster fresh on their minds, Mayor Breed & Supervisors President Peskin touted they'd finally funded a plan to return them to service soon: https://www.sf.gov/news/mayor-breed-and-board-president-peskin-announce-citywide-outdoor-public-warning-system-upgrade https://www.sf.gov/news/mayor-breed-and-board-president-pesk... In that same August 2023 timeframe, Peskin said the plan would bring this "need to have" system "up and running" & to "state of the art" by end of 2024, for $5.5M: https://www.cbsnews.com/sanfrancisco/news/san-francisco-city-leaders-look-to-bring-back-emergency-sirens-by-end-of-2024/ https://www.cbsnews.com/sanfrancisco/news/san-francisco-city... Of course, this was just more blatant self-exonerating bullshit from our local political machines immune from any real accountability for incompetence in basic public functions. A mere 6 months later in February 2024, nothing's been started, Peskin admitted "we don't even have a plan", the department is still waiting until "funding is identified", and the cost estimate has ballooned to $20.5m: https://abc7news.com/san-francisco-sirens-emergency-911-alert-system/14461668/ https://abc7news.com/san-francisco-sirens-emergency-911-aler... That works out to $170K+ for each of 119 units – units that each could probably just be a weatherized consumer-grade handheld device with multiple mobile/packet/sat radios, & a simple authenticated-playback app, mounted on existing poles that presumably already have power and even loudspeakers.
- paradox460 2y ago
  Sirens don't even need to be that complicated. A motor, a noise generator wheel (basically just a big spinning squirrelcage fan), and a horn. You could toggle them on/off with a simple relay/contactor.
  - gojomo 2y ago
    These "sirens" actually need to be able to play spoken announcements - something they did easily since about 2005, before the fumbled 2019-2024 shutdown/upgrade. More history, through January 2022, via JWZ's blog: https://www.jwz.org/blog/2022/01/the-reason-the-tuesday-noon-sirens-havent-returned/ https://www.jwz.org/blog/2022/01/the-reason-the-tuesday-noon...
Workaccount2 2y ago
My company provides new compute modules for another major global city's infrastructure that still relies on Intel CPU's from the early 80's. That is, we are building them brand new boards populated with chips that are over 40 years old. They haven't shown any interest in updating the system. It works, they can get service, and get "new" replacements for things that go bad. What they might not know though is that there is basically just one engineer we have (and probably the only one on Earth) who knows how to work on these things. He's getting old, and obviously none of the younger engineers really have an interest in learning ancient forgotten systems.
- strangattractor 2y ago
  LOL:) this story seems to resurface every couple of years. My favorite part of the article: "Jeffrey Tumlin: "It's a question of risk. The system is currently working just fine but we know that with each increasing year risk of data degradation on the floppy disks increases and that at some point there will be a catastrophic failure." This seems to imply they have been using the exact same disk for the past 20 years (absurd), they have absolutely no idea what is written on the disk and how it can be safely backed up or restored. This would be a problem regardless of the medium used. Although I hold the line at using paper tape there is nothing wrong with using floppies other than it seems antiquated. It certainly is reliable and cheap. Maybe the only thing that needs replacing is the people running the Muni.
  - asveikau 2y ago
    You make a good point. However, even with the disks backed up to a better storage medium, detecting data loss on the floppy and getting a replacement written and deployed to the correct location might take a while. During that time the system may face unacceptable transit delays.
    - jcgrillo 2y ago
      Wait, what? If this thing boots from a floppy, and that floppy has a checksum on it, and the boot sequence involves loading the checksum and the rest of the floppy contents into RAM and computing the checksum, corruption detection would be near instantaneous and the remediation would be as quick as it takes to eject the corrupted floppy, grab another floppy copy from the bin, plug it in, and flip the on switch. So the delay would be (roughly) zero.
throwaway74432 2y ago
>SFMTA's train control system relies every morning on 5 inch floppy disks. That's not the 3.5" floppy disk in the video. This is the old floppy disks[1] 1. https://www.digitaltreasures.ca/img/level2_floppy_525.jpg https://www.digitaltreasures.ca/img/level2_floppy_525.jpg
- prewett 2y ago
  And floppy floppies wasn't "cutting edge technology [in 1998]", either. The OG floppy disks were 8 inches, though, 5 inch floppies were the small ones (hence why 3.5" disks were "microfloppies"): https://images.techhive.com/images/article/2015/12/floppy_disk_2009_g1-100635495-orig.jpg https://images.techhive.com/images/article/2015/12/floppy_di...
yonran 2y ago
I believe that the article is about this RFP for “Contract No. SFMTA-2022-40 FTA” to upgrade the Communications-Based Train Control System (CBTC) or Advanced Train Control System (ATCS). The resolution to create the RFP for the supplier was approved 2023-01-17 (https://www.sfmta.com/reports/1-17-23-mtab-item-14-communications-based-train-control-system-rfp https://www.sfmta.com/reports/1-17-23-mtab-item-14-communica...), and a $36M resolution to create an RFP for a consultant was approved 2023-11-07 (https://www.sfmta.com/reports/11-7-23-mtab-item-11-train-control-upgrade-project-consultant-rfp https://www.sfmta.com/reports/11-7-23-mtab-item-11-train-con...). I’m not sure where the actual RFPs are pubished though.
time4tea 2y ago
Raspberry pi controller $30, couple of people to make it work - let's say $500k. Sure... there are loads of things that probably need attention, but maybe you don't need to fix them all at once?
ericmcer 2y ago
SF is a joke. 20 years of being one of the richest cities in the world (probably in the history of human beings) and it comes out worse than it was before.
- Smoosh 2y ago
  That assessment can probably be scaled up to the USA as a whole.
aluminum96 2y ago
SF Voters rejected Proposition A in 2022 [1], which would have included funding to upgrade Muni's control systems (among many other projects). We'll eventually have to find the money somewhere else when the system fails. [1] https://www.sfchronicle.com/sf/article/S-F-voters-narrowly-reject-Muni-s-400-million-17241868.php https://www.sfchronicle.com/sf/article/S-F-voters-narrowly-r...
fifteen1506 2y ago
Probably just a ploy to privatize the whole thing. Nothing to see here, move along.
JoyousAbandon 2y ago
Absurd errors in this article. Starting with: "it was from an era that computers didn't have a hard drive" Absolute BS. Pretty much every computer had a hard drive in 1998, and most had CD-ROM. Then they referred to the 3.5" disk as a "5-inch floppy." <sigh>
- jcgrillo 2y ago
  Saying it more times doesn't make it more true. > <sigh> </sigh>
  - JoyousAbandon 2y ago
    [dead]
asveikau 2y ago
Just a few short hours ago I was hearing some Muni workers at West Portal station talk about this story. One said to the other, "young people don't know what a floppy disk is anymore".
ornornor 2y ago
> SFMTA's train control system relies every morning on 5 inch floppy disks. And in the video, she says “on 3x 5 inch floppy disks like this one <shows a 3.5 inch floppy>”
kazinator 2y ago
> It's a question of risk. The system is currently working just fine but we know that with each increasing year risk of data degradation on the floppy disks increases and that at some point there will be a catastrophic failure. The key words in the sentence are: "it's working just fine". Data degradation of floppy discs is easy: just copy them to fresh ones, and verify that you have a good copy. The images should be safely backed up so they can be regenerated. (Plus there are emulators; a topic covered elsewhere under this submission.) I mean, are they really using the same 30 year old floppy discs over and over again until they degrade?
asdefghyk 2y ago
If SFMTA was "on the ball" they would have obtained source code from the original company.
throwaway984393 2y ago
[dead]
wakawaka28 2y ago
Trains are expected to last for decades, so this really isn't surprising. Developing a new system with the latest hardware to replace the floppy disk system would probably cost millions and might introduce bugs and security issues.
skissane 2y ago
> Turns out that in 1998, SFMTA had the latest cutting edge technology when they installed their automatic train control system. > "We were the first agency in the U.S. to adopt this particular technology but it was from an era that computers didn't have a hard drive so you have to load the software from floppy disks on to the computer," said Mariana Maguire, SFMTA Train Control Project. > SFMTA's train control system relies every morning on 5 inch floppy disks. This doesn’t make any sense. 5.25-inch floppy disks and no hard disks was not “cutting edge technology” in 1998. It arguably wasn’t even “cutting edge technology” in 1988
- jcgrillo 2y ago
  Not cutting edge personal computer technology, obviously, but was it not cutting edge train control technology? TFA isn't making any claims about PC technology. If I had to come up with a computer system that needs to last O(1 century) I'd avoid disks, I think.
- sitharus 2y ago
  It would make sense if it was 1978 which seems to be when the metro system was being brought online, but ATC was definitely 1998. Perhaps ATC was layered on top of the original signalling system, which is the part that uses the floppy disks?
bigyikes 2y ago
I grew up in the south and was fortunate enough to move to the Bay a few years ago. I was expecting it to be some kind of utopia, with futuristic technology on every corner. In reality, it is roughly the same as anywhere else in America. A bit of a let down. The innovation does not take place in the infrastructure. Beautiful place, though!
- kccqzy 2y ago
  If you count Waymo as futuristic technology* then you actually see it fairly often. *: Many people don't. Self-driving cars are better than human-driven cars, but affordable and reliable trains are even better.
- BHSPitMonkey 2y ago
  I've shared this experience/opinion with folks nearly verbatim :) Especially after having seen what I took to be a very well-integrated transit system in Chicago, the fragmented and confusing Bay Area experience was quite different from what I expected.
- patmorgan23 2y ago
  When BART was built in the 60s/70s it was innovative. The construction techniques for the Transbay Tube were unique at the time. The system was billed as a transit system built from the ground up for the space age. It was equipped with, at the time, a state of the art Automation Train Control system. Upgrading those systems is expensive and difficult, so it's not surprising old prices of equipment hung around until the end of their useful life. Especially since BART is going through a massive budget crisis at the moment (and this isn't the only one in its history).
ilya_m 2y ago
Upgrade? More like launch the process of planning for drafting a bid solicitation announcement. If I were 3.5" floppies, I wouldn't worry about retirement any time soon.
- ForOldHack 2y ago
  No, not 3.5, they said 5, but I think they mean 5 1/4. Think they could use floppy emulators, and get another 20 years out of the controls?
  - phire 2y ago
    The reporter said "it runs off three five inch floppy disks" It's unlikely a system deployed in 1998 would be using three separate 5¼" floppies. I'd believe a single 5¼" floppy, but not three of them. It's much more likely the system boots off uses a single 3.5" floppy, and the reporter forgot to read out the "point"
- ForOldHack 2y ago
  https://arstechnica.com/gadgets/2024/04/5-25-inch-floppy-disks-expected-to-help-run-san-francisco-trains-until-2030/ https://arstechnica.com/gadgets/2024/04/5-25-inch-floppy-dis... "The San Francisco Municipal Transportation Agency (SFMTA), which runs the city's Muni Metro light rail, claims to be the first US agency to adopt floppy disks. But today, the SFMTA is eager to abandon its reliance on 5¼-inch floppy disks—just give it about six more years and a few hundred more million dollars." yea for Unicode Character “¼”
mathgradthrow 2y ago
The longer a thing has not failed, the less likely it is to fail at any given moment. On the other hand, the older a thing is, the more likely it is to fail at any given moment.
deleted 2y ago
[deleted]

91 points by sidlls 2y ago

Frustratingly, though the article goes into how replacing/upgrading the whole control system will be an big, expensive project, they don't attempt to say why they can't just update the "read from floppy" part of the system to read the same info from a modern component.

inferiorhuman 2y ago
Floppies aren't the problem, the main problem is lack of support. The MTA moved off of OS/2 back in 2008. Yeah, halfway into its intended lifespan Alcatel/Thales basically stopped supporting the whole mess. If I had to guess they're using an equally unsupported version of Windows now.

sevenseventen 2y ago

Can someone just ask Foone or lcamtuf to help them get it running off an internet-connected toothbrush or something? It seems like there are so many people doing complex reverse-engineering of ancient stuff _just for kicks_ that this floppy-disk issue just shouldn't require a massive project. yes, I know safety-critical systems are different. I also expect that the floppy-disk issue is just the easiest problem to explain of a long chain of terrible legacy lock-ins. However, if they're literally holding their breath every morning when it's time to IPL the system off a floppy...that part sounds solvable.

gojomo 2y ago
The hypercompetence of tech enthusiasts and the mind-numbing idiocy & dishonesty of politial governance don't mix, so cities have to spend tens of millions with parasitic contractors for what a few clever hackers could do as a side-project.

WaitWaitWha 2y ago

>Turns out that in 1998, SFMTA had the latest cutting edge technology when they installed their automatic train control system. > "We were the first agency in the U.S. to adopt this particular technology but it was from an era that computers didn't have a hard drive so you have to load the software from floppy disks on to the computer," In 1998, most personal computers already had hard drives [0]. From Wikipedia "The IBM PC/XT in 1983 included an internal 10 MB HDD, and soon thereafter, internal HDDs proliferated on personal computers." The 3.5" floppy is from the mid 80's, again from Wiki [1] "In the early 1980s, many manufacturers introduced smaller floppy drives and media in various formats. A consortium of 21 companies eventually settled on a 3½-inch design..." [0] https://en.wikipedia.org/wiki/Hard_disk_drive https://en.wikipedia.org/wiki/Hard_disk_drive [1] https://en.wikipedia.org/wiki/Floppy_disk https://en.wikipedia.org/wiki/Floppy_disk Why do I have to do this research instead of the "journalist"?

abeppu 2y ago
Given that the agency seems not to be able to cope with change at a reasonable pace, I wonder if 1998 was just when the control system project finished, having been planned and started several years prior.
- HumblyTossed 2y ago
  > I wonder if 1998 was just when the control system project finished, having been planned and started several years prior. Probably this. Probably they got the plans worked up in 1990, but didn't finish everything until 1998.
  - rootusrootus 2y ago
    I'd say they must have started closer to 1980. By the time we had the 286, most PCs came with a hard drive, and 1990 was the era of the 486.
compuguy 2y ago
I have this feeling that they meant 1988 instead of 1998....but I could be wrong? Edit: Even the news article hints at 5 inch floppy disks...which makes 1998 make absolutely no sense to me at least. https://abc7news.com/san-francisco-train-system-has-been-running-on-floppy-disks-but-city-fears-catastrophic-failure-before-upgrade/14624828/#:~:text=SFMTA%27s%20train%20control%20system%20relies%20every%20morning%20on%205%20inch%20floppy%20disks https://abc7news.com/san-francisco-train-system-has-been-run....

t1c 2y ago

"If it ain't broke, don't fix it" definitely has its practical limits

nvahalik 2y ago

I don't understand. The retro community has proven reliably that a simple Raspberry PI can easily bit-bang floppy controllers. We have myriad floppy-to-SD card adapters. Surely a plug-and-play solution that removes the area of most concern (reliance on the media itself) should be easily achievable in a few months?

jwells89 2y ago
Speaking naively (I have no insight into the situation), I would guess that this is probably the result of the officials in charge not having technical understanding and contracting it all out. Contractors aren’t going to be choosing the quick or cost-effective option, because that’d undercut their profits.
- deweller 2y ago
  Also, it is difficult to sell a solution of "well - we aren't really modernizing the system, we are just putting a fix in place that pretends like it is a floppy drive." I would guess higher ups don't want to spend money on a band-aid without really fixing the problem.
  - krallja 2y ago
    > higher ups don't want to spend money on a band-aid without really fixing the problem I have literally never had this problem in my career. Higher-ups LOVE to push the problem off to after the next budget cycle.
jdblair 2y ago
You are correct that this would be much simpler if the only goal was to replace the floppy disk part of the system. The floppy disk angle is there to make a good headline. The article makes it clear this is a much bigger project than just replacing the floppy disks. "The detail[ed] project schedule will be finalized once we have a contractor onboard. This is effectively a multi-phase decade long project that starts with pieces of market street subway and pieces in the surface. Ultimately our goal is to have a single train control system for the entire rail system," said Tumlin.

sourcecodeplz 2y ago

Didn't it used to be that they broke often when you would keep them in the sun, even for a little?

nerdjon 2y ago

Well... I feel slightly better about Boston constantly pushing back being able to use our phones as tickets after seeing the timeline for the transition and how just that they are still doing this. What is it about public transit in the US that it is so... bad? Inadequate funding seems to be the easy one, but the MBTA (Boston) doesn't even handle the funds it has well. Yeah it needs more funding but there is also just a core issue to how it's run. It is sad to see the state of public transit in this country, particularly in dense urban areas where we should be discouraging Car use as much as possible. I am very curious what other countries are doing that we are not.

asdff 2y ago
I think about this all the time every time I see an lcd screen in my local train stations. If people who designed them actually used transit, maybe they'd have the signs you can only see as you walk out of the station say something about the bus lines you'd transfer to, instead of when the train behind you that just departed from is set to arrive. At some stations the screens they installed don't display anything useful at all, not when the next train will come, just the date and time as if everyone doesn't have that in their pocket, and this needs to be displayed every 25 feet on the platform. A lot of transit could be fixed by just taking a regular routine user, empowering them to become a dictator for a week and point out all the friction points they hit actually using the system. But then that would make the entire bureaucratic system that is the transit agency look like idiots who don't understand their own jobs, so it will unfortunately never be done.
- xcrunner529 2y ago
  Well and lots that don’t even take the transit they manage. It’s just so infuriating.
  - asdff 2y ago
    Metros would rather build a sprawling parking lot for their bus and train operators to park their cars in, than to ask why the local metro system is such a nonstarter to so many of the metros own employees.

therealmarv 2y ago

Upgrade to 3.5 inch ones!

yonran 2y ago
They do use 3.5 inch disks; this is a misstatement: “SFMTA's train control system relies every morning on 5 inch floppy disks.” The SFMTA Board presentation justifying the >$600M upgrade shows a photo of the real 3.5 inch floppy disks in use that were written in 2021. https://www.sfmta.com/sites/default/files/reports-and-documents/2023/11/11-7-23_mtab_item_11_train_control_upgrade_project_consultant_rfp_-_slide_presentation.pdf#page=3 https://www.sfmta.com/sites/default/files/reports-and-docume...

anigbrowl 2y ago

They haven't even settled on a contractor yet? Maybe the problem here is that they're trying to write one check to fix all their problems at once instead of taking an incremental approach.

wongarsu 2y ago
That seems to be common in US infrastructure projects. If it isn't a megaproject, is it even worth attempting? /s
- rtkwe 2y ago
  I get it though, it's a nightmare getting a project through approvals so by the time you have an issue big enough to push an overhaul through it's a major project and you want to pile as many things onto it as possible because it might be a decade before you get another decent funding injection to do what should be regular maintenance.
maxvt 2y ago
This takes deep expertise and a system oriented, long term view with a very precise eye towards the details of what's technically possible at every individual step. This kind of knowledge and experience doesn't come cheap, and we all know how much US city governments pay. I was at one point, very briefly, motivated to apply to a city job before I learned the pay is approximately 1/3 to 1/4 of what the private sector pays. The USDS routinely posts on "who wants to get hired" and the comments on that and other thread also mention a 66% to 75% salary reduction from baseline. This is even before we get to legal liability and political risk shifting that can happen when there is a fully responsible contracted company involved.

YawnNews 2y ago

[flagged]

_trampeltier 2y ago

Floppy disks seems common in the us. Just since 2019, they don't use 8" floppys for the nuclear rockets. https://www.theverge.com/2019/10/25/20931800/usa-nuclear-8-inch-floppy-disk-solid-state-transition https://www.theverge.com/2019/10/25/20931800/usa-nuclear-8-i...

rootusrootus 2y ago
Hardly specific to the US. Lots of stodgy old institutions across the world still run on elderly hardware. If it works, it works.

m3kw9 2y ago

If it works don’t fix it

FpUser 2y ago

>""The system is currently working just fine, but we know that with each increasing year, risk of data degradation on the floppy disks increases and that at some point there will be a catastrophic failure," Tumlin told ABC7." Do they know that floppy can be backed up?

DerekL 2y ago
Yes, but where are they going to get more blank disks? No one makes them anymore, they'll have to search for new old stock.

zitterbewegung 2y ago

Japan is doing away with 3 1/2 floppies . https://arstechnica.com/gadgets/2024/01/floppy-disk-requirements-finally-axed-from-japan-government-regulations/ https://arstechnica.com/gadgets/2024/01/floppy-disk-requirem...

wongarsu 2y ago
German high speed trains used 3.5" floppies to update seat reservations until ~2016.
cortesoft 2y ago
The article picture is of a 3.5" floppy, but the article says they are using 5.25"

HumblyTossed 2y ago

> This system was designed to last 20 to 25 years. SFMTA's director Jeffrey Tumlin said upgrading the system will take another decade and cost hundreds of millions of dollars. This is the problem, not that they're using a floppy. This isn't web dev where you get to rewrite everything every 6 mos. Systems have to have decades long life cycles BUT THEY EVENTUALLY NEED TO BE REPLACED and that's not happening quickly enough here. Edit: It was last updated in 1998, so it's due now not a decade from now.

markus_zhang 2y ago
I'd like to see a breakdown of that "hundreds of millions of dollars" and which companies provided the quotes, plus who owns the companies and who are their husbands/wives/uncles/nephews/nieces/etc.
- kion 2y ago
  I looked into this last year, a significant part of the cost is in new hardware, both on the vehicles, but also sensors spread through the system. They all need commercial grade RF systems that allow for 2 way signaling communication and integration into the vehicles control systems.
  - bobthepanda 2y ago
    also they need to be rated for the operating environment. you can't stick any random part from newegg in there. Tesla used non-automotive grade touchscreens in some cars and so they died early: https://www.thedrive.com/tech/27989/teslas-screen-saga-shows-why-automotive-grade-matters https://www.thedrive.com/tech/27989/teslas-screen-saga-shows... Automotive grade has very high requirements: > Grade 0: High Temperature Operating Life (HTOL): +175C for 408 hours. High Temperature Storage Life (HTSL): +175C for 1,000 hours. Temperature Cycling: -65C tp +175C for 500 cycles And generally speaking transit agencies aspire to buy higher-reliability, longer-lasting equipment than your average consumer car.
    - 2y ago

Aloha 2y ago

I dont really see the issue? The technology works, there is a replacement outlined, there is no shortage of floppy disks - even 5 1/4 ones.

perihelions 2y ago
Yeah, the article reads as if the real problem is a lack of a tested system of backups. Or I don't know how else to parse this line: - "It's a question of risk. The system is currently working just fine but we know that with each increasing year risk of data degradation on the floppy disks increases and that at some point there will be a catastrophic failure." "Data degradation on the floppies" should not, by itself, cause any sort of "catastrophic failure" in a sound system. If one software disk fails, you should have five identical copies in a drawer, five more in a different room, ten more off-site, plus a disk image on cloud storage that you can write fresh floppy disks from. I mean, it's probably a good idea to change the storage medium too—but that's not the root problem.
- jcgrillo 2y ago
  checksums are a whole thing

shrubble 2y ago

Does no one at the organization know what a GoTek FlashFloppy is? Sounds like they are using the floppy as an excuse to push for an upgrade that has nothing to do with the floppy drives.

rtkwe 2y ago
The entire system has reached the end of it's design life and the floppy is just the shocking bite to get people to click through. Really this is the regular replacement of an old system with a newer control system that will make the trains run better and more reliably.
- nelsondev 2y ago
  Why will a new system make the trains run better? It could also make it run worse, if they don’t cover all the current cases. I’ve ridden Muni in SF for years, drivers control most of the decision making. There’s a handful of single track tunnels/sensors/necessary software-based coordination, but as another commenter pointed out, the doors cause more issues than signal problems.
  - inferiorhuman 2y ago
    I’ve ridden Muni in SF for years, drivers control most of the decision making. All of the tunnels are under automatic operation, and the majority of the switches (street or tunnel) are computer controlled. the doors cause more issues than signal problems. While doors are a real common failure/abuse point, I'd want to see some actual numbers on that. Train control failure may not always be self-evident to a rider, especially if the remedy is to run things manually (such that things aren't ever stopped completely). Historically the cab signaling system would glitch in wet or damp weather. Transponders for the switches had a near 100% failure rate. Trains chewed up the inductive loops, forcing emergency braking. That was common enough that they had to dial back the emergency deceleration. Twenty years on they've mostly worked out the kinks but it hasn't always been that way.

VyseofArcadia 2y ago

> "It's like if you lose your memory overnight, and every morning, somebody has to tell you hey 'this is who you are and what your purpose is what you have to do today,'" said Maguire. Yeah, that's called a cold boot. Moving to not-floppies doesn't mean you can avoid this. Clearly it's off of floppies instead of ROM so you can more easily update the software, but I am wondering how often that ended up happening. Maybe EEPROMs would have been better. > Luz Pena: "How dire is it to change the system to upgrade it from a floppy disk to a wireless system?" I agree that floppies aren't the peak of reliability, but "a wireless system" also sounds like a disaster. I don't want critical urban infrastructure running on extremely hackable OTA updates. For the love of god, SF, you can avoid pretty much all potential cybersecurity problems by just not putting your trains online. I feel like neither the interviewer nor the interviewee really had the technical expertise to speak to this. This entire piece is just, "oooooo, floppies are old. Old bad! Why not new yet? New good."

JTbane 2y ago
Couldn't the city set up a private WAN for the trains, avoiding the security issues of being connected to the internet?
- VyseofArcadia 2y ago
  That doesn't avoid security issues inherent in wireless, but at least you have to be somewhat near the train instead of the other side of the planet.

ChrisArchitect 2y ago

The reporter waving around a 3.5" floppy because most viewers wouldn't even know what the 5.25" was

rainbowzootsuit 2y ago
The 8" are reserved for the ICBM targeting systems and financial institutions.

sitkack 2y ago

So, replace the floppies with a high reliability flash device that emulates a floppy drive, should have happened years ago.

jcgrillo 2y ago

> "Wow. I mean I thought we were moving on to AI. So why are we doing floppy disk," said Katie Guillen, SFMTA passenger. This naivety is not Katie's fault. We who work in tech are to blame for constantly pushing our half-baked experimental garbage as if it was "engineering" on par with civil or aeronautical systems. We can't blame people for occasionally believing the lies. The tech-washed version of this quote might go something like "wow I thought everything was moving to the multi-cloud serverless kooberneetus now, why is it still running on a computer?" > It is easy to run a secure computer system. You merely have to disconnect all dial-up connections and permit only direct-wired terminals, put the machine and its terminals in a shielded room, and post a guard at the door[1]. This is the kind of thing I want running the trains. Give it ECC RAM too, please. [1] https://en.m.wikipedia.org/wiki/Robert_Morris_(cryptographer)#cite_note-17 https://en.m.wikipedia.org/wiki/Robert_Morris_(cryptographer...

iancmceachern 2y ago

I emailed them. I live I'm SF. This is the sake, solved, problem we have in the machine tool world (milling machines, lathes, etc). Similar to old keyboards, you can just buy and use a floppy emulator. They emailed me back, they said that the floppy thing makes a good headline but is really just the tip of the iceberg. It's really the whole system that's like this at every layer, it needs replacing they say.

boringg 2y ago
Protected from hackers though!
- jmb99 2y ago
  Unless said hacker has a big magnet.
Hrnrurj 2y ago
Here is 2017 article from NY to get an idea. https://www.businessinsider.com/nyc-mta-subway-delay-2017-6 https://www.businessinsider.com/nyc-mta-subway-delay-2017-6
akira2501 2y ago
> it needs replacing they say. It sounds like they let this problem fester until it has reached this existential end. They clamor for new technology, which they get, then they use the most risk adverse management strategy and never upgrade or change it, until it reaches this problem state. They either need significant third party help deploying and managing this system, or they should go back a few generations of technology and use the simplest possible system that meets their needs. Pen and paper should be considered if it can be made more efficient. I guess the people who work on public transit aren't interested in having the best public transit system available. They're only interested in keeping it running for as long as possible with zero changes or responsibility. This is why I'm strongly doubtful on public transportation in the US. Our bureaucracy can't handle it.
- 2y ago

gojomo 2y ago

Remember when, less than 3 years ago, SF projected a $108M surplus for the next two years? Woulda been a nice time to clean up some of this technical debt! Or how about the SF Emergency Sirens, taken offline in late 2019 for a "2 year" upgrade plan that officeholders implied was already in place? In August 2023, with no progress whatsover, with the Maui fire disaster fresh on their minds, Mayor Breed & Supervisors President Peskin touted they'd finally funded a plan to return them to service soon: https://www.sf.gov/news/mayor-breed-and-board-president-peskin-announce-citywide-outdoor-public-warning-system-upgrade https://www.sf.gov/news/mayor-breed-and-board-president-pesk... In that same August 2023 timeframe, Peskin said the plan would bring this "need to have" system "up and running" & to "state of the art" by end of 2024, for $5.5M: https://www.cbsnews.com/sanfrancisco/news/san-francisco-city-leaders-look-to-bring-back-emergency-sirens-by-end-of-2024/ https://www.cbsnews.com/sanfrancisco/news/san-francisco-city... Of course, this was just more blatant self-exonerating bullshit from our local political machines immune from any real accountability for incompetence in basic public functions. A mere 6 months later in February 2024, nothing's been started, Peskin admitted "we don't even have a plan", the department is still waiting until "funding is identified", and the cost estimate has ballooned to $20.5m: https://abc7news.com/san-francisco-sirens-emergency-911-alert-system/14461668/ https://abc7news.com/san-francisco-sirens-emergency-911-aler... That works out to $170K+ for each of 119 units – units that each could probably just be a weatherized consumer-grade handheld device with multiple mobile/packet/sat radios, & a simple authenticated-playback app, mounted on existing poles that presumably already have power and even loudspeakers.

paradox460 2y ago
Sirens don't even need to be that complicated. A motor, a noise generator wheel (basically just a big spinning squirrelcage fan), and a horn. You could toggle them on/off with a simple relay/contactor.
- gojomo 2y ago
  These "sirens" actually need to be able to play spoken announcements - something they did easily since about 2005, before the fumbled 2019-2024 shutdown/upgrade. More history, through January 2022, via JWZ's blog: https://www.jwz.org/blog/2022/01/the-reason-the-tuesday-noon-sirens-havent-returned/ https://www.jwz.org/blog/2022/01/the-reason-the-tuesday-noon...

Workaccount2 2y ago

My company provides new compute modules for another major global city's infrastructure that still relies on Intel CPU's from the early 80's. That is, we are building them brand new boards populated with chips that are over 40 years old. They haven't shown any interest in updating the system. It works, they can get service, and get "new" replacements for things that go bad. What they might not know though is that there is basically just one engineer we have (and probably the only one on Earth) who knows how to work on these things. He's getting old, and obviously none of the younger engineers really have an interest in learning ancient forgotten systems.

strangattractor 2y ago
LOL:) this story seems to resurface every couple of years. My favorite part of the article: "Jeffrey Tumlin: "It's a question of risk. The system is currently working just fine but we know that with each increasing year risk of data degradation on the floppy disks increases and that at some point there will be a catastrophic failure." This seems to imply they have been using the exact same disk for the past 20 years (absurd), they have absolutely no idea what is written on the disk and how it can be safely backed up or restored. This would be a problem regardless of the medium used. Although I hold the line at using paper tape there is nothing wrong with using floppies other than it seems antiquated. It certainly is reliable and cheap. Maybe the only thing that needs replacing is the people running the Muni.
- asveikau 2y ago
  You make a good point. However, even with the disks backed up to a better storage medium, detecting data loss on the floppy and getting a replacement written and deployed to the correct location might take a while. During that time the system may face unacceptable transit delays.
  - jcgrillo 2y ago
    Wait, what? If this thing boots from a floppy, and that floppy has a checksum on it, and the boot sequence involves loading the checksum and the rest of the floppy contents into RAM and computing the checksum, corruption detection would be near instantaneous and the remediation would be as quick as it takes to eject the corrupted floppy, grab another floppy copy from the bin, plug it in, and flip the on switch. So the delay would be (roughly) zero.

throwaway74432 2y ago

>SFMTA's train control system relies every morning on 5 inch floppy disks. That's not the 3.5" floppy disk in the video. This is the old floppy disks[1] 1. https://www.digitaltreasures.ca/img/level2_floppy_525.jpg https://www.digitaltreasures.ca/img/level2_floppy_525.jpg

prewett 2y ago
And floppy floppies wasn't "cutting edge technology [in 1998]", either. The OG floppy disks were 8 inches, though, 5 inch floppies were the small ones (hence why 3.5" disks were "microfloppies"): https://images.techhive.com/images/article/2015/12/floppy_disk_2009_g1-100635495-orig.jpg https://images.techhive.com/images/article/2015/12/floppy_di...

yonran 2y ago

I believe that the article is about this RFP for “Contract No. SFMTA-2022-40 FTA” to upgrade the Communications-Based Train Control System (CBTC) or Advanced Train Control System (ATCS). The resolution to create the RFP for the supplier was approved 2023-01-17 (https://www.sfmta.com/reports/1-17-23-mtab-item-14-communications-based-train-control-system-rfp https://www.sfmta.com/reports/1-17-23-mtab-item-14-communica...), and a $36M resolution to create an RFP for a consultant was approved 2023-11-07 (https://www.sfmta.com/reports/11-7-23-mtab-item-11-train-control-upgrade-project-consultant-rfp https://www.sfmta.com/reports/11-7-23-mtab-item-11-train-con...). I’m not sure where the actual RFPs are pubished though.

time4tea 2y ago

Raspberry pi controller $30, couple of people to make it work - let's say $500k. Sure... there are loads of things that probably need attention, but maybe you don't need to fix them all at once?

ericmcer 2y ago

SF is a joke. 20 years of being one of the richest cities in the world (probably in the history of human beings) and it comes out worse than it was before.

Smoosh 2y ago
That assessment can probably be scaled up to the USA as a whole.

aluminum96 2y ago

SF Voters rejected Proposition A in 2022 [1], which would have included funding to upgrade Muni's control systems (among many other projects). We'll eventually have to find the money somewhere else when the system fails. [1] https://www.sfchronicle.com/sf/article/S-F-voters-narrowly-reject-Muni-s-400-million-17241868.php https://www.sfchronicle.com/sf/article/S-F-voters-narrowly-r...

fifteen1506 2y ago

Probably just a ploy to privatize the whole thing. Nothing to see here, move along.

JoyousAbandon 2y ago

Absurd errors in this article. Starting with: "it was from an era that computers didn't have a hard drive" Absolute BS. Pretty much every computer had a hard drive in 1998, and most had CD-ROM. Then they referred to the 3.5" disk as a "5-inch floppy." <sigh>

jcgrillo 2y ago
Saying it more times doesn't make it more true. > <sigh> </sigh>
- JoyousAbandon 2y ago
  [dead]

asveikau 2y ago

Just a few short hours ago I was hearing some Muni workers at West Portal station talk about this story. One said to the other, "young people don't know what a floppy disk is anymore".

ornornor 2y ago

> SFMTA's train control system relies every morning on 5 inch floppy disks. And in the video, she says “on 3x 5 inch floppy disks like this one <shows a 3.5 inch floppy>”

kazinator 2y ago

> It's a question of risk. The system is currently working just fine but we know that with each increasing year risk of data degradation on the floppy disks increases and that at some point there will be a catastrophic failure. The key words in the sentence are: "it's working just fine". Data degradation of floppy discs is easy: just copy them to fresh ones, and verify that you have a good copy. The images should be safely backed up so they can be regenerated. (Plus there are emulators; a topic covered elsewhere under this submission.) I mean, are they really using the same 30 year old floppy discs over and over again until they degrade?

asdefghyk 2y ago

If SFMTA was "on the ball" they would have obtained source code from the original company.

throwaway984393 2y ago

[dead]

wakawaka28 2y ago

Trains are expected to last for decades, so this really isn't surprising. Developing a new system with the latest hardware to replace the floppy disk system would probably cost millions and might introduce bugs and security issues.

skissane 2y ago

jcgrillo 2y ago
Not cutting edge personal computer technology, obviously, but was it not cutting edge train control technology? TFA isn't making any claims about PC technology. If I had to come up with a computer system that needs to last O(1 century) I'd avoid disks, I think.
sitharus 2y ago
It would make sense if it was 1978 which seems to be when the metro system was being brought online, but ATC was definitely 1998. Perhaps ATC was layered on top of the original signalling system, which is the part that uses the floppy disks?

bigyikes 2y ago

I grew up in the south and was fortunate enough to move to the Bay a few years ago. I was expecting it to be some kind of utopia, with futuristic technology on every corner. In reality, it is roughly the same as anywhere else in America. A bit of a let down. The innovation does not take place in the infrastructure. Beautiful place, though!

kccqzy 2y ago
If you count Waymo as futuristic technology* then you actually see it fairly often. *: Many people don't. Self-driving cars are better than human-driven cars, but affordable and reliable trains are even better.
BHSPitMonkey 2y ago
I've shared this experience/opinion with folks nearly verbatim :) Especially after having seen what I took to be a very well-integrated transit system in Chicago, the fragmented and confusing Bay Area experience was quite different from what I expected.
patmorgan23 2y ago
When BART was built in the 60s/70s it was innovative. The construction techniques for the Transbay Tube were unique at the time. The system was billed as a transit system built from the ground up for the space age. It was equipped with, at the time, a state of the art Automation Train Control system. Upgrading those systems is expensive and difficult, so it's not surprising old prices of equipment hung around until the end of their useful life. Especially since BART is going through a massive budget crisis at the moment (and this isn't the only one in its history).

ilya_m 2y ago

Upgrade? More like launch the process of planning for drafting a bid solicitation announcement. If I were 3.5" floppies, I wouldn't worry about retirement any time soon.

ForOldHack 2y ago
No, not 3.5, they said 5, but I think they mean 5 1/4. Think they could use floppy emulators, and get another 20 years out of the controls?
- phire 2y ago
  The reporter said "it runs off three five inch floppy disks" It's unlikely a system deployed in 1998 would be using three separate 5¼" floppies. I'd believe a single 5¼" floppy, but not three of them. It's much more likely the system boots off uses a single 3.5" floppy, and the reporter forgot to read out the "point"
ForOldHack 2y ago
https://arstechnica.com/gadgets/2024/04/5-25-inch-floppy-disks-expected-to-help-run-san-francisco-trains-until-2030/ https://arstechnica.com/gadgets/2024/04/5-25-inch-floppy-dis... "The San Francisco Municipal Transportation Agency (SFMTA), which runs the city's Muni Metro light rail, claims to be the first US agency to adopt floppy disks. But today, the SFMTA is eager to abandon its reliance on 5¼-inch floppy disks—just give it about six more years and a few hundred more million dollars." yea for Unicode Character “¼”

mathgradthrow 2y ago

The longer a thing has not failed, the less likely it is to fail at any given moment. On the other hand, the older a thing is, the more likely it is to fail at any given moment.

deleted 2y ago

[deleted]