7 ms·

Old software keeping space missions alive

80 points by alex1212 3y ago

contingencies 3y ago
The spacecraft doesn't have a fuel gauge ... Zoom call with 100 Airbus engineers ... The original software for part of the onboard navigation system was running on a Windows 98 PC that no-one could find the password to and ended up using bolt cutters to extract the hard drives. Talk about a deep space horror story!
- WalterBright 3y ago
  The Apollo capsule didn't have a fuel gauge, either, because nobody could figure out how to make a fuel gauge work in weightless conditions. Various schemes failed. The solution turned out to be to have a reserve tank with enough propellant in it to do reentry. Then when the main tank ran dry, you knew just what you could do with what was left.
  - barelyauser 3y ago
    Since you know how much fuel you start with, can't you integrate something like the time spent firing the engine and use that as gauge?
    - WalterBright 3y ago
      I'm sure they thought of that, so there must be some reason why it is inaccurate.
    - dr_orpheus 3y ago
      Yes, and that is what most satellites due to estimate fuel. But any errors in that measurement can accumulate. See my other response on this thread for more details.
    - tekla 3y ago
      No, since engines aren't perfect and wastes propellant. Also fuel boils off in space.
- WalterBright 3y ago
  Why would anyone need bolt cutters to extract the drives? I have a couple Win98 PCs in the basement, and you just unscrew the drives. The password on it didn't encrypt the drives, either. As for the password, this is the old days. Could probably google how to crack the password for it. Or just try the classic 123456, querty, password, or letmein.
  - mikestew 3y ago
    Why would anyone need bolt cutters to extract the drives? I've had PC cases that had eyelets for a lock. You'd have to get through that before "unscrew(ing) the drives".
    - WalterBright 3y ago
      The drives aren't password protected. Why would one need to take the drives apart? Besides, the text was about extracting the drives, i.e. taking them out of the computer box.
      - FetusP 3y ago
        Maybe to access the drive, they have to go through a lockable case. If someone lost the keys to that case, they might have had to pop the locks with a bolt cutter
        deleted 3y ago
        [deleted]
        mikestew 3y ago
        That was exactly what I was driving at, thanks. In order to take the drives "out of the computer box", one needs to get through the lock on that computer box first.
- dr_orpheus 3y ago
  No spacecraft has a "fuel gauge" in the traditional sense. You don't have gravity to for a float to work like any fuel gauge in a car or airplane. Fuel is driven by surface tension and pressure rather than any buoyancy/gravity forces. These leads to the development of diaphragm type tanks where the pressurized bladder pushes fuel out of the tank or propellant management device (PMD) type tanks [0/1] that use the surface tension to guide pressurant-free fuel to the thrusters. So determining how much fuel you have left is done by a combination of integrating how much time you had thrusters firing, coupled with what pressure the tanks/lines were at while the thrusters were firing. Errors in these measurements accumulate over time which is why there is a lot of effort in to determining how much fuel is left in a spacecraft. Especially critical for things like big comm birds in GEO where fuel can be limiting in operation and the more fuel you have the longer you can keep station and get revenue from the satellite. But you need to still be conservative enough to have enough fuel to get out of the GEO belt for decommissioning your satellite. [0] https://en.wikipedia.org/wiki/Propellant_management_device https://en.wikipedia.org/wiki/Propellant_management_device [1] https://www.northropgrumman.com/space/pmd-tanks/ https://www.northropgrumman.com/space/pmd-tanks/
  - RugnirViking 3y ago
    I'm sure we'll get there eventually. But there is a fun idea in there somewhere about a future where the only need for manned spaceflight is so we can get pinged by the computer once every few days and go and hug/wrap a tape measure around the fuel bladder
hilbert42 3y ago
How about traveling-wave vacuum tubes from the pre-transistor era. Doesn't Voyager use them to generate its RF transmission power?
- AlanYx 3y ago
  It does! But the really interesting thing is that travelling wave tubes are still being used for quite a few modern satellite designs. For example, SiriusXM used them in SXM-9/10, and is probably using them in SXM-11/12.
  - dr_orpheus 3y ago
    Yes travelling wave tubes are still very common on modern satellites. They tend to be used most on high power (>50W RF power)/high frequency (K-band and higher) systems. I don't know the exact rationale, but for space systems in the higher frequency and power regimes travelling wave tubes are still more reliable than solid state power amplifiers. Traveling wave tubes are still in the 40-50% efficiency range so they're not out of range with efficiency of some solid state devices as well.
    - hilbert42 3y ago
      Right, although I've worked with TWTs, klystrons, etc. in terrestrial equipment and have some familiarity with them, I've only casually thought about the TWTs in the Voyagers and in space use generally. Given the remarkable longevity of the Voyagers, what I'm surprised about is that there hasn't been much discussion about their componentry and why they've been so reliable. For example I've seen nothing written about the engineering involved in Voyagers' TWTs and why they have been so reliable. For instance, what is the cathode material, barium, strontium, thorium oxide, etc. used in these TWTs? Was its selection criteria based on emitters with the lowest work function/highest emission at the lowest temperature with preservation of the heater life foremost in mind, and or was it based on oxides with highest ruggedness—least affected by cathode poisoning, etc. Discussions about Richardson's laws and cathode emitters is something I almost never come across these days let alone how they've played a role in the engineering of Voyagers' longevity. Whilst component manufacturers consider these matters, terrestrial users generally don't, we just reach for replacement parts when components fail. Perhaps I'm just not reading the right material but given the remarkable performance of these spacecraft, I'm surprised we're not focusing on the science and engineering that's made that all possible. No doubt those who're involved in space engineering are focused on these issues but it seems to me not much information has filtered down to even people like me who have some limited knowledge of the technology let alone the general science-reading public. Using the Voyagers' history and notoriety would be an excellent way to interest students in the physics of TWTs not to mention the material science and the engineering used in their design and manufacture. When one considers it, there's a lot of fascinating science and engineering involved in making this 'relic' from the vacuum tube era function and keeping it so.
snickerbockers 3y ago
jesus christ, i dont think ive ever seen an article that beats around the bush for as long as this one does before finally getting to the point. anyways, for those who don't have the patience, the title is misleading. it's just some old software from the 90s. It's only ancient if you're one of those people who completely rewrites their entire code base from the ground up every two years because to pad out your resume with whatever bullshit new "framework" is in vogue. Contrary to popular belief you actually can just keep using the same software indefinitely, it doesn't degrade with age, and for the most part the hardware lasts pretty long too as long as you take care of it (with a few exceptions like optical drives and SSDs).
- justinator 3y ago
  There's a little more to it. If your original team retires because of old age, that's a lot of wisdom lost. Not everyone wants to learn COBOL to keep planes in the air and the IRS running, imagine needing to figure out the low level code on something millions of miles away with half the hardware broken, the battery cracked and leaking – and if the software update doesn't work, the entire mission is over? It's altogether amazing.
  - ericbarrett 3y ago
    COBOL is really quite straightforward and readable. I don't think a developer who's already familiar with a modern language or two would have a hard time picking it up. The system it's operating is domain-specific knowledge anyway, and would be something a newcomer would have to learn (or be familiar with) regardless of language.
    - justinator 3y ago
      >I don't think a developer who's already familiar with a modern language or two would have a hard time picking it up. You're probably correct, but it's more about if they would want to pick up a language that has no future, except maintenance of already existing systems. (see: Perl).
  - 3y ago
h2odragon 3y ago
From rumors I heard, for a while there before 2005, NASA was one of the biggest buyers of salvage sun4 hardware. they had multiple missions running on it and there were some particular high rate data input cards on mbus that were hard to duplicate on other platforms. "we have a closet full of sparc 10s, we figure that'll be enough to last as long as the satellite"
ubermonkey 3y ago
It's not immediatley on point, but the title reminds me that many here may enjoy the documentary about the Voyager team as it works today called It's Quieter in the Twilight.
sillywalk 3y ago
Also NASA's infrastructure is crumbling, and 25% of workers are eligible for retirement. https://arstechnica.com/space/2023/08/nasas-buildings-are-even-older-than-its-graying-workforce/ https://arstechnica.com/space/2023/08/nasas-buildings-are-ev...
paulpauper 3y ago
space-x has taken over Nasa to some degree, i think
neilv 3y ago
> Launched almost 46 years ago in 1977, the twin Voyager probes continue to send back data from beyond the Solar System. > I checked with Nasa, which has assured me that the spacecraft are still being controlled from the same beige cubicle in an annex of its Jet Propulsion Laboratory (JPL) that I visited in 2017, marked with a homemade cardboard sign reading: "Mission critical hardware – PLEASE DO NOT TOUCH". I throw around the term "mission critical" maybe too easily. Because that beige cubicle with the homemade cardboard sign sounds humbling.
clnq 3y ago
A lot of software wraps old code. There is no reason to replace what works and works well. Besides, modern doesn’t generally mean more reliable or performant. It’s often the other way around. Over the last few decades, embedded programming hasn’t changed much, for example. The rate of change is much higher with the ultra-abstracted languages.

80 points by alex1212 3y ago

The spacecraft doesn't have a fuel gauge ... Zoom call with 100 Airbus engineers ... The original software for part of the onboard navigation system was running on a Windows 98 PC that no-one could find the password to and ended up using bolt cutters to extract the hard drives. Talk about a deep space horror story!

WalterBright 3y ago
The Apollo capsule didn't have a fuel gauge, either, because nobody could figure out how to make a fuel gauge work in weightless conditions. Various schemes failed. The solution turned out to be to have a reserve tank with enough propellant in it to do reentry. Then when the main tank ran dry, you knew just what you could do with what was left.
- barelyauser 3y ago
  Since you know how much fuel you start with, can't you integrate something like the time spent firing the engine and use that as gauge?
  - WalterBright 3y ago
    I'm sure they thought of that, so there must be some reason why it is inaccurate.
  - dr_orpheus 3y ago
    Yes, and that is what most satellites due to estimate fuel. But any errors in that measurement can accumulate. See my other response on this thread for more details.
  - tekla 3y ago
    No, since engines aren't perfect and wastes propellant. Also fuel boils off in space.
WalterBright 3y ago
Why would anyone need bolt cutters to extract the drives? I have a couple Win98 PCs in the basement, and you just unscrew the drives. The password on it didn't encrypt the drives, either. As for the password, this is the old days. Could probably google how to crack the password for it. Or just try the classic 123456, querty, password, or letmein.
- mikestew 3y ago
  Why would anyone need bolt cutters to extract the drives? I've had PC cases that had eyelets for a lock. You'd have to get through that before "unscrew(ing) the drives".
  - WalterBright 3y ago
    The drives aren't password protected. Why would one need to take the drives apart? Besides, the text was about extracting the drives, i.e. taking them out of the computer box.
    - FetusP 3y ago
      Maybe to access the drive, they have to go through a lockable case. If someone lost the keys to that case, they might have had to pop the locks with a bolt cutter
      - deleted 3y ago
        [deleted]
      - mikestew 3y ago
        That was exactly what I was driving at, thanks. In order to take the drives "out of the computer box", one needs to get through the lock on that computer box first.
dr_orpheus 3y ago
No spacecraft has a "fuel gauge" in the traditional sense. You don't have gravity to for a float to work like any fuel gauge in a car or airplane. Fuel is driven by surface tension and pressure rather than any buoyancy/gravity forces. These leads to the development of diaphragm type tanks where the pressurized bladder pushes fuel out of the tank or propellant management device (PMD) type tanks [0/1] that use the surface tension to guide pressurant-free fuel to the thrusters. So determining how much fuel you have left is done by a combination of integrating how much time you had thrusters firing, coupled with what pressure the tanks/lines were at while the thrusters were firing. Errors in these measurements accumulate over time which is why there is a lot of effort in to determining how much fuel is left in a spacecraft. Especially critical for things like big comm birds in GEO where fuel can be limiting in operation and the more fuel you have the longer you can keep station and get revenue from the satellite. But you need to still be conservative enough to have enough fuel to get out of the GEO belt for decommissioning your satellite. [0] https://en.wikipedia.org/wiki/Propellant_management_device https://en.wikipedia.org/wiki/Propellant_management_device [1] https://www.northropgrumman.com/space/pmd-tanks/ https://www.northropgrumman.com/space/pmd-tanks/
- RugnirViking 3y ago
  I'm sure we'll get there eventually. But there is a fun idea in there somewhere about a future where the only need for manned spaceflight is so we can get pinged by the computer once every few days and go and hug/wrap a tape measure around the fuel bladder

hilbert42 3y ago

How about traveling-wave vacuum tubes from the pre-transistor era. Doesn't Voyager use them to generate its RF transmission power?

AlanYx 3y ago
It does! But the really interesting thing is that travelling wave tubes are still being used for quite a few modern satellite designs. For example, SiriusXM used them in SXM-9/10, and is probably using them in SXM-11/12.
- dr_orpheus 3y ago
  Yes travelling wave tubes are still very common on modern satellites. They tend to be used most on high power (>50W RF power)/high frequency (K-band and higher) systems. I don't know the exact rationale, but for space systems in the higher frequency and power regimes travelling wave tubes are still more reliable than solid state power amplifiers. Traveling wave tubes are still in the 40-50% efficiency range so they're not out of range with efficiency of some solid state devices as well.
  - hilbert42 3y ago
    Right, although I've worked with TWTs, klystrons, etc. in terrestrial equipment and have some familiarity with them, I've only casually thought about the TWTs in the Voyagers and in space use generally. Given the remarkable longevity of the Voyagers, what I'm surprised about is that there hasn't been much discussion about their componentry and why they've been so reliable. For example I've seen nothing written about the engineering involved in Voyagers' TWTs and why they have been so reliable. For instance, what is the cathode material, barium, strontium, thorium oxide, etc. used in these TWTs? Was its selection criteria based on emitters with the lowest work function/highest emission at the lowest temperature with preservation of the heater life foremost in mind, and or was it based on oxides with highest ruggedness—least affected by cathode poisoning, etc. Discussions about Richardson's laws and cathode emitters is something I almost never come across these days let alone how they've played a role in the engineering of Voyagers' longevity. Whilst component manufacturers consider these matters, terrestrial users generally don't, we just reach for replacement parts when components fail. Perhaps I'm just not reading the right material but given the remarkable performance of these spacecraft, I'm surprised we're not focusing on the science and engineering that's made that all possible. No doubt those who're involved in space engineering are focused on these issues but it seems to me not much information has filtered down to even people like me who have some limited knowledge of the technology let alone the general science-reading public. Using the Voyagers' history and notoriety would be an excellent way to interest students in the physics of TWTs not to mention the material science and the engineering used in their design and manufacture. When one considers it, there's a lot of fascinating science and engineering involved in making this 'relic' from the vacuum tube era function and keeping it so.

snickerbockers 3y ago

jesus christ, i dont think ive ever seen an article that beats around the bush for as long as this one does before finally getting to the point. anyways, for those who don't have the patience, the title is misleading. it's just some old software from the 90s. It's only ancient if you're one of those people who completely rewrites their entire code base from the ground up every two years because to pad out your resume with whatever bullshit new "framework" is in vogue. Contrary to popular belief you actually can just keep using the same software indefinitely, it doesn't degrade with age, and for the most part the hardware lasts pretty long too as long as you take care of it (with a few exceptions like optical drives and SSDs).

justinator 3y ago
There's a little more to it. If your original team retires because of old age, that's a lot of wisdom lost. Not everyone wants to learn COBOL to keep planes in the air and the IRS running, imagine needing to figure out the low level code on something millions of miles away with half the hardware broken, the battery cracked and leaking – and if the software update doesn't work, the entire mission is over? It's altogether amazing.
- ericbarrett 3y ago
  COBOL is really quite straightforward and readable. I don't think a developer who's already familiar with a modern language or two would have a hard time picking it up. The system it's operating is domain-specific knowledge anyway, and would be something a newcomer would have to learn (or be familiar with) regardless of language.
  - justinator 3y ago
    >I don't think a developer who's already familiar with a modern language or two would have a hard time picking it up. You're probably correct, but it's more about if they would want to pick up a language that has no future, except maintenance of already existing systems. (see: Perl).
- 3y ago

h2odragon 3y ago

From rumors I heard, for a while there before 2005, NASA was one of the biggest buyers of salvage sun4 hardware. they had multiple missions running on it and there were some particular high rate data input cards on mbus that were hard to duplicate on other platforms. "we have a closet full of sparc 10s, we figure that'll be enough to last as long as the satellite"

ubermonkey 3y ago

It's not immediatley on point, but the title reminds me that many here may enjoy the documentary about the Voyager team as it works today called It's Quieter in the Twilight.

sillywalk 3y ago

Also NASA's infrastructure is crumbling, and 25% of workers are eligible for retirement. https://arstechnica.com/space/2023/08/nasas-buildings-are-even-older-than-its-graying-workforce/ https://arstechnica.com/space/2023/08/nasas-buildings-are-ev...

paulpauper 3y ago

space-x has taken over Nasa to some degree, i think

neilv 3y ago

> Launched almost 46 years ago in 1977, the twin Voyager probes continue to send back data from beyond the Solar System. > I checked with Nasa, which has assured me that the spacecraft are still being controlled from the same beige cubicle in an annex of its Jet Propulsion Laboratory (JPL) that I visited in 2017, marked with a homemade cardboard sign reading: "Mission critical hardware – PLEASE DO NOT TOUCH". I throw around the term "mission critical" maybe too easily. Because that beige cubicle with the homemade cardboard sign sounds humbling.

clnq 3y ago

A lot of software wraps old code. There is no reason to replace what works and works well. Besides, modern doesn’t generally mean more reliable or performant. It’s often the other way around. Over the last few decades, embedded programming hasn’t changed much, for example. The rate of change is much higher with the ultra-abstracted languages.