The Trouble with Hubble: Payload Computer Glitch Stops Science at the Space Observatory
The Hubble Space Telescope’s remarkably long service life and its string of astonishing contributions to astronomy belie its troubled history. Long before its launch into low Earth orbit in 1990, Hubble suffered from design conflicts, funding and budgetary pressures, and even the death of seven astronauts. Long delayed, much modified, and mistakenly sent aloft with suboptimal optics, Hubble still managed to deliver results that have literally changed our view of the universe, and is perhaps responsible for more screensaver and desktop pictures than any other single source.
But all of that changed on June 13 of this year, when Hubble suffered a computer glitch that interrupted the flow of science data from the orbiting observatory. It’s not yet clear how the current issue with Hubble is going to pan out, and what it all means for the future of this nearly irreplaceable scientific asset. We all hope for the best, of course, but while we wait to see what happens, it’s worth taking the opportunity to dive inside Hubble for a look at its engineering and what exactly has gone wrong up there.
Above It All
The idea of putting a telescope in space, high above the roiling atmospheric soup we all live near the bottom of, was not exactly a new one even in the early 1960s, when humanity’s first tentative steps into space actually made such an audacious plan feasible. Before they had even put a boot on the Moon, NASA aspired to put a large reflecting telescope in Earth orbit, with a tentative goal of making it happen by the end of 1979. They recognized that such an installation would require frequent visits to maintain and upgrade it, and so the future space telescope’s design proceeded alongside NASA’s intended replacement for the mighty expendable rockets of the Apollo era: the reusable space plane that would come to be known as the Space Shuttle.
The parallel development of the Space Shuttle and the space telescope, which would eventually be named after American astronomer Edwin Hubble, resulted in design decisions that would pay enormous dividends down the road. Hubble was designed from the outset to be serviced, and specifically by space shuttle missions. As such, the spacecraft carrying the telescope and all the various scientific instruments that can be switched into its optical path has abundant handholds, tether points, and easily accessed hatches and compartments, all designed to make it easier for spacewalking astronauts to perform their tasks. Inside the spacecraft’s many equipment bays, instruments are attached with standardized fasteners designed to be manipulated with gloved hands. Hubble even came with a complete kit of specialized tools, to be used by future service missions.
Ever mindful of the march of progress, Hubble’s designers knew that the instruments they could think up and build in the 1980s would pale in comparison to what would undoubtedly follow. To prevent built-in obsolescence, Hubble was specifically designed with not only repairs in mind, but upgrades to its scientific packages. Really the only part of Hubble that has been constant over its service life has been the Optical Telescope Assembly, consisting of the famously misground primary mirror, a secondary mirror, and supporting structures like the trusses that keep them in alignment and the baffles to control reflections.
The instrument bay behind the primary mirror was designed to accommodate five scientific instruments at a time. The first service mission to the Hubble, which went up on Endeavour in 1993 and made Story Musgrave and Kathyrn Thornton household names, included the COSTAR corrective optics package, which restored Hubble’s fine focus and allowed it to explore the faintest and furthest objects in the universe. A dozen other spectrographs and cameras have occupied the equipment bays over the years, some being left in place for just a few years; others have been permanent parts of Hubble, generating data since the very beginning.
NASA Standard Spacecraft Computer: Built to be Modded
The foresight Hubble’s designers showed in recognizing the need to swap out science packages on Hubble influenced their designs for other areas of the spacecraft as well. As early as the mid-1970s, NASA realized that building custom subsystems for every mission, which had worked so well for the manned space flights of Mercury, Gemini, and Apollo, would be a hindrance to developing and fielding a wide range of spacecraft to explore near space. To that end, the NASA Standard Spacecraft Computer-1 (NSSC-1) was developed, with the goal of standardizing the tricky business of keeping pretty much any spacecraft that carried it stable and in orbit, as well as to run whatever science the mission planners could dream up, and to do all of that with basically just software changes.
NSSC-1 flew on a number of missions before finding its way into the Hubble equipment bay. By modern standards, the computer seems primitive, with discrete TTL chips and a mere 64k of 18-bit core memory (plated-wire memory was also used, and appears to be what was originally flown on Hubble’s NSSC-1). But the design was robust, radiation-hardened, and redundant at the module level, and most importantly, it gave mission planners for the first time what had largely been absent from prior spacecraft controllers: in-flight programmability.
On Hubble, the NSSC-1 forms the core of a subassembly called the Scientific Instrument Command and Data Handling Unit, or SI C&DH. This is a largish tray upon which the NSSC-1 module and other modules, like power regulators, interfaces to the scientific instruments, and remote control interfaces are attached. Everything is redundant, and the entire assembly is built to support being swapped out by spacewalking astronauts. In fact, the final service call paid to Hubble in 2009 included the replacement of the SI C&DH with a new unit, one with the original core memory modules upgraded to CMOS. The video below shows the SI C&DH undergoing ground tests before heading upstairs.
Trouble Upstairs
The current issue with Hubble centers around the SI C&DH assembly, which the media have mercifully dubbed “the payload computer.” On June 13, the NSSC-1 in the unit went into safe mode, which was the widely reported “glitch”. Initial diagnostics led operators to suspect that one of the memory modules had degraded enough that the computer couldn’t continue operating. Exactly how a single degraded memory module is causing this issue is a bit of a mystery; the NSSC-1 only needs one memory module at a time to work, leaving the other three — as well as the four memory modules in the backup NSSC-1 — in reserve. But given the available data, it seemed like the most logical explanation.
The initial attempt at recovery from the error state was pretty much what we’d all try first: turning it off and back on again. The commands to do that were sent up on June 14, to no avail. The next step was to switch the NSSC-1 to one of the three backup memory modules, but that command was met with a failure as well. This and other diagnostics led NASA to believe that the problem might not lie in the NSSC-1 or its memory modules at all, but rather in other modules on the SI C&DH, specifically the Standard Interface (STINT) hardware, or the Central Processor Module (CPM).
Testing that hypothesis required a radical step: turning on the backup NSSC-1 for the first time in space. The backup had only ever been tested on Earth, and has been sitting dormant since it was installed twelve years ago. This step was taken on June 23 and 24, and the data indicated that the backup computer suffered from the exact same fault as the primary computer.
It’s unlikely in the extreme that both the primary computer and a backup computer that has been sitting idle for more than a decade would fail in exactly the same way at exactly the same time, so that test provided strong evidence that the fault lies somewhere else. Current thinking is that some other piece of hardware on the SI C&DH, something shared by both computers, is actually causing the fault, and that the memory error is but a symptom. NASA is looking at the Command Unit/Science Data Formatter (CU/SDF), which formats and sends commands to the various science instruments and data to the ground, as the likely culprit. The other suspect is the Power Control Unit, which conditions and regulates the various voltage rails needed around the SI C&DH.
Is This the End?
While it sounds like NASA has a good handle on what’s going on with Hubble, given everything the spacecraft has been through over the last 31 years, it’s hard to say that the odds of recovery are in its favor at this point. That’s a shame, because aside from the current issue, the spacecraft and instruments are in great shape. But there’s also the fact that since the retirement of the shuttle fleet in 2011, Hubble is without the vehicle it was designed to be serviced by, and to boost it to a higher orbit from time to time to fight atmospheric drag.
What’s even more ominous is one of the last modifications made to Hubble in the 2009 service call: affixing a device called a Soft Capture Mechanism to the aft end of the spacecraft. Originally intended to provide a means for returning Hubble to Earth in the payload bay of a shuttle, the SCM’s purpose is now to provide an attachment point for a future deorbiting mission. Left to its own devices, Hubble will deorbit naturally sometime between 2028 and 2040, taking a trajectory that may leave the more than 800 kg mirror at least partially intact. The SCM will allow NASA to control reentry more precisely, giving Hubble a quick and dignified end to a decades-long career of unmatched scientific accomplishment.
It seems to us that a hypothetical deorbit mission could just as easily be an orbital boost mission, but that presumes NASA will be able to clear the current bug and get Hubble back to work. Here’s hoping that’s exactly what happens, and that the SCM doesn’t have to be used one minute earlier than necessary.
A big thanks to [David Anders] for pulling together a bunch of excellent sources on the NSSC-1 and SI C&DH and posting it on Hackaday.io. It was invaluable in pulling this article together.
[Main image via Wikimedia Commons]
Post a Comment