Today’s tanking test gathers additional data on ECO sensor (What is an ECO sensor?) failures with the ECO sensor circuits being fully instrumented. In order to gather data, an ECO sensor failure is needed. Before the start of the tanking test, it is anticipated that ECO sensor #3 will fail wet and all others will work correctly.
2:08a: The tanking test just went into fast fill mode. During the last launch attempt, ECO sensor #3 failed only a few minutes into fast fill mode.
8:14a: If I got the commentator right, there already is a failure of a sensor. That would be good.
8:21a: NASA TV commentator: “It appears that #1 sensor has failed, but the hydrogen console has not officially made that call. So they are still looking at it. Number 2 sensor has been intermittent. The engineers at the main propulsion system console are very intently looking at their data right now. The intermittent data on sensor #2 is somewhat of a surprise because we haven’t seen that before on that sensor.”
So as it looks, the ECO sensor system behaves again different from the last tanking, which was at the December, 9th launch attempt.
8:23a: NASA TV: “The propulsion console has provided confirmation that sensor #1 on liquid hydrogen has hard failed, which is what we wanted. The #2 and #3 are intermittent, we are watching for what those are going to do. We have not seen any activity of number 2 sensor on prior tankings.”
8:26a: NASA TV: “The hard fail on #1 is definitely what we need in order to do the troubleshooting at the pad later this morning.”
So far, this looks very good. Based on analysis done in the past days, sensor #1 is not expected to return to a non-failed state. So this one will hopefully a good candidate for the troubleshooting which will then be able to pinpoint the culprit. The intermittent failures of #2 and #3 are outside of expectations, at least as far as I understood the analysis done. However, if they now remain functional there still may be a good explanation for that. Let’s see how things evolve…
9:32a NASA commentary just announced that engineers have made up their troubleshooting plan. “The TDR sensor equipment will be focussed on sensors #2 and #3 as they have been intermittent and it is the greatest source of interest to see where that intermittent reading is coming from. Number 1 and 4 will be recorded continuously. … Right now they are configuring for the console activity that would be required once we go into stable replenish … very shortly. … So the final inspection team is preparing to enter into the pad, but we probably have another half an hour or so before we are into stable replenish.”
9:48a NASA TV: “The liquid hydrogen tank is full now and in stable replenish. The liquid oxygen (LOX) is at 80%. We’ve probably about another 25 to 30 minutes or so before liquid oxygen is in stable replenish.
10:00a: NASA TV: “We will begin to drain the tank between 1p and 2p and once the tank is drained the teams go back to do additional trouble shooting. A lot of data has already been collected on sensor #1 because it failed early.”
10:13a: “The problem with the liquid ox pump appears that a fuse has blown, so they are setting to the backup lox pump, … which will us back to where we were close to the stable replenish. This will take around 45 minutes to an hour. Why the fuse blow is still not clear”. So we have some delay in the troubleshooting plan.
11:45a: I needed to go out for a while, thus no updates. Atlantis is now in stable replenish state and troubleshooting is right now happening at the pad. The NASA TV commentary just announced that teams are now looking at ECO sensor #4 and the 5% sensor. Here are some pictures from the testing:
The picture above shows a room that is right inside the mobile launcher platform. This is where data today is analyzed.
11:52a: NASA TV “The … team is now on the mobile launcher platform and gathering data on sensor number 2 and 3.”
11:53a: a side-note: the ISS spacewalk just has completed, the spacewalkers are back into the airlock, which is now being repressurized. They did not find any obvious problem, but collected samples to be returned to earth in January on board STS-122.
12:06p: NASA TV: “Data will be collected remotely during de-tanking.” The equipment is currently being set up for this process. People are permitted close to the space shuttle only during stable replenishment phase (which essentially means no tank or detank operation is taking place). During detanking, they must leave for safety reasons. The set up process can be seen in this picture:
6:12p: and now those members at the controls can be seen leaving the control room. Everything is now set up for remote monitoring. The NASA TV commentary tells that good data has been gathered. The tank will be drained to 5% while instrumentation is on. Once it is fully drained, members of the red crew will return for further troubleshooting.
And on this picture, a wire can be seen. This wire is tapped into the ECO sensor circuits and connects to the time domain reflectometer (TDR) equipment that is used to do a detail analysis of the circuits. TDR is an off-the-shelf technology in wide spread use e.g. by cable companies to detect faulty lines.
around 12:25p: I was too slow for a quote, but NASA TV commentary stated that everyone seems to be quite happy with the data gathered. It sounded like a successful test. The commentator also announced that more – and official – information is available in the news briefing later today. For me (being six hours ahead), it probably is too late, but I’ll then do a wrap-up tomorrow morning. Let’s hope that they have actually managed to find the culprit and nail it down – and also let’s hope that the rest of the day (detanking test) also goes well.
12:40p: it looks like the red crew is now leaving the pad after finishing late inspections. I noticed they used binoculars and telescopes during that process. I (now) think they also visually checked for ice buildup and maybe some other things. Probably just part of the usual procedure. Here, the red crew can be seen leaving the mobile launcher platform (the last frame I was able to capture, sorry for the blur…).
The NASA TV commentator announced a few minutes before that engineers are ready for detanking once the red crew leaves. So I assume detanking will begin shortly.
12:48p: NASA TV: “The final inspection team has now left the pad … In the mean time we had a failure of ECO sensor #3, the second sensor that had failed. It was intermittent … and we are collecting data on sensor #3 … right now which is able to be done remotely much as we got the initial data from sensor #1 when it failed earlier today.”
I didn’t get the full quote, but as far as I understood, sensor #3 has now also a hard failure. That, IMHO, would be somewhat bad news because it means that the condition will change once the tank is loaded. However, this is just my uninformed guess. Also, on a actual launch, the shuttle would be long gone and fly in orbit. So that may not mean anything at all. I should stop speculating ;)
12:52pm: NASA TV: “We are close now to start the external tank draining operation. and we did indeed get the kind of data that we needed to get to the (bottom of?) that trouble.”
12:58p: NASA TV: “We did see sensor #3 also fail, so we try to get some data from it before we start draining.” So it actually has a hard fail and NASA uses the opportunity to get something out of that. To me this sounds like it failed just a few minutes ago, otherwise the red team would have looked at it (wouldn’t they?).
01:00p: NASA TV: “on the recently failed ECO sensor #3” – and indeed, it recently failed. Right in time, one could say…
3:00p: as I wrote, I have been away from my computer for a while. NASA TV has ended coverage of the tanking test in the mean time, so I can not provide any more status updates. I’ll try to do another post after the news briefing, but that will probably be tomorrow (I hope I’ll be able to get hold of an archived version of the news briefing). All in all, it looks like an excellent day at Kennedy Space Center, with good results that will bring up the root cause of the ECO sensor issue. Let’s hope that I have the right reception of today’s events.
Thanks to everyone who cared reading my post! I hope it has been useful and will be as a reference.
I’ve now also written down some results from the post-test news briefing.