Sunday Space Shuttle Launch Scrubbed!

Now it is official – today’s space shuttle Atlantis launch has been scrubbed due to a problem with ECO sensor number three.

Quote from the NASA home page:

Today’s launch of space shuttle Atlantis has been officially scrubbed. It was announced on NASA TV at 7:24 a.m. EST.

Of the four engine cutoff sensors, ECO sensor number three gave false readings. NASA’s current Launch Commit Criteria require that all four sensors function properly. The sensor system is one of several that protect the shuttle’s main engines by triggering their shut down if fuel runs unexpectedly low.

Atlantis’ scheduled launch on Thursday was delayed after two of the four engine cutoff, or ECO, sensors in the shuttle’s external fuel tank gave false readings. A third sensor failed after the tank was drained of fuel. The sensor system is one of several that protect the shuttle’s three main engines by triggering their shut down if fuel runs unexpectedly low.

Atlantis “go” for Sunday launch

The meeting today went along the lines of the last meeting. Atlantis is now set to launch on Sunday at 3:21pm ET. NASA TV launch coverage begins at 6am ET. Tanking will start at 5:55am. Any sensor problems should manifest within one hour after tanking start, but of course it may also occur at any time later. The final test on the sensors is at T-9 minutes but they will be monitored down until T-31 seconds. Then, the automatic procedures are called in. At this point, a sensor failure will no longer cause a launch scrub.

The information in my last space shuttle Atlantis launch write-up is still valid. Except, of course, that NASA is now go for launch on Sunday. Weather conditions are favorable for launch, around 80% chance of launch. The emergency landing sites are also mostly “go”. A 24 hour delay bring slightly less favorite weather, with a “only” a 70% chance for launch on Monday. Interestingly, 70% were also mentioned for Tuesday (not previously considered a potential launch day).

A launch scrub on Sunday because of the ECO sensors does not outrule a launch attempt on Monday.

Mr. Hale strongly expressed the view that even if things go really wrong after liftoff, there are ample safe abort modes. This includes landing in Europe as well as aborting to a lower orbit. He re-iterated that a launch abort scenario is highly unlikely.

Launch attempts on Sunday and Monday will preserve the potential two-day mission extension to do an extra spacewalk. Later launch attempts may also preserve it, there is some planning underway.

The number of tankings and detankings, including tests, is limited because each tanking increases the risk of foam debris. This is because the ultra-cool propellants cause stress on the tank material, cause it to shrink and expand.

Future missions will see a number of changes, for examples in the engines. Mr. Hale states that he has tasked “considerable resources” to fix the ECO sensor system issue for future flights.

Finally, let me just quote the NASA home page:

Today’s Mission Management Team meeting has concluded. We are “go” for a Sunday launch attempt.

A post MMT news conference on NASA TV is set for approximately 4:30 p.m. EST. The participants will be Wayne Hale, Space Shuttle Program manager, LeRoy Cain, MMT chairman, Doug Lyons, STS-122 launch director, and U.S Air Force Captain Chris Lovett, 45th Weather Squadron.

So let’s hope we see this tomorrow afternoon:

Update on Atlantis Launch Delay

Space Shuttle Program Manager Wayne Hale at December, 7th 2007 Press BriefingThanks to the “Space Multimedia” site, I have been able to watch yesterday’s press conference. Now the big picture clears up.

First of all, I was very wrong in my statement in my earlier STS-122 status article that NASA will launch without working ECO sensors – well, kind off…

In fact, it is quite the opposite. As NASA’s space shuttle program manager Wayne Hale stated the proposal is to require all four ECO sensors to work during launch preparations. Thus, the criteria has actually been tightened. Previously, only three good sensors were needed. Mr. Hale said that this plan came from the astronaut’s office. This goes well together with a posting I read in the Flickr STS-122 group created by Armando Perdomo. There, someone from an astronauts family posted (excerpt):

Anyway, what I heard that is interesting is that NASA offered the astronauts to go “as is” – with 2 sensors. They declined. My thought is that they didn’t decline out of fear of personal safety, but more of the consequences to NASA should something go wrong.

This not only proves what great people the astronauts are, it fits well within the press briefing picture. Mr. Hale also explained the reasoning for the new launch commit criteria: previous experience showed that failed ECO sensors always worked on the second launch attempt. Nobody knows why. Mr. Hale actually said we saw “that they magically work”.

Even if the cause is not know, one can assume that if the failure cause is the same as on previous missions, the sensors should behave the same. So seeing an additional failure on the next tanking would indicate that the failure cause is different. In this case, the launch will be scrubbed, as it would be an even less understood situation.

If the sensors work as expected during countdown, there is a good chance they will continue to function (based on previous experience). However, the problem was quite erratic and what was seen was different than before. This is the additional risk that NASA is accepting: the ECO sensor system may fail again during the launch. This is where new, still to be fully developed procedures kick in. They use new sensor instrumentation plus other systems to decide what to do in such a case. Here, some manual interaction may be required. NASA is actually reading itself to do the ascent without the ECO sensors.

So to sum it up: NASA requires 4 working sensors during countdown. If they don’t work, launch will be scrubbed. After liftoff, loss of the ECO sensors will be acceptable due to new procedures. However, it is hoped that they will continue to work if they did during countdown.

The launch has been moved to Sunday to support creation of the new procedures. The launch window has been shrunk to one minute to provide as much spare fuel as possible. Both of these measures are to address a failure scenario which will hopefully not occur due to the four-sensor launch scrub requirement.

It is important to note that this plan is not final. The mission management team meeting did not find concurrence from everyone. So folks were sent back to think about the plan, gather more data, run more simulations, get a better grip on everything – and meet again today at 1pm ET. Today, the final decision will be made. Depending on the findings, we may end up with a totally different set of options. The time of the post MMT press conference has not been announced yet. I guess they now refrain from posting any time because we’ve seen that any prediction is at least inaccurate ;)

Let’s assume NASA sticks with the plan. So what may happen? If there is a launch scrub on Sunday due to the ECO sensors, I guess it is mostly “game over” for the December launch window. If there is a scrub for weather reasons (more probable now due to the short one minute launch window), an additional launch attempt can be done on Monday. Chances of weather prohibiting launch are just around 30% for both days, so there is a fairly good chance that either of these attempts will succeed. If they don’t, the oxygen and hydrogen tanks for the fuel cell system must be topped off. That requires a three-day stand-down. Interestingly it was said that this means the next possible launch attempt would be on Thursday (is my math wrong?). It was mentioned that they could also launch on Friday. The launch time will be approximately 25 minutes earlier for each attempt.

With a Sunday or Monday launch, the two-day mission extension to do the extra ISS solar array rotary joint spacewalk is still an option. For Thursday and Friday this seems not to look well. They intentionally didn’t go into any specifics, but information I previously obtained tells me an extension would not be possible if launched late next week.

Finally, the commented that the overall shuttle launch schedule would not necessarily be affected if Atlantis’ launch would need to be deferred to early January 2008. If, however, the sensor issue would really need to be troubleshooted, the whole launch schedule would probably be affected. And it was strongly expressed that all of this post-December launch scenarios are highly hypothetical and should not be considered real for the time being. I concur to this – let’s solve today’s problems today and look at further issues when they come up.

Even though I am in no position to judge, the plan presented seems to be very sound. I hope they will not find any hidden problems and can attempt for launch on Sunday. Let’s keep our fingers crossed…

Next shuttle launch attempt Sunday

Space Shuttle Atlantis at the Pad after launch scrubThe next shuttle launch attempt for Atlantis STS-122 mission is now set for Sunday at 3:21p. As it looks, NASA now actually attempts to fly without the ECO sensors. As I have written in my essay yesterday, that is not necessarily a bad thing to do.

To do this, new guidelines (procedures) for both the astronauts as well as mission control need to be developed. This is quite complex. It could not be done to support a Saturday launch. It is hoped that the additional one day of delay allows to finish those procedures.

On close look, the launch window has also been shrunk. It is now just a one-minute launch window. It obviously is done to preserve fuel. The primary risk of launching without the ECO sensors is fuel depletion. The best mitigation is to make sure to have enough propellants on board. Depending on when the shuttle is launched, there is more or less fuel needed to reach the desired orbital position. This is a matter of seconds. So NASA is now going for the optimal launch Window with the least fuel consumption. While this increases the risk of launch scrub, it reduces the risk of running out of fuel. It obviously is an excellent decision to go for the short launch window.

There is another mission management team meeting today. They will look at new data as well as the status of the new launch procedures. Depending on how things are, they will give a “go” for Sunday launch – or not. This is how to react to a situation like the one that is faced now: do the right thing to support the mission, but keep an eye on all options until the last minute.

And a word to those of you who need to leave Kennedy Space Center before launch (like fellow launch viewer Bill Rose). I know your disappointment. I know it too well. I went through all of this myself in summer of 2006, when I tried to view Atlantis STS-115 mission. Even though I had quite a lot of time to stay there, it was delayed so often (and even because of a Hurricane!) that I finally needed to leave. That was a sad experience. However, I am glad that they only launch when it is as safe as possible and I am sure you will agree with me.

What I recommend is to go to th Kennedy Space Center today and try to get on the up close tour. I am not sure if it runs due to the current pad activity. But if it does, it will probably provide a great experience, getting you as close to a real space shuttle as you never again will be in your life. It doesn’t matter if the RSS is still blocking some of the view, it will be spectacular in any case.

Now let me quote the NASA homepage to also get the official word over to you:

NASA is targeting the launch of space shuttle Atlantis no earlier than Sunday, Dec. 9, at 3:21 p.m. EST from the Kennedy Space Center, Fla. Shuttle program managers made the decision after a meeting Friday to review data on a problem with a fuel cutoff sensor system inside the shuttle and its external fuel tank.

Because of the length of the meeting, the managers agreed that targeting Sunday would allow the launch and management teams appropriate time to rest and prepare. The Mission Management Team will meet Saturday at 1 p.m. to decide whether to make a Sunday attempt. A news conference will be held after the meeting’s conclusion.

To Launch or not to Launch?

The missing management team at NASA will have a very tough day today. As far as I know, a series of meetings have already begun. They are all about the ECO sensor issue – and how to proceed. It is hard to predict the outcome.

Let’s try to put some pieces together. Keep in mind that all of this is my personal guesswork. So tomorrow you may judge me based on what actually happens (hey, will I really do that…?).

From a technical point of view, it looks like the sensor issue can not be fixed quickly enough. The fact that there are intermittent problems in at least three (all four?) sensors makes it look like a problem with the electronics or cabling – not the sensors themselves. Fixing that would require at least three days at the pad – if it can be done there at all. Some think that a rollback is necessary. Troubleshooting the sensor system would best be done by a tanking test. However, that test has been called off by NASA in favor of consumables replenishment. To me, this is an indication that NASA has given up on fixing the ECO sensor system. In the press conference, they also mentioned that they are looking into rationale to fly as is. That also supports my argument.

But stop. Don’t say “better save than sorry”. Of course crew safety is first. But then remember that the ECO sensors are part of a backup system that kicks in when some other things already went wrong. They prevent the space shuttle main engines from running dry. So what? Do you think NASA puts not enough fuel into the tank? Obviously that isn’t the case. There is more than enough fuel in the tank for launch. So for the tank to run dry, something must be working quite wrong in the first place. Something like a leak or a similar serious issue. That’s an important point: the ECO sensors are an additional line of defense, but one that is never used in a normal flight. But, then, of course there is a reason for them to being there.

No let’s look at program constraints. If STS-22 can not make this launch window, it is not catastrophic, but will cause a wiggle in the schedule. Quite one, I think. That would cause already-much delayed international space station construction to be delayed even further. It will possibly also affect Constellation and Ares if the STS-125 hubble service mission must be moved. Pad 39B can only be handed over to Ares after the mission, because the STS-125 rescue launch-on-need mission needs to be at a pad to be able to launch it soon enough in case it is needed. So both pads are needed whenever STS-125 launches.

I guess there are also ISS constraints. We already know that construction is much delayed. Guess what? Space hardware has an expiry date. For example, the Columbus module that shall now be delivered by Atlantis is made to endure ten years in orbit. Of course, everyone hopes it will last longer. But its a simple fact: the longer hardware is in space, the less lifespan remains. Now think about all those years that other ISS components are already up there in space – waiting for the construction to finally complete. And every space ship is only as strong as its weakest part. May it be that any delay shrinks the time the ISS as whole can be productive in space (right now, its not really productive – much maintenance and construction going on and few science). Of course, a one month launch slip won’t hurt. But any larger delay will.

There is probably also one other risk with delaying the flight: if the ISS crew has to carry out vital tasks during STS-122 docked operations, their skills will fade. Astronauts are extremely well trained. But it is for a good reason that the practice until the last moment. Practicing at the ISS is rather limited. So the longer it takes, the less well-prepared the crew is. Of course, I do not know that really is an issue at this time. With STS-120, it was a very vital concern, because of all the complex staging spacewalks required by the ISS crew.

Think about it: it is not “just” the shuttle crew that must be kept save. There are also others (I have to admit that I too often overlooked that part of the picture).

As I said, its a tough decision…

Even if they launch as is, I do not think that the space shuttle’s crew life is more at risk than at any other launch (remember: spaceflight *is* a risky business). However, if something goes wrong and NASA needs to rely on the sensors, they will probably use ultra-conservative procedures. At least this is what I would expect. Thus, a launch abort would be much more likely.

And now think what happens if there is TAL (Transatlantic Abort Landing). First of all, it would be expensive. But even worse, how would the public react? Wouldn’t that be the last nail in NASA’s space shuttle program (and probably Constellation as well)? So there is a high risk in that, too. At least from an overall program perspective.

There are also political implications. Especially in this mission. Atlantis carries the European Columbus module into space. That cost roughly a billion (!) dollars. I guess the ESA would not be very amused if Columbus would be damaged due to some launch failure. Please note that I do not talk of a catastrophic failure, but of an abort, which can cause harm anyway. Given the fragile relationship between NASA and ESA, there is obviously some political thought that must be involved in the decision making.

I hope you by now have gotten an idea how many things and details need to be considered. I am sure I am still just scratching on the surface. The bottom line is that none of us outsiders will be in the position to judge it correctly. We should remind ourselves about that when we talk about the final mission management team decision.

Also, we do not have any solid data. The most important thing missing is probabilities. How probable is it that the ECO sensors are actually needed – I mean that the tank really runs dry? Is it 1 in 5,000? Or 1 in 100? The former one is probably a risk that can be accepted (the debris hit probability is much higher) while the later one obviously draws a different picture. Without solid data, you can not decide.

The whole situation looks much like the RCC panel issue we had with STS-120. However, at that time there was some more solid data at the time of decision making.

This is what the meetings are currently about: gathering data, looking at options and then deciding on what is the best thing to do after everything is put together. Such a decision, whatever it may be, easily upsets some folks (us, maybe?) who do not have the full reasoning at hand. I personally trust NASA guys to do the right thing. These are very bright people, doing a tremendous job in a very unforgiving business.

But would is my personal bet? I promised to do one… Tough, really tough. If I had to decide with just the information that is in this post… Well, I would probably fly as is on Saturday. Am I having launch fewer? Judge yourself. Here is my line of defense:

  1. The ECO sensors are “just” a second line of defense. Something else must go seriously wrong in order to need them.
  2. During launch, crew and mission control can manually monitor ECO sensor performance. If the show dry early into flight, this must be a failure. I admit that it gets harder the later we go into flight.
  3. If an abort is needed, there are established procedures and no catastrophic outcome is to be expected. However, the shuttle program and NASA itself would probably pay a big price. That risk is accepted.
  4. The launch and construction schedule can be maintained.
  5. The risk for the ISS crew is minimized

If I look at my arguments, number one and two are the strongest one. If looking from a political point of view, you may also come to the decision to postpone to avoid the program risk. And, of course, depending on probabilities, you would like to avoid a lunch because of crew safety risks. But now I am spinning in circles ;)

I’ve done my bet, now let’s see what the real outcome is.

Bill’s Launch Viewing Report

I was very pleased to find a real nice personal report from Bill Rose in my mailbox. Many thanks for it Bill, much appreciated! I hope you all enjoy it as much as I do:

We have long dreamed of seeing a shuttle launch and finally we are trying to do it. We left Philadelphia on Wednesday 5 December. Flights were delayed by a snowstorm. The airport ground crew passed the time by having snowball fights. After additional delay while our plane was de-iced, we finally left the northeastern United States and soon landed in Florida. It was 3 a.m. Thursday when the 5 of us (my wife and I, Grandma, and 2 girls ages 16 and 9) arrived at the hotel. We had phoned for causeway tickets about 20 minutes after they went on sale in October, but they sold out in 15 minutes. We plan to view the launch from Titusville.

We awoke Thursday morning and very soon learned of the fuel sensor problem. Given the past history of ECO sensors, and since two failed, we knew a scrub would occur, and soon it was official. Off we went to Sea World. What a great place! The Shamu show mixed environmentalism, patriotism, Hollywood, Las Vegas… – we loved it! The dolphins, manatees, sea lions and other creatures were a treat to see and, in some cases, touch.

Next launch attempt Saturday. We fly back to Philadelphia on Sunday night.

I hope that Bill and his family will be able to witness a great launch on Saturday! He also proves the point that a shuttle launch is a whole-family event ;) And, as I side-note, I too think that SeaWorld is the next great place to be right after Kennedy Space Center. Let me add one of my pictures to explain it:

If you yourself have an experience to share, please email me. I’d love to read your stories and share them. Maybe we can even build up a pool of launch viewing experiences!

Trying to preserve extra EVA

Now I know why there is no tanking test today. CBS’ status report (a long reading with many details) provides the reason: it is to preserve the capability to do an extra EVA (“extravehicular activity”, also known as a “spacewalk”) on the STS-122 mission.

Each launch attempt draws at least a bit from the internal consumables. Let me quote the relevant part from CBS’s page:

In the meantime, NASA managers ruled out a fueling test Friday and decided instead to top off the shuttle’s onboard supply of liquid hydrogen to power the ship’s electricity producing fuel cells. That would permit launch attempts Saturday and Sunday and still provide enough on-board supplies for a two-day mission extension and the addition of a fourth spacewalk.

This makes an awful lot of sense if you take the problems with the international space station’s solar array rotary joint into account. While that poses no problem for Columbus, I think the Kibo module can not be attached (or at least not sufficiently be powered) if the SARJ issues has not been solved until then.

NASA is hoping for a two-day mission extension to do an extra spacewalk in support of the SARJ issue. However, this is only possible if consumables permit. Having full tanks on launch is a number one requirement to achieve that goal…

STS-122 launch status Update…

After the scrub of yesterday’s launch attempt, space shuttle Atlantis will launch not earlier than Saturday, December, 8th 2007. However, even that date is under consideration.

The Rotating Service Structure is re-attached to Atlantis
The Rotating Service Structure is being reattached to Atlantis
As of the press briefing yesterday night, there were issues with two of the ECO sensors. This was known and expected. However, NASA also told that there were some anomalies with some other ECO sensors. So right now this looks like a much bigger issue than it originally appeared. I personally think the chance for a simple sensor error is weak – why then do we have these multiple failures? So the solution is probably somewhere in either electronics or their connection path to the sensors. And if that’s the case it is questionable if it can be fixed at the pad.

In the press conference, it was stated that today should be a tanking test. However, the RSS has been re-attached to Atlantis and I have been told that no tanking test can be done with the RSS attached. Whatever they do, there is only very limited ability to repair systems within the 48 hour scrub window.

Consequently, also stated on the press conference, the Saturday launch will most probably only happen if NASA finds rationale to fly with two dead sensors. If these two sensors consistently behave wrong and the other two work well, there might be good reason to fly as is. In my personal opinion, though, this is not likely, because the other sensors also seem not to work consistently.

In short words: the guessing game is open. I do not expect any more solid information until after today’s press conference, which is scheduled around 5pm ET.

RSS rotating after shuttle launch scrub

After the shuttle launch scrub on December, 6th, the rotating service structure has been rotated back. This is done both to protect space shuttle Atlantis from the weather and also to allow technicians access to it. Troubleshooting is currently in progress and the next launch attempt is no earlier than Saturday, December, 8th.

And this one is from a slightly different perspective:

The animations were created from public NASA webcam images. To start the animations, you must press the (small…) play button on the lower left! I also have a higher resolution version at my personal web site. If you are interested in details, these are probably the ones you would like to look at.