Author: Rainer Gerhards

Rainer Gerhards ist Hauptautor dieser Seite und schreibt die meisten Artikel. Weitere Informationen über Rainer Gerhards können hier über das Hauptmenü abgerufen werden. Die Biografie findet sich z.B. hier.
Posted on

New rsyslog HUP processing

There has been some discussions about rsyslog HUP processing. Traditionally, SIGHUP is used to signal the syslogd to a) close its files and b) reload its config. Rsyslog carried over this behavior from sysklogd.

However, rsyslog is much more capable than sysklogd. Among others, it is able to buffer messages that were received, but could not yet be processed. To remain compatible to the sysklogd of doing HUP, rsyslogd does a full daemon restart when it is HUPed. Among others, that means that messages from the queue are discarded, at least if the queue is configured with default settings. David Lang correctly stated that this may surprise some, if not most users. While I am still of the view that discarding the queue, under these circumstances, is the right thing to do, I agree it may be surprising (I added a hint to the man pages recently to reduce the level of surprise).

Still, there is no real need to do a full daemon restart in most cases. The typical HUP case is when logrotation wants to rotate files away and it needs to tell rsyslogd to close them. Actually, I asked if anybody knew any script that HUPs rsyslog to do a full config reload. The outcome was that nobody knew. However, some people liked to stick with the old semantics, and there may be reason to do so.

I have now implemented a lightweight HUP to address this issue. It is triggered via a new configuration directive, $HUPisRestart. If set to “on”, rsyslogd will work as usual and do a (very, very expensive) full restart. This is the default to keep folks happy that want to keep things as backwards-compatible as possible. Still, I guess most folks will set it to “off”, which is the new non-restart mode. In it, only output files are closed. Actually, the output plugin receive a HUP notification and can do whatever it likes. Currently, onle omfile acts on that and closes any open files. I can envision that other outputs, e.g. omfwd, can also be configured to do some light HUP action (for example close outbound connections).

The administrator needs to select either mode for the system. I think this is no issue at all and it safes me the trouble to define multiple signals just to do different types of HUP. My suggestion obviously is to use the new lightweight HUP for file closing, which means you need not to change anything for logrotate et al. Then, when you need to do a config reload, do a “real” restart by issuing a command like “/etc/init.d/rsyslogd restart”. And if there really exists a script that requires a config-reload HUP, that should be changed accordingly.

Posted on

rsyslog v4

Finally, it is time to think about the next major rsyslog release! I have done many enhancements in v3, but the latest performance optimization work leads to a couple of significant changes in the core engine. I think it makes sense to roll these into a new major release. That leaves folks with the option to keep at the feature-rich v3-stable branch, while avoiding some of the potential unavoidable bugs in the upcoming v4 branch.

From a feature point of few, version 3 would have been good for at least three to four major releases, which I did not do just because to prevent you from coming scared by the pace with which we are moving ;)- so I think it now is a perfect spot to begin developing v4. I hope that we will see a first beta of that branch around xmas, which, I think, is a nice gift.

Posted on

syslog appliance website online

I have now set up a first basic web site for SyslogAppliance. It is not great yet, but it provides a stable reference point for any work that comes up. So people can hopefully begin to use this site as a pointer for useful resources.

As a side-note, you may notice I am using a .de (German) domain. Thanks to the spammers, com, org and net domains are already used by spamming sites. And I thought it does not matter if we use a de domain. After all, we live in a time where domains from the Cocos Islands (cc) or Tuvalu (tv) are being abused for generic purposes, so why not use .de for a generic site, too?

Oh, and one interesting find: at least one person actually downloaded and tried the version of SyslogAppliance I uploaded yesterday. How do I know? I had forgotten to include the phpLogCon user in the README ;) [of course, this is fixed now ;)]

Posted on

rsyslog performance

Thanks to David Lang, I have been able to gather some performance data on rsyslog. More importantly, I have been able to improve rsyslog’s performance dramatically while working with David. He does not only dispense good advise, he has also a great test environment which I lack. If you would like to see how things evolve, be sure to follow this (lengthy ;) thread: http://kb.monitorware.com/rsyslog-performance-t8691.html.

But you are probably interested in actual numbers. The current v3-stable (3.18.x) manages to process around 22.000 messages per second (mps) with DNS name resolution turned on and about double that value without. That’s not bad, but obviously there is room for improvement.

Thanks to our combined effort, we have reached a state where we can process more than 100,000 mps and there is an experimental version (applying some lock-free algorithms) that goes well beyond 200,000 mps. I am not yet sure if we will pursue the lock-free algorithm. There are ample of additional ideas available and I am quite positive we can push the limit even further.

All numbers were tested with a minimal configuration (one udp input, one file output) on a capable multi-core machine. The numbers above are for sustained traffic rates. More messages can be accepted (and buffered) during bursts.

Posted on

virtual syslog appliance

I’ve just recently blogged about my syslog appliance idea. Now this has become reality. There is the first 0.0.1 version of rsyslog and phplogcon as a virtual appliance.

For starters, I have created a very simple system. While I have a number of options for the operating platform, I started to Ubuntu JeOS mainly because it had good guides for getting started with an appliance quickly. Being based based on Debian also was a plus. Some may argue that the downside is that the log appliance currently requires VMWare. While I agree this may be an issue, it is not an extremely big one especially as VMWare server runs quite well under Linux and is free to use.

I will investigate Red Hat’s AOS, but I think I need to get some results from the app point of view first and JeOS looks quite promising in this regard.

For now, I have even started with the stock rsyslog package, which is quite outdated on that platform. However, I’ll do a couple of iterations in the next days and so will come up to the current release soon. But for what the appliance currently needs, the older version is not really a problem.

I am now very interested in feedback on this new offering. The appliance can be downloaded from

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-136.phtml

One of my next actions is to set up a dedicated site, which will make finding (and providing!) information on the appliance much easier. But one thing after the other…

Oh, and one thing on the licensing: the appliance is free for non-commercial use. However, we intend to request a moderate fee for commercial use, which I think is a fair policy. Of course, all appliance components are freely available.

If you try out the appliance, please provide feedback!. I have set up a dedicated forum at

syslog appliance forum

As I said, the initial version will probably not as “plug and play” as I hope, but I am very positive we are on a good path. Besides, it is an exciting project.

Posted on

India is going to the moon…

Everybody seems to want to go the moon these days. Russia does, China does, Europe, as usual, “says” it does (if someone else provides the ferry ship ;)) and the US, of course, will, too.

Today, India has launched a moon mission, just to tell us they, too, are serious about this topic. A rocket carrying the Chandrayaan-1 probe rocketed into the skies at India’s spaceport Sriharikota. Chandrayaan-1’s mission will last two years. It is tasked to create a detailed map of minerals and chemical properties of the moon surfaces, as well as general surface structures.

The moon seems to promise big business. It is also politically quite important. With the US right in front of a very important election, it will be very interesting to see which direction the new administration will take. NASA’s constellation program is underfunded and has unrealistic goals if being worked on at the current (finance-dictated) pace.

Will the US be among the last folks to go back to the moon? The Russians are on a good path already and seem to have funding and a commercial vision. Or will a new moon race start, where the US demonstrates technical leadership? Interesting question, time will tell. At least we have a new player who seems to be serious inside this game…

Posted on

Hubble partly restored, Atlantis heading back…

The Hubble repairs go well, but unfortunately not too well. As NASA reports, the restoration succeeded only partly, some systems are still defunctional:


On Wednesday, October 14, engineers at NASA’s Goddard Space Flight Center reconfigured six components of the Hubble Data Management System and five components in the Science Instrument Command and Data Handling (SIC &DH) system to use their redundant (or B) sides. This was done to work around a failure that occurred on September 27 in the Side A Science Data Formatter in the SIC&DH and resulted in the cessation of all science observations except for astrometry with the Fine Guidance Sensors.

The reconfiguration proceeded nominally and Hubble resumed the science timeline at Noon ET on Thursday, October 16. The first activities out of that on-board science timeline were the commanding of the science instruments from their safe to operate modes. This occurred nominally for Wide Field Planetary Camera 2 and the Near Infrared Camera and Multi Object Spectrometer. However, an anomaly occurred during the last steps of the commanding to the Advanced Camera for Surveys. At 1:40 pm, when the low voltage power supply to the ACS Solar Blind Channel was commanded on, software running in a microprocessor in ACS detected an incorrect voltage level in the Solar Blind Channel and suspended ACS. Then at 5:14 pm, the Hubble spacecraft computer sensed the loss of a “keep alive” signal from the NASA Standard Spacecraft Computer in the SIC&DH and correctly responded by safing the NSSC-I and the science instruments. It is not yet known if these two events were related.

The investigation into both anomalies is underway. All data has been collected and is being analyzed. The science instruments will remain in safe mode until the NSSC-I issue is resolved. All other subsystems on the spacecraft are performing nominally and astrometry observations continue.

But at least some observations can be carried on.

At the same time, Space Shuttle Atlantis is heading back to the VAB to get to a save haven while the Hubble repair mission is postponed. Unfortunately, a rod struck parts of Atlantis while it was removed from the launch pad. It is now investigated whether or not repairs are necessary. From what I have read, the external tank probably needs some attention, the rest of the space shuttle stack seems to have not been damaged. Thankfully, there is enough time left until mid-February, which is considered the earliest launch date.

Posted on

A German rsyslog forum…

Some of you may know that I am a native German speaker. I thought I started an interesting experiment: the “deutsches rsyslog forum” (which means “german-language rsyslog forum” ;)). It is targeted to those who prefer to express themselves in German language.

The interesting question, though, is if this forum will actually attract much attention. In German IT, there is a tendency to think that almost everyone speaks sufficiently well English so that he or she can obtain enough information to get the job done. If that proves true, there would be very little benefit in localizing any of the documentation into German language. So before seriously considering that, it is probably a good idea to do some testing. For the very same reason, my buddy Tom Bergfeld currently translates the rsyslog home page, mainly the announcements, into German. We will do a similar experiment to phpLogCon and evaluate both together after some time has progressed.

Please drop me a note if you have an opinion on this, or on localization at all.

Thanks,
Rainer

Posted on

NASA’s Ares I Rocket Passes Review To Reach Critical Milestone

NASA has taken a major step toward building the nation’s next generation launch vehicle with Wednesday’s successful completion of the Ares I rocket preliminary design review.

Starting in 2015, the Ares I rocket will launch the Orion crew exploration vehicle, its crew of four to six astronauts, and small cargo payloads to the International Space Station. The rocket also will be used for missions to explore the moon and beyond in the coming decades.

The preliminary design review is the first such milestone in more than 35 years for a U.S. rocket that will carry astronauts into space. The review was conducted at NASA’s Marshall Space Flight Center in Huntsville, Ala. It examined the current design for the Ares I launch vehicle to assess that the planned technical approach will meet NASA’s requirements for the fully integrated vehicle. That ensures all components of the vehicle and supporting systems are designed to work together.

“This is a critical step for development of the Ares I rocket,” said Rick Gilbrech, associate administrator of the Exploration Systems Mission Directorate in Washington. “Completing the preliminary design review of the integrated vehicle demonstrates our engineering design and development are on sound footing, and the Ares I design work is taking us another step closer to building America’s next mode of space transportation.”

The preliminary design review included more than 1,100 reviewers from seven NASA field centers and multiple industry partners. The review is the final step of this design process. Teams representing each major part of the Ares I rocket — the upper stage engine, first stage and upper stage — all have conducted similar reviews during the past year.

The preliminary design review is one of a series of reviews that occurs before actual flight hardware can be built. As the review process progresses, more detailed parts of the vehicle design are assessed to ensure the overall system can meet all NASA requirements for safe and reliable flight. This process also identifies technical and management challenges and addresses ways to reduce potential risks as the project goes forward.

“Risk assessment is a very important part of the process,” said Steve Cook, manager of the Ares I rocket at Marshall. “It allows us to identify issues that might impact the Ares I rocket. For example, we identified thrust oscillation – vibration in the first stage – as a risk. In response to this issue, we formed an engineering team. The team conducted detailed analyses and reviewed previous test data, and then recommended options to correct the problem.”

“We intend to hold a limited follow-up review next summer to fully incorporate the thrust oscillation recommendations into the stacked vehicle design,” Cook added. “Identifying risks that can impact the project and resolving them is a necessary and vital part of the development process.”

With the completion of this review, each element of the Ares I rocket will move to the detailed design phase. A critical design review will mark the completion of the detailed design phase and allows for a more thorough review of each system element to ensure the vehicle design can achieve requirements of the Ares program.

This week, the J-2X engine will be the first Ares I element to kick off the critical design review process. The engine will power the Ares I upper stage to orbit after separation from the first stage.

“We’re excited about getting into full system engine tests with the new J-2X engine,” Cook said. “This will be one of the safest, most affordable and highest performing rocket engines ever built, and testing is critical as we begin preparation for future flights.”

Marshall manages the Ares projects and is responsible for design and development of the Ares I rocket and Ares V heavy cargo launch vehicle. NASA’s Johnson Space Center in Houston manages the Constellation Program, which includes the Ares I rocket, the Ares V vehicle, the Orion crew capsule and the Altair lunar lander. NASA’s Kennedy Space Center in Florida is responsible for ground and launch operations. The program also includes multiple project element teams at NASA centers and contract organizations around the U.S.

For more information about the Ares rockets, visit:

http://www.nasa.gov/ares

For more information about NASA’s Constellation Program, visit:

http://www.nasa.gov/constellation