Author: Rainer Gerhards

Past day’s rsyslog work log:
2008-05-06
– added documentation for TLS
– merge TLS branch into main devel branch
– released 3.19.0 (MILESTONE, we now have TLS! :-D )
– some cleanup (gotten rid of some more plain chars)
– fixed some issues thanks to darix
– fixed problem with man pages thanks to Michael Biebl’s help
2008-05-07
– fixed some issues in liblogging (thanks to darix for pointing out)
– limited number of unavoidable compiler warnings when compiling with
GnuTLS
– released 3.19.1
2008-05-08
– bugfix: gtls netstram driver did not specify threading model (can
possbly lead to “interesting effects” ;))
– server’s X509 cert fingerprint is obtained by client on connect
– thought hard about syslog-transport-tls-12 (obviously, this does
not manifest in code ;))

I am very pleased to announce rsyslog 3.19.0.

It is the first release that natively supports TLS for plain TCP syslog. Actually, it is the world’s first implementation of the upcoming syslog-transport-tls IETF standard. As this standard is not yet finished, the implementation is obviously experimental.

Native TLS is a big improvement over existing functionality. For example, rsyslog can now be used without the help of stunnel, which relieves us of some problems from those configurations. To the best of my knowledge, rsyslog is the first open-source syslogd offering TLS support natively.

The current TLS functionality is limited to the bare minimum. During the next few weeks, I will improve it based on my own spec and feedback (hopefully received). My hope is to have a production-grade implementation by summer at latest. Please note rsyslog’s premium and ultra-reliable RELP protocol does not yet support TLS (but can be used with stunnel without the real problems legacy tcp had with it). My plan is to let TLS mature with legacy syslog and then move it to RELP. Thus I can limit my development to one major use case, which I think will facilitate things.

There is some documentation on how to use the new TLS mode:

http://www.rsyslog.com/doc-rsyslog_tls.html

Currently, TLS is provided via GnuTLS. As I outlined earlier on the list, GnuTLS offered much more support to getting started (documentation and sample-code wise). I will focus on GnuTLS until I am fully satisfied with the TLS implementation). I’ll then see that I can also integrate NSS. Advise in this regard would be highly welcome, so if you have some knowledge in this area, please contribute.

In order to support TLS (and multiple libraries!), a major rewrite of the networking components has been done. Rsyslog now supports a so-called “network streams” (netstreams) driver interface. This interface enables app-level functionality (like the legacy tcp syslog sender and receiver) to work with dynamically selectable netstream drivers (like plain (unencrypted) TCP) and TLS). This interface will enable rsyslog to utilize other TLS drivers (and even other protocols) in the future. Different drivers can even be used concurrently.

Rsyslog now has been split into a runtime system and tools (with currently rsyslogd being the only tool). This design will further strengthen modularization and help make rsyslog functionality available in small parts.

Finally, the RFC 3195 input has been rewritten in the form of an input plugin. It can now be build as part of the normal build procedure.

Please note that there were a couple of major changes. I expect the initial 3.19.0 to be quite Unstable. I recommend it for testing environments, only. Even those parts that were not directly touched may have become a bit destabilized due to the runtime split. So please use it with care. Feedback, however, would be more than welcome, because I need to start somewhere to stabilize this release. That can only be done with your help. So please use it on test systems, try to break it and file bug reports when it fails.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-102.phtml

Changelog:

http://www.rsyslog.com/Article221.phtml

File your bug reports here ;) :

http://bugzilla.adiscon.com/rsyslog-bugs.html

I hope this release is useful. Feedback is much appreciated.

Yesterday’s rsyslog work log:
2008-05-05
– made imgssapi work with new netstrm driver model
there were a couple of things where imgssapi was not compatible
with the new encapsulation. I did a somewhat dirty fix. The real
solution would be to turn gssapi functionality into a netstream
driver, which is too much for now (after all, we want to release
some time AND we need to have the code mature in practice
– added $DefaultNetstreamDriverCAFile config directive
– added $DefaultNetstreamDriverCertFile config directive
– added $DefaultNetstreamDriverKeyFile config directive
– added $ActionSendStreamDriver config directive

Yesterday’s rsyslog work log:
2008-03-04
– changed module interface to support querying obj interface (stage work)
– changed module interface version, as the interface change is quite large
– tweak omsnmp doc a bit (to cover Andre’s changed oid)
– did some portability changes to make rsyslog compile on HP UX
I couldn’t resist: I finally found a suitable HP UX machine on HP’s
testdrive system. So I looked at what it takes to make rsyslog
compile. Got this going after a relatively short while. The core
engine also seems to run, but there seem to be some issues. So far,
rsyslog seems to compile but it is questionable if it can acutally
run. I’ll look into this later (or as need arises), but will now
focus again on new features (portability, as a side-effect, also
often shows code that can be improved, so it is useful to look
at different platforms even if we do not eagerly need to support
them).

Past day’s rsyslog work log:
2008-04-30
– added server-side TLS capability for plain tcp server
– added $InputTCPServerStreamDriverMode config directive
2008-05-02
– fixed a bug in imklog which lead to startup problems (including
segfault) on some platforms under some circumsances. Thanks to
Vieri for reporting this bug and helping to troubleshoot it.

Hi all,

again, I have been quite busy the past days. A lot was rsyslog related, but phpLogCon also kept me busy. I am still working on TLS support with some occasional bug-hunting in between.

TLS support grew too a much larger project than it initially looked. In short, it caused a more or less complete rewrite of the base networking classes. Quite complex work, even though it doesn’t involve lots of code (I am always surprised how few code changes actually happen). I am going slowly but steadly through it and hope that we will have an even better abstraction of the networking classes when I am done. In the long term, it will probably also affect gssapi (in a positive way) which could finally lead to a fully generic, driver-based and highly flexible networking model. Well worth the wait, I’d say… ;)

So here is the past day’s rsyslog work log:
2008-04-21
– worked on netstrm abstraction (server side)
2008-04-22
– got socket abstraction to work for server part
– fixed newly introduced memory leaks
– released 3.16.0
– -c option no longer must be the first option – thanks to varmjofekoj
for the patch
2008-04-25
– added missing copyright statements (thanks to Michael Biebl for noticing)
– added select() driver for GnuTls
– made gtls server driver work in plain tcp mode
2008-04-28
– added $DefaultNetstreamDriver config directive
– added $ActionSendStreamDriverMode config directive
2008-04-29
– worked on klogd problem on gentoo
– moved netstrms, netstrm and nssel into a single loadble module
because they belong together
– fixed “loadbale module leak”
– fixed problem with module unload sequence

Past day’s rsyslog work log:
2008-04-18
– abstracted a driver level for netstream
– converted netstrm into generic netstrm and the nsd_pctp driver
– prepared everything so that I finally can begin to implement
TLS sender part (in plain tcp output)
– bugfix: a recent change effectively disabled error messages
– implemented a first working version of a TLS-enabled plain TCP
sender (but, of course, the implementation is insecure as it is)
2008-04-21
– primarily phpLogCon design

I found a good blog post describing the problems that go along with reliable logging. It also offers most of the options to resolve them:

http://mschuette.name/wp/2008/03/31/thoughts-on-reliable-syslog/

The author missed one thing: a buffer must not necessarily exist only in main memory. When the in-core buffer runs out of space, you may also use a disk-based buffer, which offers much more capacity. Of course, even the largest disk-based buffer may be exhausted at some point, where one needs to resort to other strategies. But a disk-based buffer is an excellent solution for temporary (but lengthy) receiver outages.

Rsyslog has implemented everything mentioned in the paper, plus more. You can directly apply the knowledge you got from this paper to rsyslog. And then, you can dig down into the dirty details:

http://www.rsyslog.com/doc-queues.html