A good blog post on syslog reliability

I found a good blog post describing the problems that go along with reliable logging. It also offers most of the options to resolve them:

http://mschuette.name/wp/2008/03/31/thoughts-on-reliable-syslog/

The author missed one thing: a buffer must not necessarily exist only in main memory. When the in-core buffer runs out of space, you may also use a disk-based buffer, which offers much more capacity. Of course, even the largest disk-based buffer may be exhausted at some point, where one needs to resort to other strategies. But a disk-based buffer is an excellent solution for temporary (but lengthy) receiver outages.

Rsyslog has implemented everything mentioned in the paper, plus more. You can directly apply the knowledge you got from this paper to rsyslog. And then, you can dig down into the dirty details:

http://www.rsyslog.com/doc-queues.html