Category: rsyslog

Past days’ rsyslog work log:
2008-02-14
– enhanced file monitor doc
– implemented $InputFilePollInterval config directive
– error handling and cleanup in imfile
– improved file polling algorithm for more rapid file data delivery
– some more cleanup
– moved decoding of syslog names to a more appropriate place
– added new facility and severity syntaxes to cfsysline handler
– implemented $InputFileFacility config directive
– implemented $InputFileSeverity config directive
– created a bare template for omlibdbi (dbi output action)
– created an initial version of omlibdbi (does not yet work)
2008-02-15
– did some more work on omlibdbi, but did not yet get libdbi working.
I guess its a compile problem, but have not found it so far.
– the libdbi problem was actually related to libdbi/distro packages;
fixed that by installing from source, now omlibdbi basically works
removed some debug code
– cleaned up omlibdbi – works now
– implemented $ActionLibdbiDriverDirectory config directive
– some cleanup
– doc improvements
2008-02-16
– patched libdbi to work better with plugins
– adopted omlibdbi to use new version of libdbi
– improved man page a bit for the novice user

Yesterday’s rsyslog work log:
2008-02-14
– enhanced file monitor doc
– implemented $InputFilePollInterval config directive
– error handling and cleanup in imfile
– improved file polling algorithm for more rapid file data delivery
– some more cleanup
– moved decoding of syslog names to a more appropriate place
– added new facility and severity syntaxes to cfsysline handler
– implemented $InputFileFacility config directive
– implemented $InputFileSeverity config directive
– created a bare template for omlibdbi (dbi output action)
– created an initial version of omlibdbi (does not yet work)

Yesterday’s rsyslog work log:
2008-02-13
– added some code to expr.c – not yet to be used
– cleaned up imfile.c
– changed interface of logmsg() to make it more straightforward
– introduced a new, more powerful, message submission interface
submitMsg() in additon to logmsg()
– a first, rough implementation of imfile that is able to read files
(but does not persist or handle rotation or whatever)
– removed some left-over unnecessary dbgprintf’s
– added ability to monitor file accross rotation
– fixed a race condition in DoDie() – cosmetic issue in debugging mode,
could not happen in production
– added the ability to persist current read location for the file
monitor
– some cleanup
– created initial doc for imfile plugin
– released a preview of 3.11.2 (with the file monitor)

Yesterday’s rsyslog work log:
2008-02-12
– still helped a bit with omsnmp
– applied patch from Michael Biebl that fixed my doc change from yesterday
which was somewhat incomplete
– fixed a bug in stringbuf.c related to STRINGBUF_TRIM_ALLOCSIZE, which
wasn’t supposed to be used with rsyslog. Put a warning message up that
tells this feature is not tested and probably not worth the effort.
Thanks to Anders Blomdell fro bringing this to our attention
– somewhat improved performance of rsCStr obj
– fixed bug that caused invalid treatment of tabs (HT) in rsyslog.conf
– released 2.0.2
– reduced volume of debug output
– bugfix: setting for $EscapeCopntrolCharactersOnReceive was not
properly initialized
– clarified usage of space-cc property replacer option
– bugfix: discard action and backup actions did not work due to
problem in direct queue mode. Now fixed. Tracker was
http://sourceforge.net/tracker/index.php?func=detail&aid=1886931&group_id=123448&atid=696552
– improved diagnostic information for abort cases
– some initial effort for malloc/free debugging support
– bugfix: using dynafile actions caused rsyslogd abort
– fixed man errors thanks to Michael Biebl
– released 3.11.1

I admit it is just a quick note, but I guess it tell a lot to those that are working in the protocol area:

In the long term, I’ll move the protocol stack to RFC3080/3081 (including RFC 3195) as the primary rsyslog-to-rsyslog mode. I thought hard about it, but it is the best choice to have a) plugins only loaded on demand but b) the ability to decide upon runtime on the highest level of features/confidentiality. With liblogging, I already made serious investment in that protocol suite. It still not a trivial thing to do it in the way I intend to (with profile plugins) and it will probably an at least 50% rewrite of the 3080/81 code, but I came to the conclusion it is worth it. I think you’ll like it when you see it in all its glory ;) But other things are currently more important…

If you don’t get the idea what this is all about, just forget it. Really… it’s not important to you then ;)

Yesterday’s rsyslog work log:
2008-02-11
– added x-info field to rsyslogd startup/shutdown message. Hopefully
points users to right location for further info (many don’t even know
they run rsyslog ;))
– bugfix: trailing “:” of tag was lost while parsing legacy syslog messages
without timestamp – thanks to Anders Blomdell for providing a patch!
– worked on integrating omsnmp

OK, I am guilty of lazyness… I worked quite a bit on RELP and by doing so, I always thought “it’s not much for the rsyslog work log, so let’s do it tomorrow” ;) Well, tomorrow is now two weeks old and I think it is a good plan to come back to my habit of posting work logs. This time, of course, a bit more than usual.

Please note that the initial version with relp support, 3.15.0 has been released. I am currently evaluating the next focus feature, which probably is native TLS support. If so, I’ll probably again need to put some work into a utility library, which will not directly be rsyslog related. But I hope I’ll still continue to post something useful here. In the mean time, please be reminded that any implementation reports are still most welcome.

So here we go, the past ca. two-weeks worth of rsyslog work log:
2008-03-19
– added flow control options to other input sources
– worked on librelp
2008-03-20
– bugfix: some minor memory leaks
– bugfix: some slightly invalid memory accesses
– made librelp and rsyslog relp system send the first messages
to the remote peer (but it then discards them ;))
– made debug module free some memory on exit to make memory debugger
happy
2008-03-21
– added capability to receive RELP messages and forward them to the
main message queue to imrelp (not yet fully finished)
– cleanup of omrelp
– prepared omrelp for real “relp action”
2008-03-22
– changed queue’s discard severities default value to 8 (do not discard)
to prevent unintentional message loss
– removed a now-longer needed callback from the output module
interface. Results in reducing code complexity.
– Greatly enhanced rsyslogd’s file write performance by disabling
file syncing capability of output modules by default. This
feature is usually not required, not useful and an extreme performance
hit (both to rsyslogd as well as the system at large). Unfortunately,
most users enable it by default, because it was most intuitive to enable
it in plain old sysklogd syslog.conf format. There is now a new config
setting which must be enabled in order to support syncing. By default it
is off. So even if the old-format config lines request syncing, it is
not done unless explicitely enabled. I am sure this is a very useful
change and not a risk at all. I need to think if I undo it under
compatibility mode, but currently this does not happen (I fear a lot of
lazy users will run rsyslogd in compatibility mode, again bringing up
this performance problem…).
– added $ActionfileEnableSync config directive
– bugfix: internally generated messages had “FROMHOST” property not set
2008-03-25
– bugfix: continue parsing if tag is oversize (discard oversize part) – thanks
to mclaughlin77@gmail.com for the patch
– added $HHOUR and $QHOUR system properties – can be used for half- and
quarter-hour logfile rotation
– bugfix: QHOUR and HHOUR properties were wrongly calculated
2008-03-27
– made relp modules use new relpengine-provided feature selection functions
– bugfix: fixed memory leaks in stream class and imfile
– bugfix: $ModDir did invalid bounds checking, potential overlow in
dbgprintf() – thanks to varmojfekoj for the patch
– changed default for “last message repeated n times”, which is now
off by default
– implemented backward compatibility commandline option parsing
– bugfix: -t and -g legacy options max number of sessions had a wrong
and much too high value
– automatically generated compatibility config lines are now also
logged so that a user can diagnose problems with them
– added compatibility mode for -a, -o and -p options
– MILESTONE: compatibility mode processing finished
2008-03-28
– updated man pages
– changed default file output format to include high-precision timestamps
– added a buid-in template for previous syslogd file format
– added new $ActionFileDefaultTemplate directive
– added support for high-precision timestamps when receiving legacy
syslog messages
– added new $ActionForwardDefaultTemplate directive
– added new $ActionGSSForwardDefaultTemplate directive
– added build-in templates
– fixed small memory leak in tcpclt.c
– bugfix: fixed small memory leak in template regular expressions
– bugfix: regular expressions inside property replacer did not work
properly
2008-03-29
– removed –enable-mudflap, added –enable-valgrind ./configure setting
2008-03-31
– bugfix: tcp receiver could segfault due to uninitialized variable
– docfix: queue doc had a wrong directive name that prevented max worker
threads to be correctly set
– worked a bit on atomic memory operations to support problem-free
threading
– added a –enable/disable-rsyslogd configure option so that
source-based packaging systems can build plugins without the need
to compile rsyslogd
– released 3.13.0-dev0
– begun 3.17.0 [focus: TLS for plain tcp syslog]
2008-04-01
– bugfix: rsyslogd was no longer build by default; man pages are
only installed if corresponding option is selected. Thanks to
Michael Biebl for pointing these problems out.

Today’s rsyslog work log:
2008-02-01
– applied documentation fix by Michael Biebl — many thanks!
– added input-plugin interface specification in form of a (copy) template
input module
– cleaned up some no longer needed files, thanks to Michael Biebl for pointing
this out
– added sample config file to distribution tarball
– bugfix: immark did not have MARK flags set…
– very quickly hacked a rought outline of the file monitor (without any
guarantees)

Yesterday’s rsyslog work log:
2008-01-31
– added some advanced features usually in demand to sample rsyslog.conf,
but left it commented out
– rename $TimoutWorkerThreadShutdown to $WorkerTimoutThreadShutdown
for consistency reasons
– changed default for action queue size to 1000 elements (more reasonable here)
– fixed bug in sample rsyslog.conf
– fixed wrong action suspend/resume handling
– we have some issue with the mutx in dbgoprint, but that is acceptable for
the time being, I just removed the deadlock codition (debug system only)
– bugfix: dbgoprint mutex – was too simple once I wrote the tracker item ;)
– bugfix: having fun with 32/64 bit portability – after 15 years, I finally
was trapped again ;) — now fixed, sizes > 2GB supported on 32bit platforms