Category: rsyslog

I made big progress, even though the work log seems not to indicate it. The issues I worked on were quite complex. And, most frustratingly, there were no simple way to even compile rsyslog until the change was completed. So I hacked for about 6 hours without any feedback on the effect. Of course, after the first compile things were really bad. But over time, I managed to fix the bugs. Now I am quite happy with the result. The output module interface really begins to materialize. The next big thing is handling of configuration system line directives ($-lines). Stay tuned…

The work log for Friday:
– released 1.17.4
– added omsr object (objomsr.c, objomsr.h) – template request for output
modules
– changed doAction() interface
– templates and output string generation for doActon() is now fully
– removed selector_t f references from output modules
– MILESTONE reached: no more access to selector_t from any module, at
least at this layer we communicate via clean interfaces. However,there
remains the topic of global variable access and calling to functions
housed somewhere else (e.g. in syslogd.c). A new code review is now due,
many changes happened, many TODO’s added.

During the large rsyslog modularization effort, I take a more detailed audit log of what I am doing. I hope that this log will allow others to both follow the progress as well as help to understand what I am doing. I was not sure (and I still am not) where to post that log. I’ve now decided to post it to my blog, because it doesn’t look really suitable for the “offcial” rsyslog site.

Please note that the work audit contains more detail than the ChangeLog. This is intentional. The ChangeLog shall provide the average user with an idea of what’s now. My audit here provides a finer-grained information for those that are really interested in it.

Here come yesterday’s changes. They are listed in the same order I have done them.

– applied patch from mildew to avoid zombies
– applied patch from Michel Samia to fix compilation when NOT
compiled for pthreads
– implemented onSelectReadyWrite() interface
– MILESTONE reached: no more access to f->f_un in syslogd.c
– shuffled code from tcpsyslog.c to omfwd.c. It looks like it belongs more
to that file. But we need to look at it some time later. The move was
absolutely necessary so that no access to f->f_un happened in
tcpsyslog.c (which was evil)
– MILESTONE reached: no more access to f->f_un from non-output modules
– changed doAction() interface to include module data pointer
– removed references to f_un from omusrmsg.c
– changed module template for parseSelectorAct() [code reduction,
consitency]
– removed references to f_un from ommysql.c
– removed references to f_un from omfwd.c
– removed references to f_un from omshell.c
– removed references to f_un from omfile.c
– MILESTONE reached: f->f_un has gone away!
– removed f_type from omshell.c, omdiscard.c, omusrmsg.c, ommysql.c
– removed f_type from syslogd.c/cflineParseFileName()
– fixed bug in omfile.c which could lead to invalid addressing if “-” was
given to not sync file
– removed f_type from omfile.c
– implemented needUDPSocket() interface
– replaced (mis) use of f_prevcount in omfwd.c -> now data element in
instance data is used for retry counting
– removed f->f_type from syslogd.c, omfwd.c
– removed f->f_file from omfwd.c, omfile.c
– f->f_flags is gone away
– changed doAction() interface to contain the full message string
– f_iov and its handling has been removed
– added IDs to selector_t

If you are interested in even more details, you can go to the rsyslog cvs and see the changes on a file-by-file basis.

As the day closed, I identified a problem with the current interface definition: Modules need to access the template pointer in selector_t. They may even need to have multiple templates (e.g. dynaFiles, a hypothetical email action [subject and message text]). I need to address this soon.

I hope this audit is useful. Yesterday’s changes will be released as 1.17.4 this morning. Then, I continue to work on modularization.

Rsyslog‘s output module interface begins to materialize. I have even begun to restructure the code modules, which currently mostly means shifting code to different places. However, there is much more behind this code-shuffling. I’ve been thinking quite a while about modularization of rsyslog. What happens now is the result of this thinking. In the end, we will have output modules running on independent threads, each being able to queue data when the output for some reason is suspended (e.g. the remote syslog server it sends data to is unavailable). And, of course, the module interface will also support plug-ins.

The current MySQL action will become such a plug-in. I need to adopt a way to tell current users a way to migrate to the loadable module interface. I guess, I’ll add a dummy statement like

$ModLoad MySQL

To the current configuration. Well, yes, let’s do that – I’ve created a feature tracker as I write down this blog entry.

The only effect it will have in current code is that it tells the config engine that the user cared about modules. In builds that will later support loadable modules, it will actually load the mysql plugin. Currently, its only function will be to warn users to apply it, when they did not do it. That should take care of a smooth transition.