rsyslog 6.3.3 config format improvements

In rsyslog 6.3.3, the config processor has finally changed. The old legacy processor (and with it the early RainerScript implementation) is thrown out and has been replaced by the so-called RainerScript processor (why that crazy name?). This is an extremely important step for rsyslog, as it now has the foundation for a much better and intuitive rsyslog.conf format. However, most of that can not be seen in 6.3.3, as it requires more work, especially in the plugin arena. Still, there are a couple of smaller improvements available.

Most importantly, the performance of script based filters has been considerably enhanced. Preliminary testing shows a three times speedup (we’ll do more benchmarking at a later stage; there is also still lots of room for optimization ;-)).

The ugliness of continuation lines has been removed. They may still be used, and this may make a lot of sense with some actions, but you are usually no longer forced to use continuation lines. Take this config snippet from a leading distro:

if ( 
     /* kernel up to warning except of firewall  */ 
     ($syslogfacility-text == 'kern')      and      
     ($syslogseverity <= 4 /* warning */ ) and not  
     ($msg contains 'IN=' and $msg contains 'OUT=') 
 ) or ( 
     /* up to errors except of facility authpriv */ 
     ($syslogseverity <= 3 /* errors  */ ) and not  
     ($syslogfacility-text == 'authpriv')           
 ) 
then /dev/tty10
& |/dev/xconsole

This can now be written as follows:

if ( 
     /* kernel up to warning except of firewall  */
     ($syslogfacility-text == 'kern')      and     
     ($syslogseverity <= 4 /* warning */ ) and not 
     ($msg contains 'IN=' and $msg contains 'OUT=')
 ) or (
     /* up to errors except of facility authpriv */
     ($syslogseverity <= 3 /* errors  */ ) and not 
     ($syslogfacility-text == 'authpriv')          
 ) 
then /dev/tty10
& |/dev/xconsole

Of course, this is not a real big advantage, but can be very useful during day-to-day operations. Forgetting the continuation marker is easy and has happend quite often, causing many more problems than necessary.

Also, the somewhat unintuitive use of “&” to chain actions together can now (optionally) be replaced by so-called blocks. For example,

authpriv.err /dev/tty10
&            |/dev/xconsole

can now be written as

authpriv.err { /dev/tty10
               |/dev/xconsole
             }

This looks much more familiar and thus intuitive to many users. Of course, both the old style as well as the new style is supported.

Finally, the need to use single quote characters (‘) over the usual double quotes (“) in script based filters was often a source of confusion. You may now use both, so ‘string’ and “string” works both. However, strings in double quotes will support parameter replacement in later versions of rsyslog. That is “Message is $msg” will evaluate to exactly this string in 6.3.3, but $msg will be resolved to the actual message content some time in the future. So be careful if you use double quotes.

Of course, none of these changes are the important ones so many users are waiting for, most importantly an intuitively-usable scoping for actions and inputs. These will be coming up shortly. I need to write some more engine code *and* need to enhance plugins to support that. I’ll probably start with actions as first. Note that the RainerScript processor already parses some of these constructs, but the rest of the engine simply ignores them. In order to get you an idea of how it will look, see this hypothetical example:

if $msg contains "error" then {
    action(type="omfwd" protocol="tcp" target="10.0.0.1:514"
           action.retryCount="-1"
           queue.type="linkedList" queue.fileName="fwdRule" queue.maxDiskSpace="1g"
           queue.saveOnShutdown="on"
          )
    action(type="omfile" target="/var/log/somelog.log")
    action(type="omuser" target="all" action.onceInterval="30")
}

I hope the example is intuitive enough to grasp it’s meaning ;) In current format, you need to write

$ActionQueueFileName fwdRule1
$ActionQueueMaxDiskSpace 1g 
$ActionQueueSaveOnShutdown on 
$ActionQueueType LinkedList
$ActionResumeRetryCount -1
if $msg contains 'error' then @@10.0.0.1:514
& /var/log/somelog.log
$ActionExecOnlyOnceEveryInterval 30
& :omusrmsg:*

At least to me, the upcoming new way looks much nicer ;)

In regard to the distro-example given above, I’ll try to simplify it towards this form:

if ( /* kernel up to warning except of firewall  */
     hasPRI("kern.warn") and not 
     ($msg contains 'IN=' and $msg contains 'OUT=')
    ) or hasPRI("authpriv.err")
    then { /dev/tty10
          |/dev/xconsole
         }

But that’s the second step after introducing the new action statements.

Please note that the final format selection was very carefully based on many discussions both on the mailing list and inside the forum as well as needs to preserve backwards compatibility. For example, on Debian, packages drop rsyslog-specific configs into the /etc/rsyslog.d directory and expect them to be understood. In order to break things here, we needed to remain compatible with the legacy format and extend it. Only thanks to the good user feedback we could finally come up with a solution that the majority of users hopefully will find good.

With that said, I’ll now see that I create the actual release. For obvious reasons, 6.3.3 will be a bit shaky as far as the config is concerned. Most probably it will also not run the full testbench successfully (due to some very esoteric tests that are broken by actual functionality changes). However, you can be sure that the engine works well as long as it passed the config stage, because there were almost no changes during runtime (well… script filter expression evaluation has been rewritten from scratch).