Category: rsyslog

Posted on

rsyslog: configure syslog UDP reception

This tutorials tells how rsyslog is configured to accept syslog messages over the network via UDP. No advanced topics are covered. We use CentOS 7. This is part of a rsyslog tutorial series.

Scope

We will configure the relay system to accept UDP based syslog from remote ends.  We do not, however, configure any sender to connect to it. We will use LC as UDP server, just so that we get some more variety into our lab with limited systems. In our base lab scenario, this will lead to the following configuration:

Note that we will accept incoming logs and store them into the same location as we do for local logs. Handling them different will be part of a later tutorial. Continue reading “rsyslog: configure syslog UDP reception”

Posted on

How to start, stop and query the status of rsyslog (on a systemd system)

This short tutorial explains everyday service management. While it claims to address management of rsyslog, it actually describes the tools for all services. The tutorial is written for CentOS 7, but should work equally well on other systemd-based systems like CentOS 8, recent Fedora, recent Debian and recent Ubuntu. Continue reading “How to start, stop and query the status of rsyslog (on a systemd system)”

Posted on

rsyslog: forward messages to remote server

This tutorials tells how rsyslog is configured to send syslog messages over the network via TCP to a remote server. No advanced topics are covered. We use CentOS 7. This is part of a rsyslog tutorial series.

Scope

We will configure an end node (here: LR) to send messages via TCP to a remote syslog server. We do not apply local pre-filtering and we want to make only minimal changes to the CentOS 7 default configuration. In our base lab scenario, this will lead to the following configuration:

Continue reading “rsyslog: forward messages to remote server”

Posted on

rsyslog: configure syslog TCP reception

This tutorials tells how rsyslog is configured to accept syslog messages over the network via TCP. No advanced topics are covered. We use CentOS 7. This is part of a rsyslog tutorial series.

Scope

We will configure the relay system to accept TCP based syslog from remote ends. We do not, however, configure any sender to connect to it. In our base lab scenario, this will lead to the following configuration:

Note that we will accept incoming logs and store them into the same location as we do for local logs. Handling them different will be part of a later tutorial. Continue reading “rsyslog: configure syslog TCP reception”

Posted on

Tutorials for rsyslog

While I have been in Tallinn to give some lectures about syslog technology in general as well as rsyslog in specific I had the idea to use that opportunity to think about crafting rsyslog tutorials in general.

For the practical session at TALTECH IT-College I have identified a couple of typical configuration tasks. As experience shows, to carry them out successfully not only rsyslog knowledge is required but general sysadmin know-how as well. Continue reading “Tutorials for rsyslog”

Posted on

rsyslog’s daily stable

Did you know? The rsyslog project offers a stable release every day! The world is changing and getting faster each time. Especially software. If a bug is fixed, you want to have the fix as soon as possible. Even more so if it is security related.

Development happens in active code. When we fix something, this is done in so-called “master branch”. If you use master branch, you are covered as soon as the fix is applied.

Traditionally there are “stable builds” which are released relatively infrequently. Sometimes many months. In rsyslog’s case, only 6 weeks. For bugfixes, someone needs to backport the fix to that “stable build”. Backporting comes with its own risks, as the code is integrated into a version it was never written for.

We have much more frequently updated versions as well. They are crafted each day and contain the current latest and greatest. Including all known fixes. These daily version are usually considered as experimental or development version. Quite honestly, this is no longer the case.

Before I continue, please consider that rsyslog is a relatively small project. What is true for it may not be true for much larger ones.

In rsyslog, we have two important policies:

  • new versions never break existing configurations (except for extremely important reasons) – this means you can always update to the latest version without risking that your config blows up
  • we rely on our CI – if a change passes the testbench, it basically is good to go.To complement, we also have manual reviews of critical changes. Only things we are pretty confident in go into master branch.

What does that mean? First of all, master branch actually is a stable version. The code passed all of our checks and safeguards. We do not question the stability and continue to work on new features or bug fixes. Note that the strong position here sometimes upsets contributors. We have and had PRs who take month to ripen before they are finally sufficiently good to be merged to master.

The second thing is that daily stable builds are built from master and so are also stable.

Now let’s consider what happens when it is time to create the 6-weekly stable release. Code-wise it’s pretty simple: as master is stable, we simply take master branch and declare it as the new stable version. It’s the same version as the daily build from that day. What is different is that the doc is consolidated and we prepare files nicely for package build. Then, packages are built and tested. Except for the doc, one could also have used the daily stable build.

Think about it. What it ultimately means is that the 6-week “stable” release is just a way to avoid doing more frequent updates. But the daily build is actually as stable as the stable release.

In a world of rapidly moving development, using the daily stable build has a lot of advantages. Most importantly, one gets fixes as soon as possible. Not to mention new features.

I understand that the scheduled release may be the better option for some environments. But for most, the daily stable is actually to be preferred.

Please note: daily stable builds are currently only available for Ubuntu. With our efforts towards OpenSuse Build Service we aim to make them available for a wider number of platforms.

Posted on

Finally … rsyslog Minimum Batch Sizes

Today’s release of rsyslog 8.1901.0 contains a small but important feature: the ability to specify a minimum batch size. It is much-needed for some outputs, with ElasticSearch (and ClickHouse) being prime examples. While I am happy I finally implemented it, I am also a bit ashamed it took me almost three and a half year since Radu Gheorghe proposed that feature in 2015.

Quick reminder on how rsyslog batches work: we receive messages and put them into queues. From these queues, we pull so-called batches (sets of messages) and have them processed by output modules. A batch can contain a given maximum number of messages (by default and depending on case around 1024 or below). If there are that many messages inside the queue, a full batch is extracted and processed. If the queue does not contain that many, whatever it currently has is taken and forms the batch. As such a batch contain as few messages as one. Continue reading “Finally … rsyslog Minimum Batch Sizes”

Posted on

rsyslog version numbering change

As we know, rsyslog uses a version number scheme of

8.<real-version>.0

where we increment <real-version> every 6 weeks with each release. The 8 and 0 are constant (well, the 0 could change to 1 with a very important patch, but in practice we have only done this once).

While this scheme has worked pretty well since we introduced it, I often see people not understanding that there is really a big difference between 8.24 and e.g. 8.40. Looking at recent trends in software versioning, we see

  1. single-number versions, e.g. in systemd
    This is actually what we use, except that we make it look like and old-style version number by the prefix 8 and suffix 0.
  2. date-based versions, e.g. by distros (Ubuntu 18.04)

With the next release, will will make more clear how old a version really is. To do so, we change the version number slightly  to

8.yymm.0

where yy is the two-digit year and mm the two-digit month of the release date. We release every 6 weeks, so we will never have two releases within the same month.

So the next version will be 8.1901.0 instead of 8.41.0. To make things even more clear, rsyslog visible version output will be even more up to the point: rsyslog -v will now report “8.1901.0 (aka 2019.01)“. I am right now implementing these changes. Continue reading “rsyslog version numbering change”

Posted on

rsyslog’s first signature provider: why Guardtime?

The new has already spread: rsyslog 7.3 is the first version that natively supports log signatures, and does so via a newly introduced open signature provider interface. A lot of my followers quickly realized that and begun to play with it. To make sense of the provider interface, one obviously also needs a signature provider. I selected the keyless signature infrastructure (KSI), which is being engineered by the OpenKSI group. Quickly, I was asked what were the compelling reasons to provide the first signature provider for this technology.

So rather than writing many mails, I thought I blog about the reason ;) Continue reading “rsyslog’s first signature provider: why Guardtime?”

Posted on

Which data does the Guardtime signature provider transfer to external parties?

With the interest in privacy concerns currently having a “PRISM-induced high”, I wanted to elaborate a little bit about what rsyslog’s Guardtime signature provider actually transmits to the signature authority.

This is a condensed post of what the provider does, highlighting the main points. If you are really concerned, remember that everything is open source. So you are invited to read the actual signature provider source, all of which is available at the rsyslog git.

The most interesting question first: the provider does only send a top-level hash to the signature authority. No actual log record will ever be sent or otherwise disclosed. Continue reading “Which data does the Guardtime signature provider transfer to external parties?”