This tutorials tells how rsyslog is configured to accept syslog messages over the network via TCP. No advanced topics are covered. We use CentOS 7. This is part of a rsyslog tutorial series.
Scope
We will configure the relay system to accept TCP based syslog from remote ends. We do not, however, configure any sender to connect to it. In our base lab scenario, this will lead to the following configuration:
Note that we will accept incoming logs and store them into the same location as we do for local logs. Handling them different will be part of a later tutorial.
Side-note: If you think closely about syslog terms, you will notice that the word “relay” is not correct for this lab. The system actually is a syslog collector at the end of this tutorial. We use the term “relay” to convey the intent at the end of the core tutorial series. This information is given for clarity. You can safely ignore it at this stage.
Prerequisites
You need to
- know how to start, stop and query the status of rsyslog
- find rsyslog configuration files
- have some basic network understanding
- have a basic understanding on how a firewall works and is configured under CentOS 7
Setup
We will start by making minimal changes to /etc/rsyslog.conf. The default configuration already contains commented-out configuration bits that we can use for our needs:
The selected text shows the basic elements you need to use to create the syslog server. In rsyslog terms, this is called a “listener”. If you uncomment these lines, you will create the necessary listener.
Note: in the default sample, port 514 is used. This is common for plain syslog over TCP, but it is not the standard port number for it. The port is actually assigned to a different use case by networking authorities. In almost all cases, you do not need to be concerned about it. Using 514/tcp is a good choice and will possibly safe you some hassle.
The CentOS default configuration unfortunately is provided in obsolete legacy format. As we have no more complex needs, we will still use it as-is for now.
To do the configuration change
- uncomment the lines
- save the configuration
- make rsyslog use the new configuration
Checking everything works
In order to ensure everything works correctly, you need to verify proper operation. This should be done in two steps. We will use netcat for both.
Locally
Firstly, on the local machine, connect via netcat and send a test “message” (we know that this is not a perfectly formatted syslog message, but we do not care now). Then, verify that it is recorded in /var/log/messages.log:
Your output should look similar to above screen-shot. If it is not, ensure that the configuration change was actually applied to /etc/rsyslog.conf and that rsyslog has successfully been restarted. Inside /var/log/messages you should see rsyslog’s startup message. Watch carefully out for error messages and correct them if there are any.
Remotely
The second step will be carried out at the remote machine, e.g. LC as we call it in this tutorial series. You basically do the same netcat command on LC, except that the target IP address is now the address of LR (or it’s name if name resolution is available). If you have an unaltered CentOS 7 install (as we assume), you will see an error message:
Think a moment about what this means.
The issue is that the remote machine cannot connect to rsyslogd on LR. On the other hand, we know that rsyslogd runs properly on LR. We also tested that we can successfully connect to it locally. Consequently, the problem is somewhere in the network. For the lab setup, we can probably rule out real network connectivity issues. So the problem must be somewhere in system configuration.
Remember the prerequisites? Yes, we said “firewall”. And, yes, this exactly is what gets in our way. Configuring rsyslog itself is not sufficient for the remote use case. We also need to permit remote systems to connect to it. To do so, we need to tell the firewall that traffic via port 514 over the TCP protocol should be permitted. In a real-world scenario you may want to add more precise rules, giving e.g. only a limited set of source IPs the permission to connect. For our tutorial use, we just permit any sender to reach LR. After this is done, you should be able to connect to LR from LC. Retry it.
If you still experience problems, do network troubleshooting. If you did not use port 514, be advised that CentOS 7 default SELinux policies permit rsyslog to listen to only ports 514, 601, and 10514. If you selected another port, you need to also permit that in SELinux. How this is done is outside the scope of this tutorial, but can be found in a Red Hat tutorial.
Extra Check
You may already know that the procedure we used to configure the firewall only adds a temporary permission to the system. If you do not know this, we recommend to get some experience with this fact. To do so
- reboot LR and wait until it is fully started up
- retry sending a message from LC (via netcat)
- you should see the same error message again
- you can permanently fix the issue by adding “–permanent” to the firewall-cmd command
Make the change permanent, verify everthing works and do another reboot. Retry sending the message again. This time it should work immediately. If not, double-check you did everything correctly and did not receive an error message.
Result
Both messages from the local system as well as messages received via TCP over the network will now be recorded into the local log files. Please note that what we did not does not split local from remote traffic. This is done in a separate tutorial.
Please also note that messages from remote systems, namely LC, will not yet automatically appear in LR’s log files. This will be done in the next tutorial.