Engineering notebook
The typical logging problem as viewed from syslog
- Published
- Updated
- AI use
- Human-created
I run into different syslog use cases from time to time. So I thought it is a good idea to express what I think the typical logging problem is. As I consider it the typical problem, syslog (and WinSyslog and rsyslog in specific) address most needs very well. What they spare is the analysis and correlation part, but other members of the family (like our log analyzer) and third parties care well for that.
So the typical logging problem, as seen from the syslog perspective, is:
- there exists events that need to be logged
- a single "higher-level" event E may consist of a number of fine-grained lower level events e_i
- each of the e_i's may be on different systems / proxies
- each e_i consists of a subset of properties p_j from a set of all possible common properties P
- in order to gain higher-level knowledge, the high-level event E must be reconstructed from e_i's obtained from *various* sources
- a transport mechanism must exist to move event e_i records from one system to another, e.g., to a central correlator
- systems from many different suppliers may be involved, resulting in different syntax and semantic of the higher-level objects
- there is potentially a massive amount of events
- events potentially need to be stored for an extended period of time
- quick review of at least the current event data (today, past week) is often desired
- there exists lots of noise data
- the data needs to be fed into backend processes, like billing systems