Today, I have added support for dynamically taking search index and type from the message itself. It works much like dynafiles do. If dynamic mode is selected (via config params), the index name and type given in the configuration are template names instead of literal ones. That means the rsyslog core will generate the actual name at runtime, based on actual message content. These dynamically generated values will then be used inside the request.
Here is a quick howto: you need to define the templates first, like this:
$template srchidx,”%hostname%”
$template srchtype,”%programname%”
In this example, the hostname (from the message) is used as name for the search index, and the programname (usually part of the syslog tag) is used as searchtype. The full power of templates, including all properties, can be used, so this is highly flexible.
To actually use these dynamic values, omelasticsearch must be called like this (line break do not matter):
*.* action(type=”omelasticsearch”
searchIndex=”srchidx” dynSearchIndex=”on”
searchType=”srchtype” dynSearchType=”on”)
The “dynSearch…” setting tells the engine if it is a dynamic name or not. If set to on, like above, a template name is given and the actual name will be dynamically generated. If set to “off”, the literal value will be used (“off” is the default, so the parameters do not need to be specified if “off” is desired). For example:
*.* action(type=”omelasticsearch”
searchIndex=”srchidx” searchType=”srchtype”)
will use the index “srchidx” and type “srchtype” literally. No template-based name extension will happen in this case. Please note that it is ok for only one parameter to by dynamic. It’s not “all or nothing”.
I hope this is a useful addition for omelasticsearch. Several folks have indicated that this method can increase throughput in a couple of use cases, for example to split of indexes based on host names, host functions and the like.