Harden First, Don't Wait for the Exploit

Yesterday OpenSSH released 10.6p2. The team learned of a serious password authentication problem on some Linux systems with musl libc and no PAM, and shipped a workaround the next day. Damien Miller explained why: they recently saw several cases where AI agents rediscovered bugs very quickly, sometimes within hours, so they now consider long embargoes a dangerous practice that mostly benefits attackers. The release notes also say they will, for now, release more often instead of batching fixes until the next planned release.

I think this is exactly right. And it touches something I have wanted to write about for a while: how we handle hardening in rsyslog.

We do not wait for a proof of exploit

A lot of our recent work is hardening. Parsers become more robust against malformed input, TLS paths verify more strictly, limits get added where there were none. Often nobody has shown that the old code could actually be exploited.

Some people ask: is it really exploitable? Honestly, for most of these changes we do not know, and we do not try hard to find out. Checking every hardening change for exploitability is simply not possible with the resources we have. It is also not very useful. Building a working exploit takes much longer than fixing the weak spot. And many more defects are abused in the wild than are ever reported, so "no known exploit" does not tell you much.

So we strongly prefer to harden as fast as possible. The exception is when exploitation is obvious, or when something comes up in testing. Then we treat it accordingly.

Why this needs frequent updates

Hardening only helps when it runs on your systems. This is where release practice matters.

Many distributions backport selected fixes into older versions. That is valuable work. But a selective backport usually needs a reason, typically a concrete reported issue. Proactive hardening without such an issue can easily stay out until something specific is found later.

That is why I recommend staying current. rsyslog has scheduled releases roughly every two months. But main is kept stable, and our daily stable builds are main from that day. I wrote back in 2019 that daily stable is just as stable as a scheduled release. That still holds, and with today's pace of bug discovery it matters much more. With daily stable, a hardening change can be on your systems the next day.

For our Windows products at Adiscon we follow the same idea with a monthly Patch + Feature Drop.

Some environments really need the scheduled release, and that is fine. But please do not wait for a CVE before you update. By then the fix may already be late.