OSSEC and log analysis

OSSEC has been reading logs and deciding what matters since 2004. This page covers how it does that, what its rule language can express, and the gap that opens up when you need the logs themselves rather than the alerts.

What OSSEC is

OSSEC is an open source host based intrusion detection system. It runs an agent on each machine, ships events to a central manager, and applies a rule set to decide whether anything happened that somebody should know about. Alongside log analysis it does file integrity monitoring, rootkit detection, policy checks and active response.

A note on where we are standing: Trunc was founded by Daniel Cid, who wrote OSSEC and released it in 2004. The project was later acquired by Third Brigade and then Trend Micro, and is now maintained by the community. We are not neutral about it, and the categorization logic in Trunc is a descendant of the OSSEC rule set. What follows is a description of what the tool does well and where it stops, written by people who know exactly where it stops.

How OSSEC reads a log

Log analysis in OSSEC happens in two stages, and the separation between them is the part worth understanding.

Decoders extract the fields

A decoder matches the shape of a log line and pulls named values out of it. Take a common one:

Jan 14 09:21:44 web01 sshd[4823]: Failed password for invalid user admin from 203.0.113.5 port 44122 ssh2

The sshd decoder recognizes the program name, then a child decoder matches the "Failed password for invalid user" phrasing and extracts admin as the user and 203.0.113.5 as the source address. Decoders are declared in XML with regular expressions and can be chained, so a parent handles the general format and children handle the variations.

Rules decide what it means

Rules run against the decoded output. Each has a numeric ID, a severity level from 0 to 15, and a set of conditions. Level 0 means ignore this entirely, level 5 is a routine failure, level 10 or above is something you probably want to know about tonight.

What makes the rule language useful is that rules can depend on each other. if_sid makes a rule fire only when another rule has already matched, which lets you write a general rule for "sshd authentication failure" and then a specific child for "authentication failure against a nonexistent user". if_matched_sid combined with frequency and timeframe fires when the same rule has matched a given number of times in a window, which is how the brute force rules work.

That composition is the reason OSSEC catches things a search cannot. Fifty failed logins from one address is a rule with a frequency of 50. Fifty failed logins followed by a successful one from that same address is a second rule that depends on the first having fired, and it is a materially different event: the first is somebody trying, the second is somebody who got in. No amount of searching a log store will surface the second, because it is not a property of any single line.

The rest of OSSEC

Feature What it does
Syscheck File integrity monitoring. Hashes a configured set of files and directories on a schedule and reports changes to contents, ownership or permissions. On Windows it also watches registry keys. This is the feature most compliance frameworks ask for by name.
Rootcheck Rootkit and anomaly detection. Looks for hidden processes, hidden ports, unusual file permissions and known rootkit signatures. Also runs policy checks against a configured baseline.
Active response Runs a script when a rule of sufficient severity fires. Most commonly a firewall drop for the offending address, with an automatic timeout. Powerful and worth configuring carefully, since a badly scoped response can lock you out of your own infrastructure.
Agentless Checks against devices that cannot run an agent, over SSH. Useful for network hardware and appliances.

The gap: OSSEC discards the logs

This is the part that surprises people, and it is a design decision rather than an oversight.

By default OSSEC writes alerts and nothing else. A log line that matches no rule, or matches a level 0 rule, is read, evaluated and dropped. That is correct behaviour for an intrusion detection system: the job is to tell you when something happened, and storing the ninety nine percent of lines where nothing happened is not part of it.

It becomes a problem the moment you need the logs for anything else. An auditor asking for twelve months of authentication records does not want your alerts. An investigation into what an attacker did after they got in needs the successful logins, the commands, the file accesses, all of which are routine events that generated no alert at the time. Threat hunting needs the uninteresting logs by definition, since you are looking for patterns nobody has written a rule for yet.

OSSEC has a logall option that writes every received event to disk. It produces a flat file, or a gzipped one, organized by date. There is no index, no search beyond grep, no retention management, and no interface. On a busy manager it grows quickly and the practical experience of using it during an incident is running grep against a multi-gigabyte file while somebody asks you for an answer.

Other things worth knowing

  • The web interface is long gone. The original OSSEC WUI was deprecated years ago and should not be deployed. Most installations read alerts.log, forward alerts to syslog, or send them somewhere else entirely.
  • Analysis is largely single threaded. The analysis daemon processes events in sequence, which sets a ceiling on how much a single manager handles. Large deployments shard across managers.
  • Rules are XML. Verbose, and the ordering and inheritance rules take time to internalize. Testing with ossec-logtest before deploying is not optional.
  • Alert fatigue is real. A default install on a busy web server produces a great deal of level 5 noise. Tuning the rule set to your environment is the work, and it never quite finishes.

Using OSSEC with Trunc

These fit together rather than competing. OSSEC decides what happened on the host. Trunc keeps everything, makes it searchable, and gives you somewhere to look.

The straightforward arrangement is to keep OSSEC doing what it is good at, file integrity monitoring and host level detection, and send its alerts to Trunc alongside the raw logs from everything else. OSSEC can forward alerts to a syslog destination, so this is a configuration change rather than a project:

<syslog_output> <server>your Trunc endpoint</server> <port>your port</port> <level>3</level> </syslog_output>

Then point your log sources at Trunc directly, either with the Trunc agent or with syslog from whatever is already forwarding. You end up with OSSEC's judgement about what mattered, stored beside the full record of what happened, in one searchable place with retention that satisfies an auditor.

What each side contributes

OSSEC Trunc
Host detection Yes Log based only
File integrity monitoring Yes No, keep OSSEC for this
Rootkit detection Yes No
Stores every log Only with logall, unsearchable Yes, indexed
Full text search grep Yes, with field filters
Interface None current Built for logs
Retention management Your own scripts Configurable, included
Compliance reporting Build it yourself PCI DSS and ISO 27001 built in
Correlation across events Yes, if_sid and frequency Yes, same lineage
Something to run Manager, agents, rule set Nothing

If you are running OSSEC purely for log analysis and have never used syscheck or rootcheck, Trunc covers that ground on its own and removes a manager from your estate. If you rely on file integrity monitoring, keep OSSEC and let Trunc handle storage, search and the audit trail.

Common questions

Is OSSEC still maintained?

Yes, by the community, though development is slower than it was and slower than Wazuh. If you want an actively developed fork with commercial backing, look at Wazuh. If you want something small, stable and well understood, OSSEC still does the job it was designed for.

OSSEC or Wazuh?

Wazuh is a fork with considerably more surface area: a REST API, vulnerability detection, configuration assessment, cloud integrations and a dashboard. It also brings an OpenSearch cluster to run. OSSEC is smaller and does less. We have a separate page on Wazuh that goes into the logging side in detail.

Can Trunc read my existing alerts.log?

Yes. Point the Trunc agent at /var/ossec/logs/alerts/alerts.log and the alerts arrive as events like anything else. Forwarding through syslog_output is usually tidier because the alerts arrive structured, but either works.

Do I need both agents on a server?

If you want OSSEC's file integrity monitoring on that host, yes, and they coexist without trouble. If you only care about the logs, the Trunc agent alone is enough.

Simple, Affordable, Log Management and Analysis.

14 days free trial. No credit card required.