Syscheck

Syscheck is the name of the integrity checking process inside OSSEC. It runs periodically to check if any configured file (or registry entry on Windows) has changed.

Why Integrity checking?

This is the explanation from the OSSEC book:

There are multiple types of attacks and many attack vectors, but there is one thing unique about all of them: they leave traces and always change the system in some way. From viruses that modify a few files, to kernel-level rootkits that alters the kernel, there is always some change in the integrity of the system.

Integrity checking is an essential part of intrusion detection, that detects changes in the integrity of the system. OSSEC does that by looking for changes in the MD5/SHA1 checksums of the key files in the system and on the Windows registry.

The way it works is that the agent scans the system every few hours (user defined) and send all the checksums to the server. The server stores the checksums and look for modifications on them. An alert is sent if anything changes.

Quick facts

  • How often does it run?

    • By default every 6 hours, but the frequency or time/day are configurable.

  • Where is the database stored?

    • On the manager in /var/ossec/queue/syscheck.

  • How does it help with compliance? (PCI DSS, etc)

    • It helps with sections 11.5 (install FIM software) and 10.5 (integrity checking of log files) of PCI.

  • How much CPU does it use?

    • The scans are performed slowly to avoid using too much CPU/memory.

  • How are false positives handled?

    • Files can be ignored manually in the configuration or using rules. By default when a file has changed 3 times further changes are automatically ignored.

Realtime options

ossec-syscheckd is able to check file integrity in near realtime on Windows and modern Linux distros. Windows comes with support out of the box, but on Linux systems inotify packages may need to be installed. Check for inotify dev packages, and possibly an inotify-tools package.

Configuration options

These configuration options can be specified in each agent’s ossec.conf file, except for the auto_ignore and alert_new_file which apply to manager and local installs. The ignore option applies to all agents if specified on the manager.

directories

Use this option to add or remove directories to be monitored (they must be comma separated). All files and subdirectories will also be monitored. Drive letters without directories are not valid. At a minimum the ‘.’ should be included (D:\.). This should be set on the system you wish to monitor (or in the agent.conf if appropriate).

Default: /etc,/usr/bin,/usr/sbin,/bin,/sbin

Attributes:

  • realtime: Value=yes

    • This will enable realtime/continuous monitoring on Linux (using the inotify system calls) and Windows systems.

  • report_changes: Value=yes

    • Report diffs of file changes. This is limited to text files at this time.

    Note

    This option is only available on Unix-like systems.

  • check_all: Value=yes

    • All the following check_* options are used together unless a specific option is explicitly overridden with “no”.

  • check_sum: Value=yes

    • Check the md5 and sha1 hashes of the of the files will be checked.

      This is the same as using both check_sha1sum=”yes” and check_md5sum=”yes”

  • check_sha1sum: Value=yes

    • When used only the sha1 hash of the files will be checked.

  • check_md5sum: Value=yes

    • The md5 hash of the files will be checked.

  • check_size: Value=yes

    • The size of the files will be checked.

  • check_owner: Value=yes

    • Check the owner of the files selected.

  • check_group: Value=yes

    • Check the group owner of the files/directories selected.

  • check_perm: Value=yes

    • Check the UNIX permission of the files/directories selected. On windows this will only check the POSIX permissions.

  • restrict: Value=string

    • A string that will limit checks to files containing that string in the file name.

    Allowed: Any directory or file name (but not a path)

  • no_recurse: Value=no

    New in version 3.2.

    • Do not recurse into the defined directory.

    Allowed: yes/no

ignore

List of files or directories to be ignored (one entry per element). The files and directories are still checked, but the results are ignored.

Default: /etc/mtab

Attributes:

  • type: Value=sregex

    • This is a simple regex pattern to filter out files so alerts are not generated.

Allowed: Any directory or file name

nodiff

New in version 3.0.

List of files to not attach a diff. The files are still checked, but no diff is computed. This allows to monitor sensitive files like private key or database configuration without leaking sensitive data through alerts.

Attributes:

  • type: Value=sregex

    • This is a simple regex pattern to filter out files so alerts are not generated.

Allowed: Any directory or file name

frequency

Frequency that the syscheck is going to be executed (in seconds).

The default is 6 hours or 21600 seconds

Default: 21600

Allowed: Time in seconds

scan_time

Time to run the scans (can be in the formats of 21pm, 8:30, 12am, etc).

Allowed: Time to run scan

Note

This may delay the initialization of realtime scans.

scan_day

Day of the week to run the scans (can be in the format of sunday, saturday, monday, etc)

Allowed: Day of the week

auto_ignore

Specifies if syscheck will ignore files that change too often (after the third change)

Default: yes

Allowed: yes/no

Valid: server, local

alert_new_files

Specifies if syscheck should alert on new files created.

Default: no

Allowed: yes/no

Valid: server, local

Note

New files will only be detected on a full scan, this option does not work in realtime.

scan_on_start

Specifies if syscheck should do the first scan as soon as it is started.

Default: yes

Allowed: yes/no

windows_registry

Use this option to add Windows registry entries to be monitored (Windows-only).

Default: HKEY_LOCAL_MACHINESoftware

Allowed: Any registry entry (one per element)

Note

New entries will not trigger alerts, only changes to existing entries.

registry_ignore

List of registry entries to be ignored.

Default: ..CryptographyRNG

Allowed: Any registry entry (one per element)

prefilter_cmd

Command to run to prevent prelinking from creating false positives.

Example:

<prefilter_cmd>/usr/sbin/prelink -y</prefilter_cmd>

Note

This option can potentially impact performance negatively. The configured command will be run for each and every file checked.

skip_nfs

New in version 2.9.0.

Specifies if syscheck should scan network mounted filesystems. Works on Linux and FreeBSD. Currently skip_nfs will abort checks running against CIFS or NFS mounts.

Default: no

Allowed: yes/no

Warning

This option was added in OSSEC 2.9.

Configuration Examples

To configure syscheck, a list of files and directories must be provided. The check_all option checks md5, sha1, owner, and permissions of the file.

Example:

<syscheck>
    <directories check_all="yes">/etc,/usr/bin,/usr/sbin</directories>
    <directories check_all="yes">/root/users.txt,/bsd,/root/db.html</directories>
</syscheck>

Files and directories can be ignored using the ignore option (or registry_ignore for Windows registry entries):

<syscheck>
    <ignore>/etc/random-seed</ignore>
    <ignore>/root/dir</ignore>
    <ignore type="sregex">.log$|.tmp</ignore>
</syscheck>

The type attribute can be set to sregex to specify a Regular Expression Syntax in the ignore option.

<syscheck>
    <ignore type="sregex">^/opt/application/log</ignore>
</syscheck>

A local rule can be used to modify the severity for changes to specific files or directories:

<rule id="100345" level="12">
    <if_matched_group>syscheck</if_matched_group>
    <match>/var/www/htdocs</match>
    <description>Changes to /var/www/htdocs - Critical file!</description>
</rule>

In the above example, a rule was created to alert with high severity (12) for changes to the files in the htdocs directory.

Real time Monitoring

OSSEC supports realtime (continuous) file integrity monitoring on Linux (support was added kernel version 2.6.13) and Windows systems.

The configuration is very simple. In the <directories> option where you specify what directories to monitor, adding realtime="yes" will enable it. For example:

<syscheck>
    <directories realtime="yes" check_all="yes">/etc,/usr/bin,/usr/sbin</directories>
    <directories check_all="yes">/bin,/sbin</directories>
</syscheck>

In this case, the directories /etc, /usr/bin and /usr/sbin will be monitored in real time. The same applies to Windows too.

Warning

The real time monitoring will not start immediately. First ossec-syscheckd needs to scan the file system and add each sub-directory to the realtime queue. It can take a while for this to finish (wait for the log “ossec-syscheckd: INFO: Starting real time file monitoring” ).

Note

Real time only works with directories, not individual files. So you can monitor the /etc or C:\program files directory, but not an individual file like /etc/file.txt.

Note

Both rootcheck and syscheck runs on the same thread, so when rootcheck is running, the inotify events would get queued until it finishes.

Report Changes

OSSEC supports sending diffs when changes are made to text files on Linux and unix systems.

Configuring syscheck to show diffs is simple, add report_changes="yes" to the <directories option. For example:

<syscheck>
    <directories report_changes="yes" check_all="yes">/etc</directories>
    <directories check_all="yes">/bin,/sbin</directories>
</syscheck>

Note

Report Changes can only work with text files, and the changes are stored on the agent inside /var/ossec/queue/diff/local/dir/file.

If OSSEC has not been compiled with libmagic support, report_changes will copy any file designated, e.g. mp3, iso, executable, /chroot/dev/urandom (which would fill your hard drive). So unless libmagic is used, be very carefull on which directory you enable report_changes.

MD5 whitelist database

ossec-analysisd can query an sqlite database for known-good md5 hashes. The database should contain the hashes, file names, and an optional date.

This feature uses code from Xavier Mertens.

Configure the database in ossec.conf:

</global>
  <md5_whitelist>/rules/lists/md5whitelist.db</md5_whitelist>
</global>

Schema:

CREATE TABLE files (
    md5sum VARCHAR(32),
    file VARCHAR(256),
    time DATETIME
);
CREATE UNIQUE INDEX files_idx ON files(md5sum);

Syscheck: FAQ

How to force an immediate syscheck scan?

Run agent control tool to perform a integrity checking immediately (option -a to run on all the agents and -u to specify an agent id)

# /var/ossec/bin/agent_control -r -a
# /var/ossec/bin/agent_control -r -u <agent_id>

For more information see the agent_control documentation.

How to tell syscheck not to scan the system when OSSEC starts?

Set the option <scan_on_start> to “no” on ossec.conf

How to ignore a file that changes too often?

Set the file/directory name in the <ignore> option or create a simple local rule.

The following one will ignore files /etc/a and /etc/b and the directory /etc/dir for agents mswin1 and ubuntu-dns:

<rule id="100345" level="0" >
    <if_group>syscheck</if_group>
    <description>Changes ignored.</description>
    <match>/etc/a|/etc/b|/etc/dir</match>
    <hostname>mswin1|ubuntu-dns</hostname>
</rule>

Why does OSSEC still scan a file even though it’s been ignored?

No idea. So if there are some directories you do not want scanned at all, make sure they are not included in a <directories> configuration.

How to know when the syscheck scan ran?

Use the agent_control tool on the manager, to see this information.

More information see the agent_control documentation.

How to get detailed reporting on the changes?

Use the syscheck_control tool on the manager or the web ui for that.

More information see the syscheck_control documentation.

Syscheck not sending any file data to the server?

With ossec 1.3 and Fedora you may run into this problem:

You have named files you’d like ossec to monitor so you add:

<ossec_config>
    <syscheck>
        <directories check_all="yes">/var/named</directories>

to ossec.conf on the client. Fedora – at least as of version 7 – runs named in a chroot jail under /var/named/chroot. However, part of that chroot jail includes /var/named/chroot/proc. The contents of that directory are purely ephemeral; there is no value to checking their integrity. And, at least in ossec 1.3, your syscheck may stall trying to read those files.

The symptom is a syscheck database on the server that never grows beyond a file or two per restart of the client. The log monitoring continues to work, so you know it’s not a communication issue, and you will often see a slight increase in syscheck database file size after the client has restarted (in one case about 20 minutes after). But the database will never be completely built; there will only be a couple files listed in database.

The solution is to add an ignore clause to ossec.conf on the client:

<ossec_config>
    <syscheck>
        <ignore>/var/named/chroot/proc</ignore>

Why aren’t new files creating an alert?

By default OSSEC does not alert on new files. To enable this functionality, <alert_new_files> must be set to yes inside the <syscheck> section of the manager’s ossec.conf. Also, the rule to alert on new files (rule 554) is set to level 0 by default. The alert level will need to be raised in order to see the alert. Alerting on new files does not work in realtime, a full scan will be necessary to detect them.

Add the following to local_rules.xml:

<rule id="554" level="10" overwrite="yes">
  <category>ossec</category>
  <decoded_as>syscheck_new_entry</decoded_as>
  <description>File added to the system.</description>
  <group>syscheck,</group>
</rule>

The <alert_new_files> entry should look something like this:

<syscheck>
  <frequency>7200</frequency>
  <alert_new_files>yes</alert_new_files>
  <directories check_all="yes">/etc,/bin,/sbin</directories>
</syscheck>

Can OSSEC include information on who changed a file in the alert?

In short, no. OSSEC does not track this information. You could use your OS’s auditing facilities to track this information, and create a rule to alert when an appropriate log is created.

How do I stop syscheck alerts during system updates?

There is no easy way to do this, but there are work-arounds. Stop the OSSEC processes on the manager, and run /var/ossec/bin/syscheck_control -u AGENT_ID. This will clear the syscheck database for the agent, and the next time syscheck runs it will create a new baseline. Next, start the OSSEC processes on the manager. Once the system update is complete, run a syscheck scan on that agent. The database will be populated with new values, and should not trigger “file modified” alarms.