27. General Settings » Data Retention

In Configuration » General Settings » Data Retention you can see how much disk space is used by the software and configure for how long the data should be kept.

DATA_RETENTION

Data Retention Settings parameters:

● IP Accounting Data – This feature can create over time thousands of tables because each table holds accounting information for only one day and one Sensor interface. Keep the number of retention days for these tables to a minimum
● IP & Sensor Tops – These tables usually grow the fastest because they store binary data for Sensor Tops, so it’s best to set a low retention period when the disk space is limited
● Console Events – These events provide information about configuration changes, user logins, etc. Keep this data for as long as possible
● Other Events – These events are stored in a single table called events. Keep the number of rows in this table under a few million to avoid Console performance issues
● Anomaly Data – These tables hold attack-related information. Keep this data for as long as possible
● Response Data – These tables don’t grow very fast, and you should keep the data for as long as possible
● System Commands – These tables are useful for troubleshooting. They don’t grow up fast in normal conditions
● Packet Dumps – The filesystem paths for packet dumps can be changed in General Settings » Graphs & Storage
● Raw Flow Data – The filesystem paths for flows can be changed in General Settings » Graphs & Storage
● InfluxDB Retention Policy – If you use InfluxDB as a Graph Storage Engine in General Settings » Graphs & Storage, you can set its data retention period here
● RRD files – If you use RRDTool as a Graph Storage Engine in General Settings » Graphs & Storage, expand this fieldset to see the disk space consumed by it

Note

✔ If the data is stored on an SSD disk, it’s essential to keep the disk usage below 80-90% to avoid performance losses due to Over Provisioning
✔ If the number of rows used by the database tables is in the millions, expect sudden performance losses caused by MySQL/MariaDB indexes
✔ As a rule of thumb, it’s best to only keep the data you actually need, for as long as you need it

The database is purged from old data once every hour, so the changes in Data Retention might not be immediately visible. The command for purging old data is:

[root@console ~]# sudo -u andrisoft /opt/andrisoft/bin/WANmaintenance

On some systems, MySQL/MariaDB will not free up the deleted data and instead it will reserve the resulting disk space for future use. To reclaim the newly freed up disk space, execute the following command on the Console server (it might take a very long time to execute):

[root@console ~]# /opt/andrisoft/bin/WANmaintenance optimize_db

You can backup the whole database by executing on the Console server:

[root@console ~]# /opt/andrisoft/bin/WANmaintenance backup_db

You can backup only the software configuration by executing on the Console server:

[root@console ~]# /opt/andrisoft/bin/WANmaintenance backup_config

The resulting backup file will be placed in the current directory. You can import it with:

[root@new_console ~]# mysql -p andrisoft < wanguard_*.sql