7. General Settings » Graphs & Storage

In a later chapter, you will see how to configure Sensors to generate traffic graphs, top statistics, and accounting information for every IP in the monitored network. In Configuration » General Settings » Graphs & Storage, you can configure how much disk space will be used to store that data, as well as other storage-related settings.

The Graph Storage Engine parameters let you select the third-party software used to store graph data:

► InfluxDB is a specialized database designed for time series data. It uses more RAM than RRDTool but requires less storage space, is faster in most cases, can be configured for High Availability, and does not delete existing data when you change the configuration

► RRDTool is a very stable and old solution for time series data. However, it can be slow in some cases (due to disk seek times), uses more storage space (due to pre-allocation of data), and some configuration changes (e.g., adding a decoder) reset existing graph data

Wansight uses Graph Storage Engine 1 as the only backend by default. However, it can be configured to use two backends simultaneously in these combinations: RRDTool + InfluxDB, or InfluxDB (primary) + InfluxDB (secondary)

Click the Options button on the right-hand side to configure the selected Graph Storage Engine:

InfluxDBRRDTool (On Disk)RRDTool (In RAM)

InfluxDB 1.9 is the preferred choice for new installations.

● IP Graph Granularity sets the level of detail for IP, AS, and Country graphs. The default value is 5 minutes. When using Flow Sensor, avoid setting it lower than the flow delay

● Storage Restrictions excludes certain classes of graphs from using the selected Graph Storage Engine

● InfluxDB URL should be the full URL of the InfluxDB (HTTP/HTTPS) server, for example: http://127.0.0.1:8086

● InfluxDB Username and InfluxDB Password are optional but must be set if auth-enabled=true in influxdb.conf

● InfluxDB Database is also optional. The default database name is andrisoft. Change it only when using a single InfluxDB instance for multiple Consoles

● InfluxDB Connector lets you initialize, delete, or check the status of the database. The most important status value is the “cardinality estimation,” which is directly related to InfluxDB’s RAM usage. High-cardinality data includes IP and AS graphs. If RAM usage is too high, consider disabling IP graphing for large subnets, then deleting and reinitializing the database

Graphing Engine Backend lets you choose from which Graph Storage Engine to retrieve data when rendering graphs. The options button lets you select which graph file format to use: SVG or PNG. On macOS or Linux, you may need to install the Consolas font to properly display SVG files.

Decoders are internal functions (traffic dissectors) that identify and classify the underlying protocols of each packet or flow. Each enabled decoder increases the size of IP graphs, tops, and accounting data, and introduces a minimal performance penalty. Only enable the decoders you need. You can create custom decoders under General Settings » Custom Decoders. The built-in decoders are:

IP	Matches all IP packets, regardless of higher-layer protocols. Always enabled.
TCP	Matches TCP traffic
TCP+SYN	Matches TCP traffic with the SYN flag set and ACK unset. Flow Sensor counts one packet per flow
UDP	Matches UDP traffic
ICMP	Matches ICMP traffic
OTHER	Matches IP protocols other than TCP, UDP, and ICMP
INVALID	Matches TCP or UDP port 0, or IP protocol 0
FLOWS	Matches flow records and replaces packets/s with flows/s. Works only with Flow Sensor
FLOW+SYN	Matches flow records with the SYN flag set. Flow Sensor counts all packets per flow
FRAGMENT	Matches fragmented IP packets. Works only with Packet Sensor
TCP-NULL	Matches TCP traffic without TCP flags (indicative of reconnaissance sweeps)
TCP+RST	Matches TCP traffic with the RST flag set
TCP+ACK	Matches TCP traffic with the SYN flag unset and ACK set
TCP+SYNACK	Matches TCP traffic with both SYN and ACK flag set
NETBIOS	Matches TCP traffic on source or destination port 139
QUIC	Matches Google’s QUIC protocol on UDP ports 80 and 443
UDP-QUIC	Matches UDP traffic that is not part of the QUIC protocol
MEMCACHED	Matches UDP traffic on port 11211
HTTP	Matches TCP traffic on source or destination port 80
HTTPS	Matches TCP traffic on source or destination port 443
MAIL	Matches TCP traffic on source or destination ports 25, 110, 143, 465, 585, 587, 993, 995
DNS	Matches UDP traffic on source or destination port 53
SIP	Matches TCP or UDP traffic on source or destination port 5060
IPSEC	Matches IP traffic on IP protocols 50 or 51
WWW	Matches TCP traffic on source or destination ports 80 or 443
SSH	Matches TCP traffic on source or destination port 22
NTP	Matches UDP traffic on source or destination port 123
SNMP	Matches UDP traffic on source or destination ports 161 or 163
RDP	Matches TCP or UDP traffic on source or destination port 3389
YOUTUBE	Matches IP traffic to or from YouTube AS 43515, 36561, or YouTube subnets
NETFLIX	Matches IP traffic to or from Netflix AS 55095, 40027, 2906, or Netflix subnets
HULU	Matches IP traffic to or from Hulu AS 23286 or Hulu subnets
FACEBOOK	Matches IP traffic to or from Facebook AS 54115, 32934, or Facebook subnets

IP Sweep Graphing allows storing IP graph data for IPv4 and/or IPv6 addresses receiving traffic without returning any. Avoid setting it to Off when monitoring unidirectional links or asymmetric traffic. Enabling it for IPv6 is generally not recommended because the vast number of possible IPv6 addresses can quickly exhaust RAM (when using InfluxDB) or disk space (when using RRDTool).

Sensor Top N (default: 20) Specifies the maximum number of items stored for each ordered data set, such as top talkers, external IPs, ASNs, countries, TCP/UDP ports, IP protocols, etc.

Flow Sensor saves flow data on the local disk in the path defined by the Flow Collector Path, using a directory structure determined by Flow Data Hierarchy.

Packet Sensor saves packet dumps in the path set by the Packet Dump Path.

RRDTool stores all graph files on the Console server in the Graphs Disk Path. InfluxDB stores graph data in /var/lib/influxdb/data/. If you want to use another path or drive, symlink it to the original location.

Note

It is strongly recommended to automate the deletion of old data and to monitor disk usage for IP graphs in General Settings » Data Retention..

7.1. Sensor and Applications Graph Troubleshooting

✔ Ensure that all Sensors are running correctly by checking the event log and live statistics in Reports » Devices » Overview. Refer to the troubleshooting guides for Packet Sensor, Flow Sensor, and SNMP Sensor

✔ Discontinuous Sensor graphs can happen if IP Accounting is enabled for too many or too large subnets, especially when there’s a slow connection between the Sensor and the MySQL/MariaDB instance on the Console server.

7.2. IP/Subnet and Profiling Graph Troubleshooting

✔ Ensure that all Sensors are running correctly by checking the event log and live statistics in Reports » Devices » Overview Refer to the troubleshooting guides for Packet Sensor, Flow Sensor, and SNMP Sensor

✔ Generating IP and profiling graph data causes the biggest impact on the load of the Console server. Enable each feature (IP graphing, IP accounting, IP profiling) sequentially for each subnet, ensuring the Console server can handle it. The storage requirements for each subnet are listed in the IP Zone, and current disk usage is in General Settings » Data Retention

✔ The internal process used for saving IP graph data is /opt/andrisoft/bin/genrrds_ip. If it is overloading the Console server or the event log contains warnings such as “Updating IP graph data takes longer than 5 minutes”, use InfluxDB or RRDCacheD, RAM/SSD updating method, faster disk drivers, enable IP graphing for fewer subnets, or deploy a Sensor Cluster configured to aggregate IP graph data

✔ The internal process used for generating IP or subnet graphs is /opt/andrisoft/bin/gengraph_ip. Console users launch the process for each requested IP or subnet graph. If the Console server gets too loaded by gengraph_ip, execute “killall gengraph_ip” and configure InfluxDB or RRDCacheD. When launched, the process stops only when the graph is generated. This process can be slow when users request subnet graphs for subnets not specifically defined in the IP Zone. It is not possible to throttle the number of graphs requested by users

7.3. AS and Country Graph Troubleshooting

✔ Ensure that all Sensors are running correctly by checking the event log and live statistics in Reports » Devices » Overview Refer to the troubleshooting guides for Packet Sensor, Flow Sensor, and SNMP Sensor

✔ To enable AS and Country graphs, set the Stats Engine parameter to either Extended in Flow Sensor’s configuration, or to Full in Packet Sensor’s configuration

✔ SNMP Sensor cannot generate AS or Country graphs due to SNMP protocol constraints.