16. Components » Packet Sensor

Packet Sensor inspects every packet it receives in order to perform packet-based traffic analysis. The advantages and disadvantages of packet-based traffic monitoring versus flow-based traffic monitoring are listed in the Choosing a Method of Traffic Monitoring chapter.

In switched networks, only the packets for a specific device reach the device’s network card. If the server running the Packet Sensor is not deployed inline in the main data path, a network TAP or a switch/router that offers a “monitoring port” or “mirroring port” has to be used. In this case, the network device sends copies of data packets traveling through selected ports or VLANs to the monitoring port.

To add a Packet Sensor, click the [+] button from the title bar of the Configuration » Components panel. To modify an existing Packet Sensor, go to Configuration » Components and click its name.

Packet Sensor Configuration parameters:

● Sensor Name – A short name to help you identify the Packet Sensor

● Sensor Color – The color used in graphs for the Packet Sensor. The default color is a random one. You can change it from the drop-down menu

● Sensor Visibility – Toggles the listing inside the Reports » Devices panel

● Device Group – Optional parameter used to group components (e.g., by location or role). You can use it to restrict the access of Guest accounts

● Sensor Server – Select a server that fulfills the minimum system requirements for running the Packet Sensor. If this is not the Console server, follow the NFS configuration steps to make the raw packet data visible in the web interface

● Capture Engine – Select the preferred packet capturing engine:

• Embedded Libpcap – Select to use the built-in libpcap library. Libpcap requires no additional setup, but it cannot run on multiple threads, and it may be too slow for multi-gigabit sniffing

• System Libpcap – Select to use the libpcap library provided by the Linux distribution

• PF_RING – Select to use the PF_RING framework to speed up packet processing. PF_RING is much faster than Libpcap, but it requires the installation of additional kernel modules and supports only a limited number of NICs. Click the button on the right for PF_RING-specific settings. PF_RING ZC is faster than “vanilla” PF_RING, but it requires purchasing a ZC license from ntop.org. When an interface enters PF_RING ZC mode, it is no longer available to the OS for other operations

• Netmap – Select to use the Netmap framework to speed up packet processing. Netmap requires the installation of additional kernel modules, and it only supports a limited number of NICs. When an interface enters Netmap mode, it is no longer available to the OS for other operations

• DPDK – Select to use the DPDK framework, then click the button on the right of the Capture Engine field to configure DPDK-specific parameters as described in Appendix 1

● Capture Engine Options – This button shows specific options for the selected Capture Engine if such options are available

● Sniffing Interface – The network interface(s) listened by the Packet Sensor. Libpcap accepts only a single interface. PF_RING allows listening to multiple physical interfaces simultaneously when the interfaces are separated by semicolons (“;”)

● Sniffing Interface Options – If the Sniffing Interface is a TUN/TAP interface, set the Interface Type field accordingly. If GRE Decapsulation is enabled, Packet Sensor will analyze the encapsulated packet, not the GRE packet itself

● CPU Threads – Packet Sensor can run multi-threaded on a given set of CPU cores. Each thread increases the RAM usage. On most systems, activating more than 6 CPU threads hurts performance

● Link Speed IN / OUT – Enter the monitored link’s speed (bandwidth, capacity). The values are used for percentage-based reports and percentage-based bits/s thresholds

● Sensor License – The license used by the Packet Sensor. Wanguard provides all features; Wansight does not provide traffic anomaly detection and reaction

● Stats Engine – Collects traffic tops and AS graphs:

• Basic – Enables tops for internal IPs, IP protocols, IP versions, and TCP/UDP ports. It adds a minimal performance penalty, so it’s the recommended value on most systems

• Extended – Enables the tops from Basic as well as tops for external IPs (IPs not included in the IP Zone). It adds a performance penalty of over 20%, especially during spoofed attacks with randomized sources. Permits the detection of threshold violations for external IPs

• Full – Enables the tops from Extended as well as tops and graphs for autonomous systems. It adds a performance penalty of over 20%, especially during spoofed attacks with randomized sources. Permits the detection of threshold violations for external IPs

● Stats Engine Options – When Stats Engine is set to Full or Extended, you can click the button next to it. The Max. Source IPs parameter can limit the amount of RAM used to keep track of external IPs, or prevent interruptions during very powerful attacks from randomized sources. To enable Transit AS tops and graphs, enter the path to an existing BGP Dump File exported by BGPd in MTR format, and the IPv4 and optionally IPv6 address of the BGP router. MTR files can also be downloaded from RIPE

● IP Zone – Packet Sensor needs an IP Zone from which to learn the network’s boundaries and to extract per-subnet settings

● BPF Expression – You can use this parameter to filter the type of traffic received by the Packet Sensor. The tcpdump-style syntax is described when clicking the star button

● IP Validation – This option is the most frequently used method of establishing the direction of packets in relation to your network:

• Off – Packet Sensor analyzes all traffic and uses MAC Validation to identify the direction of traffic

• On – Packet Sensor analyzes the traffic that has the source and/or the destination IP in the selected IP Zone. This is the default value and the recommended one for most configurations

• Strict – Packet Sensor analyzes the traffic that has either the source or the destination IP in the selected IP Zone

• Exclusive – Packet Sensor analyzes the traffic that has the destination IP in the selected IP Zone, but not the source IP

● MAC Validation/Options – This rarely-used option can be used to distinguish the direction of the packets or to ignore unwanted OSI Layer 2 traffic:

• None – Packet Sensor analyzes all traffic and uses IP Validation to identify the direction of traffic

• Upstream MAC – MAC validation is active, and the MAC Address belongs to the upstream router

• Downstream MAC – MAC validation is active, and the MAC Address belongs to the downstream router

The MAC Address has to be written using the Linux convention – six groups of two hexadecimal values separated by colons (:)

● Granularity – Interval between successive updates for traffic parameters and anomalies. A granularity of 1 second ensures detection of anomalies in under one second at the expense of the SQL server’s load. The default value is 5 seconds

● Sampling (1/N) – Must contain the packet sampling rate. On most systems, the correct value is 1

● Comments – Comments about the Packet Sensor can be saved here. They are not visible elsewhere

To start the Packet Sensor, click the small button displayed next to its name in Configuration » Components. Ensure that the Packet Sensor starts correctly by watching the event log. If the Packet Sensor starts, but you can’t see any data collected by it in Reports » Devices » Overview after 10 seconds, follow the troubleshooting steps listed below.

Note

To obtain detailed information about the sources of the attacks detected by Packet Sensor, add a Packet Filter, set the same Sniffing Interface and Capture Engine, and then activate it in the “When an anomaly is detected…” panel from the Response.

16.1. Packet Sensor Troubleshooting

✔ Look for warnings or errors produced by the Packet Sensor in the event log

✔ Ensure that you have correctly configured the Packet Sensor. Each configuration field is described above in this chapter

✔ The event log error “License key not compatible with the existing server” indicates that the server is unregistered and you need to send the string from Configuration » Servers » [Server] » Hardware Key to sales@andrisoft.com

✔ When IP Validation is not disabled, make sure that the selected IP Zone contains all your subnets

✔ Ensure that you have correctly configured the switch/TAP to send packets to the server on the configured interface

✔ Make sure that the sniffing interface is up:

[root@localhost ~]# ip link show <interface>

✔ Verify whether the server is receiving packets via the configured interface:

[root@localhost ~]# tcpdump -i <interface_usually_eth1_or_p1p2> -n -c 100

✔ If the CPU usage of the Packet Sensor is too high, set the Stats Engine parameter to “Basic”, install PF_RING, Netmap or DPDK to enable multi-threading, or use a network adapter that allows distributing Packet Sensors over multiple CPU cores

✔ To troubleshoot Sensor graph or IP graph issues, follow the Graphs Troubleshooting guide

✔ For PF_RING-related issues, contact ntop.org

✔ The system process responsible for capturing packets is called WANtrafficlogger. There will be as many such processes active as the number of packet dumps active in Reports » Tools » Packets

✔ Go to Help » Software Updates to make sure that you are running the latest version of the software

16.2. Optimization Steps for Intel XL710

To distribute the packet-processing tasks of the Packet Sensor over multiple CPU cores when using the Intel XL710 adapter and a ZC license:

➔ Install PF_RING ZC and switch to the PF_RING-aware Intel driver

➔ Create a separate Packet Sensor for every queue of your multi-queue NIC. For example, when having 12 queues, define 12 Packet Sensors defined named ‘queue0’ thru ‘queue11’. Each Packet Sensor must be configured to listen on an interface such as zc:enp134s0f0@0 for queue0, or zc:enp134s0f0@7 for queue7, and so forth. Each defined Packet Sensor needs to have the PF_RING Capture Engine selected, ZC checked, and further, needs a unique Cluster ID. So, ‘queue0’ has cluster id 0, while ‘queue7’ has cluster id 7, and so forth

➔ Add all Packet Sensors to a single Sensor Cluster to have a unified reporting and anomaly detection domain. No additional licensing is needed because all Packet Sensors defined to listen to a single interface use a single Sensor license

16.3. Optimization Steps for Intel 82599

To distribute the packet-processing tasks of the Packet Sensor over multiple CPU cores when using an adapter with the Intel 82599 chipset (Intel X520, Intel X540, HP X560, etc.):

➔ Follow the documentation and optimization guides provided by the network adapter vendor

➔ Install PF_RING and switch to the PF_RING-aware ixgbe driver

➔ See the number of RSS queues allocated by the ixgbe driver by executing dmesg, or by listing /var/log/messages or /var/log/syslog. By default, the number of RSS queues equals the number of CPU cores when hyper-threading is off, or double the number of CPU cores when hyper-threading is on. You can set the number of RSS queues manually by loading ixgbe.ko with the RSS=<number> option

➔ Enable multi-threading in the Packet Sensor configuration or define multiple Packet Sensors, each listening to ethX@queue_id or ethX@queue_range and add them to a Sensor Cluster to have a unified reporting and anomaly detection domain. No additional licensing is needed because all Packet Sensors defined to listen to a single interface use a single Sensor license

On a quad-core CPU with multi-threading, the ixgbe driver allocates 8 RSS queues. In this case, if you define a Packet Sensor for ethX@0-3 and another one for ethX@4-7, the packet-processing task will be distributed over 2 CPU cores. PF_RING exposes up to 32 RSS queues.