Install and verify softflowd for Netflow on OpenWrt
softflow is a great opensource Netflow exporter that is support on OperWrt :) This allows us to sample the flows running through our OpenWrt router, export them to a netflow analyzer (more on those in a future post) and "see" our traffic. Netflow is much richer in information than just SNMP and allows for detailed analysis of individual traffic flows.
Install softflowd
SSH to the router, update package lists and install with opkg install softflowd
BusyBox v1.30.1 () built-in shell (ash)
_______ ________ __
| |.-----.-----.-----.| | | |.----.| |_
| - || _ | -__| || | | || _|| _|
|_______|| __|_____|__|__||________||__| |____|
|__| W I R E L E S S F R E E D O M
-----------------------------------------------------
OpenWrt 19.07.2, r10947-65030d81f3
-----------------------------------------------------
root@Them00n:~# opkg update
Downloading http://downloads.openwrt.org/releases/19.07.2/targets/lantiq/xrx200/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_core
Downloading http://downloads.openwrt.org/releases/19.07.2/targets/lantiq/xrx200/packages/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/releases/19.07.2/targets/lantiq/xrx200/kmods/4.14.171-1-0f59e90218b95a909e229a713d3da157/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_kmods
Downloading http://downloads.openwrt.org/releases/19.07.2/targets/lantiq/xrx200/kmods/4.14.171-1-0f59e90218b95a909e229a713d3da157/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/releases/19.07.2/packages/mips_24kc/base/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_base
Downloading http://downloads.openwrt.org/releases/19.07.2/packages/mips_24kc/base/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/releases/19.07.2/packages/mips_24kc/luci/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_luci
Downloading http://downloads.openwrt.org/releases/19.07.2/packages/mips_24kc/luci/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/releases/19.07.2/packages/mips_24kc/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_packages
Downloading http://downloads.openwrt.org/releases/19.07.2/packages/mips_24kc/packages/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/releases/19.07.2/packages/mips_24kc/routing/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_routing
Downloading http://downloads.openwrt.org/releases/19.07.2/packages/mips_24kc/routing/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/releases/19.07.2/packages/mips_24kc/telephony/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_telephony
Downloading http://downloads.openwrt.org/releases/19.07.2/packages/mips_24kc/telephony/Packages.sig
Signature check passed.
root@Them00n:~# opkg install softflowd
Installing softflowd (0.9.9-2) to root...
Downloading http://downloads.openwrt.org/releases/19.07.2/packages/mips_24kc/packages/softflowd_0.9.9-2_mips_24kc.ipk
Configuring softflowd.
root@Them00n:~#
Configue softflowd
softflowd by default reads its config from the file /etc/config/softflowd. As with most applications on OpenWrt we can either manually edit this file or use UCI. We'll look at the options for both below but just use the option you prefer (I prefer UCI since it does some sanity checking).
You can add multiple interfaces to be monitored and you can also monitor any interface that appears in the output of ifconfig. I have two interfaces to monitor so the examples below include two, just modify the examples to suit your own setup.
Some details you'll need to know before you can configure softflowd though:
- Interface(s) to monitor - (e.g pick from ifconfig)
- Netflow collector IP address and port(s)
- Netflow collector netflow version support - If you're not sure I'd suggest trying v9 these days
- Check CPU load before and after enabling softflow! (This has a minimal impact on my systems but this will add some load)
- The sampling rate you would like to set - Default is 1 in 100 packets and I'd suggest leaving it around that level
Configure using UCI
To configure softflowd using uci just replace your collector IP and interfaces below. Also remove the references to softflow[1] if you dont have more than one interface.
uci set softflowd.@softflowd[0].enabled='1'
uci set softflowd.@softflowd[0].interface='pppoe-wan'
uci set softflowd.@softflowd[0].timeout='maxlife=60'
uci set softflowd.@softflowd[0].max_flows='8192'
uci set softflowd.@softflowd[0].host_port='10.1.1.1:9995'
uci set softflowd.@softflowd[0].pid_file='/var/run/softflowd.pid'
uci set softflowd.@softflowd[0].control_socket='/var/run/softflowd.ctl'
uci set softflowd.@softflowd[0].export_version='9'
uci set softflowd.@softflowd[0].tracking_level='full'
uci set softflowd.@softflowd[0].track_ipv6='1'
uci set softflowd.@softflowd[0].sampling_rate='100'
If you need more than one interface monitored just add multiple versions of the below and modify the [1] accordingly. So my second interface:
uci add softflowd softflowd
uci set softflowd.@softflowd[1].enabled='1'
uci set softflowd.@softflowd[1].interface='br-iot'
uci set softflowd.@softflowd[1].timeout='maxlife=60'
uci set softflowd.@softflowd[1].max_flows='8192'
uci set softflowd.@softflowd[1].host_port='10.1.1.1:9996'
uci set softflowd.@softflowd[1].pid_file='/var/run/softflowd.pid'
uci set softflowd.@softflowd[1].control_socket='/var/run/softflowd.ctl'
uci set softflowd.@softflowd[1].export_version='9'
uci set softflowd.@softflowd[1].tracking_level='full'
uci set softflowd.@softflowd[1].track_ipv6='1'
uci set softflowd.@softflowd[1].sampling_rate='100'
Commit the changes using uci commit and restart the service using /etc/init.d/softflowd restart. Config done :) See below for how to confirm its working.
Configured editing config file
To do this simply edit /etc/config/softflowd directly and edit it similar to the below:
config softflowd
option enabled '1'
option interface 'pppoe-wan'
option timeout 'maxlife=60'
option max_flows '8192'
option host_port '10.1.1.1:9995'
option pid_file '/var/run/softflowd.pid'
option control_socket '/var/run/softflowd.ctl'
option export_version '9'
option tracking_level 'full'
option track_ipv6 '1'
option sampling_rate '100'
config softflowd
option enabled '1'
option interface 'br-iot'
option timeout 'maxlife=60'
option max_flows '8192'
option host_port '10.1.1.1:9996'
option pid_file '/var/run/softflowd.pid'
option control_socket '/var/run/softflowd.ctl'
option export_version '9'
option tracking_level 'full'
option track_ipv6 '1'
option sampling_rate '100'
Save the file and restart the service using /etc/init.d/softflowd restart. Config done :) See below for how to confirm its working.
Verify it works
softflowd has a sister application called softflowctl we can use to analyze the currently cached flows in softflowd. Just run softflowctl dump-flows.
root@Them00n:~# softflowctl dump-flows
softflowd[11100]: Dumping flow data:
ACTIVE seq:24 [1.2.3.4]:53592 <> [10.11.12.13]:443 proto:6 octets>:208 packets>:4 octets<:825 packets<:2 start:2020-04-05T09:38:39.821 finish:2020-04-05T09:39:37.483 tcp>:10 tcp<:18 flowlabel>:00000000 flowlabel<:00000000
EXPIRY EVENT for flow 24 in 1 seconds
ACTIVE seq:25 [1.2.3.4]:47916 <> [6.7.8.9]:443 proto:6 octets>:104 packets>:2 octets<:4476 packets<:3 start:2020-04-05T09:38:46.596 finish:2020-04-05T09:38:46.711 tcp>:10 tcp<:10 flowlabel>:00000000 flowlabel<:00000000
EXPIRY EVENT for flow 25 in 8 seconds
ACTIVE seq:26 [1.2.3.4]:47768 <> [6.7.8.9]:443 proto:6 octets>:936 packets>:18 octets<:88028 packets<:59 start:2020-04-05T09:38:49.294 finish:2020-04-05T09:39:37.531 tcp>:10 tcp<:10 flowlabel>:00000000 flowlabel<:00000000
EXPIRY EVENT for flow 26 in 11 seconds
ACTIVE seq:27 [1.2.3.4]:59374 <> [14.15.16.17]:80 proto:6 octets>:0 packets>:0 octets<:52 packets<:1 start:2020-04-05T09:39:09.036 finish:2020-04-05T09:39:09.036 tcp>:00 tcp<:10 flowlabel>:00000000 flowlabel<:00000000
EXPIRY EVENT for flow 27 in 31 seconds
Netflow version 9 sends a template every so often (I think ~ every 30mins or so) to describe the structured of the exported flows. If your analyzer hasnt received this you can force softflowd to include the template in the next update with softflowctl send-template.
root@Them00n:~# softflowctl send-template
softflowd[11100]: Template will be sent at next flow export
You can also view the statistics of the application using softflowctl statistics
root@Them00n:~# softflowctl statistics
softflowd[11100]: Accumulated statistics since 2020-04-05T09:35:42 UTC:
Number of active flows: 10
Packets processed: 4896
Packets non-sampled: 484646
Fragments: 0
Ignored packets: 0 (0 non-IP, 0 too short)
Flows expired: 567 (0 forced)
Flows exported: 593 in 133 packets (18 failures)
Packets received by libpcap: 489624
Packets dropped by libpcap: 0
Packets dropped by interface: 0
Expired flow statistics: minimum average maximum
Flow bytes: 36 9768 119091
Flow packets: 1 9 105
Duration: 0.00s 15.61s 81.51s
Expired flow reasons:
tcp = 398 tcp.rst = 5 tcp.fin = 0
udp = 44 icmp = 9 general = 0
maxlife = 111
over 2 GiB = 0
maxflows = 0
flushed = 0
Per-protocol statistics: Octets Packets Avg Life Max Life
icmp (1): 686 9 0.00s 0.00s
tcp (6): 5473973 4746 17.06s 81.51s
udp (17): 63547 91 1.82s 48.37s
root@Them00n:~#
The final (and as ever best) proof we're actually exporting flows can be done with tcpdump. If its not installed already, install it and listen for the flows being exported:
root@Them00n:~# opkg install tcpdump
Installing tcpdump (4.9.3-1) to root...
Downloading http://downloads.openwrt.org/releases/19.07.2/packages/mips_24kc/base/tcpdump_4.9.3-1_mips_24kc.ipk
Configuring tcpdump.
root@Them00n:~# tcpdump -i br-lan port 9995 or port 9996
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-lan, link-type EN10MB (Ethernet), capture size 262144 bytes
10:31:06.421755 IP Netflow.home.45100 > OpenWrt.home.9995: UDP, length 264
10:31:26.224629 IP Netflow.home.45100 > OpenWrt.home.9995: UDP, length 168
10:32:01.154466 IP Netflow.home.45100 > OpenWrt.home.9995: UDP, length 72
10:32:01.206544 IP Netflow.home.59400 > OpenWrt.home.9996: UDP, length 328
10:32:10.522646 IP Netflow.home.45100 > OpenWrt.home.9995: UDP, length 216
There we go job done :)
m00nie
More detailed info can be found on these links: