ngrep fails when using libpcap v1.8.1 and newer kernel version with warning kernel filter failed #23
Description
Using ngrep with more than 6 filters on a newer kernel version such as 4.18, 5.0, 5.3 or 5.4 will result in a complete fail when libpcap v1.8.1 is used (e.g. ubuntu 18.04). ngrep with more than 6 filters with libpcap 1.9.1 will run but still returns a warning about kernel filter failed: cannot allocate memory.
The complete fail with libpcap v1.8.1 appears to have been fixed around 1.9.0. As for the warnings "Warning: Kernel filter failed: Cannot allocate memory", which persists even with libpcap 1.9.1, I suspect this is due to recent bug fixes and improvements to the bpf verifier as well as ngrep attempting to apply filters in a non-optimal way. I do not experience any of these same failures or warnings when attempting to use tcpdump with more than 6 filters.
- https://www.tcpdump.org/libpcap-changes.txt
- 1.9.0-rc1 - the-tcpdump-group/libpcap@995849a
Fix some Linux memory-mapped capture buffer size issues
Don't fail if kernel filter can't be set on Linux (GitHub issue #549) - pcap-linux.c - if kernel filter cannot be set, BPF filter setup will fail with "Machine is not on the network" the-tcpdump-group/libpcap#549
Some bug fixes to bpf verifier:
- https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=979d63d50c0c0f7bc537bf821e056cc9fe5abd38
- https://nvd.nist.gov/vuln/detail/CVE-2019-7308
- https://blog.cloudflare.com/ebpf-cant-count/
libpcap 1.8.1 + kernel 4.18, 5.0, 5.3, 5.4 Output:
# uname -sr
Linux 5.4.0-42-generic
# /usr/bin/ngrep -V
ngrep: V1.47.1-git, libpcap version 1.8.1
# ngrep -q -t -d any -W byline '' port 123 or 6001 or 6010 or 6002 or 6003 or 6004 or 6005
interface: any
Warning: Kernel filter failed: Cannot allocate memory
Warning: Kernel filter failed: Cannot allocate memory
pcap: can't remove kernel filter: No such file or directory
libpcap 1.9.1 + kernel 4.18, 5.0, 5.3, 5.4 Output:
# uname -sr
Linux 4.18.0-1020-aws
# LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH ./ngrep -V
ngrep: V1.47.1-git, libpcap version 1.9.1 (with TPACKET_V3)
# LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH /usr/local/src/ngrep*/ngrep -q -t -d any -W byline '' \(port 123 or 5999 or 6000 or 6001 or 6002 or 6003 or 6004\)
interface: any
Warning: Kernel filter failed: Cannot allocate memory
filter: ( (port 123 or 5999 or 6000 or 6001 or 6002 or 6003 or 6004) ) and (ip)
U 2020/09/01 13:46:22.778659 10.10.11.123:42044 -> 91.189.89.199:123 #1
#............................................i..
U 2020/09/01 13:46:22.852287 91.189.89.199:123 -> 10.10.11.123:42044 #2
$......U......"{...a.W.X.....i........a8......y.
libpcap 1.8.1 + kernel 4.15
# uname -sr
Linux 4.15.0-1080-aws
# ngrep -q -t -d any -W byline '' \(port 123 or 5999 or 6000 or 6001 or 6002 or 6003 or 6004\)
interface: any
filter: ( (port 123 or 5999 or 6000 or 6001 or 6002 or 6003 or 6004) ) and (ip)
U 2020/09/01 13:57:07.137353 10.10.11.123:54195 -> 91.189.94.4:123 #1
#............................................/..
U 2020/09/01 13:57:07.212640 91.189.94.4:123 -> 10.10.11.123:54195 #2
$...........^..
.....n......./......-..)....-...