Packet drop numbers incorrect on Linux with new libpcap

[pw@…]

This issue is a followup to the mailinglist post http://mailman.icsi.berkeley.edu/pipermail/bro/2008-November/003843.html
The number of packet drops reported by Bro is not always correct. Here is a detailed explanation:

Bro queries the libpcap component in order to get the number of dropped packets, using the pcap_stats function. The responsible code in PktSrc::Statistics() distinguishes between Linux and FreeBSD systems. It is assumed that for Linux the value delivered by pcap_stats gives the number of drops since the last query, whereas on FreeBSD it gives the number of drops since the start of the current Bro run. It seems, that for certain older libpcap versions this assumption was correct, even though, by specification, pcap_stats should always give a cumulative count of drops. However, due to a bug in libpcap, its behavior deviated on Linux systems, requiring the mentioned distinction of the OSes in the Bro code.

It seems, that this bug in libpcap has been fixed in the meantime, and starting from version 0.9 the Linux behavior correctly matches the specification again, and thus, also matches the behavior of FreeBSD. Check: http://www.archivum.info/debian-bugs-closed@lists.debian.org/2005-06/msg00488.html

Therefore, on Linux systems running libpcap versions > 0.9 the packet drop numbers reported by Bro are wrong. I have experienced this problem on my machine running Debian Lenny and libpcap 0.9.8. I was able to fix it by simply using the FreeBSD branch of the code. Probably, it is no longer suitable to make the distinction based solely on the OS, but one should take into consideration the libpcap version as well.

[robin]

#116 has some more details. Please continue discussion here.

[robin]

From William L. Jones:

I would base it on the pcap version


#define PCAP_VERSION_MAJOR 2
#define PCAP_VERSION_MINOR 4

or greater and check for PACKET_STATISTIC getsockopt function.

[robin]

More from Bill:

I looked at the libcap source over the past 6 years and found the change to correct linux pcap stats behavior took place in release libpcap-0.9.1 around 05-Jul-2005.

The bad news is I that the major and minor version numbers weren't change, sigh.

[robin]

The bad news is I that the major and minor version numbers weren't change, sigh.

:-(

Does anybody have an idea how we can check the libpcap version?

[gregor]

Only with some hacks I think.

1)
There's the pcap_lib_version() function. Unfortunately it returns a string, which must then be parsed (and different linux distros might have fancy ideas about using there own versioning for libpcap....)

2)
Another option is to look for changes in the API between 0.8.3 and 0.9.1. A quick check indicates that the following functions are new in 0.9.1

int pcap_setdirection(pcap_t *, direction_t);
pcap_t *pcap_fopen_offline(FILE *, char *);
int pcap_inject(pcap_t *, const void *, size_t);
int pcap_sendpacket(pcap_t *, const u_char *, int);
pcap_dumper_t *pcap_dump_fopen(pcap_t *, FILE *fp);
long pcap_dump_ftell(pcap_dumper_t *);

So we could use autoconf to check whether one of these exist. A problem could arise if distributions already included some of these functions without including the change to the drop counter.

3)
Another way is to check for DLT types that are defined in 0.9.1 but not in 0.8.3. Here the changes are:
DLT 139-142 are new (although there's a comment in 0.8.3 indicating that these DLTs are resevered. However, they are not defined in 0.8.3)
DLTs 165--171 are new in 0.9.1

So one could check whether those are defined with an #ifdef or one can use pcap_datalink_val_to_name() to check whether one of these DLTs
are defined. (Note: DLTs are defined in pcap-bpf.h and I don't know whether the OS might have its own version of header files defining DLTs)

cu
gregor

[robin]

Seth, and others, suggests to just remove the code:

Would it make sense to just remove the linux specific code? Is anyone
running older Linux distros for their analyzers?

Based on one of the links in the ticket, it looks like Debian updated
their libpcap and got rid of this issue in 2005.

[robin]

I'm going to remove the Linux-specific branch but I don't have any good way to test it so I'll have to hope that it works fine. Also, there's more Linux-specific code in PktSrc? which I'm not going to touch because I don't know whether that has changed as well (seems less important though).

[robin]

Patch attached. Somebody here who can test whether numbers reported make sense on Linux?

[vern]

Integrated in 1.5-pre-release.10