Clone(skb), The inside story on Packet Sniffing Efficiently on a Linux Platform

Even if you are not a  complete geek you might find this interesting.

The two common tools in standard Linux used in many commercial packet sniffing firewalls are, IPtables, and  the Layer7 Packet Classifier.  These low level rule sets are often used in commercial firewalls to identify protocols ( Youtube, Netflix etc) and  then to take action by blocking them or reducing their footprint;  however in their current form, they can bog down your firewall when exposed to higher throughput levels.  The  basic problems as you  run at high line speeds are

  •  The path through the Linux Kernel is bottle necked around an Interface port. What this means is that for every packet that must be analyzed for a specific protocol, the interface port where packets arrive, is put on hold while the analysis is completed. Think of a line of cars being searched going through a border patrol check point. Picture the back up as each car is completely searched at the gate while other cars wait in line. This is essentially what happens in the standard Linux-based packet classifier, all packets are searched while other packets wait in line. Eventually this can cause latency.
  • The publicly available protocol patterns are not owned and supported by any entity and  they are somewhat unreliable. I know, because I wrote and tested many of them over 10 years ago and they are still published and re-used. In fairness, protocol accuracy will always be the Achilles heel of layer 7 detection. There is however some good news in this area which I will cover shortly.

Technology Changes in the Kernel to alleviate the bottleneck

A couple of years ago we had an idea to create a low-cost turn-key intrusion detection device. To build something that could stand up to todays commercial line speeds we would require a better layer 7 detection engine that the standard IPtables solution.  We ended building a very nice Intrusion detection device called the NetGladiator.  One of the stumbling blocks of building this device that we overcame was to maintain a commercial grade line speed of up to 1 gigabyte while still being able to inspect packets.  How did we do it?

Okay so I am a geek, but while poking around in the Linux Kernel I  noticed an interesting call titled Clone(skb). What clone skb does, is allow you to make a very fast copy of an IP packet and its data as it comes through the kernel.  I also noticed in  the newer Linux kernel there was a mechanism for multi-threading.  If you go back to my analogy of cars lined up the border  you can think of  multi-threading and cloning each car such that:

1) Car comes to the border station,

2) clone (copy) it, wave it through without delay

3) send the clone off to a processing lab for analysis ,  a really close by lab near the border

4) If the analysis  from the lab comes back with contraband in the clone, then send a helicopter after the original car and arrest the occupants

5) Throw the clone away

We have taken the cloning and multi-threading elements of the Linux Kernel and produced a low cost, accurate packet classifier that can run at 10 times the line speeds as the standard tools. It will be released in Mid February


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: