NetEqualizer News: December 2013

December 2013


Enjoy another issue of NetEqualizer News! This month, we discuss new features planned for 2014, announce our FlyAway Contest winner, give you a heads-up on some options for your old NE2000 devices, and highlight NetGladiator enhancements. As always, feel free to pass this along to others who might be interested in NetEqualizer News.

A message from Art…
Art Reisman, CTO – APconnections

As the year comes to a close, we are wrapping up our 2013 goals and now starting to look ahead to 2014! I am excited about art_canoe_picturewhere I see 2014 taking APconnections and the NetEqualizer and NetGladiator. You will see our continued commitment to investing in our platforms, from our 2014 planned features for NetEqualizer, to our strengthening of the NetGladiator product, and finally our ongoing work to enhance the NCO caching module. Once you read about our plans, I think you will be excited too! We share them in this newsletter, so that you can start mapping out your plan for 2014 as well…

We love it when we hear back from you – so if you have a story you would like to share with us of how we have helped you, let us know. Email me directly at I would love to hear from you!

Planned NEW Features for 2014

The New Year is the perfect time to start thinking about new features for NetEqualizer! It is also a great time for you to start thinking about upgrading your device to the latest software.

2013 saw a lot of changes for NetEqualizer and for 2014 we plan on building on that base even more. In 2014, keep an eye out for some of these exciting new ideas:

1) Expanded caching – We’ve been enhancing our NetEqualizer Caching Option (NCO) for the the last several months and you should expect even more from this add-on feature in 2014.  We are testing out larger SSD drives, assessing whether Netflix can be cached, and looking for even more caching opportunities.

2) Heuristic-based identification – This is a really cool concept that we are currently developing. It is based on the idea that each user has their own unique “path” once they join the network. Knowing that path can help to identify users. The principles apply to both bandwidth optimization and security. Over the next year we’ll be implementing this idea and seeing what value it could add to both our NetEqualizer and NetGladiator product lines. See the next article, NetGladiator Continues to Grow, for more information.

3) Bigger, better, faster Reporting (RTR) – We have received very positive feedback on our initial RTR rollout, and our enhanced RTR Traffic Reports, which are currently in Beta.

We now feel it is time to expand RTR, with a goal to completely replace our ntop historical reporting by end of 2014.

We spent a lot of time in 2013 improving our user interface, and our commitment to making NetEqualizer easy to use will show in 2014 as well. Expect new features in our Dynamic Real-Time Reporting tool including, but not limited to:
– ntop-like historical data tracking
– Pool and VLAN drill-down reports
– Time of day configuration interface
– Penalty graphs over time
– and more!

These features will be free to customers with valid NetEqualizer Software and Support who are running version 7.0+ (NCO features will require NCO). If you are not current with NSS, contact us today!



NetGladiator Continues to Grow

Our investment in IPS continues!
We are starting to plan some new features for NetGladiator in 2014, including some exciting heuristic-based identification capability. Deep in the world of network authentication lies a hidden signature. The hidden signature of behavior – the websites you visit, the paths you take, the things you pause on. Just like your fingerprints, your signature when you enter a Network is unique. We’ll be implementing this idea of heuristic-based identification throughout the year – let us know what you think!

Also, we have talked to some of you in 2013 regarding your IPS needs. If you are looking for a simple, elegant, and affordable way to protect your web applications, you should think about the NetGladiator. You should also consider taking our Hacking Challenge to see if your web applications are safe and secure!netgladiator_logo Contact us at:



to discuss your security needs.

Our Next Local Linux Talk

Our CTO, Art Reisman, will be speaking at another local linux user group in early January.

The Boulder Linux Users Group will host the event in downtown Boulder, CO on January 9, 2014 at 6pm. Boulder is one of the biggest technology hotbeds outside of Silicon Valley, and we think there will be a lot of interesting discussion and ideas that come out of this meeting.

If you are in the Boulder, CO area at the time, feel free to stop on by!

And the FlyAway Contest Winner is…

Every few months, we have a drawing to give away two round-trip domestic airline tickets from Frontier Airlines to one lucky person who’s recently tried out our online NetEqualizer demo.

The time has come to announce this round’s airlines

And the winner is…

Jeff Gay at Morrisville State College! 

Congratulations, Jeff!

Please contact us within 30 days (by January 17, 2014) at:



to claim your prize!

Some Options for Your NE2000

Earlier this year, we announced that we are discontinuing our NE2000 series, and are moving the NE2000 license levels (20, 50, 100, and 150Mbps) onto the NE3000 platform. This change was made to get ready for our 7.0+ 64-bit releases, and also to take advantage of multi-core processing. We also felt that it was time to consolidate on the NE3000 platform.

We have talked to many of you regarding this change. However, if you have not already talked to us about trading in your NE2000, we offer a generous 50% trade-in credit of your original unit purchase price towards a new unit (1 trade-in credit per unit purchased please).NE2000 options differ depending on when your NE2000 was purchased. Some of the more recent NE2000’s (purchases from August 2011 and later) can run our 7.0+ software, and these customers will be able to get support AFTER 12/31/2014 on these units.  For units purchased prior to August 20011 that cannot run 7.0+, support will be offered through 12/13/2014.

Contact us at:



to discuss your options.

Best Of The Blog

Latest Notes on the Peer to Peer Front and DMCA Notices

By Art Reisman – CTO – APconnections

Just getting back from our tech talk seminar today at Western Michigan University. The topic of DMCA requests came up in our discussions, and here are some of my notes on the subject.

Background: The DMCA, which is the enforcement arm of the motion picture copyright conglomerate, tracks down users with illegal content.

They seem to sometimes shoot first and ask questions later when sending out their notices more specific detail to follow.

Unconfirmed Rumor has it, that one very large University in the State of Michigan just tosses the requests in the garbage and does nothing with them, I have heard of other organizations taking this tact. They basically claim this problem for the DMCA is not the responsibility of the ISP.

I also am aware of a sovereign Caribbean country that also ignores them. I am not advocating this as a solution just an observation…

Photo Of The Month
Happy Holidays!
Our CTO, Art Reisman, entered this truck in the Louisville, CO Holiday Parade. It was about 5 degrees below zero (Fahrenheit) when it was in the parade. This is the 2nd year that Art has created a “Christmas Truck,” and he uses it to deliver cookies to neighbors as well during the Holiday Season.

How to Block Frostwire, utorrent and Other P2P Protocols

By Art Reisman, CTO,

Art Reisman CTO

Disclaimer: It is considered controversial and by some definitions illegal for a US-based ISP to use deep packet inspection on the public Internet.

At APconnections, we subscribe to the philosophy that there is more to be gained by explaining your technology secrets than by obfuscating them with marketing babble. Read on to learn how I hunt down aggressive P2P traffic.

In order to create a successful tool for blocking a P2P application, you must first figure out how to identify P2P traffic. I do this by looking at the output data dump from a P2P session.

To see what is inside the data packets I use a custom sniffer that we developed. Then to create a traffic load, I use a basic Windows computer loaded up with the latest utorrent client.

Editors Note: The last time I used a P2P engine on a Windows computer, I ended up reloading my Windows OS once a week. Downloading random P2P files is sure to bring in the latest viruses, and unimaginable filth will populate your computer.

The custom sniffer is built into our NetGladiator device, and it does several things:

1) It detects and dumps the data inside packets as they cross the wire to a file that I can look at later.

2) It maps non printable ASCII characters to printable ASCII characters. In this way, when I dump the contents of an IP packet to a file, I don’t get all kinds of special characters embedded in the file. Since P2P data is encoded random music files and video, you can’t view data without this filter. If you try, you’ll get all kinds of garbled scrolling on the screen when you look at the raw data with a text editor.

So what does the raw data output dump of a P2P client look like ?

Here is a snippet of some of the utorrent raw data I was looking at just this morning. The sniffer has converted the non printable characters to “x”.
You can clearly see some repeating data patterns forming below. That is the key to identifying anything with layer 7. Sometimes it is obvious, while sometimes you really have work to find a pattern.

Packet 1 exx_0ixx`12fb*!s[`|#l0fwxkf)d1:ad2:id20:c;&h45h”2x#5wg;|l{j{e1:q4:ping1:t4:ka 31:v4:utk21:y1:qe
Packet 2 exx_0jxx`1kmb*!su,fsl0’_xk<)d1:ad2:id20:c;&h45h”2x#5wg;|l{j{e1:q4:ping1:t4:xv4^1:v4:utk21:y1:qe
Packet 3 exx_0kxx`1exb*!sz{)8l0|!xkvid1:ad2:id20:c;&h45h”2x#5wg;|l{j{e1:q4:ping1:t4:09hd1:v4:utk21:y1:qe
Packet 4 exx_0lxx`19-b*!sq%^:l0tpxk-ld1:ad2:id20:c;&h45h”2x#5wg;|l{j{e1:q4:ping1:t4:=x{j1:v4:utk21:y1:qe

The next step is to develop a layer 7 regular expression to identify the patterns in the data. In the output you’ll notice the string “exx” appears in line, and that is what you look for. A repeating pattern is a good place to start.

The regular expression I decided to use looks something like:


This translates to: match any string starting with “exx” followed, by any character “.” followed by “0”, followed by “xx”, followed by any sequence of characters ending with “qe”.

Note: When I tested this regular expression it turns out to only catch a fraction of the Utorrent, but it is a start. What you don’t want to do is make your regular expression so simple that you get false positives. A layer 7 product that creates a high degree of false positives is pretty useless.

The next thing I do with my new regular expression is a test for accuracy of target detection and false positives.

Accuracy of detection is done by clearing your test network of everything except the p2p target you are trying to catch, and then running your layer 7 device with your new regular expression and see how well it does.

Below is an example from my NetGladiator in a new sniffer mode. In this mode I have the layer 7 detection on, and I can analyze the detection accuracy. In the output below, the sniffer puts a tag on every connection that matches my utorrent regular expression. In this case, my tag is indicated by the word “dad” at the end of the row. Notice how every connection is tagged. This means I am getting 100 percent hit rate for utorrent. Obviously I doctored the output for this post :)

ndex SRCP DSTP Wavg Avg IP1 IP2 Ptcl Port Pool TOS
0 0 0 17 53 — 2 99 dad
1 0 0 16 48 — 2 99 dad
2 0 0 16 48 — 2 99 dad
3 0 0 18 52 — 2 99 dad
4 0 0 12 24 — 2 99 dad
5 0 0 18 52 — 2 99 dad
6 0 0 10 0 — 2 99 dad
7 0 0 88 732 — 2 99 dad
8 0 0 12 0 — 2 99 dad
9 0 0 12 24 — 2 99 dad
10 0 0 16 48 — 2 99 dad
11 0 0 11 16 — 2 99 dad
12 0 0 17 52 — 2 99 dad
13 0 0 27 54 — 2 99 dad
14 0 0 10 0 — 2 99 dad
15 0 0 14 28 — 2 99 dad
16 0 0 14 32 — 2 99 dad
17 0 0 10 0 — 2 99 dad
18 0 0 24 33 — 2 99 dad
19 0 0 17 53 — 2 99 dad

A bit more on reading this sniffer output…

Notice columns 4 and 5, which indicate data transfer rates in bytes per second. These columns contain numbers that are less than 100 bytes per second – Very small data transfers. This is mostly because as soon as that connection is identified as utorrent, the NetGladiator drops all future packets on the connection and it never really gets going. One thing I did notice is that the modern utorrent protocol hops around very quickly from connection to connection. It attempts not to show it’s cards. Why do I mention this? Because in layer 7 shaping of P2P, speed of detection is everything. If you wait a few milliseconds too long to analyze and detect a torrent, it is already too late because the torrent has transferred enough data to keep it going. It’s just a conjecture, but I suspect this is one of the main reasons why this utorrent is so popular. By hopping from source to source, it is very hard for an ISP to block this one without the latest equipment. I recently wrote a companion article regarding the speed of the technology behind a good layer 7 device.

The last part of testing a regular expression involves looking for false positives. For this we use a commercial grade simulator. Our simulator uses a series of pre-programmed web crawlers that visit tens of thousands of web pages an hour at our test facility. We then take our layer 7 device with our new regular expression and make sure that none of the web crawlers accidentally get blocked while reading thousands of web pages. If this test passes we are good to go with our new regular expression.

Editors Note: Our primary bandwidth shaping product manages P2P without using deep packet inspection.
The following layer 7 techniques can be run on our NetGladiator Intrusion Prevention System. We also advise that public ISPs check their country regulations before deploying a deep packet inspection device on a public network.

NetGladiator: A Layer 7 Shaper in Sheep’s Clothing

When explaining our NetGladiator technology the other day, a customer was very intrigued with our Layer 7 engine. He likened it to a caged tiger under the hood, gobbling up and spitting out data packets with the speed and cunning of the world’s most powerful feline.

He was surprised to see this level of capability in equipment offered at our prices.  He was impressed with the speed attained for the price point of our solution (more on this later in the article)…

In order to create a rock-solid IPS (Intrusion Prevention System), capable of handling network speeds of up to 1 gigabit with standard Intel hardware, we had to devise a technology breakthrough in Layer 7 processing. Existing technologies were just too slow to keep up with network speed expectations.

In order to support higher speeds, most vendors use semi-custom chip sets and a technology called “ASIC“. This works well but is very expensive to manufacture.

How do typical Layer 7 engines work?

Our IPS story starts with our old Layer 7 engine. It was sitting idle on our NetEqualizer product. We had shelved it when we got away from from Layer 7 shaping in favor of Equalizing technology, which is a superior solution for traffic shaping.  However, when we decided to move ahead with our new IPS this year, we realized we needed a fast-class analysis engine, one that could look at all data packets in real time. Our existing Layer 7 shaper only analyzed headers because that was adequate for its previous mission (detecting P2P streams).  For our new IPS system, we needed a solution that could do a deep dive into the data packets.  The IPS mission requires that you look at all the data – every packet crossing into a customer network.

The first step was to revamp the older engine and configure it to look at every packet. The results were disappointing.  With the load of analyzing every packet, we could not get throughput any higher than about 20 megabits, far short of our goal of 1 gigabit.

What do we do differently with our updated Layer 7 engine?

Necessity is the mother of invention, and so we invented a better Layer 7 engine.

The key was to take advantage of multiple processors for analysis of data without delaying data packets. The way the old technology worked was that it would intercept a data packet on a data link, hold it, analyze it for P2P patterns, and then send it on.  With this method, as packets come faster and faster you end up not having enough CPU time to do the analysis and still send the packet on without adding latency.  Many customers find this out the hard way when they update their data speeds from older slower T1 technology.  Typical analysis engines on affordable routers and firewalls often just can’t keep up with line speeds.

What we did was take advantage of a utility in the Linux Kernel called “clone skb”.  This allows you to make a temporary copy of the data packet without the overhead of copying.  More importantly, it allows us to send the packet on without delay and do the analysis within a millisecond (not quite line speed, but fast enough to stop an intruder).

We then combined the cloning with a new technology in the Linux kernel called Kernel Threading.  This is different than the technology that large multi-threaded HTTP servers use because it happens at the kernel level, and we do not have to copy the packet up to some higher-level server for analysis. Copying a packet for analysis is a huge bottleneck and very time-consuming.

What were our Results?

With kernel threading, cloning, and a high-end Intel SMP processor, we can make use of 16 CPU’s doing packet analysis at the same time and we now have attained speeds close to our 1 gigabit target.

When we developed our bandwidth shaping technology in 2003/2004, we leveraged technology innovation to create a superior bandwidth control appliance (read our NetEqualizer Story).  With the NetGladiator IPS, we have once again leveraged technology innovation to enable us to provide an intrusion prevention system at a very compelling price (register to get our price list), hence our customer’s remark about great speed for the price.

What other benefits does our low cost, high-speed layer 7 engine allow for? Is it just for IPS?

The sky is the limit here.  Any type of pattern you want to look at in real-time can now be done at one tenth (1/10th) the cost of the ASIC class of shapers.  Although we are not a fan of unauthorized intrusion into private data of the public Internet (we support Net Neutrality), there are hundreds of other uses which can be configured with our engine.

Some that we might consider in the future include:

– Spam filtering
– Unwanted protocols in your business
– Content blocking
– Keyword spotting

If you are interested in testing and experimenting in any of these areas with our raw technology, feel free to contact us

A Smarter Way to Limit P2P Traffic

By Art Reisman

Art Reisman CTO

Editor’s note: Art Reisman is the CTO of APconnections. APconnections designs and manufactures the popular NetEqualizer bandwidth shaper.

If you are an IT professional interested in the ethical treatment of P2P (which we define as keeping it in check without invading the privacy of your customers by looking at their private data), you’ll appreciate our next generation approach to containing P2P usage. Thanks to some key input by a leading-edge ISP in South Africa, we have developed a next-generation P2P control that balances the resources of an ISP, and yet allows their end customers to use Bittorent without bringing down the network.

First a quick review of how P2P affects a network

A signature of a typical P2P user is that they can open hundreds of small connections while downloading files. A P2P client, such as Kazaa, is designed to find as many sources to a file as possible. For efficiency and speed, P2P clients operate as multi-threaded download engines, where each download stream captures a different segment of the requested file. When all the segments are complete they are re-assembled into a complete usable media file on your hard drive. The multiple downloads cause a strain on network bandwidth resources. They also create extreme overhead on wireless routers. Extreme P2P usage by just a subset of users can crowd out web pages, VoIP, YouTube and many other less aggressive applications.

Current P2P Limiting Solution: Connection Limits

Our current generation of P2P control involves intelligently looking at the number of connections generated from a user on your network. Based on the persistence and number of connections, we can reliably tell if a user is currently using P2P. The current P2P remedy, deployed on our NetEqualizer equipment, involves limiting the number of connections of suspected P2P users; this works well to limit p2p usage.  Thus, it keeps the P2P users from overwhelming a shared network.

Next-Generation P2P Limiting: Smart Connection Limits

While we have retained the connection-limiting aspects of our current P2P limiting technology, our new technology goes a step further. With Smart Connection Limits, limiting is done by also slowly starving the P2P connections for bandwidth. The bandwidth reduction is based on a formula which takes into a account two main factors:

1) the number of connections a user has open.
2) the load on the network.

I like to think of this technology as more of a “reward system”, resulting in a higher quality of service for non-P2P users.  In this case, the reward is that non-P2P users’ connections are not experiencing this reduction in bandwidth (although they may get equalized on any connection that is hogging bandwidth).  P2P users will slowly see less bandwidth allocated to their P2P traffic, which should discourage them from using P2P on your network.  Basically, this helps to train them to use better behavior – sharing the network resource more fairly with others.

This philosophyof fairness is aligned with the primary goal of the NetEqualizer – to ensure fairness for all network users. It follows that if a user has 20 concurrent streams and another user only has 5, to ensure equal  use of bandwidth under network load, the user with 20 streams should have his streams operate at 1/4 the speed of the user that has 5. While you may configure Smart Connection Limits at various levels, you could enforce the example indicated above.

The reason this technology is important is that, on a network pressed for bandwidth, the P2P users are often taking an unfair share. Even with basic rate caps per user in place, you often must augment that restriction by limiting the total number of connections per user. And now with our latest technology, we also temporarily restrict the bandwidth per connection (only applied to the P2P users).

If you are interested in learning more about Smart Connection Limits, to see if they are a fit for your network, contact us.

Some common questions and answers:

Is it possible to completely block P2P?

It is never safe to try to completely block p2p for a couple of reasons.

1) Although it is always possible to identify P2P, it is often expensive and not foolproof. To block it based on hearsay will cause problems. Our solution, although targeted on limiting P2P, focuses on the resource footprint of the P2P user, and does not attempt to outright block types of traffic. In other words, whether or not the traffic is actually P2P is not the issue. The issue is, is this user abusing resources? If yes, they get punished.

2) Devices that attempt to identify P2P traffic often use a technique called deep packet inspection (DPI), which is frowned upon as an invasion of privacy.  Additionally, we are finding that the latest P2P tools (such as utorrent) encrypt P2P streams as their default behavior, which defeats deep packet inspection.  Not so with our solutions; both Connection Limits and Smart Connection Limits will throttle encrypted P2P traffic.

Who do we recommend move from Connection Limits to Smart Connection Limits ?

If you are in a business where you charge for bandwidth usage (ISP, WISP, satellite provider), you should consider implementing Smart Connection Limits.  We also recommend looking at Smart Connection Limits if you have repeat offenders – basically, the same users are consistently running P2P traffic on your network and you want to change their behavior.

Can I continue using the Connection Limits or do I need to move to Smart Connection Limits?

Both solutions to Limit P2P traffic are being supported. If you do not have a lot of P2P traffic on your network, you may opt to stay with Connection Limits, as a quick-and-easy implementation. Smart Connection Limits take a little more thought to implement and have additional complexity, which you may not wish to take on at this point.

NetEqualizer News: October 2011

NetEqualizer News

October 2011


Enjoy another issue of NetEqualizer News! This month, we present a video demonstration detailing how active connections behave on a live network. The video utilizes a real-time reporting tool that you can leverage with your own NetEqualizer data! We also preview some new features coming this fall (IPv6 Visibility and ToS Priority), announce our FlyAway Contest winner, and discuss P2P blocking! As always, feel free to pass this along to others who might be interested in NetEqualizer News.

Our Website     Contact Us      NetEqualizer Demo      Price List      Join Our Mailing List

In This Issue:

:: Demo: How Active Connections Behave in Real Time

:: And The Fly Away Contest Winner Is…

:: Update on New Features Coming This Fall

:: Best Of The Blog

Demo: How Active Connections Behave in Real Time

We often get asked about active connections and how they are handled by the NetEqualizer. The answer to this question is fundamental to how equalizing and behavior-based bandwidth shaping works.

In early August, we posted an article on our blog that discussed how you could generate real-time reports using Excel and your NetEqualizer data. The video linked to below references that project, and uses it to demonstrate how active connections behave in real-time on a live network.

There are some interesting observations you can take away from this video, even if you don’t implement the reporting tool on your own device. You will come away from it with a better understanding of how users are connected through your network, and what types of connections are occurring every second.

Click the image below to view the video.  Note: real-time reports using Excel functionality has been replaced by Dynamic Real-Time Reporting in software update 7.1:

Some key points from the video are:

  • For every user, there are many connections occurring that most people are probably not aware of. The OS might be checking for updates, A/V could be checking for new signatures, an email program is reloading its inbox, etc.
  • Most connections have a very short life, and they are also mostly very small. 90% of connections will only utilize 10 to 1000 bytes/second.
  • Flows change dynamically. Even for a single user, 2 to 20 connections (or more) can exist at any moment in time.
  • Contention can occur quickly. Because of the variability in connections (especially with a broad user base), network contention can occur quickly. If large downloads are part of the active connections, this contention happens even faster.
  • The NetEqualizer instantly responds to this problem by taking a Robin Hood approach to the hogging connections. It shaves off bandwidth from the large connections and gives that much-needed resource to the thousands of other connections that require it.

View the blog article referenced in the video above here:
Dynamic Reporting With The NetEqualizer.

And The FlyAway Contest Winner Is…

frontier airlinesEvery few months, we have a drawing to give away two roundtrip domestic airline tickets from Frontier Airlines to one lucky person who’s recently tried out our online NetEqualizer demo.
The time has come to announce this round’s winner.
And the winner is…Mohammed O. Ibrahim of Zanzibar Connections.  Congratulations, Mohammed!
Please contact us within 30 days (by November 10th, 2011) at: email
admin -or- 303-997-1300 to claim your prize.

Update on New Features
Coming This Fall!

We are very excited about the new features coming in our Fall 2011 Software Update!

IPv6 Visibility

As we await the need to handle significant amounts of IPv6 traffic, NetEqualizer is already implementing solutions to meet the shift head-on. The Fall 2011 Software Update will include features that will provide enhanced visibility to IPv6 traffic.

This feature will help our customers that are experimenting with IPv6/IPv4 dual stacks, as they start to see IPv6 Internet traffic on their networks.

The enhanced IPv6 capabilities that we are implementing in the NetEqualizer this Fall include:

  • Providing you with visibility to current IPv6 connections so that you to determine if you need to start shaping IPv6 traffic.
  • Logging the IPv6 traffic so that you can obtain a historical snapshot to help in your IPv6 planning efforts.

ToS Priority

We are now seeing an influx of customers looking to provide priority bandwidth to VoIP connections on their links without all the hassle of complex router rules. NetEqualizer’s new Type of Service (ToS) Priority feature is the solution. Included in the Fall 2011 Software Update, the ToS Priority feature will automatically prioritize connections that are utilizing services like VoIPas well as a host of other types of important connections. This will provide improved quality of service (QoS) on your network.

Larger SSD Drives

We will now be shipping with larger SSD drives to customers waiting to try our NetEqualizer Caching Option (NCO).

As always, the Fall 2011 Software Update will be available at no charge to customers with valid NetEqualizer Software Subscriptions (NSS).

For more information on the NetEqualizer or the upcoming release, visit our blog or contact us at: email sales -or- toll-free U.S.(800-918-2763), worldwide (303) 997-1300 x. 103.

Best of the Blog

How Effective is P2P Blocking?
by Art Reisman – CTO – NetEqualizer

This past week, a discussion about peer-to-peer (P2P) blocking tools came up in a user group that I follow. In the course of the discussion, different IT administrators chimed in, citing their favorite tools for blocking P2P traffic.

At some point in the discussion, somebody posed the question, “How do you know your peer-to-peer tool is being effective?” For the next several hours the room went eerily silent.

The reason why this question was so intriguing to me is that for years I collaborated with various developers on creating an open-source P2P blocking tool using layer 7 technology (the Application Layer of the OSI Model). During this time period, we released several iterations of our technology as freeware. Our testing and trials showed some successes, but we also learned how fragile the technology was and we were reluctant to push it out commercially.

To keep reading, click here.

Photo Of The Month

NetEqualizer CF Card

New Design!

As of August 10th, 2011, our Compact Flash Cards are being shipped with a new label design and card case!

View our videos on YouTube

How Effective is P2P Blocking?

This past week, a discussion about peer-to-peer (P2P) blocking tools came up in a user group that I follow. In the course of the discussion, different IT administrators chimed in, citing their favorite tools for blocking P2P traffic.

At some point in the discussion, somebody posed the question, “How do you know your peer-to-peer tool is being effective?” For the next several hours the room went eerily silent.

The reason why this question was so intriguing to me is that for years I collaborated with various developers on creating an open-source P2P blocking tool using layer 7 technology (the Application Layer of the OSI Model). During this time period, we released several iterations of our technology as freeware. Our testing and trials showed some successes, but we also learned how fragile the technology was and we were reluctant to push it out commercially. I had always wondered if other privately-distributed layer 7 blocking tools had found some magic key to perfection?

Sometimes, written words can be taken as fact even though the same spoken words might be dismissed as gossip; and so it was with our published open source technology. We started getting indications that it was getting picked up and integrated in other solutions and touted as gospel.

Our experience with P2P blocking:

Our free P2P blocking tool worked most of the time – maybe eighty percent. Eighty percent accuracy is fine for an experimental open-source tool. Intuitively, a blocking tool is expected to be 99.9 percent effective. Even though most customers would likely not conclusively measure our accuracy, eighty percent was too low to ethically sell this technology without disclosures.

The on-line discussion ended fairly quickly when the question of accuracy was brought up, and I think it is safe to assume the silence is an indication that nobody else was achieving better than eighty percent.

How do you validate the effectiveness of a P2P tool?

1) Brute force testing:

I am not aware of too many IT administrators that have the time to load up six or seven different P2P clients on their laptops, and download bootlegged Madonna videos all day.

In testing P2P clients, we infected several computers with just about every virus in circulation. Over time, you can get a rough idea of how deep you must go to expose weaknesses in your tool set. To be thorough, you can’t stop at the first P2P client tool. In the real world, users on your network will likely search for multiple P2P clients, especially if the first one fails. Once they find a kink in the armor, they will yap to others, exposing your Achilles heel.

2) Reduction of RIAA requests:

Most small-to-medium ISP’s don’t really think about P2P unless they get RIAA requests or their network is saturated.

RIAA requests seem to be a big motivator in purchasing technology to block P2P. If you are getting RIAA requests (these are letters from lawyers threatening to sue you for copyright infringement), you can install your P2P blocking tool, and if in the next week your notifications of copyright violations are way down, you can assume that you have put a good dent in your P2P downloading issue.

3) Reduced congestion:

Plug your P2P tool in and see if your network utilization drops.

4) Lower connection rates through your router:

One of the signatures of P2P is that clients will open up hundreds of connections per minute to P2P servers in order to download content. There are ways to measure and quantify these connection rates empirically.

Other observations:

Many times we’ll hear from an ISP/operator claiming they have P2P users run amok on their network, however analysis often shows most of their traffic is video – Netflix, YouTube, Hulu, etc.

Total P2P traffic has really dropped off quite a bit in the last three or four years. We attribute this decline to:

1) Legal iTunes. 99 cent songs have eliminated the need for pirated music.

2) RIAA enforcement and education of copyright laws.

3) The invention of the iPad and iPhone. These devices control the applications which run on them (they are not going to distribute P2P clients as readily).

One method to handle P2P problems is to control all the computers in your environment, scan them before granting network access, and then block access to P2P sites (the sites where the client utilities are loaded from).

Note: once a P2P client is loaded on a computer you cannot block any single remote site, as the essence of P2P is that the content is not centralized.


Results of different P2P blocking techniques are often temporary, especially when you have an aggressive user base with motivation to download free content.

NetEqualizer P2P Locator Technology

Editor’s NoteThe NetEqualizer has always been able to thwart P2P behavior on a network. However, our new utility can now pinpoint an individual P2P user or gamer without any controversial layer-7 packet inspectionThis is an extremely important step from a privacy point of view as we can actually spot P2P users without looking at any private data.

A couple of months ago, I was doing a basic health check on a customer’s heavily used residential network. In the process, I instructed the NetEqualizer to take a few live snapshots. I then used the network data to do some filtering with custom software scripts. Within just a few minutes, I was able to inform the administrator that eight users on his network were doing some heavy P2P, and one in particular looked to be hosting a gaming session. This was news to the customer, as his previous tools didn’t provide that kind of detail.

A few days later, I decided to formally write up my notes and techniques for monitoring a live system to share on the blog. But, as I got started, another lightbulb went on…in the end, many customers just want to know the basics — who is using P2P, hosting game servers, etc. They don’t always have the time to follow a manual diagnostic recipe.

So, with this in mind, instead of writing up the manual notes, I spent the next few weeks automating and testing an intelligent utility to provide this information. The utility is now available with NetEqualizer 5.0.

The utility provides: 

  • A list of users that are suspected of using P2P
  • A list of users that are likely hosting gaming servers
  • A confidence rating for each user (from high to low)
  • The option of tracking users by IP and MAC address

The key to determining a user’s behavior is the analysis of the fluctuations in their connection counts and total number of connections. We take snapshots over a few seconds, and like a good detective, we’ve learned how to differentiate P2P use from gaming, Web browsing and even video. We can do this without using any deep packet inspection. It’s all based on human-factor heuristics and years of practice.

Enclosed is a screen shot of the new P2P Locator, available under our Reports & Graphing menu.

Our new P2P Locator technology

Contact us to learn more about the NetEqualizer P2P Locator Technology or NetEqualizer 5.0. For more information about ongoing changes and challenges with BitTorrent and P2P, see Ars Technica’s “BitTorrent Has New Plan to Shape Up P2P Behavior.”

%d bloggers like this: