How to Block Frostwire, utorrent and Other P2P Protocols

July 18, 2012 — netequalizer

By Art Reisman, CTO, http://www.netequalizer.com

Art Reisman CTO www.netequalizer.com

Disclaimer: It is considered controversial and by some definitions illegal for a US-based ISP to use deep packet inspection on the public Internet.

At APconnections, we subscribe to the philosophy that there is more to be gained by explaining your technology secrets than by obfuscating them with marketing babble. Read on to learn how I hunt down aggressive P2P traffic.

In order to create a successful tool for blocking a P2P application, you must first figure out how to identify P2P traffic. I do this by looking at the output data dump from a P2P session.

To see what is inside the data packets I use a custom sniffer that we developed. Then to create a traffic load, I use a basic Windows computer loaded up with the latest utorrent client.

Editors Note: The last time I used a P2P engine on a Windows computer, I ended up reloading my Windows OS once a week. Downloading random P2P files is sure to bring in the latest viruses, and unimaginable filth will populate your computer.

The custom sniffer is built into our NetGladiator device, and it does several things:

1) It detects and dumps the data inside packets as they cross the wire to a file that I can look at later.

2) It maps non printable ASCII characters to printable ASCII characters. In this way, when I dump the contents of an IP packet to a file, I don’t get all kinds of special characters embedded in the file. Since P2P data is encoded random music files and video, you can’t view data without this filter. If you try, you’ll get all kinds of garbled scrolling on the screen when you look at the raw data with a text editor.

So what does the raw data output dump of a P2P client look like ?

Here is a snippet of some of the utorrent raw data I was looking at just this morning. The sniffer has converted the non printable characters to “x”.
You can clearly see some repeating data patterns forming below. That is the key to identifying anything with layer 7. Sometimes it is obvious, while sometimes you really have work to find a pattern.

Packet 1 exx_0ixx`12fb*!s[`|#l0fwxkf)d1:ad2:id20:c;&h45h”2x#5wg;|l{j{e1:q4:ping1:t4:ka 31:v4:utk21:y1:qe
Packet 2 exx_0jxx`1kmb*!su,fsl0′_xk<)d1:ad2:id20:c;&h45h”2x#5wg;|l{j{e1:q4:ping1:t4:xv4^1:v4:utk21:y1:qe
Packet 3 exx_0kxx`1exb*!sz{)8l0|!xkvid1:ad2:id20:c;&h45h”2x#5wg;|l{j{e1:q4:ping1:t4:09hd1:v4:utk21:y1:qe
Packet 4 exx_0lxx`19-b*!sq%^:l0tpxk-ld1:ad2:id20:c;&h45h”2x#5wg;|l{j{e1:q4:ping1:t4:=x{j1:v4:utk21:y1:qe

The next step is to develop a layer 7 regular expression to identify the patterns in the data. In the output you’ll notice the string “exx” appears in line, and that is what you look for. A repeating pattern is a good place to start.

The regular expression I decided to use looks something like:

exx.0.xx.*qe

This translates to: match any string starting with “exx” followed, by any character “.” followed by “0″, followed by “xx”, followed by any sequence of characters ending with “qe”.

Note: When I tested this regular expression it turns out to only catch a fraction of the Utorrent, but it is a start. What you don’t want to do is make your regular expression so simple that you get false positives. A layer 7 product that creates a high degree of false positives is pretty useless.

The next thing I do with my new regular expression is a test for accuracy of target detection and false positives.

Accuracy of detection is done by clearing your test network of everything except the p2p target you are trying to catch, and then running your layer 7 device with your new regular expression and see how well it does.

Below is an example from my NetGladiator in a new sniffer mode. In this mode I have the layer 7 detection on, and I can analyze the detection accuracy. In the output below, the sniffer puts a tag on every connection that matches my utorrent regular expression. In this case, my tag is indicated by the word “dad” at the end of the row. Notice how every connection is tagged. This means I am getting 100 percent hit rate for utorrent. Obviously I doctored the output for this post :)

ndex SRCP DSTP Wavg Avg IP1 IP2 Ptcl Port Pool TOS
0 0 0 17 53 255.255.255.255 95.85.150.34 — 2 99 dad
1 0 0 16 48 255.255.255.255 95.82.250.60 — 2 99 dad
2 0 0 16 48 255.255.255.255 95.147.1.179 — 2 99 dad
3 0 0 18 52 255.255.255.255 95.252.60.94 — 2 99 dad
4 0 0 12 24 255.255.255.255 201.250.236.194 — 2 99 dad
5 0 0 18 52 255.255.255.255 2.3.200.165 — 2 99 dad
6 0 0 10 0 255.255.255.255 99.251.180.164 — 2 99 dad
7 0 0 88 732 255.255.255.255 95.146.136.13 — 2 99 dad
8 0 0 12 0 255.255.255.255 189.202.6.133 — 2 99 dad
9 0 0 12 24 255.255.255.255 79.180.76.172 — 2 99 dad
10 0 0 16 48 255.255.255.255 95.96.179.38 — 2 99 dad
11 0 0 11 16 255.255.255.255 189.111.5.238 — 2 99 dad
12 0 0 17 52 255.255.255.255 201.160.220.251 — 2 99 dad
13 0 0 27 54 255.255.255.255 95.73.104.105 — 2 99 dad
14 0 0 10 0 255.255.255.255 95.83.176.3 — 2 99 dad
15 0 0 14 28 255.255.255.255 123.193.132.219 — 2 99 dad
16 0 0 14 32 255.255.255.255 188.191.192.157 — 2 99 dad
17 0 0 10 0 255.255.255.255 95.83.132.169 — 2 99 dad
18 0 0 24 33 255.255.255.255 99.244.128.223 — 2 99 dad
19 0 0 17 53 255.255.255.255 97.90.124.181 — 2 99 dad

A bit more on reading this sniffer output…

Notice columns 4 and 5, which indicate data transfer rates in bytes per second. These columns contain numbers that are less than 100 bytes per second – Very small data transfers. This is mostly because as soon as that connection is identified as utorrent, the NetGladiator drops all future packets on the connection and it never really gets going. One thing I did notice is that the modern utorrent protocol hops around very quickly from connection to connection. It attempts not to show it’s cards. Why do I mention this? Because in layer 7 shaping of P2P, speed of detection is everything. If you wait a few milliseconds too long to analyze and detect a torrent, it is already too late because the torrent has transferred enough data to keep it going. It’s just a conjecture, but I suspect this is one of the main reasons why this utorrent is so popular. By hopping from source to source, it is very hard for an ISP to block this one without the latest equipment. I recently wrote a companion article regarding the speed of the technology behind a good layer 7 device.

The last part of testing a regular expression involves looking for false positives. For this we use a commercial grade simulator. Our simulator uses a series of pre-programmed web crawlers that visit tens of thousands of web pages an hour at our test facility. We then take our layer 7 device with our new regular expression and make sure that none of the web crawlers accidentally get blocked while reading thousands of web pages. If this test passes we are good to go with our new regular expression.

Editors Note: Our primary bandwidth shaping product manages P2P without using deep packet inspection.
The following layer 7 techniques can be run on our NetGladiator Intrusion Prevention System. We also advise that public ISPs check their country regulations before deploying a deep packet inspection device on a public network.

Posted in Bandwidth Management, Educational Articles, Net Neutrality, Research, Security, White Papers. Tags: , , , , . Leave a Comment »

NetGladiator: A Layer 7 Shaper in Sheep’s Clothing

June 28, 2012 — netequalizer

When explaining our NetGladiator technology the other day, a customer was very intrigued with our Layer 7 engine. He likened it to a caged tiger under the hood, gobbling up and spitting out data packets with the speed and cunning of the world’s most powerful feline.

He was surprised to see this level of capability in equipment offered at our prices.  He was impressed with the speed attained for the price point of our solution (more on this later in the article)…

In order to create a rock-solid IPS (Intrusion Prevention System), capable of handling network speeds of up to 1 gigabit with standard Intel hardware, we had to devise a technology breakthrough in Layer 7 processing. Existing technologies were just too slow to keep up with network speed expectations.

In order to support higher speeds, most vendors use semi-custom chip sets and a technology called “ASIC“. This works well but is very expensive to manufacture.

How do typical Layer 7 engines work?

Our IPS story starts with our old Layer 7 engine. It was sitting idle on our NetEqualizer product. We had shelved it when we got away from from Layer 7 shaping in favor of Equalizing technology, which is a superior solution for traffic shaping.  However, when we decided to move ahead with our new IPS this year, we realized we needed a fast-class analysis engine, one that could look at all data packets in real time. Our existing Layer 7 shaper only analyzed headers because that was adequate for its previous mission (detecting P2P streams).  For our new IPS system, we needed a solution that could do a deep dive into the data packets.  The IPS mission requires that you look at all the data – every packet crossing into a customer network.

The first step was to revamp the older engine and configure it to look at every packet. The results were disappointing.  With the load of analyzing every packet, we could not get throughput any higher than about 20 megabits, far short of our goal of 1 gigabit.

What do we do differently with our updated Layer 7 engine?

Necessity is the mother of invention, and so we invented a better Layer 7 engine.

The key was to take advantage of multiple processors for analysis of data without delaying data packets. The way the old technology worked was that it would intercept a data packet on a data link, hold it, analyze it for P2P patterns, and then send it on.  With this method, as packets come faster and faster you end up not having enough CPU time to do the analysis and still send the packet on without adding latency.  Many customers find this out the hard way when they update their data speeds from older slower T1 technology.  Typical analysis engines on affordable routers and firewalls often just can’t keep up with line speeds.

What we did was take advantage of a utility in the Linux Kernel called “clone skb”.  This allows you to make a temporary copy of the data packet without the overhead of copying.  More importantly, it allows us to send the packet on without delay and do the analysis within a millisecond (not quite line speed, but fast enough to stop an intruder).

We then combined the cloning with a new technology in the Linux kernel called Kernel Threading.  This is different than the technology that large multi-threaded HTTP servers use because it happens at the kernel level, and we do not have to copy the packet up to some higher-level server for analysis. Copying a packet for analysis is a huge bottleneck and very time-consuming.

What were our Results?

With kernel threading, cloning, and a high-end Intel SMP processor, we can make use of 16 CPU’s doing packet analysis at the same time and we now have attained speeds close to our 1 gigabit target.

When we developed our bandwidth shaping technology in 2003/2004, we leveraged technology innovation to create a superior bandwidth control appliance (read our NetEqualizer Story).  With the NetGladiator IPS, we have once again leveraged technology innovation to enable us to provide an intrusion prevention system at a very compelling price (register to get our price list), hence our customer’s remark about great speed for the price.

What other benefits does our low cost, high-speed layer 7 engine allow for? Is it just for IPS?

The sky is the limit here.  Any type of pattern you want to look at in real-time can now be done at one tenth (1/10th) the cost of the ASIC class of shapers.  Although we are not a fan of unauthorized intrusion into private data of the public Internet (we support Net Neutrality), there are hundreds of other uses which can be configured with our engine.

Some that we might consider in the future include:

- Spam filtering
- Unwanted protocols in your business
- Content blocking
- Keyword spotting

If you are interested in testing and experimenting in any of these areas with our raw technology, feel free to contact us ips@netgladiator.net.

Posted in Bandwidth Monitoring, Security, Topics. Tags: , , , , . 8 Comments »

A Smarter Way to Limit P2P Traffic

June 20, 2012 — netequalizer

 

By Art Reisman

Art Reisman CTO www.netequalizer.com

Editor’s note: Art Reisman is the CTO of APconnections. APconnections designs and manufactures the popular NetEqualizer bandwidth shaper.

 

If you are an IT professional interested in the ethical treatment of P2P (which we define as keeping it in check without invading the privacy of your customers by looking at their private data), you’ll appreciate our next generation approach to containing P2P usage. Thanks to some key input by a leading-edge ISP in South Africa, we have developed a next-generation P2P control that balances the resources of an ISP, and yet allows their end customers to use Bittorent without bringing down the network.

First a quick review of how P2P affects a network

A signature of a typical P2P user is that they can open hundreds of small connections while downloading files. A P2P client, such as Kazaa, is designed to find as many sources to a file as possible. For efficiency and speed, P2P clients operate as multi-threaded download engines, where each download stream captures a different segment of the requested file. When all the segments are complete they are re-assembled into a complete usable media file on your hard drive. The multiple downloads cause a strain on network bandwidth resources. They also create extreme overhead on wireless routers. Extreme P2P usage by just a subset of users can crowd out web pages, VoIP, YouTube and many other less aggressive applications.

Current P2P Limiting Solution: Connection Limits

Our current generation of P2P control involves intelligently looking at the number of connections generated from a user on your network. Based on the persistence and number of connections, we can reliably tell if a user is currently using P2P. The current P2P remedy, deployed on our NetEqualizer equipment, involves limiting the number of connections of suspected P2P users; this works well to limit p2p usage.  Thus, it keeps the P2P users from overwhelming a shared network.

Next-Generation P2P Limiting: Smart Connection Limits

While we have retained the connection-limiting aspects of our current P2P limiting technology, our new technology goes a step further. With Smart Connection Limits, limiting is done by also slowly starving the P2P connections for bandwidth. The bandwidth reduction is based on a formula which takes into a account two main factors:

1) the number of connections a user has open.
2) the load on the network.

I like to think of this technology as more of a “reward system”, resulting in a higher quality of service for non-P2P users.  In this case, the reward is that non-P2P users’ connections are not experiencing this reduction in bandwidth (although they may get equalized on any connection that is hogging bandwidth).  P2P users will slowly see less bandwidth allocated to their P2P traffic, which should discourage them from using P2P on your network.  Basically, this helps to train them to use better behavior – sharing the network resource more fairly with others.

This philosophyof fairness is aligned with the primary goal of the NetEqualizer – to ensure fairness for all network users. It follows that if a user has 20 concurrent streams and another user only has 5, to ensure equal  use of bandwidth under network load, the user with 20 streams should have his streams operate at 1/4 the speed of the user that has 5. While you may configure Smart Connection Limits at various levels, you could enforce the example indicated above.

The reason this technology is important is that, on a network pressed for bandwidth, the P2P users are often taking an unfair share. Even with basic rate caps per user in place, you often must augment that restriction by limiting the total number of connections per user. And now with our latest technology, we also temporarily restrict the bandwidth per connection (only applied to the P2P users).

If you are interested in learning more about Smart Connection Limits, to see if they are a fit for your network, contact us.

Some common questions and answers:

Is it possible to completely block P2P?

It is never safe to try to completely block p2p for a couple of reasons.

1) Although it is always possible to identify P2P, it is often expensive and not foolproof. To block it based on hearsay will cause problems. Our solution, although targeted on limiting P2P, focuses on the resource footprint of the P2P user, and does not attempt to outright block types of traffic. In other words, whether or not the traffic is actually P2P is not the issue. The issue is, is this user abusing resources? If yes, they get punished.

2) Devices that attempt to identify P2P traffic often use a technique called deep packet inspection (DPI), which is frowned upon as an invasion of privacy.  Additionally, we are finding that the latest P2P tools (such as utorrent) encrypt P2P streams as their default behavior, which defeats deep packet inspection.  Not so with our solutions; both Connection Limits and Smart Connection Limits will throttle encrypted P2P traffic.

Who do we recommend move from Connection Limits to Smart Connection Limits ?

If you are in a business where you charge for bandwidth usage (ISP, WISP, satellite provider), you should consider implementing Smart Connection Limits.  We also recommend looking at Smart Connection Limits if you have repeat offenders – basically, the same users are consistently running P2P traffic on your network and you want to change their behavior.

Can I continue using the Connection Limits or do I need to move to Smart Connection Limits?

Both solutions to Limit P2P traffic are being supported. If you do not have a lot of P2P traffic on your network, you may opt to stay with Connection Limits, as a quick-and-easy implementation. Smart Connection Limits take a little more thought to implement and have additional complexity, which you may not wish to take on at this point.

Posted in Bandwidth Management. Tags: , , , . 4 Comments »

NetEqualizer News: October 2011

October 12, 2011 — netequalizer

NetEqualizer News

October 2011

Greetings!

Enjoy another issue of NetEqualizer News! This month, we present a video demonstration detailing how active connections behave on a live network. The video utilizes a real-time reporting tool that you can leverage with your own NetEqualizer data! We also preview some new features coming this fall (IPv6 Visibility and ToS Priority), announce our FlyAway Contest winner, and discuss P2P blocking! As always, feel free to pass this along to others who might be interested in NetEqualizer News.

Our Website     Contact Us      NetEqualizer Demo      Price List      Join Our Mailing List

In This Issue:

:: Demo: How Active Connections Behave in Real Time

:: And The Fly Away Contest Winner Is…

:: Update on New Features Coming This Fall

:: Best Of The Blog

Demo: How Active Connections Behave in Real Time

We often get asked about active connections and how they are handled by the NetEqualizer. The answer to this question is fundamental to how equalizing and behavior-based bandwidth shaping works.

In early August, we posted an article on our blog that discussed how you could generate real-time reports using Excel and your NetEqualizer data. The video linked to below references that project, and uses it to demonstrate how active connections behave in real-time on a live network.

There are some interesting observations you can take away from this video, even if you don’t implement the reporting tool on your own device. You will come away from it with a better understanding of how users are connected through your network, and what types of connections are occurring every second.

Click the image below to view the video:

Some key points from the video are:

  • For every user, there are many connections occurring that most people are probably not aware of. The OS might be checking for updates, A/V could be checking for new signatures, an email program is reloading its inbox, etc.
  • Most connections have a very short life, and they are also mostly very small. 90% of connections will only utilize 10 to 1000 bytes/second.
  • Flows change dynamically. Even for a single user, 2 to 20 connections (or more) can exist at any moment in time.
  • Contention can occur quickly. Because of the variability in connections (especially with a broad user base), network contention can occur quickly. If large downloads are part of the active connections, this contention happens even faster.
  • The NetEqualizer instantly responds to this problem by taking a Robin Hood approach to the hogging connections. It shaves off bandwidth from the large connections and gives that much-needed resource to the thousands of other connections that require it.

View the blog article referenced in the video above here:
Dynamic Reporting With The NetEqualizer.

And The FlyAway Contest Winner Is…

frontier airlinesEvery few months, we have a drawing to give away two roundtrip domestic airline tickets from Frontier Airlines to one lucky person who’s recently tried out our online NetEqualizer demo.
The time has come to announce this round’s winner.
And the winner is…Mohammed O. Ibrahim of Zanzibar Connections.  Congratulations, Mohammed!
Please contact us within 30 days (by November 10th, 2011) at: email admin -or- 303-997-1300 to claim your prize.

Update on New Features
Coming This Fall!

We are very excited about the new features coming in our Fall 2011 Software Update!

IPv6 Visibility

As we await the need to handle significant amounts of IPv6 traffic, NetEqualizer is already implementing solutions to meet the shift head-on. The Fall 2011 Software Update will include features that will provide enhanced visibility to IPv6 traffic.

This feature will help our customers that are experimenting with IPv6/IPv4 dual stacks, as they start to see IPv6 Internet traffic on their networks.

The enhanced IPv6 capabilities that we are implementing in the NetEqualizer this Fall include:

  • Providing you with visibility to current IPv6 connections so that you to determine if you need to start shaping IPv6 traffic.
  • Logging the IPv6 traffic so that you can obtain a historical snapshot to help in your IPv6 planning efforts.

ToS Priority

We are now seeing an influx of customers looking to provide priority bandwidth to VoIP connections on their links without all the hassle of complex router rules. NetEqualizer’s new Type of Service (ToS) Priority feature is the solution. Included in the Fall 2011 Software Update, the ToS Priority feature will automatically prioritize connections that are utilizing services like VoIP – as well as a host of other types of important connections. This will provide improved quality of service (QoS) on your network.

Larger SSD Drives

We will now be shipping with larger SSD drives to customers waiting to try our NetEqualizer Caching Option (NCO).

As always, the Fall 2011 Software Update will be available at no charge to customers with valid NetEqualizer Software Subscriptions (NSS).

For more information on the NetEqualizer or the upcoming release, visit our blog or contact us at: email sales -or- toll-free U.S.(800-918-2763), worldwide (303) 997-1300 x. 103.

Best of the Blog

How Effective is P2P Blocking?
by Art Reisman – CTO – NetEqualizer

This past week, a discussion about peer-to-peer (P2P) blocking tools came up in a user group that I follow. In the course of the discussion, different IT administrators chimed in, citing their favorite tools for blocking P2P traffic.

At some point in the discussion, somebody posed the question, “How do you know your peer-to-peer tool is being effective?” For the next several hours the room went eerily silent.

The reason why this question was so intriguing to me is that for years I collaborated with various developers on creating an open-source P2P blocking tool using layer 7 technology (the Application Layer of the OSI Model). During this time period, we released several iterations of our technology as freeware. Our testing and trials showed some successes, but we also learned how fragile the technology was and we were reluctant to push it out commercially.

To keep reading, click here.

Photo Of The Month

NetEqualizer CF Card

New Design!

As of August 10th, 2011, our Compact Flash Cards are being shipped with a new label design and card case!

View our videos on YouTube
Posted in Newsletters. Tags: , , , . Leave a Comment »

How Effective is P2P Blocking?

October 3, 2011 — netequalizer

This past week, a discussion about peer-to-peer (P2P) blocking tools came up in a user group that I follow. In the course of the discussion, different IT administrators chimed in, citing their favorite tools for blocking P2P traffic.

At some point in the discussion, somebody posed the question, “How do you know your peer-to-peer tool is being effective?” For the next several hours the room went eerily silent.

The reason why this question was so intriguing to me is that for years I collaborated with various developers on creating an open-source P2P blocking tool using layer 7 technology (the Application Layer of the OSI Model). During this time period, we released several iterations of our technology as freeware. Our testing and trials showed some successes, but we also learned how fragile the technology was and we were reluctant to push it out commercially. I had always wondered if other privately-distributed layer 7 blocking tools had found some magic key to perfection?

Sometimes, written words can be taken as fact even though the same spoken words might be dismissed as gossip; and so it was with our published open source technology. We started getting indications that it was getting picked up and integrated in other solutions and touted as gospel.

Our experience with P2P blocking:

Our free P2P blocking tool worked most of the time – maybe eighty percent. Eighty percent accuracy is fine for an experimental open-source tool. Intuitively, a blocking tool is expected to be 99.9 percent effective. Even though most customers would likely not conclusively measure our accuracy, eighty percent was too low to ethically sell this technology without disclosures.

The on-line discussion ended fairly quickly when the question of accuracy was brought up, and I think it is safe to assume the silence is an indication that nobody else was achieving better than eighty percent.

How do you validate the effectiveness of a P2P tool?

1) Brute force testing:

I am not aware of too many IT administrators that have the time to load up six or seven different P2P clients on their laptops, and download bootlegged Madonna videos all day.

In testing P2P clients, we infected several computers with just about every virus in circulation. Over time, you can get a rough idea of how deep you must go to expose weaknesses in your tool set. To be thorough, you can’t stop at the first P2P client tool. In the real world, users on your network will likely search for multiple P2P clients, especially if the first one fails. Once they find a kink in the armor, they will yap to others, exposing your Achilles heel.

2) Reduction of RIAA requests:

Most small-to-medium ISP’s don’t really think about P2P unless they get RIAA requests or their network is saturated.

RIAA requests seem to be a big motivator in purchasing technology to block P2P. If you are getting RIAA requests (these are letters from lawyers threatening to sue you for copyright infringement), you can install your P2P blocking tool, and if in the next week your notifications of copyright violations are way down, you can assume that you have put a good dent in your P2P downloading issue.

3) Reduced congestion:

Plug your P2P tool in and see if your network utilization drops.

4) Lower connection rates through your router:

One of the signatures of P2P is that clients will open up hundreds of connections per minute to P2P servers in order to download content. There are ways to measure and quantify these connection rates empirically.

Other observations:

Many times we’ll hear from an ISP/operator claiming they have P2P users run amok on their network, however analysis often shows most of their traffic is video – Netflix, YouTube, Hulu, etc.

Total P2P traffic has really dropped off quite a bit in the last three or four years. We attribute this decline to:

1) Legal iTunes. 99 cent songs have eliminated the need for pirated music.

2) RIAA enforcement and education of copyright laws.

3) The invention of the iPad and iPhone. These devices control the applications which run on them (they are not going to distribute P2P clients as readily).

One method to handle P2P problems is to control all the computers in your environment, scan them before granting network access, and then block access to P2P sites (the sites where the client utilities are loaded from).

Note: once a P2P client is loaded on a computer you cannot block any single remote site, as the essence of P2P is that the content is not centralized.

Summary:

Results of different P2P blocking techniques are often temporary, especially when you have an aggressive user base with motivation to download free content.

Posted in Bandwidth Management. Tags: , , . 3 Comments »

NetEqualizer P2P Locator Technology

May 21, 2011 — netequalizer

Editor’s Note: The NetEqualizer has always been able to thwart P2P behavior on a network. However, our new utility can now pinpoint an individual P2P user or gamer without any controversial layer-7 packet inspection. This is an extremely important step from a privacy point of view as we can actually spot P2P users without looking at any private data.

A couple of months ago, I was doing a basic health check on a customer’s heavily used residential network. In the process, I instructed the NetEqualizer to take a few live snapshots. I then used the network data to do some filtering with custom software scripts. Within just a few minutes, I was able to inform the administrator that eight users on his network were doing some heavy P2P, and one in particular looked to be hosting a gaming session. This was news to the customer, as his previous tools didn’t provide that kind of detail.

A few days later, I decided to formally write up my notes and techniques for monitoring a live system to share on the blog. But, as I got started, another lightbulb went on…in the end, many customers just want to know the basics — who is using P2P, hosting game servers, etc. They don’t always have the time to follow a manual diagnostic recipe.

So, with this in mind, instead of writing up the manual notes, I spent the next few weeks automating and testing an intelligent utility to provide this information. The utility is now available with NetEqualizer 5.0.

The utility provides: 

  • A list of users that are suspected of using P2P
  • A list of users that are likely hosting gaming servers
  • A confidence rating for each user (from high to low)
  • The option of tracking users by IP and MAC address

The key to determining a user’s behavior is the analysis of the fluctuations in their connection counts and total number of connections. We take snapshots over a few seconds, and like a good detective, we’ve learned how to differentiate P2P use from gaming, Web browsing and even video. We can do this without using any deep packet inspection. It’s all based on human-factor heuristics and years of practice.

Enclosed is a screen shot of the new P2P Locator, available under our Reports & Graphing menu.

Our new P2P Locator technology

Contact us to learn more about the NetEqualizer P2P Locator Technology or NetEqualizer 5.0. For more information about ongoing changes and challenges with BitTorrent and P2P, see Ars Technica’s ”BitTorrent Has New Plan to Shape Up P2P Behavior.”

Posted in Bandwidth Monitoring, New Features. Tags: , , . 3 Comments »

Looking To Block P2P?

May 27, 2010 — netequalizer

Article reprinted from Slyck.com

By Thomas Mennecke

Let’s face it. P2P networking takes up a considerable amount of bandwidth. Whether you are a network administrator for a college LAN (Local Area Network) or an ISP, some estimates place P2P consumption (especially BitTorrent) at 60%. However, figuring out the most practical solution for various administrators varies. Should you outright ban P2P traffic? Incorporate bandwidth throttles? NetEqualizer lets you choose from these options and much more.

ISPs are in a more precarious situation than say a college LAN administrator. If P2P traffic begins to saturate a college network, the LAN administrator does not have to worry all that much if the decision is made to filter or block file-sharing traffic. Perhaps some people may complain, but the loss of revenue is not a concern.

ISPs on the other hand must take this into heavy consideration. P2P traffic consumes an enormous amount of bandwidth compared to the amount of individuals that use it. For example, CacheLogic, a P2P measuring and network solutions firm, states P2P traffic can consume a majority of the ISPs bandwidth, easily blowing away HTTP. Comparatively, only a relative few individuals actually utilize such high consumption protocols.

So here is the tricky part. ISPs know that P2P has helped fuel the broadband revolution. While not everyone uses BitTorrent; eDonkey2000, FastTrack, Gnutella, etc. are very popular. Block P2P users, an the ISP might face a significant backlash. Throttle their bandwidth, and the ISP might have similar results.

One of the more compromising solutions has been offered from CacheLogic, which aims to make everyone happy. CacheLogic’s function is to “cache” or store common P2P files based on the frequency of search queries. Instead of P2P traffic bogging down and ISPs network, it simple searches the cache server. P2P fans are left to enjoy their file-sharing bliss and web surfers can happily surf the World Wide Web.

However, say you are not interested in making the P2P crowd happy, and catering to web surfers is the priority. Say you want to throttle or block P2P traffic completely…then APConnection’s Net Equalizer comes into play.

According to a press release issued by APConnection today, their product “Net Equalizer” will now be distributed on a worldwide scale. Net Equalizer aims to give priority to web based traffic, while throttling back those who utilized P2P software. When file-sharing traffic begins to slow down those surfing the web, its “fairness” algorithing kicks. For more information on Net Equalizer, read the FAQ here.

“The recently signed distributors have selected NetEqualizer primarily for its ability to deliver automated bandwidth control. Other features that have driven adoption include the enhancement of security offerings with the ability to block and control p2p traffic and unique quality of service (QoS) capabilities that enable distributors to include NetEqualizer as part of a service provider VoIP package.”

NetEqualizer is a stark contrast to CacheLogic, which aims to compromise rather than block or throttle P2P traffic. Regardless, NetEqaulizer’s solution is straight forward and offers and immediate solution to a network that is bogged down with P2P traffic. However, as file-sharing and P2P traffic becomes more mainstream, consumers may take into consideration whether an ISP uses NetEqualizer or CacheLogic as a network management solution.

Posted in Bandwidth Management, News. Tags: , . Leave a Comment »

Behind the Scenes on the latest Comcast Ruling on Net Neutrality

April 7, 2010 — netequalizer

Yesterday the FCC ruled in favor of Comcast regarding their rights to manipulate consumer traffic . As usual, the news coverage was a bit oversimplified and generic. Below we present a breakdown of the players involved, and our educated opinion as to their motivations.

1) The Large Service Providers for Internet Service: Comcast, Time Warner, Quest

From the perspective of Large Service Providers, these companies all want to get a return on their investment, charging the most money the market will tolerate. They will also try to increase market share by consolidating provider choices in local markets. Since they are directly visible to the public, they will also be trying to serve the public’s interest at heart; for without popular support, they will get regulated into oblivion. Case in point, the original Comcast problems stemmed from angry consumers after learning their p2p downloads were being redirected and/or  blocked.

Any and all government regulation will be opposed at every turn, as it is generally not good for private business. In the face of a strong headwind, don’t be surprised if Large Service Providers might try to reach a compromise quickly to alleviate any uncertainty.  Uncertainty can be more costly than regulation.

To be fair, Large Service Providers are staffed top to bottom with honest, hard-working people but, their decision-making as an entity will ultimately be based on profit.  To be the most profitable they will want to prevent third-party Traditional Content Providers from flooding  their networks with videos.  That was the original reason why Comcast thwarted bittorrent traffic. All of the Large Service Providers are currently, or plotting  to be, content providers, and hence they have two motives to restrict unwanted traffic. Motive one, is to keep their capacities in line with their capabilities for all generic traffic. Motive two, would be to thwart other content providers, thus making their content more attractive. For example who’s movie service are you going to subscribe with?  A generic cloud provider such as Netflix whose movies run choppy or your local provider with better quality by design?

2) The Traditional Content Providers:  Google, YouTube, Netflix etc.

They have a vested interest in expanding their reach by providing expanded video content.  Google, with nowhere to go for new revenue in the search engine and advertising business, will be attempting  an end-run around Large Service Providers to take market share.   The only thing standing in their way is the shortcomings in the delivery mechanism. They have even gone so far as to build out an extensive, heavily subsidized, fiber test network of their own.  Much of the hubbub about Net Neutrality is  based on a market play to force Large Service Providers to shoulder the Traditional Content Providers’ delivery costs.  An analogy from the bird world would be the brown-headed cowbird, where the mother lays her eggs in another bird’s nest, and then lets her chicks be raised by an unknowing other species.  Without their own delivery mechanism direct-to-the-consumer, the Traditional Content Providers  must keep pounding at the FCC  for rulings in their favor.  Part of the strategy is to rile consumers against the Large Service Providers, with the Net Neutrality cry.

3) The FCC

The FCC is a government organization trying to take their existing powers, which were granted for airwaves, and extend them to the Internet. As with any regulatory body, things start out well-intentioned, protection of consumers etc., but then quickly they become self-absorbed with their mission.  The original reason for the FCC was that the public airways for television and radio have limited frequencies for broadcasts. You can’t make a bigger pipe than what frequencies will allow, and hence it made sense to have a regulatory body oversee this vital  resource. In  the early stages of commercial radio, there was a real issue of competing entities  broadcasting  over each other in an arms race for the most powerful signal.  Along those lines, the regulatory entity (FCC) has forever expanded their mission.  For example, the government deciding what words can be uttered on primetime is an extension of this power.

Now with Internet, the FCC’s goal will be to regulate whatever they can, slowly creating rules for the “good of the people”. Will these rules be for the better?  Most likely the net effect is no; left alone the Internet was fine, but agencies will be agencies.

4) The Administration and current Congress

The current Administration has touted their support of Net Neutrality, and perhaps have been so overburdened with the battle on health care and other pressing matters that there has not been any regulation passed.  In the face of the aftermath of the FCC getting slapped down in court to limit their current powers, I would not be surprised to see a round of legislation on this issue to regulate Large Service Providers in the near future.  The Administraton will be painted as consumer protection against big greedy companies that need to be reigned in, as we have seen with banks, insurance companies, etc…. I hope that we do not end up with an Internet Czar, but some regulation is inevitable, if nothing else for a revenue stream to tap into.

5) The Public

The Public will be the dupes in all of this, ignorant voting blocks lobbied by various scare tactics.   The big demographic difference on swaying this opinion will be much different from the health care lobby.  People concerned for and against Internet Regulation will be in income brackets that have a higher education and employment rate than the typical entitlement lobbies that support regulation.  It is certainly not going to be the AARP or a Union Lobbyist leading the charge to regulate the Internet; hence legislation may be a bit delayed.

6) Al Gore

Not sure if he has a dog in this fight; we just threw him in here for fun.

7) NetEqualizer

Honestly, bandwidth control will always be needed, as long as there is more demand for bandwidth than there is bandwidth available.  We will not be lobbying for or against Net Neutrality.

8) The Courts

This is an area where I am a bit weak in understanding how a Court will follow legal precedent.  However, it seems to me that almost any court can rule from the bench, by finding the precedent they want and ignoring others if they so choose?  Ultimately, Congress can pass new laws to regulate just about anything with impunity.  There is no constitutional protection regarding Internet access.  Most likely the FCC will be the agency carrying out enforcement once the laws are in place.

Posted in Commentary and Editorials, Net Neutrality, News. Tags: , , , , , . Leave a Comment »

APconnections Announces 50-Percent-Off Sale of New NetEqualizer-Lite

May 26, 2009 — netequalizer

Beginning May 26, all customers purchasing a full size NetEqualizer 2000/3000 model will qualify for a 50-percent discount on the NetEqualizer-Lite. In addition, the offer will be extended to all existing NetEqualizer users who will also be entitled to the 50-percent discount on their first NetEqualizer-Lite purchase. This offer is valid until June 30, 2009. Limit two per customer.

As well as offering users the same services available through previously released NetEqualizer models, the NetEqualizer-Lite is Power-over-Ethernet (PoE), handling up to 10 megabits of traffic and 200 users. Furthermore, the NetEqualizer-Lite also serves to solve hidden node issues without customers having to change their existing access points.*

Although the core technology behind the NetEqualizer has not changed, with the latest release price point, many ISPs and businesses are deploying the NetEqualizer-Lite closer to end users, often directly behind congested access points.

After just over a month in the field, NetEqualizer-Lite users are reporting they can now easily increase Internet subscribers by 30 to 50 percent at once congested towers and AP sites. For example, a customer with an 802.11b radio now has 100 subscribers on his network and is still running smoothly. In the past, this customer’s norm for saturation stood at roughly 20 users, but he is now enjoying a 500-percent increase after installing the NetEqualizer-Lite. This is translating into both higher revenues and a more satisfied customer base.

The NetEqualizer-Lite lists at $1499. In addition to the 50-percent discount, we are also currently offering volume discounts. Pricing information on all other NetEqualizer models is available online at http://www.netequalizer.com. For more information, please contact APconnections at 1-800-918-2763 or admin@apconnections.net.

*Hidden nodes are a problem frequently encountered by commercial wireless operators that has previously been solved using APconnections’ AirEqualizer technology. The NetEqualizer-Lite’s capability to offer similar solutions is simply one of the multiple benefits of the technology for administrators of networks of many different types and sizes.

Posted in Press Releases. Tags: , , , , , , , , , , , , , , . Leave a Comment »

NetEqualizer-Lite Revolutionizing WISP Performance

May 21, 2009 — netequalizer

After just over a month in the field, NetEqualizer-Lite users are reporting they can now easily increase Internet subscribers by 30 to 50 percent at once congested towers and access point (AP) sites. For example, a customer with an 802.11 B radio now has 100 subscribers on his network and is still running smoothly. In the past, this customer’s norm for saturation stood at roughly 20 users, but he is now enjoying a 500-percent increase after installing the NetEqualizer-Lite. This is translating into both higher revenues and a more satisfied customer base.

Although the core technology behind the NetEqualizer has not changed, with the latest release price point, many users are deploying the NetEqualizer-Lite closer to customers or just behind their congested wireless access points. Customer satisfaction with the new release has been consistent across the board, with users voicing their reviews to us directly as well as online. One user on DSLReports.com commented:

“The Netequalizer has resulted in dramatically improved service to our customers….Bottom line to this is that we can deliver significantly more data through the same AP. The customers hitting web pages, checking e-mail, etc. virtually always see full bandwidth, and the hogs don’t impact these customers. Even the hogs see better performance” (dslreports.com).

In addition to offering users the same services available through previously released NetEqualizer models, the NetEqualizer-Lite is Power-over-Ethernet (PoE), handling up to 10 megabits of traffic and 200 users. Furthermore, the NetEqualizer-Lite also serves to solve hidden node issues without customers having to change their existing APs.*

The NetEqualizer-Lite lists at $1499, but we are currently offering volume discounts. Please contact us for more information at 1-800-918-2763 or admin@apconnections.net.

*Hidden nodes are a problem frequently encountered by commercial wireless operators that has previously been solved using APconnections’ AirEqualizer technology. The NetEqualizer-Lite’s capability to offer similar solutions is simply one of the multiple benefits of the technology for administrators of networks of many different types and sizes.
Posted in Press Releases, Testimonials. Tags: , , , , , , , , , , . Leave a Comment »

NetEqualizer-Lite Is Now Available!

May 11, 2009 — netequalizer

Last month, we introduced our newest release, a Power-over-Ethernet NetEqualizer. Since then, with your help, we’ve titled the new release the NetEqualizer-Lite and are already getting positive feedback from users. Here’s a little background about what led us to release the NetEqualizer-Lite…Over the years, we’d had several customers express interest in placing a NetEqualizer as close as possible to their towers in order to relieve congestion. However, in many cases, this would require both a weatherproof and low-power NetEqualizer unit – two features that were not available up to this point. However, in the midst of a growing demand for this type of technology, we spent the last few months working to meet this need and thus developed the NetEqualizer-Lite.

Here’s what you can expect from the NetEqualizer-Lite:

  • Power over Ethernet
  • Up to 10 megabits of shaping
  • Up to 200 users
  • Comes complete with all standard NetEqualizer features

And, early feedback on the new release has been positive. Here’s what one user recently posted on DSLReports.com:

We’ve ordered 4 of these and deployed 2 so far. They work exactly like the 1U rackmount NE2000 that we have in our NOC, only the form factor is much smaller (about 6x6x1) and they use POE or a DC power supply. I amp clamped one of the units, and it draws about 7 watts….The Netequalizer has resulted in dramatically improved service to our customers. Most of the time, our customers are seeing their full bandwidth. The only time they don’t see it now is when they’re downloading big files. And, when they don’t see full performance, its only for the brief period that the AP is approaching saturation. The available bandwidth is re-evaulated every 2 seconds, so the throttling periods are often brief. Bottom line to this is that we can deliver significantly more data through the same AP. The customers hitting web pages, checking e-mail, etc. virtually always see full bandwidth, and the hogs don’t impact these customers. Even the hogs see better performance (although that wasn’t one of my priorities). (DSLReports.com)

Pricing for the new model will be $1,200 for existing NetEqualizer users and $1,550 for non-customers purchasing their first unit. However, the price for subsequent units will be $1,200 for users and nonusers alike.

For more information about the new release, contact us at admin@apconnections.net or 1-800-918-2763.

Posted in New Features. Tags: , , , , , , , , , , , , . 1 Comment »

Speeding up Your T1, DS3, or Cable Internet Connection with an Optimizing Appliance

April 30, 2009 — netequalizer

By Art Reisman, CTO, APconnections (www.netequalizer.com)

Whether you are a home user or a large multinational corporation, you likely want to get the most out of your Internet connection. In previous articles, we have  briefly covered using Equalizing (Fairness)  as a tool to speed up your connection without purchasing additional bandwidth. In the following sections, we’ll break down  exactly how this is accomplished in layman’s terms.

First , what is an optimizing appliance?

An optimizing appliance is a piece of networking equipment that has one Ethernet input and one Ethernet output. It is normally located between the router that terminates your Internet connection and the users on your network. From this location, all Internet traffic must pass through the device. When activated, the optimizing appliance can rearrange traffic loads for optimal service, thus preventing the need for costly new bandwidth upgrades.

Next, we’ll summarize equalizing and behavior-based shaping.

Overall, equalizing is a simple concept. It is the art form of looking at the usage patterns on the network, and when things get congested, robbing from the rich to give to the poor. In other words, heavy users are limited in the amount of badwidth to which they have access in order to ensure that ALL users on the network can utilize the network effectively. Rather than writing hundreds of rules to specify allocations to specific traffic as in traditional application shaping, you can simply assume that large downloads are bad, short quick traffic is good, and be done with it.

How is Fairness implemented?

If you have multiple users sharing your Internet trunk and somebody mentions “fairness,” it probably conjures up the image of each user waiting in line for their turn. And while a device that enforces fairness in this way would certainly be better than doing nothing, Equalizing goes a few steps further than this.

We don’t just divide the bandwidth equally like a “brain dead” controller. Equalizing is a system of dynamic priorities that reward smaller users at the expense of heavy users. It is very very dynamic, and there is no pre-set limit on any user. In fact, the NetEqualizer does not keep track of users at all. Instead, we monitor user streams. So, a user may be getting one stream (FTP Download) slowed down while at the same time having another stream untouched(e-mail).

Another key element in behavior-based shaping is connections. Equalizing takes care of instances of congestion caused by single-source bandwidth hogs. However, the other main cause of Internet gridlock (as well as bringing down routers and access points) is p2p and its propensity to open hundreds or perhaps thousands of connections to different sources on the Internet. Over the years, the NetEqualizer engineers have developed very specific algorithms to spot connection abuse and avert its side effects.

What is the result?

The end result is that applications such as Web surfing, IM, short downloads, and voice all naturally receive higher priority, while large downloads and p2p receive lower priority. Also, situations where we cut back large streams is  generally for a short duration. As an added advantage, this behavior-based shaping does not need to be updated constantly as applications change.

Trusting a heuristic solution such as NetEqualizer is not always an easy step. Oftentimes, customers are concerned with accidentally throttling important traffic that might not fit the NetEqualizer model, such as video. Although there are exceptions, it is rare for the network operator not to know about these potential issues in advance, and there are generally relatively few to consider. In fact, the only exception that we run into is video, and the NetEqualizer has a low level routine that easily allows you to give overriding priority to a specific server on your network, hence solving the problem. The NetEqualizer also has a special feature whereby you can exempt and give priority to any IP address specifically in the event that a large stream such as video must be given priority.

Through the implementation of Equalizing technology, network administrators are able to get the most out of their network. Users of the NetEqualizer are often surprised to find that their network problems were not a result of a lack of bandwidth, but rather a lack of bandwidth control.

See who else is using this technology.

Created by APconnections, the NetEqualizer is a plug-and-play bandwidth control and WAN/Internet optimization appliance that is flexible and scalable. When the network is congested, NetEqualizer’s unique “behavior shaping” technology dynamically and automatically gives priority to latency sensitive applications, such as VoIP and email. Click here for a full price list.

Posted in Bandwidth Management, Educational Articles, Network Speed. Tags: , , , , , , , , , , , , . 2 Comments »

APconnections Releases NetEqualizer for Small Business and WISP Market

April 13, 2009 — netequalizer 

LAFAYETTE, Colo., April 13 /PRNewswire/ -- APconnections (http://www.netequalizer.com),
a leading supplier of plug-and-play bandwidth shaping products,
today announced the release of its newest NetEqualizer model,
developed specifically with WISPs and small business users in mind.

This newest NetEqualizer release easily handles up to 10 megabits of traffic and up to 100 users, allowing room for expansion for growing demand. Furthermore, in addition to offering all standard NetEqualizer features, this smaller model will be Power over Ethernet, providing administrators greater flexibility in placing the unit within their network.

The model was developed to meet a growing demand both for an affordable traffic shaping device to help small businesses run VoIP concurrent with data traffic over their Internet link as well as a need for a shaping unit with PoE for the WISP market.

In a large wireless network, congestion often occurs at tower locations. However, with a low-cost PoE version of the NetEqualizer, wireless providers can now afford to have advanced bandwidth control at or near their access distribution points.

“About half of wireless network slowness comes from p2p (Bit Torrent) and video users overloading the access points,” said Joe D’Esopo, vice president of business development at APconnections. “We have had great success with our NE2000 series, but the price point of $2,500 was a bit too high to duplicate all over the network.”

For a small- or medium-sized office with a hosted VoIP PBX solution, the NetEqualizer is one of the few products on the market that can provide QoS for VoIP over an Internet link. And now, with volume pricing approaching $1,000, the NetEqualizer will help revolutionize the way offices use their Internet connection.

Pricing for the new model will be $1,200 for existing NetEqualizer users and $1,499 for non-customers purchasing their first unit. However, the price for subsequent units will be $1,200 for users and nonusers alike.

The NetEqualizer is a plug-and-play bandwidth control and WAN/Internet optimization appliance that is flexible and scalable. When the network is congested, NetEqualizer’s unique “behavior shaping” technology gives priority to latency sensitive applications, such as VoIP and email. It does it all dynamically and automatically, improving on other available bandwidth shaping technology. It controls network flow for the best WAN optimization.

APconnections is a privately held company founded in 2003 and is based in Lafayette, Colorado.

Full Article
Posted in Press Releases. Tags: , , , , , , , , , , . Leave a Comment »

Finally a Bandwidth Control appliance for under $1500

April 9, 2009 — netequalizer

Lafayette Colorado April 9th 2009

APconnections today Announced a small business bandwidth control device that  lists at $1499. (for single unit orders)

This new offer  handles up to 10 megabits and 100 users with room to spare for some expansion. It comes complete with all the standard features of the NetEqualizer, but in a smaller  low power format  with Power over Ethernet.

Demand for this new offer came from two sources

1) There was huge demand for an affordable traffic shaping device to  help small business run their VOIP concurrent with their data traffic over their internet link.

2) There was also a need  for a low end unit, with POE,  for the WISP market .

In  a large wireless network, congestion often occurs at tower locations.  With a low cost POE version of the NetEqualizer,  wireless providers can  now afford to have advanced bandwidth control at or near their Access distribution points.

According to Joe DeSopo from NetEqualizer, “About half of wireless network slowness comes from p2p (bittorrent)  and video users overloading the access points. We have had great success with our  NE2000 series  but the price point of $2500 was a bit too high to duplicate all over the network.”

For a small or medium sized office with a hosted VOIP PBX solution the NetEqualizer works like a genie in a bottle. It is one of the few products on the market that can provide QOS for voip over an Internet link. And now, with volume pricing approaching $1000,  it will help revolutions the way offices use their Internet connection.

The NetEqualizer is a plug-and-play bandwidth control and WAN/Internet optimization appliance that is flexible and scalable. When the network is congested, NetEqualizer’s unique “behavior shaping” technology gives priority to latency-sensitive applications, such as VoIP and email. It does it all dynamically and automatically, improving on other available bandwidth shaping technology. It controls network flow for the best WAN optimization.

APconnections is a privately held company founded in 2003 and is based in Lafayette, Colorado.

Related Articles

Posted in Press Releases. Tags: , , , , , , , , , , , , , . Leave a Comment »

NetEqualizer Bandwidth Control Tech Seminar Video Highlights

February 4, 2009 — netequalizer

Tech Seminar, Eastern Michigan University, January 27, 2009

This 10-minute clip was professionally produced January 27, 2009. It  gives a nice quick overview of how the NetEqualizer does bandwidth control while providing priority for VoIP and video.

The video specifically covers:

1) Basic traffic shaping technology and NetEqualizer’s behavior-based methods

2) Internet congestion and gridlock avoidance on a network

3) How peer-to-peer file sharing operates

4) How to counter the effects of peer-to-peer file sharing

5) Providing QoS and priority for voice and video on a network

6) A short comparison by a user (a university admin) who prefers NetEqualizer to layer-7 deep packet inspection techniques

Posted in Conferences. Tags: , , , , , , , , , , , , , , , , , . 1 Comment »
« Older posts
Follow

Get every new post delivered to your Inbox.

Join 31 other followers

%d bloggers like this: