NetEqualizer White Paper Comparison with Traditional Layer-7 (Deep Packet Inspection Products)


Updated with new reference material May 4th 2009

How NetEqualizer compares to Packeteer, Allot, Cymphonics, Exinda

We often get asked how NetEqualizer compares to Packeteer, Allot, Cymphonics, Exinda and a plethora of other well-known companies that do layer 7 application shaping (packet shaping). After several years of these questions, and discussing different aspects with former and current application shaping IT administrators, we’ve developed a response that should clarify the differences between NetEqualizers behavior based approach and the rest of the pack.

We thought of putting our response into a short, bullet-by-bullet table format, but then decided that since this decision often involves tens of thousands of dollars, 15 minutes of education on the subject with content to support the bullet chart was in order. If you want to see just the bullet chart, you can skip to the end now, but if you’re looking to have the question answered as objectively as possible, please take a few minutes to read on

In the following sections, we will cover specifically when and where application shaping (deep packet inspection) is used, how it can be used to your advantage, and also when it may not be a good option for what you are trying to accomplish. We will also discuss how the NetEqualizer and its behavior-based shaping fits into the landscape of application shaping, and how in some cases the NetEqualizer is a much better alternative.

First off, let’s discuss the accuracy of application shaping. To do this, we need to review the basic mechanics of how it works.

Application shaping is defined as the ability to identify traffic on your network by type and then set customized policies to control the flow rates for each particular type. For example, Citrix, AIM, Youtube, and BearShare are all applications that can be uniquely identified.

As you are likely aware, all traffic on the Internet travels around in what is called an IP packet. An IP packet can very simply be thought of as a string of characters moving from computer A to computer B. The string of characters is called the “payload,” much like the freight inside a railroad car. On the outside of this payload is the address where it is being sent. On the inside is the data/payload that is being transmitted. These two elements, the address and the payload, comprise the complete IP packet. In the case of different applications on the Internet, we would expect to see different kinds of payloads.

At the heart of all current application shaping products is special software that examines the content of Internet packets as they pass through the packet shaper. Through various pattern matching techniques, the packet shaper determines in real time what type of application a particular flow is. It then proceeds to take action to possibly restrict or allow the data based on a rule set designed by the system administrator.

For example, the popular peer-to-peer application Kazaa actually has the ASCII characters “Kazaa” appear in the payload, and hence a packet shaper can use this keyword to identify a Kazaa application. Seems simple enough, but suppose that somebody was downloading a Word document discussing the virtues of peer-to-peer and the title had the character string “Kazaa” in it. Well, it is very likely that this download would be identified as Kazaa and hence misclassified. After all, downloading a Word document from a Web server is not the same thing as the file sharing application Kazaa.

The other issue that constantly brings the accuracy of application shaping under fire is that some application writers find it in their best interest not be classified. In a mini arms race that plays out everyday across the world, some application developers are constantly changing their signature and some have gone as far as to encrypt their data entirely.

Yes, it is possible for the makers of application shapers to counter each move, and that is exactly what the top companies do, but it can take a heroic effort to keep pace. The constant engineering and upgrading required has an escalating cost factor. In the case of encrypted applications, the amount of CPU power required for decryption is quite intensive and impractical and other methods will be needed to identify encrypted p2p.

But, this is not to say that application shaping doesn’t work in all cases or provide some value. So, let’s break down where it has potential and where it may bring false promises. First off, the realities of what really happens when you deploy and depend on this technology need to be discussed.

Accuracy and False Positives

As of early 2003, we had a top engineer and executive join APConnections direct from a company that offered application shaping as one of their many value-added technologies. He had first hand knowledge from working with hundreds of customers who were big supporters of application shaping:

The application shaper his company offered could identify 90 percent of the spectrum of applications, which means they left 10 percent as unclassified. So, right off the bat, 10 percent of the traffic is unknown by the traffic shaper. Is this traffic important? Is it garbage that you can ignore? Well, there is no way to know with out any intelligence, so you are forced to let it go by without any restriction. Or, you could put one general rule over all of the traffic – perhaps limiting it to 1 megabit per second max, for example. Essentially, if your intention was 100-percent understanding and control of your network traffic, right out the gate you must compromise this standard.

In fairness, this 90-percent identification actually is an amazing number with regard to accuracy when you understand how daunting application shaping is. Regardless, there is still room for improvement.

So, that covers the admitted problem of unclassifiable traffic, but how accurate can a packet shaper be with the traffic it does claim to classify? Does it make mistakes? There really isn’t any reliable data on how often an application shaper will misidentify an application. To our knowledge, there is no independent consumer reporting company that has ever created a lab capable of generating several thousand different applications types with a mix of random traffic, and then took this mix and identified how often traffic was misclassified. Yes, there are trivial tests done one application at a time, but misclassification becomes more likely with real-world complex and diverse application mixes.

From our own testing of application technology freely available on the Internet, we discovered false positives can occur up to 25 percent of the time. A random FTP file download can be classified as something more specific. Obviously commercial packet shapers do not rely on the free technology in open source and they actually may improve on it. So, if we had to estimate based on our experience, perhaps 5 percent of Internet traffic will likely get misclassified. This brings our overall accuracy down to 85 percent (combining the traffic they don’t claim to classify with an estimated error rate for the traffic they do classify).

Constantly Evolving Traffic

Our sources say (mentioned above) that 70 percent of their customers that purchased application shaping equipment were using the equipment primarily as a reporting tool after one year. This means that they had stopped keeping up with shaping policies altogether and were just looking at the reports to understand their network (nothing proactive to change the traffic).

This is an interesting fact. From what we have seen, many people are just unable, or unwilling, to put in the time necessary to continuously update and change their application rules to keep up with the evolving traffic. The reason for the constant changing of rules is that with traditional application shaping you are dealing with a cunning and wise foe. For example, if you notice that there is a large contingent of users using Bittorrent and you put a rule in to quash that traffic, within perhaps days, those users will have moved on to something new: perhaps a new application or encrypted p2p. If you do not go back and reanalyze and reprogram your rule set, your packet shaper slowly becomes ineffective.

And finally lest we not forget that application shaping is considered by some to be a a violation of Net Neutrality.

When is application shaping the right solution?

There is a large set of businesses that use application shaping quite successfully along with other technologies. This area is WAN optimization. Thus far, we have discussed the issues with using an application shaper on the wide open Internet where the types and variations of traffic are unbounded. However, in a corporate environment with a finite set and type of traffic between offices, an application shaper can be set up and used with fantastic results.

There is also the political side to application shaping. It is human nature to want to see and control what takes place in your environment. Finding the best tool available to actually show what is on your network, and the ability to contain it, plays well with just about any CIO or IT director on the planet. An industry leading packet shaper brings visibility to your network and a pie chart showing 300 different kinds of traffic. Whether or not the tool is practical or accurate over time isn’t often brought into the buying decision. The decision to buy can usually be “intuitively” justified. By intuitively, we mean that it is easier to get approval for a tool that is simple to conceptually understand by a busy executive looking for a quick-fix solution.

As the cost of bandwidth continues to fall, the question becomes how much a CIO should spend to analyze a network. This is especially true when you consider that as the Internet expands, the complexity of shaping applications grows. As bandwidth prices drop, the cost of implementing such a product is either flat or increasing. In cases such as this, it often does not make sense to purchase a $15,000 bandwidth shaper to stave off a bandwidth upgrade that might cost an additional $200 a month.

What about the reporting aspects of an application shaper? Even if it can only accurately report 90 percent of the actual traffic, isn’t this useful data in itself?

Yes and no. Obviously analyzing 90 percent of the data on your network might be useful, but if you really look at what is going on, it is hard to feel like you have control or understanding of something that is so dynamic and changing. By the time you get a handle on what is happening, the system has likely changed. Unless you can take action in real time, the network usage trends (on a wide open Internet trunk) will vary from day to day.1 It turns out that the most useful information you can determine regarding your network is an overall usage patter for each individual. The goof-off employee/user will stick out like a sore thumb when you look at a simple usage report since the amount of data transferred can be 10-times the average for everybody else. The behavior is the indicator here, but the specific data types and applications will change from day to day and week to week

How does the NetEqualizer differ and what are its advantages and weaknesses?

First, we’ll summarize equalizing and behavior-based shaping. Overall, it is a simple concept. Equalizing is the art form of looking at the usage patterns on the network, and then when things get congested, robbing from the rich to give to the poor. Rather than writing hundreds of rules to specify allocations to specific traffic as in traditional application shaping, you can simply assume that large downloads are bad, short quick traffic is good, and be done with it.

This behavior-based approach usually mirrors what you would end up doing if you could see and identify all of the traffic on your network, but doesn’t require the labor and cost of classifying everything. Applications such as Web surfing, IM, short downloads, and voice all naturally receive higher priority while large downloads and p2p receive lower priority. This behavior-based shaping does not need to be updated constantly as applications change.

Trusting a heuristic solution such as NetEqualizer is not always an easy step. Oftentimes, customers are concerned with accidentally throttling important traffic that might not fit the NetEqualizer model, such as video. Although there are exceptions, it is rare for the network operator not to know about these potential issues in advance, and there are generally relatively few to consider. In fact, the only exception that we run into is video, and the NetEqualizer has a low level routine that easily allows you to give overriding priority to a specific server on your network, hence solving the problem.

Another key element in behavior-based shaping is connections. Equalizing takes care of instances of congestion caused by single-source bandwidth hogs. However, the other main cause of Internet gridlock (as well as bringing down routers and access points) is p2p and its propensity to open hundreds or perhaps thousands of connections to different sources on the Internet. Over the years, the NetEqualizer engineers have developed very specific algorithms to spot connection abuse and avert its side effects.

This overview, along with the summary table below, should give you a good idea of where the NetEqualizer stands in relation to packet shaping.

Summary Table

Application based shaping

  • good for static links where traffic patterns are constant

  • good for intuitive presentations makes sense and easy to explain to non technical people
  • detailed reporting by application type
  • not the best fit for wide open Internet trunks
    • costly to maintain in terms of licensing

    • high initial cost

    • constant labor to tune with changing application spectrum

    • expect approximately 15 percent of traffic to be unclassified

  • only a static snapshot of a changing spectrum may not be useful
  • false positives may show data incorrectly no easy way to confirm accuracy
  • violates Net Neutrality

Equalizing

  • not the best for dedicated WAN trunks

  • the most cost effective for shared Internet trunks
  • little or no recurring cost or labor
  • low entry cost
  • conceptual takes some getting used to
  • basic reporting by behavior used to stop abuse
  • handles encrypted p2p without modifications or upgrades
  • Supports Net Neutrality

1 The exception is a corporate WAN link with relatively static usage patterns.

Note: Since we first published this article, deep packet inspection also known as layer 7 shaping has taken some serious industry hits with respect to US based ISPs

Related articles:

Why is NetEqualizer the low price leader in bandwidth control

When is deep packet inspection a good thing?

NetEqualizer offers deep packet inspection comprimise.

Internet users attempt to thwart Deep Packet Inspection using encryption.

Why the controversy over deep Packet inspection?

World wide web founder denounces deep packet inspection

What NetEqualizer Users Are Saying (updated June 2009)


Editor’s Note: As NetEqualizer’s popularity has grown, more and more users have been sharing their experiences on message boards and listservs across the Internet. Just to give you an idea of what they’re saying, here a few of the reviews and discussion excerpts that have been posted online over the past several months…

Wade LeBeau — The Daily Journal Network Operations Manager

NetEqualizer is one of the most cost-effective management units on the market, and we found the unit easy to install—right out of the box. We made three setting changes to match our network using the web (browser) interface, connected the unit, and right away traffic shaping started, about 10minutes total setup time. The unit has two Ethernet ports…one port toward your user network, the other ports toward your broadband connection/server if applicable. A couple of simple clicks and you can see reporting live as it happens. In testing, we ran our unit for 30-days and saw our broadband reports stabilize and our users receiving the same slices of broadband access. With the NetEqualizer, there is no burden of extensive policies to manage….The NetEqualizer is a nice tool to add to any network of any size. Businesses can see how important the Internet is and how hungry users can be for information.

__________________________________________________________________________________________________

DSL Reports, April 2009

The Netequalizer has resulted in dramatically improved service to our customers. Most of the time, our customers are seeing their full bandwidth. The only time they don’t see it now is when they’re downloading big files. And, when they don’t see full performance, its only for the brief period that the AP is approaching saturation. The available bandwidth is re-evaluated every 2 seconds, so the throttling periods are often brief.

Bottom line to this is that we can deliver significantly more data through the same AP. The customers hitting web pages, checking e-mail, etc. virtually always see full bandwidth, and the hogs don’t impact these customers. Even the hogs see better performance (although that wasn’t one of my priorities).

__________________________________________________________________________________________________

Loyola University — Chicago

At Loyola University Chicago, we are on our 2nd iteration of the NetEqualizer. We used the product happily for a number of years when we had a T3. We upgraded our internet pipe to 100MB and after about 6 months we noticed 100% saturation and students complaining of slow internet for various applications. We knew then that we needed another NetEqualizer. Once we plugged the box in it started managing the bandwidth, our pipe has not been saturated since, and more importantly the complaints have ceased.

__________________________________________________________________________________________________

Alan Leech, Orlean Invest West Africa Limited, January 24, 2009

Gentlemen

We purchased 3 of your devices last year and I have to say we are very impressed by them.

They have matched our requirement perfectly and allow us to provide fair usage to our clients whilst reducing our overall OPEX.

You can be sure we will be purchasing in the future.

Alan Leech

__________________________________________________________________________________________________

Illinois Wesleyan Replaces Packeteer with NetEqualizer as Part of Bandwidth Upgrade, January 19, 2009

By tshort

Network Services has completed the Network Upgrade Project.  The Internet bandwidth available to the Campus was doubled from 45MBs (DS3) to 90MBs in December.  Along with the additional bandwidth, a new bandwidth sharing device call a NetEqualizer replaced the existing Packeteer.  The NetEqualizer uses bandwidth sharing fairness rules based on network usage to share bandwidth and balance the available bandwidth between all users.  The project made a dramatic improvement to Internet access for the campus community.

__________________________________________________________________________________________________

Chris Chamberlain, Oakland University in Detroit

Doug,

Because Netequalizer simply makes things fair, i.e. gives everyone on the link the same percentage of the bandwidth “pie” the netequalizer can handle any type of traffic, because it isn’t classifying anything.

Chris Chamberlain

Oakland University

>On Apr 30, 2008, at 4:42 PM, Green, Doug wrote:

>We are considering Netequalizer. They are claiming to be able to manage  encrypted BitTorrent. Can anyone verify this?

>Thank you,

>Doug Green

>Manager, Network Services & Security

>University of New Hampshire

>50 College Rd

__________________________________________________________________________________________________

Charlie Prothero, CIO, Keystone College

I have written on a couple of Educause lists about our experience with the Netequalizer, which has been invariably positive.  It’s a snap to set up and doesn’t require anywhere near the tuning effort that a Packeteer does.  For general Internet circuit coverage, I’m very pleased with it.

__________________________________________________________________________________________________

Ben Schworm, The Independent School Educators’ List, ISED-L

We just re-evaluated our systems after realizing that even with the Packetshaper in place, we’d need to increase the amount of bandwidth that we offer the community. First of all, the new Packetshaper hardware we’d need was going to cost $18,000. Second, over the 5 years that we’ve had the Packetshaper, we’ve seen its effectiveness decrease with the increased availability and academic usage of real-time streaming apps and the increasing amount of traffic that is classified as either pure web browsing traffic (whether it is or not) or “default”, the traffic class that catches all the other traffic that the Packetshaper can’t specifically identify. Furthermore, the Packetshaper can tend to be a pretty admin-intensive system to keep working effectively.

The NetEqualizer really only deals with end-user behavior in that it looks at the bandwidth that a given user is trying to utilize relative to what’s available and throttles “bad” users in order to try to maintain fair access to the bandwidth. It also throttles “bad” applications like P2P that open many connections to and from a given user. The box is nearly configuration and maintenance-free and costs a fraction of what the Packetshaper does.

__________________________________________________________________________________________________

Ed Loebach, UVMRESNET

I was asked to tell our experience with NetEqualizer. We purchased the box about 3 weeks into first semester when our old bandwidth control server died and support was not forthcoming from the company.

We put NetEqualizer in place and fired it up with little to no problem. For the first 5-6 hours it worked as we were told it would with NO configuration. After the first day we noticed problems with students exceeding the connection limits we set. We called the company and within 24 hours we had the configuration modified to the specific needs of our network and our bandwidth was under our control again.

In the last 4 months I have not had to make any additional changes to the configuration. In fact we have not even had the need to restart the box. The NetEqualizer box has some very good algorithms to have controlled our heavy bandwidth users with not adding significant network overhead to the rest of our low bandwidth users. Our students have seen an increase in bandwidth when they need it. The gamers are happy because the latency we used to have under our old bandwidth system has disappeared.

__________________________________________________________________________________________________

Douglas Hedges, EDUCAUSE Small College Constituent Group Listserv

We’ve dumped our Packeteer device about 18 mos. ago for a NetEqualizer. It has worked as advertised and has required virtually no maintenance after initial setup (which took just a few minutes as well). There are some good technical papers on their site (http://www.netequalizer.com) describing its operation and comparing it to other products. I believe they’re worth a read if you want to see if it’s a good fit for your campus. It sure was for ours.

__________________________________________________________________________________________________

Russ Leathe, EDUCAUSE Security Constituent Group Listserv

Gordon College switched from Packeteer to netEQ a while ago. It works flawlessly and our daily management of bandwidth decreased significantly.

They also have a CALEA probe.

__________________________________________________________________________________________________

Superdog, DSLReports.com

When you plug in the Neteq box, it doesn’t care about IP’s or what range it is on. You set the bandwidth maximum limit for whatever your pipe size is and then plug it inline between your core router and your first main switch and you are done…

…I love this unit and I can not say enough about it. With M0n0wall and Packetteer, you have to manually setup all of the rules in order for the units to be effective. After you spend a few hours getting them setup, it only takes the user/program 10 seconds to switch ports on you and that rule is then invalid and you need to go back and redo it.

This type of setup requires you to monitor your box constantly, creating even more work. The Neteq unit doesn’t need to know all of this. It just counts connections per user (A limit you set) and the amount of bandwidth each user consumes. If the bandwidth is there and no one else is using it, that person gets it. If they are running Limewire at full throttle and another user logs in and starts to surf the net?, that user gets full priority and their pages will load quickly while the Limewire download has delay added to their packets.

IMHO, using this unit is a no-brainer for any ISP. It is a hands off setup that really works.

__________________________________________________________________________________________________

Josh Heller, Sr. Network Analyst — Information Technology, Kutztown University

Our University started with PacketShapers, but also made an investment in NetEqualizer when we found the PacketShaper wasn’t completely doing the job. Today we use both products.

We have been pleased with NetEqualizer  as it does what it advertises – it makes a noticeable difference in congested network.

__________________________________________________________________________________________________

Nathan P. Hay, Network Engineer — Computer Services, Cedarville University

We switched from PacketShaper to NetEqualizer this summer.  NetEq is much simpler to manage and much cheaper.

__________________________________________________________________________________________________

George Flowers, Flint River Tech

We currently have the NE2000, and it works great!  No other product can do what the NetEqualizer does at a great price!

NetEqualizer Bandwidth Controller POE unit a hit with customers


Editors Note:  Just pulled this post off of DSL reports.

NetEqualizer POE units list at $1499 and serve as a great QOS devise for the SOHO small business user.

We’ve ordered 4 of these and deployed 2 so far. They work exactly like the 1U rackmount NE2000 that we have in our NOC, only the form factor is much smaller (about 6x6x1) and they use POE or a DC power supply. I amp clamped one of the units, and it draws about 7 watts.

We have a number of remote APs where we don’t have the physical space and/or power sources (i.e., solar powered) to accommodate the full size Netequalizer. Also, because of our network topology, it makes sense to have these units close to the AP and not at our border. These units are the perfect solution for these locations.

Our service area is mostly in a forest, so have a number of Trango 900 Mhz APs. These units can cut through the trees well, but they only have about 2.5 Mbps available on them (they’re rated at 3 Mbps, but we’ve tested their actual throughput at 2.5 Mbps). We have our customers set for 768k, so it doesn’t take too many Youtube and Netflix streams to kill the performance on these APs. We were using Mikrotiks to throttle the customers (using bursting to give them about 10 minutes @768k, then throttling them to around 300k). While this helped to keep the bandwidth hogs from individually killing the performance, it sometimes made matters worse.

For example, if a customer started downloading some 2 GB file at 10:00pm, it would take them until 1:00pm the next day to finish. As such, they would have disrupted services in the morning and early afternoon. If we had given this customer their full 768k, they would have finished this download before 4:00am and would never have been a disruption.

With the Mikrotik solution, we also had too many times that there was less than 768k available for the next customer, because there were a number of customers locked at 300k tying up much of the bandwidth. So, the customer that was hitting the casual web page was seeing poor performance (as were the hogs). In general, I wasn’t happy with the service we were delivering.

The Netequalizer has resulted in dramatically improved service to our customers. Most of the time, our customers are seeing their full bandwidth. The only time they don’t see it now is when they’re downloading big files. And, when they don’t see full performance, its only for the brief period that the AP is approaching saturation. The available bandwidth is re-evaulated every 2 seconds, so the throttling periods are often brief.

Bottom line to this is that we can deliver significantly more data through the same AP. The customers hitting web pages, checking e-mail, etc. virtually always see full bandwidth, and the hogs don’t impact these customers. Even the hogs see better performance (although that wasn’t one of my priorities).

I didn’t tell any customers that I was deploying the Netequalizers. Without solicitation, I’ve had a number of them comment that the service seems faster lately. It sure is fun to hear unsolicited compliments…

The only tweak of significance I made to the default setup was to change the MOVING_AVG from 8 to 29 (it can be set higher, but you can’t do it in the web interface). This makes it so that the Netequalizer considers someone to be a hog when their average data rate over the last 29 seconds is greater than HOGMIN (which we’ve left at 12,000 – 96 kbps). Given that our customers are set for 768k, this means that they can burst at full rate for a little under 4 seconds before they are considered a hog (approximately 350 KiloBytes of data). The default setting of 8 would allow approximately 1 second at full bandwidth (a little under 100K). By making this change, almost all web pages would never be subject to throttling. It also makes it so that most bandwidth test servers will not see any throttling. The change makes us more at risk that we can peak out the AP (since less customers may be subject to throttling), but we’ve seen that the throttling usually kicks in long before we see that problem.

The only feature I’d like to see in these units is to have a “half duplex” mode. The Netequalizers have separate upload and download pools. This works fine for most ISPs using typical full duplex circuits. However, most hardware that WISPs use are half duplex. So, our Trangos have 2.5 Mbps available TOTAL of upload and download. In order to have the Netequalizer throttle well, I configured it so that the Trangos had 1.9 Mbps down and .6 Mbps up. I would prefer to have a single 2.5 Mbps pool that throttles only when download + upload approaches 2.5 Mbps. If we had this feature, we could move even more data through the Trangos

Related Article

Speeding up Your T1, DS3, or Cable Internet Connection with an Optimizing Appliance


By Art Reisman, CTO, APconnections (www.netequalizer.com)

Whether you are a home user or a large multinational corporation, you likely want to get the most out of your Internet connection. In previous articles, we have  briefly covered using Equalizing (Fairness)  as a tool to speed up your connection without purchasing additional bandwidth. In the following sections, we’ll break down  exactly how this is accomplished in layman’s terms.

First , what is an optimizing appliance?

An optimizing appliance is a piece of networking equipment that has one Ethernet input and one Ethernet output. It is normally located between the router that terminates your Internet connection and the users on your network. From this location, all Internet traffic must pass through the device. When activated, the optimizing appliance can rearrange traffic loads for optimal service, thus preventing the need for costly new bandwidth upgrades.

Next, we’ll summarize equalizing and behavior-based shaping.

Overall, equalizing is a simple concept. It is the art form of looking at the usage patterns on the network, and when things get congested, robbing from the rich to give to the poor. In other words, heavy users are limited in the amount of badwidth to which they have access in order to ensure that ALL users on the network can utilize the network effectively. Rather than writing hundreds of rules to specify allocations to specific traffic as in traditional application shaping, you can simply assume that large downloads are bad, short quick traffic is good, and be done with it.

How is Fairness implemented?

If you have multiple users sharing your Internet trunk and somebody mentions “fairness,” it probably conjures up the image of each user waiting in line for their turn. And while a device that enforces fairness in this way would certainly be better than doing nothing, Equalizing goes a few steps further than this.

We don’t just divide the bandwidth equally like a “brain dead” controller. Equalizing is a system of dynamic priorities that reward smaller users at the expense of heavy users. It is very very dynamic, and there is no pre-set limit on any user. In fact, the NetEqualizer does not keep track of users at all. Instead, we monitor user streams. So, a user may be getting one stream (FTP Download) slowed down while at the same time having another stream untouched(e-mail).

Another key element in behavior-based shaping is connections. Equalizing takes care of instances of congestion caused by single-source bandwidth hogs. However, the other main cause of Internet gridlock (as well as bringing down routers and access points) is p2p and its propensity to open hundreds or perhaps thousands of connections to different sources on the Internet. Over the years, the NetEqualizer engineers have developed very specific algorithms to spot connection abuse and avert its side effects.

What is the result?

The end result is that applications such as Web surfing, IM, short downloads, and voice all naturally receive higher priority, while large downloads and p2p receive lower priority. Also, situations where we cut back large streams is  generally for a short duration. As an added advantage, this behavior-based shaping does not need to be updated constantly as applications change.

Trusting a heuristic solution such as NetEqualizer is not always an easy step. Oftentimes, customers are concerned with accidentally throttling important traffic that might not fit the NetEqualizer model, such as video. Although there are exceptions, it is rare for the network operator not to know about these potential issues in advance, and there are generally relatively few to consider. In fact, the only exception that we run into is video, and the NetEqualizer has a low level routine that easily allows you to give overriding priority to a specific server on your network, hence solving the problem. The NetEqualizer also has a special feature whereby you can exempt and give priority to any IP address specifically in the event that a large stream such as video must be given priority.

Through the implementation of Equalizing technology, network administrators are able to get the most out of their network. Users of the NetEqualizer are often surprised to find that their network problems were not a result of a lack of bandwidth, but rather a lack of bandwidth control.

See who else is using this technology.

Created by APconnections, the NetEqualizer is a plug-and-play bandwidth control and WAN/Internet optimization appliance that is flexible and scalable. When the network is congested, NetEqualizer’s unique “behavior shaping” technology dynamically and automatically gives priority to latency sensitive applications, such as VoIP and email. Click here for a full price list.

Using a Load Generator/Emulator to Test Your Network


By Art Reisman, CTO, APconnections (www.netequalizer.com)

One of the most challenging aspects of technology development has always been the process of bridging the gap between theory and application.  What may seem to work on paper, and even in limited trials, was never guaranteed when dealing with real-world scenarios and often unforeseen problems.

Several members of our engineering team just returned from a week of  testing with Candela Technologies’ network load emulator, and once again, we’ve not been dissapointed.  At the touch of a button, we were able to create unbelievably realistic worst-case load scenarios. Candela’s LANforge equipment not only stressed our network elements, but did so with variation, creating an environment that successfully simulated the challenges our technology will face on a regular basis in the field.

Judging by the numerous trials we’ve run, it’s become clear that simply driving a fixed load across a network is not enough to ensure reliability. Instead, you need a simulation with a multitude of elements (different packet sizes, UDP , TCP, broadcast traffic, etc.) and traffic streams, including those that refuse to back down such as with a bad denial of service attack or virus.  Fortunatley, this is exactly the quality of service that Candela Tech offers.

In addition to giving you peace of mind, this type of simulation can also save you and your company time and money.  When implementing a network upgrade, the normal method of operation goes a little like this:

  1. Work late at night and over the weekends
  2. Implement the change
  3. Put staff on standby for the next business day
  4. Have a fallback strategy to revert to a previously proven configuration should things go south

While these steps eventually may do the trick, they’re not without their costs — both financial and otherwise. Aside from the overtime you’ll end up paying your admin, perhaps more importantly, you also run the risk of negatively impacting the service of clients and customers during the hit-and-miss setup process.

Yet, the costs that come with this type of strategy can easily be reduced with a sophisticated load generation device. Network choke points can be stressed and limits determined before unwittingly making  guinea pigs out of your network users.  And, the staff from Candela Tech is more than knowledgeable and eager to help, which has allowed us to be up and running right out of the box on more than one occasion.

Ultimately, using Candela Technologies has been a lot like looking into a crystal ball. After the LANforge simulations, we’re able to identify and address any issues before they affect our customers. What was once a process of bringing our technology to the real world has now become a process of Candela bringing the real world to us.

Note: There are other competitive network load generators on the market, Fluke being the market leader.

Created by APconnections, the NetEqualizer is a plug-and-play bandwidth control and WAN/Internet optimization appliance that is flexible and scalable. When the network is congested, NetEqualizer’s unique “behavior shaping” technology dynamically and automatically gives priority to latency sensitive applications, such as VoIP and email. Click here for a full price list.

Five Key Marketing Tips for Entrepreneur and Tech Start-Up Companies


By Art Reisman, CEO and co-founder of APconnections, makers of the NetEqualizer

Updated April 25th , 2010

Aside from a few freakish start ups, very few products will take off with out some form of promotion. However, since founding APconnections in 2003,  we’ve learned that marketing can be a double-edged sword. Over time, we’ve been able to build upon both our successes as well as mistakes, coming up with a few dos and don’ts of marketing a tech start up along the way.  Here are a few of the key points:

1) Make sure your marketing company has skin in the game

Most marketing firms are staffed by people who went to college and took soft course work, meaning they were not into the black and white of the scientific method.  Perhaps they had a course or two with this emphasis, but it’s not likely to be as ingrained as perhaps a physics or psychology major whose course work included extensive lab experiments showing cause and effect.

Although some creative skill is necessary to be a good marketing person, the down side is most people in this industry tend to remain artsy and vague with how they can measure results. When negotiating with marketing companies  (or people), we came up with a simple formula to measure results and provide a metric which was easy to quantify — hits to the web site.

In our case it was very simple.  We had a baseline established already and we were only going to change one variable  — marketing.  Hence, it would be easy to tie any increase in web traffic to a marketing effort.  To make sure that  any benefit of doubt went to our marketing firm, we decided  any increase in web traffic, regardless of cause, would be credited to their efforts.

Once we tied marketing fees exclusively to a metric that could be measured, we were able to eliminate several marketing firms, many of which headed for the hills never to be seen from again.

2) Round two — good web traffic verses bad traffic

Having solved the problems of paying for results, we came across another hurdle. We’ll just call it good traffic verses bad traffic. The easiest way to describe this is by example.

Our product, the NetEqualizer, is meant to  be sold to commercial operators and businesses where there is Internet congestion.   An obvious catch-all key phrase to lead with in marketing literature would be, “speed up your Internet connection”.

If you throw this type of tag line into a generic advertisement to a broad audience,  perhaps 99 percent of the people who follow up on it will be home users, kids playing World of Warcraft, looking for some tool they can load up on their Windows machine for $25. In other words, the majority of these follow ups would certainly not be our target market.

What we found was that our consignment-based marketing people were not  screening this traffic out. We believe this disconnect  goes back to their inability to use the scientific method to control variables. So, as you can imagine, we initially had a flood of inquiries outside our target market which turned out to be a big waste of our time. The solution to ending this march of unqualified leads was to put a higher price in any literature or teasers and to emphasize our product was for commercial users etc. So, instead of just promoting the potential to speed up Internet traffic, we made NetEqualizer’s starting price clear from the beginning.

3) Make sure your marketing people understand  how your product is used and take an honest interest in it

We would spend hours explaining our target market and details about our product only to find out that this information would go in one ear and out the other. When we finally found somebody who had the capacity to understand our product we doubled their pay.  It was worth it in time saved.

4) Consignment ads only

Once you have decided on an effective message in an advertisement, follow this rule — Never pay a dime up front to bolster your ad’s exposure just to increase your market presence. Yes market presence is good, but unless you can measure this in terms of some metric,  just don’t do it.

Any advertisement you place should only charge you when somebody clicks on it. I am not talking about discount coupons for a local business here. I am talking about selling a product to a broad national or global audience. This edict pretty much rules out print advertising. To be fair, I have heard from other CEO’s of tech companies that if you stick with print ads and spend a good deal of cash, they will pay-off, but this shouldn’t be your first or only option. Until you have exhausted every conceivable outlet for consignment based advertising, why risk digging any dry wells?

Our experience with six weeks of continuous quarter- and half-page ads in tech magazines brought zero impact. Nothing. Nada. We measured no increase in web hits. Maybe we did increase awareness, but awareness has no value if you go bankrupt establishing it and don’t see any returns.

Of course, while there are no guarantees for successful marketing, these four tips have been tested and proven effective at APconnections over the past several years.

5) Avoid being strung out

Perhaps this tip should be number one as it is essential to understand how companies will string you out. As a start up, with an idea you will likely get conditional inquiries. Can you product do “this”, can we we customize it ? Often times the more questions, the more uncertain the customer is about their own business. You MUST establish the customers willingness to pay before getting  wrapped up in the promise of future sales. Obviously you cannot demand payment on a first consultation with a prospect, but this is a good time to  set some expecations that your time is valuable. I bring this up because at this very moment I am in talks with a large customer interested in our product that has been asking questions for over a month. This morning I basically told  them  (nicely)I will continue the conversation if and when they purchase their first unit from us. For all I know I am dealing with an underling that has time on his hands but no ability to influence a purchase decision. Getting them to purchase something is a big first step toward qualification. If you fail to master qualification you will find yourself borrowing money from relatives to pay your rent or out of business very quickly.

I’d also suggest you look at our tips for using google ads.

Another great site for start ups is entrepreneurship.org

Here is a marketing company (outspoken Media) that  I ran across last Friday , I have not talked to them yet but I really like their bio’s and attitude from their web site.

Good luck!

Looking for a new tech Job? You’ll need to embrace some fear and step out


By Art Reisman, CTO, http://www.netequalizer.com

Art Reisman CTO www.netequalizer.com

Art Reisman

As the CEO of a  tech company and former employee of large stifling unimaginative telecom company here is the advice I recently provided a former colleague on how to land the job at a smaller tech firm.

Contrary to popular media  belief very few (if any employers) care  about your age and race when hiring for a tech job ; however if your former company was  large and stifling, that will create  serious baggage  with a more progressive company.

Nobody ever takes the blame for being part of the problem at a sputtering old line company.  The fact is if you worked there for a long time, then  you were part of the problem. Even if you had a million great ideas on how the company should run itself and the bureaucracy held you down. You will be viewed as complicit by association.

Perhaps it was the high salary that kept you there, or the lack of other opportunity in your region. It does not matter , the smaller start up company will view your past with suspicion.

I meet with several CEOs of other tech companies once a quarter and the consensus is that people with ambition go someplace where they can make an impact.

The perception on the street  is that your old company is still sputtering off its legacy channel left over from what it retained from days past,  and that all the abuse and mind games it uses to stifle employees leaves permanent scars on ingenuity etc.

The kind of people that settled for climbing the ladder in a larger organization are not what new tech companies are  looking for.

You can break that mold by saying hey look how confident I am that I can help you , I am an go getter stuck in an “old company” body type of thing.

So how do you compete and break out?


I’d throw out any expectation of a salary  based on your previous compensation. The company you previously worked for likely  paid you very well. High pay creates a false sense of market value. You’ll notice when times got tough they had to cut massive staff to make ends meet.  So evidentally, the high paying jobs were more of retainer in a good job market rather than a measure against your productivity. Please don’t misconstrue the point I am making,  you do have the potential to make a difference and perhaps make millions, its just that your drive and creativity has been misdirected toward internal corporate games and must be revitalized.

For example, if you are comfortable doing technical sales support and really like a  new company then:

Approach the  CEO (hiring manager) of an established small tech company:

Note: the following specific advice was geared toward an individual and company he was approaching.

Offer to work for $10 an hour  with  conditions:

A Large upside commission and a future salary based on some measurable metric.

I would only make this type  of offer to somebody with integrity that would follow through. Yes there are people with integrity out there , the media plays up the slime balls but most businesses do not function that way at the top.

He  (the CEO)  has long complicated sales cycles with large Fortune 1000  clients.  A mature person who could explain the technology, demonstrate value, could be a huge benefit to his business;  but in this market,  he would not want  the risk of hiring you unless you had skin in the game . On the other hand he might not be able to do something like that because it would be upsetting to his other sales staff who knows ? I am sure it would get his wheels turning as to how he could make it a win win.

This approach also allows you to have some control of where and what you do rather than just take the rescue job that some VP musters  at your current company.

My experience in the real world is that you have to scrap down in the dreggs to get anything going and over time work your  way up. For example when we  first got started there were quite a few super star sales people from the likes of  XXXX and XXXX that I would run into and put to work on commission.  They were completely and totally useless to a small tech startup.  Yes they were nice people but without a ready made channel and customer  base who calls them wanting a bake off between them and the competition they were lost. So we developed a model of guerilla web marketing  and slowly built my oour own direct channel very slowly , but it cost very little and now has very little overhead.

Does your ISP block Web Crawling?


By Art Reisman

Art Reisman CTO www.netequalizer.com

Editor’s note: Art Reisman is the CTO of APconnections. APconnections designs and manufactures the popular NetEqualizer bandwidth shaper.

About one year ago I got the idea to see if I could build a Web Crawler (robot) with the Specific mission of finding references to our brand name on the Internet.

I admit to being a complete amateur to the art of writing a Web Crawler, and certainly it might make more sense to  do Google search on “NetEqualizer” , but I wanted to see if any occurances were  out there,  in Cyber space, that Google ignored or missed.

If you are a hack and want to try this for yourself, I have included my beta Web Crawler source code below.

Back on topic, Does your ISP block Web Crawling?

First a little background on how my Web Crawler works.

1) It takes a seed , a set of web pages to start on

2) It systematically reads those seed Web Pages looking for URL’s amongst them

3) When it finds a URL, it reads it as text, looking for additional URLS within the text.

4) It ranks URLs as Interesting if it finds certain keywords ( a List I created) in the Text of the new URL

5) The more Interesting a URL the more likely it is to get read and so forth.

6) If no keywords are found at all on the searched page it tosses it out as not to be searched. (I think better check this)

7) Ultimately it will stop when it finds “NetEqualizer” or loops a whole bunch of times without finding any new keywords whichever comes first

So you can imagine when this thing is running it is sucking bandwidth as fast as it can read pages, and also hitting random web pages more than humanly possible, after all it is a crawler.

I only ran this script two or three times in its present form because each time I ran it within an hour or so my Internet service would crash and stop altogether. It may just be coincidence that I was having problems with my line at the time as within the next month I did have to have the external cable to the pole replaced by my provider. So honestly I am not postive if my Provider shut me down, but I think so.

At the time, I had not really given it much thought, but if my provider had any watch dog type big brother metric keeping tabs on me, surely this thing would have set off a code Red at the main office. I would assume that residential Internet accounts that start scanning the web at high speed are considered infected with a virus ? Is there a formal clause that by my provider that says they can shut me down if I write a crawler ? I don’t know , as I did not push the issue.

Below is the code. It did start with a perl program written by somebody else, but critical pieces seemed to be omitted (Specific Perl calls on the original) so I had stripped it way down and then built it back up to crawl. I honestly have no idea where I got the original code from as it was over a year ago. Apologies for not giving credit.

See also a generic flow diagram of a Web Crawler.

Sorry about the formatting in the blog.

Use at your won risk etc.

#!/usr/bin/perl -w
##
# spider.pl Set tabstops to 3.
#
$| = 1;

if(scalar(@ARGV) < 2){
print “Usage: $0 <fully-qualified- seed URL> <search-phrase> <keywords>\n”;
exit 1;
}

# Initialize.
%URLqueue = ();
chop($client_host=`hostname`);
$been = 0;
$search_phrase = $ARGV[1];
if (scalar(@ARGV) > 2 ) {
$kicker1 = $ARGV[2]; }
if (scalar (@ARGV) > 3 ) {
$kicker2 = $ARGV[3];
}
if(scalar (@ARGV) > 4 ) {
$kicker3 = $ARGV[4]; }

# Load the queue with the first URL to hit.
$URLqueue{$ARGV[0]} = 0;

# While there’s a URL in our queue which we haven’t looked at …
$total_sites=0;
while ($total_sites < 10000)
{
$x= `echo total sites loop $total_sites >> visited `;
# Progress report.
if ($total_sites > 1000) { exit 1; }
for ( $sites=0; $sites < 200; ) # keep looping hundred times in this beta version
{
$x= `echo sites loop $sites >> visited `;
while(($key,$value) = each(%URLqueue)){
if ( $URLqueue{$key} < 0 ){ if ($URLqueue{$key} == -1)
{ delete $URLqueue{$key}; } # garbage collection
next; } # already been there
if ($sites > 50 && $value < 1 ) {$sites ++; next; }
if ($sites > 100 && $value < 2 ) {$sites ++ ;next;}
if ($sites > 50)
{
$x=`echo primo sites $sites value $value site $key`;
}
($protocol, $rest) = $key =~ m|^([^:/]*):(.*)$|;

# If the protocol is http, fetch the page and process it.
if ( !defined ($protocol)) {next;}
if($protocol eq “http”){
$URLqueue{$key}=-1 ; # mark as visited
$sites++;
$total_sites++;
# Split out the hostname, port and document.
# ($server_host, $port, $document) =
# $rest =~ m|^//([^:/]*):*([0-9]*)/*([^:]*)$|;
print “getting $key \n”;
$x = `cd /tmp; wget -nd -Q 10000 –timeout=2 –tries=1 $key` ;
print “done wget \n”;
$x= `echo $key >> ./visited`;
$page_text = `cat /tmp/* 2> /dev/null`;
$x=`rm /tmp/* 2> /dev/null`;

$page_text =~ tr/\r\n//d;
$page_text =~ s|<!–[^>]*–>||g;
# Report if our search string is found here.
$kick=0;
if($page_text =~ m|$search_phrase|i){
print “found phrase $key $search_phrase ,total sites $total_sites \n”;
exit ;
}
if ( defined $kicker1) {
if($page_text =~ m|$kicker1|i){
#rank this page higher if it has this key word
$x= `echo found kicker $key $kicker1 total sites $total_sites >> visited`;
$kick++;
}
}
if ( defined $kicker2 ) {
if($page_text =~ m|$kicker2|i){
# rank this page higher if it has this key word
$x= `echo found kicker $key $kicker2 ,total sites $sites >> visited`;
$kick++;
}
}
if (defined $kicker3 ) {
if($page_text =~ m|$kicker3|i){
# rank this page higher if it has this key word
print “found kicker $key $kicker3 ,total sites $sites \n”;
$kick++;
}
}
else
{
delete $URLqueue{$key}; # not http
}

# Find anchors in the HTML and update our list of URLs..
(@anchors) = $page_text =~ m|<A[^>]*HREF\s*=\s*”([^
“>]*)”|gi;
foreach $anchor (@anchors){

$newURL = &fqURL($key, $anchor);

if ( exists $URLqueue{$newURL} )
{
$URLqueue{$newURL}= $URLqueue{$newURL} -1;
#don’t garbage collect low numbers
print “duplicate $newURL \n”;
}
else
{
print “new anchor $newURL \n”;
if ($kick > 0 ) {
$x=`echo kick $kick $key $newURL >> ./anchors`; }
$URLqueue{$newURL} =$kick; #new url added to queu
}
}
} #end of while URLqueue
} # end of sites
} #end of total sites
}

sub fqURL
{
local($thisURL, $anchor) = @_;
local($has_proto, $has_lead_slash, $currprot, $currhost, $newURL);

# Strip anything following a number sign ‘#’, because its
# just a reference to a position within a page.
$anchor =~ s|^.*#[^#]*$|$1|;

# Examine anchor to see what parts of the URL are specified.
$has_proto = 0;
$has_lead_slash=0;
$has_proto = 1 if($anchor =~ m|^[^/:]+:|);
$has_lead_slash = 1 if ($anchor =~ m|^/|);

if($has_proto == 1){

# If protocol specified, assume anchor is fully qualified.
$newURL = $anchor;

}
elsif($has_lead_slash == 1){

# If document has a leading slash, it just needs protocol and host.
($currprot, $currhost) = $thisURL =~ m|^([^:/]*):/+([^:/]*)|;
$newURL = $currprot . “://” . $currhost . $anchor;

}
else{

# Anchor must be just relative pathname, so append it to current URL.
($newURL) = $thisURL =~ m|^(.*)/[^/]*$|;
$newURL .= “/” if (! ($newURL =~ m|/$|));
$newURL .= $anchor;

}
return $newURL;
}
The disclaimers:

Use this code at your own risk. I am not even sure if it follows the moral and ethic standards that the major players who crawl the web for living abide by; but since I was only doing this as a weekend experiment I did not worry too much about the standard.

In other words it is experimental and not for commerical use. Do not walk away and leave it running unattended lest you get censured and black listed from the Internet.

Where are the safe tech jobs ?


By Art Reisman, CTO, http://www.netequalizer.com

Art Reisman CTO www.netequalizer.com

Art Reisman

Article Type: Opinion

As the CEO of a small (yet growing) tech company in the current recession I often get calls from former colleagues working at larger corporations. Amidst their companies insincere rhetoric, inaction, and falling revenues, good people wait around wondering who will be next to get the ax.

The underlying problem at most of these companies is that they are continue to push products into a stagnant or declining market. The only way to have any relative security is to get on board with an industry or niche with solid growth potential.

So if your wondering where to turn for potential job security here are some tips that might help

Look for a company that is doing something with real value for society and not just jumping on the latest bandwagon.

1) Renewable energy is hot , and certainly a job in renewable energy is better than selling steam engines running off coal. Renewable energy, although here to stay is being over hyped . Right now the success of renewable energy is dependent on battery technology. Fossil fuels are nothing more than the Suns energy stored up and retrieved at will when needed. For renewable (wind, solar) energy is to compete easily with traditional fossil fuels we must come up with a clean effective battery to store energy. My advice seek out a company that specializes in battery technology and then help them make a difference.

2) Network and Internet Optimization

Internet Infrastructure companies are being forced by their stock holders to turn a profit. The days of free falling bandwidth contracts are slowing down, hence the new hot market will be companies with products that optimize internet bandwidth. Bandwidth control , WAN optimization and compression although not on the front pages, are areas of value and are holding their own in the recession. Some companies to look at , are

APconnection (NetEqualizer)

Packeteer

Allot

RiverBed

Exinda

3) Medical Technology,

From newer and better and MRI machines to prosthetics , Americans will spare no expense for anything that will make their lives more comfortable. So when will this party end and the associated demand for jobs in the Medical Industry flatten out?

Although I do not expect a crash in this field as we might have seen in other boom and bust industries, I do expect a slowdown. Every bubble has its end, and the Medical technology industry is due for a slow down. As consumers push back on medical care pricing, high end technology research will slow down. Still a better prospect than steam engines though.

4) Auto Industry

If you are entering into the field of Mechanical Engineering or electronics controls now would be a good time to focus on the Auto Industry. For the next 5 to 10 years I expect that auto makers will be looking for new innovative ideas in their engineering departments. They will also be looking for new talent. Don’t let the down turn discourage you this is an opportunity.

APconnections Releases NetEqualizer for Small Business and WISP Market


LAFAYETTE, Colo., April 13 /PRNewswire/ -- APconnections (http://www.netequalizer.com),
a leading supplier of plug-and-play bandwidth shaping products,
today announced the release of its newest NetEqualizer model,
developed specifically with WISPs and small business users in mind.

This newest NetEqualizer release easily handles up to 10 megabits of traffic and up to 100 users, allowing room for expansion for growing demand. Furthermore, in addition to offering all standard NetEqualizer features, this smaller model will be Power over Ethernet, providing administrators greater flexibility in placing the unit within their network.

The model was developed to meet a growing demand both for an affordable traffic shaping device to help small businesses run VoIP concurrent with data traffic over their Internet link as well as a need for a shaping unit with PoE for the WISP market.

In a large wireless network, congestion often occurs at tower locations. However, with a low-cost PoE version of the NetEqualizer, wireless providers can now afford to have advanced bandwidth control at or near their access distribution points.

“About half of wireless network slowness comes from p2p (Bit Torrent) and video users overloading the access points,” said Joe D’Esopo, vice president of business development at APconnections. “We have had great success with our NE2000 series, but the price point of $2,500 was a bit too high to duplicate all over the network.”

For a small- or medium-sized office with a hosted VoIP PBX solution, the NetEqualizer is one of the few products on the market that can provide QoS for VoIP over an Internet link. And now, with volume pricing approaching $1,000, the NetEqualizer will help revolutionize the way offices use their Internet connection.

Pricing for the new model will be $1,200 for existing NetEqualizer users and $1,499 for non-customers purchasing their first unit. However, the price for subsequent units will be $1,200 for users and nonusers alike.

The NetEqualizer is a plug-and-play bandwidth control and WAN/Internet optimization appliance that is flexible and scalable. When the network is congested, NetEqualizer’s unique “behavior shaping” technology gives priority to latency sensitive applications, such as VoIP and email. It does it all dynamically and automatically, improving on other available bandwidth shaping technology. It controls network flow for the best WAN optimization.

APconnections is a privately held company founded in 2003 and is based in Lafayette, Colorado.

Full Article

The Pros and Cons of Bonded DSL and Load Balancing Multiple WAN links


Editor’s Note:We often get asked if our NetEqualizer bandwidth shapers can do load balancing. The answer is yes -maybe if we wanted to integrate into one of the public domain load balancing devices freely available. It seems that to do it correctly without issues is extremely expensive. 

In the following excerpt, we have reprinted some thoughts and experience from a user who has a wide breadth of knowledge in this area.  He gives detailed examples of the trade-offs involved in bonding multiple WAN connections.

When bonding is done by your provider, it is essentially seamless and requires no extra effort (or risks to the customer). It is normally done using bonded T1 links, but also can come in the form of a bonded DSL. The technology discussed below is applicable to users who are bonding two or more lines together without the knowledge (or help) of their upstream provider.

As for Linux freeware Load Balancing devices, they are NOT any sort of true bonding at all.  If you have 3 x 1.5 Mbit lines, then you do NOT have a 4.5 Mbit line with these products. If you really want a 4.5Mbit Bonded line, then I’m not aware of any way to do it without having BGP or some method of coordinating with someone upstream on the other side of the link.  However, what a multi-WAN-router will do is try to equally spread sessions out over the three lines, so that if your users are collectively doing 3Mbit of collective downloads, that should be about 1Mbit on each line. For the most part, it does a pretty good job.

It does this by using fairly dumb round-robin NATing.  So, it’s much like a regular NAT router – everyone behind it is a private 192.168 number (which is the 1st downside) – and it will NAT the privates to one of the 3 Public IP’s on the WAN ports. The side effect of that is broken sessions, where some websites (particularly SSL) will complain that your IP address has changed, for example, while you’re inside the shopping cart or whatever.

To counteract that problem, they have ‘session persistence’ which tries to track each ‘Session Pair’ and keep the same WAN IP in effect for that ‘Session Pair’. That means that the 1st time one of the private IP:port accesses some particular public ip:port, the router will remember that and use that same WAN port for that same public/private pair. The result of this is that ‘most’ of the time, we don’t have these broken sessions, but the downside of this is that the fairness of the load balancing is offset.

For example, if you had 2 lines connected:

  • User1 comes to speakeasy and does a speedtest – the router says ‘speakeasy is out WAN1 forevermore’.
  • User2 comes and looks up google, and the router says ‘google is out WAN2 forevermore’
  • User3 goes to Download.com and the router decides ‘Download.com is on WAN1′.
  • User4 goes to smalltextsite.com (WAN2)
  • User5 goes to YouTube (WAN1)

And so on. With session persistence turned on, User300 will get SpeakEasy, Download.com and YouTube across WAN1 because that’s what it originally learned to be persistent about.

So, the tradeoff is if you don’t use the session persistence, then you’ll have angry customers because things break. If you do use persistence, then there may be an unbalancing.

Also, there are still some broken sites, even with persistence on. For example, some online stores have the customer shopping at www.StoreSite.com and when they checkout it transfers their cart contents to www.PaymentProcessor.com, which may flag an IP security violation. Any time the router sees different IP’s out in the public side, it figures it can use a new WAN port and doesn’t know it’s the same user and application. There are a few game launchers that kids load a ‘launcher’ program and select a server to connect to, but when they actually click ‘connect’, the server complains because the WAN addresses have changed.

In all honesty, it works quite well and there are few problems. We also can make our own exception list, so in my shopping cart example, we can manually add ‘storesite.com‘ and ‘paymentprocessor.com‘ to the same WAN address and that will ensure that it always uses the same WAN for those sites. This requires that users complain first before you would even know that there is a problem, AND also requires some tricks to figure out what’s going on.  However, the exception list can ultimately handle these problems if you make enough exceptions.

Created by APconnections, the NetEqualizer is a plug-and-play bandwidth control and WAN/Internet optimization appliance that is flexible and scalable. When the network is congested, NetEqualizer’s unique “behavior shaping” technology dynamically and automatically gives priority to latency-sensitive applications, such as VoIP and email. Click here to request a full price list.

Additional articles

How to inexpensively increase internet bandwidth by bonding cable and dsl.

From BusinessPhoneNews.com a great guide to access bandwidth needs, Bandwidth Management Buyers Guide.

Finally a Bandwidth Control appliance for under $1500


Lafayette Colorado April 9th 2009

APconnections today Announced a small business bandwidth control device that  lists at $1499. (for single unit orders)

This new offer  handles up to 10 megabits and 100 users with room to spare for some expansion. It comes complete with all the standard features of the NetEqualizer, but in a smaller  low power format  with Power over Ethernet.

Demand for this new offer came from two sources

1) There was huge demand for an affordable traffic shaping device to  help small business run their VOIP concurrent with their data traffic over their internet link.

2) There was also a need  for a low end unit, with POE,  for the WISP market .

In  a large wireless network, congestion often occurs at tower locations.  With a low cost POE version of the NetEqualizer,  wireless providers can  now afford to have advanced bandwidth control at or near their Access distribution points.

According to Joe DeSopo from NetEqualizer, “About half of wireless network slowness comes from p2p (bittorrent)  and video users overloading the access points. We have had great success with our  NE2000 series  but the price point of $2500 was a bit too high to duplicate all over the network.”

For a small or medium sized office with a hosted VOIP PBX solution the NetEqualizer works like a genie in a bottle. It is one of the few products on the market that can provide QOS for voip over an Internet link. And now, with volume pricing approaching $1000,  it will help revolutions the way offices use their Internet connection.

The NetEqualizer is a plug-and-play bandwidth control and WAN/Internet optimization appliance that is flexible and scalable. When the network is congested, NetEqualizer’s unique “behavior shaping” technology gives priority to latency-sensitive applications, such as VoIP and email. It does it all dynamically and automatically, improving on other available bandwidth shaping technology. It controls network flow for the best WAN optimization.

APconnections is a privately held company founded in 2003 and is based in Lafayette, Colorado.

Related Articles

Cisco Bandwidth Control for Education Networks


The Cisco method is outlined below. However, you might also want to check out the NetEqualizer video filmed in front of the IT staffs at Eastern Michigan and Western Michigan Universities for a perspective on a simple alternate philosophy.

There is quite a bit of history with traffic classification  in the higher-ed market, so you can research some of the pros and cons of Layer 7 shaping before investing. You might also find some of these higher ed testimonials on the NetEqualizer worth reading.

The following was pulled from Cisco  marketing material specific to their bandwidth control solution for educational networks:

A fundamental requirement of any bandwidth control solution is the ability to apply QoS mechanisms. These mechanisms control the bandwidth of specific users and prioritize traffic to help ensure appropriate handling of delay-sensitive applications. QoS capabilities are essential for carrying delay-sensitive IP voice and video traffic over an institution’s ISP link, as well as for rate limiting recreational P2P traffic.
The Cisco SCE uses three levels of QoS:

Hierarchical bandwidth control: The Cisco SCE supports granular bandwidth control by allocating part of a link’s bandwidth for groups of specific application flows. Academic IT departments can define these groups according to categories such as “all P2P traffic,” “browsing and streaming traffic,” “all traffic flowing off net,” and so on. In addition, colleges and universities can use the Cisco SCE to enforce minimum and maximum bandwidth limits and priorities for the total traffic that is produced by a given user, as well as for the specific applications (browsing, gaming, and so on) in which the user engages. These advanced mechanisms are used in a tiered fashion.

Differentiated Services (DiffServ) queuing: Internet applications use DiffServ to help ensure that packets from delay-sensitive applications are prioritized over other packets. The Cisco SCE includes DiffServ-compliant transmit queues using “Best Effort Forwarding,” four levels of “Assured Forwarding,” and “Expedited Forwarding” for delay-sensitive applications.

DiffServ marking:  The Cisco SCE’s advanced classification capabilities can also be used for marking the IP type of service (ToS)/DiffServ codepoint (DSCP) byte of the associated traffic. Each flow or group of flows can be marked with a relevant DiffServ value based on the application or service. The next-hop Layer 3 device, such as a switch or router, then uses this marking to carry the delay-sensitive traffic appropriately. As a result, the Cisco SCE, crucial to the Cisco Bandwidth Control Solution, can serve as the ideal network element for classifying and marking application traffic for other DiffServ-enabled network elements.

Canadians request comments on traffic shaping practices


Art Reisman CTO www.netequalizer.com

I am not sure if this is open to Canadians only, but the CRTC (the Canadian equivalent of the FCC) has set up a site for comments regarding their policies on Internet traffic shaping. The site is open from now till April 30th and can be found at

http://isppractices.econsultation.ca/

So if you get the chance chime in and give them your thoughts.

For the fun of it (see below) I grabbed a few of the existing comments truely at random. After reading them it is funny how the consumer sentiments so far are in total agreement with what we NetEqualizer have been proselytizing  which is:  “Traffic management is fine as long as there is full disclosure of policies”. Nobody wants to pump gas without knowing the grade and the price and the same goes for their Internet service.

——————-comments—————————————————-

“Any traffic management practices deviating from complete network neutrality, that is to say, any practices that single out one protocol over another, should certainly be disclosed to the user in the service agreement. To disclose anything less would be consumer fraud.”

“Traffic management has a real impact on the product that a consumer is paying for. All ISPs are not created equal and consumers aren’t in a position to analyze the complexities of network management and the possible impacts on their usage.”

“All traffic shaping practices should be disclosed, in plain English, online and as a part of the terms of service.”

“I agree with the other posters thus far — if ISPs are allowed to get away with uncompetitive throttling of Internet traffic, those techniques and the effect on the customer should be fully disclosed in plain versions of both official languages.”

“Any new communication technologies can be thwarted if ISPs deem them to be competitive with any of their services, stifling innovation. Even the CBC has used BitTorrent to distribute programming, and..”

5 Tips to speed up your business T1/DS3 to the Internet


By Art Reisman

Art Reisman CTO www.netequalizer.com

In tight times expanding your corporate Internet pipe is a hard pill to swallow, especially when your instincts tell you the core business should be able to live within the current allotment.

Here are some tips and hard facts that you  you may want to consider  to help stretch your business Internet pipe

1) Layer 7 application shaping.

The market place is crawling with solutions that allow you to set policies on bandwidth based on type of application.  Application shaping allows an administrator to restrict lower priority activities, while allowing mission critical Apps favorable consideration. This methodology is very seductive , but from our experience it can send your IT department into a nanny state, constantly trying to figure out what to allow and what to restrict. Also the cost of an Internet link expansion is dropping, while many of the application shaping solutions start around $10,000 and go up from there.

The up side is Layer 7 application shaping does work well when it comes to internal WAN links that do not carry Internet traffic. An administrator can get a handle on the fixed traffic running privately within their network quite easily.

2) Using your router to restrict specific IP and ports

If your core business utilization can be isolated to a single server or group of servers a few simple rules to allocate a large chunk of the pipe to these resources (by IP address) may be a good fit.

In an environment where business priorities change and are not isolated to a fixed server or two, this solution can backfire, but if your resource allocation requirements are stable doing something on your router to restrict one particular subnet over another can be useful in stretching your bandwidth.

One thing to be careful is that it often takes a skilled technician to set up specialty rules on your router. You can easilyu rack  up  $$ to your IT consultants if  your set up is not static.

3) Behavior based shaping

Editors note: We are the makers of the NetEqualizer which specializes in this technology; however our intent in this article is to be objective.

Behavior based shaping works well and affordably in most situations. Most business related applications will get priority as they tend to use small amounts of data or web pages.  Occasionally there are exceptions that need to override the basic behavior based shaping such as video.  Video can easily  be excluded from the generic policies.  Implementing a few exclusions is far less cumbersome than trying to classify all traffic all the time such as with application shaping.

4) Add more bandwidth and by pass your local loop carrier

T1’s and T3’s from your local telco may not be the only options for bandwidth in your area. Many of our customers get creative by purchasing bandwidth directly from a tier one provider (such as Level 3) and then using a Microwave back haul the bandwidth to their location. The Telco’s make a killing with what they call a loop charge (before they put any bandwidth on your line) With Microwave backhaul technology you can by-pass this charge for significant savings.

5) Clean up the laptops and computers on your network.  Many robots and viruses run in the background on your windows machines and can generate a cacophony of back ground traffic.  A business wide license for good virus protection may be worth the investment.  Stay away from the free ware versions of virus protection they tend to miss quite a bit.