Obama’s Revival of Net Neutrality Revisits An Issue Hardly Forgotten

June 1, 2009 — netequalizer

Last Friday, President Obama reinvigorated (for many people, at least) the debate over net neutrality during a speech from the White House on cybersecurity. The president made it clear that users’ privacy and net neutrality would not be threatened under the guise of cybersecurity measures. President Obama stated:

“Let me also be clear about what we will not do. Our pursuit of cyber-security will not — I repeat, will not include — monitoring private sector networks or Internet traffic. We will preserve and protect the personal privacy and civil liberties that we cherish as Americans. Indeed, I remain firmly committed to net neutrality so we can keep the Internet as it should be — open and free.”

While this is certainly an important issue on the security front, for many ISPs and networks administrators, it didn’t take the president’s comments to put user privacy or net neutrality back in the spotlight.  In may cases, ISPs and network administrators constantly must walk the fine line between net neutrality, user privacy, and ultimately the well being of their own networks, something that can be compromised on a number of fronts (security, bandwidth, economics, etc.).

Therefore, despite the president’s on-going commitment to net neturality, the issue will continue to be debated and remain at the forefront of the minds of ISPs, administrators, and many users. Over the past few years, we at NetEqualizer have been working to provide a compromise for these interested parties, ensuring network quality and neutrality while protecting the privacy of users. It will be interesting to see how this debate plays out, and what it will mean for policy, as the philosophy of network neutrality continues to be challenged — both by individuals and network demands.

Further Reading

Posted in Bandwidth Monitoring, Commentary and Editorials, Net Neutrality, News. Tags: , , , , , , , , , , , , , . Leave a Comment »

New Speed Test Tools from M-Lab Expose ISP Bandwidth Throttling Practices

March 21, 2009 — netequalizer

In a recent article, we wrote about the “The White Lies ISPs tell about their bandwidth speeds“.  We even hinted at how they (your ISP)  might be inclined to give preferential treatment to normal speed test sites.  Well, now there is a speed test site from M-Lab that goes beyond simple speed tests. M-lab gives the consumer sophisticated results and exposes any tricks your ISP might be up to.

Features provided include:

  • Network Diagnostic Tool – Test your connection speed and receive sophisticated diagnosis of problems limiting speed.
  • Glasnost – Test whether BitTorrent is being blocked or throttled.
  • Network Path and Application Diagnosis – Diagnose common problems that impact last-mile broadband networks.
  • DiffProbe (coming soon) – Determine whether an ISP is giving some traffic a lower priority than other traffic.
  • NANO (coming soon) – Determine whether an ISP is degrading the performance of a certain subset of users, applications, or destinations.

Click here to learn more about M-Lab.

Related article on how to determine your true video speed over the Internet.

Posted in Bandwidth Monitoring, Network Speed, News. Tags: , , , , , . 5 Comments »

How to set up a computer for network monitoring – for free!

February 14, 2009 — netequalizer

By Art Reisman, CTO, http://www.netequalizer.com

Art Reisman CTO www.netequalizer.com

Art Reisman

Editors note:

We often get asked where to find a simple network monitoring tool.  Well, you can get more economical than this!   All you need is some elbow grease and perseverance.  Note: We are not the original authors of this idea and have adopted it to our blog, unfortunately I was unable to trace back to the original to give credit.

How to set up a computer for network monitoring – All for free!

This is not as hard as it looks, once you have done it a time or two you can be up and running in less than 5 minutes, assuming you have high-speed access to the Internet.

Do you have a computer with the following?

  • 2 network cards installed or two on-board LAN ports
  • 1 CDRom drive
  • 256 to 1,024 or more Meg of RAM
  • monitor
  • keyboard
  • mouse (optional — it is necessary if you boot into the graphics mode, though)


If so, you can be minutes away from having a network monitoring machine up and running that you can insert in your network and see what is going on.  If you follow these instructions it will act as a transparent bridge so no other machines or routers will know the difference.

The Knoppix CD is a live CD distribution which does not need a floppy or hard drive to run. It is all self contained on the CD. It uses your RAM as a read/writeable area so you can still install a few programs if need be and edit most of the configuration files.

You can get the Knoppix iso image from http://knopper.net/knoppix/ or the English version at http://knopper.net/knoppix/index-en.html

The download page for English reading when this article was written is at http://knopper.net/knoppix-mirrors/index-en.html

Download a CD image of 4.02 or better. A typical file name will be:

KNOPPIX_V4.0.2CD-2005-09-23-EN.iso

The filename ends in -EN and if you speak english then get that one. If you speak German, then get the -DE one.

Now burn that .iso file to a CD using your program of choice (burning the CD image to make a bootable CDRom is not covered here).

You should insert your machine into the network so it is between the Router and the switch, assuming you want to monitor traffic going from or to your network and the Internet. You may have to use a crossover cable from one of your machines LAN ports to the router and a standard network cable from the other LAN port on your machine and the switch.

Internet or Router or ???
_________
| |
| | eth0
_____| |_______________________
|     Monitoring Unit                          |
|_____________________________|
||
eth1  ||
_______________________||______
|     Internal Network Switch                |
|_______________________________|
|| || || || || || || || || || ||
Your internal network users or whatever you want to monitor

Once that is done you can run Knoppix by placing it into your machine and booting up. If you have a limited video card or an old monitor then you can hit the appropriate key when Knoppix boots up and find the option to boot into text mode only. You may also want to do this if you have limited RAM.

Once you are booted up and running Knoppix you can do the following:

If you booted to an XWindows look then click on the little computer screen icon next to the house on the bottom tool bar. If you booted into text mode you do not have to do that.

Now gain root access by typing the following and then Enter:

su

Now that you are root you can run the following commands to start up your transparent bridge and get traffic flowing through the machine from one lan port to the other. The IP 192.168.1.153 below was use as an example along with the default gateway being 192.168.1.1 so change those if your network is on a different IP range. You will want to give it an IP so that you can get into the machine from another machine on the network. In some cases you might want to be able to get to it from the Internet so in that case you would have to give it an IP that can be reached from the Internet and not a 10.x.x.x or 192.168.x.x number.

ifconfig br0 down
brctl delbr br0
ifconfig eth1 down
ifconfig eth0 down
ifconfig eth0 promisc 0.0.0.0 up
ifconfig eth1 promisc 0.0.0.0 up
brctl addbr br0
brctl addif br0 eth0
brctl addif br0 eth1
ifconfig br0 192.168.1.153 netmask 255.255.255.0 up
route add default gw 192.168.1.1

It may take a half a minute for traffic to start flowing through the transparent bridge br0.

Once you can do something like:

ping http://www.yahoo.com

and it comes back with ping times then you are ready to continue. BTW: hit CTRL C to stop ping.

apt-get update
apt-get install ntop

Say Y or hit enter to install ntop. When it is all done do the following:

mkdir /var/log/ntop/rrd
chmod -R 777 /var/lib/ntop
chmod -R 777 /var/log/ntop

warning: the chmod commands above allow anyone to read/write to those directories that can get to the machine so keep your machine safe with firewalls or passwords accordingly.

You can now run ntop. You need to run ntop from the console or via SSH first by just running the command:

ntop

It will ask you for an admin password and then again to,verify it. This is for the admin interface in ntop.

Once ntop is up and running in  a window you can leave that up and just go to your web browser and put in the URL of:

http://192.168.1.153:3000/

The :3000 is the special port that the ntop web server runs on.

If you choose the menu item Summary and then Network Load you should see a graph of your traffic. Not all ntop menu items are used on every system. Most of the time you will only be using the items under Summary or All Protocols.

You now have a running bandwidth monitoring system. ntop is the only application mentioned here but there are others installed on the default Knoppix CD already too.

Related post

The systems I have installed this routine on vary from a system with a Celeron to one with a P4 CPU Running on 10/100 Realtek chipsets to 10/100/1000 Intel chipsets. From 256 Meg of RAM to 2 GIG. Knoppix runs very well on a variety of hardware but your mileage may differ.

Created by APconnections, the NetEqualizer is a plug-and-play bandwidth control and WAN/Internet optimization appliance that is flexible and scalable. When the network is congested, NetEqualizer’s unique “behavior shaping” technology dynamically and automatically gives priority to latency sensitive applications, such as VoIP and email. Click here for a full price list.

Posted in Bandwidth Monitoring, Educational Articles. Tags: , , , , , . 1 Comment »

Do Internet Service Providers give home field advantage to their VOIP?

January 21, 2009 — netequalizer

By Art Reisman

Art Reisman CTO www.netequalizer.com

Editor’s note: Art Reisman is the CTO of APconnections. APconnections designs and manufactures the popular NetEqualizer bandwidth shaper.

The following article caught my attention this morning. Many of the ISPs that deploy our technology also provide their own VOIP service. Most have asked the question; can they make their in house VOIP offering work better than that offered by third parties such as Skype ? Fortunately, to date, we have taken the high road and talked them out of such a policy. We contend that protectionist strategies will eventually backfire. We have always proselytized if you have VOIP offering make sure it works well, price it well and your customers will stick with you.
Here is an excerpt from the Ars Technica article:

FCC wants to know if Comcast is interfering with VoIP

By Matthew Lasar | Published: January 19, 2009 – 10:25PM CT

Does Comcast give its own Internet phone service special treatment compared to VoIP competitors who use the ISP’s network? That’s basically the question that the Federal Communications Commission posed in a letter sent to the cable giant on Sunday.

Read on for the full article

Related Articles

The White lies ISPs tell about broadband speeds


Posted in Bandwidth Monitoring, News. Tags: , , , , , , . Leave a Comment »

Is Barack Obama going to turn the tide toward Net Neutrality ?

January 19, 2009 — netequalizer

NetWork World of Canada discusses some interesting scenarios about possible policy changes with the new adminstration.

In the article the author (Howard Solomon) specifically sites Obama’s leaning…

Meanwhile, the new President favours net neutrality, the principle that Internet service providers (ISPs) shouldn’t interfere with content traveling online, which could hurt Sandvine, a builder of deep packet inspection appliances for ISPs. At least one Senator is expected to introduce limiting legislation this month.

Will this help NetEqualizer sales and our support for behavior-based Net Neutral policy shaping?

According to Eli Riles vice president of sales at APconnections, “I don’t think it will change things much, we are already seeing steady growth, and I don’t expect a rush to purchase our equipment due to a government policy change. We sell mostly to Tier2 and Tier3 providers who have already generally stopped purchasing Layer 7 solutions mostly due to the higher cost and less so due to moral high ground or government mandate.”

related article

Stay tuned…

Posted in Bandwidth Monitoring, Net Neutrality, News. Tags: , , , , . Leave a Comment »

Network Access Control Module Screenshots

January 12, 2009 — netequalizer

Step 1

Step 1

Step 2

Step 2

Step 3

Step 3

Final Step

Final Step

Posted in Bandwidth Monitoring, NAC. Tags: , . Leave a Comment »

The True Cost of Bandwidth Monitoring

December 15, 2008 — netequalizer

By Art Reisman

Art Reisman CTO www.netequalizer.com

For most IT administrators, bandwidth monitoring of some sort is an essential part of keeping track of, as well as justifying, network expenses. Without visibility into a network load, an administrator’s job would degrade into a quagmire of random guesswork. Or would it?

The traditional way of  looking at monitoring your Internet has two parts: the fixed cost of the monitoring tool used to identify traffic, and the labor associated with devising a remedy. In an ironic inverse correlation, we assert that costs increase with the complexity of the monitoring tool. Obviously, the more detailed the reporting tool, the more expensive its initial price tag. The kicker comes with part two. The more expensive the tool, the more  detail  it will provide, and the more time an administrator is likely to spend adjusting and mucking, looking for optimal performance.

But, is it a fair to assume higher labor costs with  more advanced monitoring and information?

Well, obviously it would not make sense to pay more for an advanced tool if there was no intention of doing anything with the detailed information it provides. Why have the reporting tool in the first place if the only output was to stare at reports and do nothing? Typically, the more information an admin has about a network, the more inclined he might be to spend time making adjustments.

On a similar note, an oversight often made with labor costs is the belief  that when  the work needed to adjust the network comes to fruition, the associated adjustments can remain statically in place. However, in reality, network traffic changes constantly, and thus the tuning so meticulously performed on Monday may be obsolete by Friday.

Does this mean that the overall productivity of using a bandwidth tool is a loss? Not at all. Bandwidth monitoring and network mucking can certainly result in a cost-effective solution. But, where is the tipping point? When does a monitoring solution create more costs than it saves?

A review of recent history reveals that technologies with a path similar to bandwidth monitoring have become commodities and shunned the overhead of most human intervention.  For example, computer operators disappeared off the face of the earth with the invention of cheaper computing in the late 1980’s.  The function of a computer operator did not disappear completely, it just got automated and rolled into the computer itself. The point is, anytime the cost of a resource is falling, the attention and costs used to manage it should be revisited.

An effective compromise with many of our customers is that they are stepping down from expensive complex reporting tools to a simpler approach. Instead of trying to determine every type of traffic on a network by type, time of day, etc., an admin can spot trouble by simply checking overall usage numbers once a week or so. With a basic bandwidth control solution in place (such as a NetEqualizer), the acute problems of a network locking up will go away, leaving what we would call only “chronic” problems, which may need to be addressed eventually, but do not require immediate action.

For example, with a simple reporting tool you can plot network usage by user.  Such a report, although limited in detail, will often reveal a very distinct bell curve of usage behavior. Most users will be near the mean, and then there are perhaps one or two percent of users that will be well above the mean. You don’t need a fancy tool to see what they are doing; abuse becomes obvious just looking at the usage (a simple report).

However, there is also the personal control factor, which often does not follow clear lines of ROI (return on investment).

What we have experienced when proposing a more hands-off model to network management is that a customer’s comfort depends on their bias for needing to know, which is an unquantifiable personal preference. Even in a world where bandwidth is free, it is still human nature to want to know specifically what bandwidth is being used for, with detailed information regarding the type of traffic. There is nothing wrong with this desire, but we wonder how strong it might be if the savings obtained from using simpler monitoring tools were converted into a trip to Hawaii.

In our next article, we’ll put some real world numbers to the test for actual break downs, so stay tuned. In the mean time, here are some other articles on bandwidth monitoring that we recommend. And, don’t forget to take our poll.

Planetmy
Linux Tips
How to set up a monitor for free

Posted in Bandwidth Monitoring, Educational Articles, ROI. Tags: , , , , , , . 3 Comments »

Will the New UDP-based Bittorrent Thwart Traffic Shaping?

December 2, 2008 — netequalizer

A customer asked us today how the newer Bittorrent methods using UDP will affect our ability to keep traffic in check. Here is our first take on this subject (See the related article “Bittorrent declares war on VoIP, gamers”).

The change from TCP to UDP transfer will have some effect on our methods to throttle bandwidth, however, at
the IP level there is no difference between the two and we have never based our shaping techniques on whether packets were UDP or TCP. The ISP mentioned in the  article mentioned above likely uses TCP window-size manipulation to slow downloads. You can’t do that with UDP, and I think that is what the author was eluding to.

The only difference for the NetEqualizer will be that UDP streams are harder to knock down, so it may require a tuning change if it is really an issue. By this, I mean we may have to hit them harder with more latency than our standard defaults when throttling packets.

On a side note, we are seeing some interesting trends with regard to Bittorrent.

When looking at our customer networks, we are just not seeing the same levels of Bittorrent that we have seen in the past  (circa 2006).

We believe the drop is due to a couple of factors:

1)  The RIAA’s enforcement — The high school and university crowd has been sufficiently spanked with copyright prosecutions. Most people now think twice about downloading copyrighted material.

2) Legal alternatives — The popularity of online purchase music  sites has replaced some of the illegal transfers (These also take up bandwidth, but they are not distributed by bittorrent).

The recent trends do not mean that bittorrent is going away, but rather that viable alternatives are emerging.  However, while legal distribution of content is here to stay and will likely grow over time, we do not expect an explosion that will completely replace bittorrent.

Posted in Bandwidth Monitoring, Commentary and Editorials, HEOA/RIAA. Tags: , , , , , . 1 Comment »

NetEqualizer Offers Net Neutrality, User Privacy Compromise

August 12, 2008 — netequalizer

Although the debates surrounding net neutrality and user privacy are nothing new, the recent involvement of the Federal Communications Commission is forcing ISPs and network administrators to rethink their strategies for network optimization. The potential benefits of layer-7 bandwidth shaping and deep packet inspection are coming into conflict with the rights of Internet users to surf the net unimpeded while maintaining their privacy.

Despite the obvious potential relationship between net neutrality, deep packet inspection and bandwidth shaping, the issues are not inherently intertwined and must be judged separately. This has been the outlook at APconnections since the development of the network optimization appliance NetEqualizer five years ago.

On the surface, net neutrality seems to be a reasonable and ultimately beneficial goal for the Internet. In a perfect world, all consumers would be able to use the Internet to the extent they saw fit, absent of any bandwidth regulation. However, that perfect world does not exist.

In many cases, net neutrality can become a threat to equal access. Whether this is true for larger ISPs is debatable, however it cannot be denied when considering the circumstances surrounding smaller Internet providers. For example, administrators at rural ISPs, libraries, universities, and businesses often have no choice but to implement bandwidth shaping in order to ensure both reliable service and their own survival. When budgets allow only a certain amount of bandwidth to be purchased, once that supply is depleted, oftentimes due to the heavy usage of a small number of users, options are limited. Shaping in no longer a choice, but a necessity.

However, this does not mean that a free pass should be given for Internet providers to accomplish network optimization through any means available even at the expense of customer privacy. This is especially true considering that it’s possible to achieve network optimization without compromising privacy or equal access to the Internet. The NetEqualizer is a proven example.

Rather than relying on techniques such as deep packet inspection, NetEqualizer regulates bandwidth usage by connection limits and, through its fairness algorithm, ensures that all users are given equal access when the network is congested (Click here for a more detailed explanation of the NetEqualizer technology).

Therefore, a heavy bandwidth user that might be slowing Internet access for other customers can be kept in check without having to actually examine or completely block the data that is being sent. The end result is that the large majority of users will be able to access the Internet unhindered, while the privacy of all users is protected.

In the midst of the ongoing debates over net neutrality and privacy, the NetEqualizer approach is gaining popularity. This is apparent in both an increase in sales as well as on message boards and forums across the Internet. A recent Broadband Reports post reads:

“I don’t think anyone’s going to argue with you if you’re simply prioritizing real time traffic over non-real time. Just so long as you’re agnostic as to who’s sending the traffic, not making deals behind people’s backs, etc. then I’d have no problem with my ISP letting me surf the web or e-mail or stream at full speed, even if it meant that, when another person was doing the same, I could only get 100 KBs on a torrent instead of 150.

“I’d much rather have a NetEq’d open connection than a NATed nonmanaged one, that’s for sure.”

It is this agnostic approach that differentiates NetEqualizer from other network optimization appliances. While network administrators are able to prioritize applications such as VoIP in order to prevent latency, other activity, such as BitTorrent, is still able to take place – just at a slower speed when the network is congested. This is all done without deep packet inspection.

“NetEqualizer never opens up any customer data and thus cannot be accused of spying. Connections are treated as a metered resource,” said Art Reisman, CEO of APconnections. “The ISPs that use NetEqualizer simply put a policy in their service contracts stating how many connections they support, end of story. BitTorrent is still allowed to run, albeit not as wide with unlimited connections.”

Although not a proponent of bandwidth shaping, TorrentFreak.com editor-in-chief and founder Ernesto differentiates NetEqualizer from other bandwidth shaping appliances.

“I am not a fan of bandwidth control, the correct solution is for providers to build out more capacity by reinvesting their profits, however I’ll concede a solution such as a NetEqualizer is much more palatable than redirecting or specially blocking bittorrent and also seems to be more acceptable to consumers than bandwidth caps or metered plans.

“There is a risk though, who decides what the ‘peaks times’ are, how much bandwidth / connections would that be? Let me reiterate, I would rather see that ISPs invest in network capacity than network managing hardware.

“The Internet is growing rapidly, and if networks ‘crash’ already, they are clearly doing something wrong.”

The ultimate capacity of individual networks will vary on a case-by-case basis, with some having little choice but to employ bandwidth shaping and others doing so for reasons other than necessity. It has never been the intention of APconnections to pass judgment on how or why users implement shaping technology. The NetEqualizer is simply providing a bandwidth optimization alternative to deep packet inspection that gives administrators the opportunity to manage their networks with respect to both net neutrality and customer privacy.

Posted in Bandwidth Monitoring, Commentary and Editorials, Net Neutrality. Tags: , , , , , , , , . 11 Comments »

Curbing RIAA Requests on Your Student Network

August 11, 2008 — netequalizer

Editor’s Note: We often get asked by college administrators how the NetEqualizer can block p2p with our behavior-based rules. Since the NetEqualizer is containment based, it is effective in stopping approximately 80 to 90 percent of all p2p (see comparison with layer 7 shapers). Yet, questions and fears still remain about RIAA requests. Since the NetEqualizer is not a complete block, not that anything is, customers wonder how they can be safe from those intimidating lawyers.

In short, here’s the answer. The RIAA finds copyright violators by downloading files from your network. Since these downloads must be initiated from the outside, you simply need to block all outside initiated requests for data. Obviously you would still allow requests to your Web servers and other legitimate well known content servers on your network. Understanding this, administrators can configure their routers to work in conjunction with their NetEqualizers to largely curb RIAA requests.

Below, NetEqualizer user Ted Fines, the network administrator at Macalester College, shares his methods for preventing RIAA requests on his university network.

A few years ago, we implemented a rule on our firewall to improve our overall security. However, it has also had the added effect of stopping RIAA notices almost entirely.

The rule simply blocks all inbound connections to all ports on all residence hall computers. Here are some sample config lines from our firewall (aCisco PIX) that show how the rule works:

name 111.112.113.0 Kirk description Kirk Res Hall
object-group network Res_Halls
description All Residence Halls
network-object Kirk 255.255.255.0
network-object Bigelow 255.255.255.0
network-object Wallace 255.255.255.0
access-list 101 extended deny ip any object-group Res_Halls

Even though it may appear this rule would interfere with normal user Web browsing, etc., this rule actually has no effect at all on what systems the student computers in our residence halls may access. This is because the firewall tracks what computer initiates the connection.

For instance, when a student tries to access “http://www.cnn.com”, they are initiating the connection to CNN’s server. So when CNN’s server replies and send back news content, etc., the firewall knows that the student computer requested it and the incoming connection is allowed.

However, if a student is running a server, such as a Web server or a file sharing server, outside computers are not able to connect to it. The firewall knows that the outside computer is trying to initiate a connection, so it is blocked.

Our student body makes great use of our resources and we have a very open and unrestricted campus life, so I was pleasantly surprised that making this change did not ruffle any feathers. We do make exceptions when students request that a port be unblocked for a particular need. I have found that the ones who are savvy enough to know that they need a particular port opened are not typically the ones we have to be worried about, so we’re usually happy to accommodate them.

–Ted Fines, Macalester College, St. Paul, MN


Editor’s Note cont’d: This recent tip was given on the ResNet mailing list by Sidney Eaton of Ferris State University…

If you want to minimize your notices, just block these address ranges on your firewalls (in and out):

64.34.160.0/20

64.124.145.0/25

These are MediaSentry IP addresses (the company scanning your network to determine if your users are sharing copyprotected materials). They are not the only company hired by the RIAA and MPAA but they are the largest one. So you may still get some but hopefully not as many.

Sidney Eaton, Ferris State University, Big Rapids, MI

Posted in Bandwidth Monitoring, HEOA/RIAA, Support, Testimonials. Tags: , , . 1 Comment »

Failover and NetEqualizer: The Whys and Why Nots

April 9, 2008 — netequalizer

Do you want failover on your NetEqualizer or wondered why it’s not available? Let me share a story with you that has developed our philosophy on failover.

A long time ago, back in 1993 or so, I was the Unix and operating system point person for the popular AT&T (i.e. Lucent and Avaya) voice messaging product called Audix. It was my job to make sure that the Unix operating system was bug free and to trouble shoot any issues.

At the time, Audix sales accounted for about $300 million in business and included many Fortune 500 companies around the world. One of the features which I investigated, tested, and certified was our RAID technology. The data on our systems consisted of the archives of all those saved messages that were so important, even more so before e-mail became the standard.

I had a lab setup with all sorts of disk arrays and would routinely yank one from the rack while an Audix system was running. The RAID software we’d integrated worked flawlessly in every test. We were one of the largest companies in the world and we spared no expense to ensure quality in our equipment, and we also charged a premium for everything we sold. If the RAID line item feature was included with an Audix system, it could run as high as $100,000.

Flash forward to the future. We get a call that a customer has lost all their data. A RAID system had failed. It was a well-known insurance company in the Northeast. Needless to say, they were not pleased that their 100 K insurance policy against disk failure did not pan out.

I had certified this mechanism and stood behind it. So, I called together the RAID manufacturer and several Unix kernel experts to do a postmortem. After several days locked in a room, we found was that the real world failure did not follow the lab testing where we had pulled live disk drives in our lab. In fact, it failed in such a way as to slowly corrupt the customer data on all disk drives rendering it useless.

I did some follow up research on failover strategies over the years and discovered that many people implement them for political reasons to cover their asses. I do not mean to demean people covering their asses, it is an important part of business, but the problem is the real cost of testing and validating failover is not practical for most manufacturers.

Many customers ask, “If a NetEqualizer fails, will the LAN cards still pass data?” The answer is, we could certainly engineer our product this way, but there is no guarantee for fail safe systems.

Here are the pros and cons of such a technology:

1) Just like my disk drive failure experience, a system can fail many different ways and the failover mechanism is likely not foolproof. So, I don’t want to recreate history for something we cannot (nor can anybody) reliably real-world test.

2) NetEqualizer’s failure rate is about two percent over two years, which is mostly attributed to harsh operating conditions. That means you have a 1 in 50 chance of having a failure over a two-year period. Put simply, the odds are against this happening.

3) If a NetEqualizer fails, it is usually a matter of moving a cable, which can be easily fixed. So, if you, or anyone with access to the NetEqualizer, are within an hour of your facility, that means you have a 1 in 50 chance of your network being down for one hour every two years because of a NetEqualizer.

4) Customers that really need a fully redundant failover for their operation duplicate their entire infrastructure and purchase two NetEqualizers. These customers are typically brokerage houses where large revenue could be lost. Since they already have a fully tested strategy at the macro level, a failover card on the NetEqualizer is not needed.

5) For customer that is just starting to dabble, they have gone to Cisco spanning tree protocol. Cisco has many years and billions of dollars invested in their switching technology and is rock solid.

6) Putting LAN failover cards in our product would likely raise our base price by about $1000. That would be a significant price increase for most customers, and one that would most likely not be worth paying for.

7) Most equipment failures are software or system related. We take pride in the fact that our boxes run forever and don’t lock up or need rebooting. A failover LAN card does not typically protect against system-type failures.

So, yes, we could sell our system as failsafe with a failover LAN card, but we would rather educate than exploit fears and misunderstandings. Hopefully we’ve accomplished that here.

Posted in Bandwidth Monitoring, Commentary and Editorials. Tags: , , , . Leave a Comment »

APconnections Field Guide to Contention Ratios

November 26, 2007 — netequalizer

In a recent article titled “The White Lies ISPs Tell about Broadband Speeds,” we discussed some of the methods ISPs use when overselling their bandwidth in order to put on their best face for their customers. To recap a bit, oversold bandwidth is a condition that occurs when an ISP promises more bandwidth to its users than it can actually deliver. Since the act of “overselling” is a relative term, with some ISPs pushing the limit to greater extremes than others, we thought it a good idea to do a quick follow-up and define some parameters for measuring the oversold condition. 

For this purpose we use the term contention ratio. A contention ratio is simply the size of an Internet trunk divided by the number of users. We normally think of Internet trunks in units of megabits. For example, 10 users sharing a one megabit trunk would have a 10-to- 1 contention ratio. If sharing the bandwidth on the trunk equally and simultaneously, each user could sustain a constant feed of 100kbs, which is exactly 1/10 of the overall bandwidth.

So what is an acceptable contention ratio?

From a business standpoint, it is whatever a customer will put up with and pay for without canceling their service. This definition may seem ethically suspect, but whether in the bygone days of telecommunications phone service or contemporary Internet bandwidth business, there are long-standing precedents for overselling. What do you think a circuit busy signal is caused by? Or a dropped cell phone call? It’s best to leave the moral debate to a university assignment or a Sunday sermon.

So, without pulling any punches, what exactly will a customer tolerate before pulling the plug?
Here are some basic observations:
  • Rural customers in the US and Canada: Contention ratios of 50 to 1 are common
  • International customers in remote areas of the world: Contention ratios of 80 to 1 are common
  • Internet providers in urban areas: Contention ratios of 20 to 1 are to be expected

    • The numbers above are a good, rough starting point, but things are not as simple as they look. There is a statistical twist as bandwidth amounts get higher.

    Contention ratios can actually increase as the overall Internet trunk size gets larger. For example, if 50 people can share one megabit without mutiny, it should follow that 100 people can share two megabits without mutiny as the ratio has not changed. It is still 50 to 1.

    However, from observations of hundreds of ISPs, we can easily conclude that perhaps 110 people can share two megabits with the same tolerance as 50 people sharing one megabit. What this means is that the larger the ISP, the more bandwidth at a fixed cost per megabit, and thus the larger the contention ratios you can get away with.

    Is this really true? And if so, what are its implications for your business?

    This is simply an empirical observation, backed up by talking to literally thousands of ISPs over the course of four years and noticing how their oversubscription ratios increase with the size of their trunk.

    A conservative estimate is that, starting with the baseline ratio listed above, you can safely add 10 percent more subscribers above and beyond the original contention ratio for each megabit of trunk they share.

    Thus, to provide an illustration, 50 people sharing one megabit can safely be increased to 110 people sharing two megabits, and at four megabits you can easily handle 280 customers. With this understanding, getting more from your bandwidth becomes that much easier.

    Posted in Bandwidth Management, Bandwidth Monitoring, Educational Articles, Network Capacity. 12 Comments »

    CALEA Update

    May 2, 2007 — netequalizer

    CALEAAs promised, NetEqualizer is now offering the utilities necessary to meet requirements set forth this month by CALEA, or the Communications Assistance for Law Enforcement Act. This law oversees telecommunication security and has now been expanded to Internet security. There are some fairly harsh federal penalties for noncompliance that became effective May 1.

    In the spirit of protecting our nation, the mission is not to make life miserable and expensive for operators and thwart communications, but rather to give the FBI and homeland security tools to wire tap (if we can borrow the term) Internet conversation on a moment’s notice. We suspect it would be a rare occurrence for a small WISP to receive a warrant to comply, but it would be potentially devastating to security should the means to monitor conversation not be available.

    The following updated Q&A will address NetEqualizer’s capabilities in reference to CALEA compliance.

    1. Functionally, what does the Netequalizer CALEA release provide?

    We provide a network probe with the following capabilities:

    • It will allow an ISP or other operator to comply with a basic warrant for information about a user by capturing and sending IP communications in real time to a third party.
    • Communication may be captured by headers or headers and content.

    2. In what format is the data portion sent to a law enforcement agency?

    We will provide basic descriptive tags identifying headers, data, and time stamps, along with HEX or ASCII representation of content data.

    3. Do you meet the standards of the receiving law enforcement agency?

    The law and specifications on “how” to deliver to a law enforcement agency are somewhat ambiguous. The FBI has created some detailed specifications, but the reality is that there are some 40,000 law enforcement agencies and they are given autonomy on how they receive data. We do provide samples on how to receive NetEqualizer-captured data on a third party server, but are unable to guarantee definite compliance with any specific agency.

    4. Does the NetEqualizer do any analysis of the data?

    No. We are only providing a probe function.

    5. Is the NetEqualizer release fully CALEA compliant?

    Although the law (see CALEA sections 103 and 107(a)(2)) is fairly specific on what needs to be done, the how is not addressed to any level of detail to which we can engineer our solution. Many people are following the ATIS specification which was put forth by the FBI, and we have read and attempted to comply with the probe portion of that specification. But, the reality is that there is no one agency given the authority to test a solution and bless it as compliant. So, if faced with a warrant for information, the law enforcement agency in charge may indeed want something in slightly different formats. If this is the case, there may be additional consulting.

    As best we can tell at this time, there is no one government agency that can fully declare our technology CALEA compliant. However, we do pledge to work with our customers should they be faced with a warrant for information to adjust and even customize our solution; however additional fees may apply.

    For more information on NetEqualizer and CALEA, visit our extended Q&A page at http://www.netequalizer.com/caleafaq.php. Additional information on CALEA itself can be found at http://www.askcalea.org.

    Posted in Bandwidth Monitoring, CALEA, New Features, Support. Tags: , , . Leave a Comment »

    NetEqualizer and CALEA: A Short Q&A

    March 15, 2007 — netequalizer

    What is CALEA?

    CALEA, or the Communications Assistance for Law Enforcement Act, is the law that oversees telecommunication security which has now been expanded to Internet security. The FBI has been working to specify what is expected of wired and wireless ISPs, which has yet to be released in final form. There are some fairly harsh federal penalties for noncompliance that become effective in May 2007 (the stick). In the spirit of protecting our nation, the mission is not to make life miserable and expensive for operators and thwart communications, but rather to give the FBI and homeland security tools to wire tap (if I can borrow the term) Internet conversation on a moment’s notice.  I suspect it would be a rare occurrence for a small WISP to receive a warrant to comply, but it would be potentially devastating to security should the means to monitor conversation not be available. In the words of a consultant working for CALEA and the FBI, here is the verbatim minimal requirement as we obtained via e-mail in order to determine our obligations as a Network Tool supplier.

    Norm wrote:

    “Basically, an interception warrant would need to isolate and capture all communications to or from the subject of the warrant.  The warrant could specify that only header information is to be provided (i.e., a Pen Register/Trap and Tracee) or that header information and communications content should both be provided. “The Packet Technologies and Services Committee (PTSC) has developed standard ATIS-1000013.2007 for CALEA compliance for landline ISPs (including WiFi and WiMAX). Unfortunately, ATIS has not yet posted the standard on its web site (www.atis.org).”

    Our promise to our customers will be to provide a minimal compliance utility on our NetEqualizer Platform and support these utilities without adding additional cost to the product, if possible.

    Below is a Q&A regarding our plans.

    When will the NetEqualizer CALEA compliance module be available?

    We will have a “best effort” unit available for trial as of May 1. We caveat this as best effort because there may be some lag time to comply exactly with the requirement once the requirement is finalized and posted. However, there is enough information right now to get close to compliance, which is what we plan to do.

    Will there be any additional cost?

    At this time all customers with current NSS (software upgrade licenses) will not be charged. The NSS license for one year runs approximately 10 percent of the purchase cost of a new unit. Typically this would be in the $200 to $300 range.

    Will the CALEA module ship with newly purchased units?

    Yes, in fact any units purchased after March 20 will be eligible to receive the upgrade at no extra cost.

    Will the upgrade cost for the CALEA module always remain the same?

    We cannot promise a fixed price for future upgrades. If the complexity of this feature gets “out of hand,” we may have to label a “nonstandard” upgrade and charge, essentially making it a new product rather than an upgrade and charge accordingly.At this time our plans are to keep it as a standard upgrade.

    Will the standard NetEqualizer feature and the CALEA utility run on the same hardware at the same time?

    Due to the sensitive nature of the information should a warrant be requested for a tap, we have decided it would be best to focus on getting the stream to the federal agency. For this reason, the NetEqualizer will fall back to standard bridge mode. Obviously this may slow or degrade service to all customers, however this will be a rare event if ever and we’d rather do it this way than force customers to purchase an all new standalone appliance.

    Additional Questions… If you have any questions please, contact us at support@apconnections.net or 1-888-287-2492. For additional information on CALEA, visit www.askcalea.net.

    Posted in Bandwidth Monitoring, CALEA, Educational Articles, New Features. Tags: , . Leave a Comment »
    Newer posts »