QoS Over The Internet – Is it possible? Five Must-Know Facts

August 29, 2010 — netequalizer

I had an inquiry from a potential customer yesterday asking if we could monitor their QoS. I was a bit miffed as to what to tell them. At first, the question struck me as if they’d asked if we can monitor electrons on their power grid. In other words, it was a legitimate question in a sense, but of what use would it be to monitor QoS? I then asked him why he had implemented QoS in the first place. How did he know he needed it?

After inquiring a bit deeper, I also found out this customer was using extensive VPNs to remote offices over DSL internet circuits. His WAN traffic from the remote offices was sharing links with regular Internet data traffic, and all of it was traversing the public Internet. Then it hit me – he did not realize his QoS mechanisms were useless outside of his internal network.

Where there is one customer with confusion there are usually others. Hence, I’ve put together a quick fact sheet on QoS over an Internet link. Below, you’ll find five quick facts that should help clarify QoS and answer the primary question of “is it possible over the Internet?”.

Fact #1

If your QoS mechanism involves modifying packets with special instructions (ToS bits) on how it should be treated, it will only work on links where you control both ends of the circuit and everything in between.

Fact #2

Most Internet congestion is caused by incoming traffic. For data originating at your facility, you can certainly have your local router give priority to it on its way out, but you cannot set QoS bits on traffic coming into your network (We assume from a third party). Regulating outgoing traffic with ToS bits will not have any effect on incoming traffic.

Fact #3

Your public Internet provider will not treat ToS bits with any form of priority (The exception would be a contracted MPLS type network). Yes, they could, but if they did then everybody would game the system to get an advantage and they would not have much meaning anyway.

Fact #4

The next two facts address our initial question — Is QoS over the Internet possible? The answer is, yes, QoS on an Internet link is possible. We have spent the better part of seven years practicing this art form and it is not rocket science, but it does require a philosophical shift in thinking to get your arms around it.

We call it “equalizing,” or behavior-based shaping, and it involves monitoring incoming and outgoing streams on your Internet link. Priority or QoS is nothing more than favoring one stream’s packets over another stream’s. You can accomplish priority QoS on incoming streams by queuing (slowing down) one stream over another without relying on ToS bits.

Fact #5

Surprisingly, behavior-based methods such as those used by our NetEqualizer do provide a level QoS for VoIP on the public Internet. Although you can’t tell the Internet to send your VoIP packets faster, most people don’t realize the problem with congested VoIP is due to the fact that their VoIP packets are getting crowded out by large downloads. Often, the offending downloads are initiated by their own employees or users. A good behavior-based shaper will be able to favor VoIP streams over less essential data streams without any reliance on the sending party adhering to a QoS scheme.

For more information, check out Using NetEqualizer To Ensure Clean Clear VOIP.

Posted in Bandwidth Management, Educational Articles. Tags: qos internet, tos bits, video over internet, voip over internet. 22 Comments »

NetEqualizer News: August 2010

August 19, 2010 — netequalizer

August 2010

NetEqualizer News

NetEqualizer Lite Sale; Ireland Tech Seminar Details

Greetings!Enjoy another issue of the NetEqualizerNewsletter. This month, we kick off our two-week NetEqualizer Lite overstock sale and announce details for our upcoming Ireland seminar. As always, feel free to pass this along to others who might be interested in NetEqualizer or AirEqualizer news.In This Issue:

NetEqualizer Lite Overstock Sale
NetEqualizer To Be Featured At CCSA Conference
APconnections Is Coming To Ireland
We Want Your Feedback
Best Of The Blog

NetEqualizer Lite Overstock Sale

Over the next two weeks, we’ll be offering a 50-percent discount on the NetEqualizer Lite when you purchase one unit at full price.*Offering many of the same services available through other NetEqualizer models, the NetEqualizer Lite is a perfect entry level unit for administrators that don’t yet need the advanced capabilities of higher-level NetEqualizer models, such as NTOP reporting.Furthermore, the NetEqualizer Lite is Power-over-Ethernet (PoE), handling up to 10 megabits of traffic and 100 users. It offers a great solution for those remote links where you don’t need a full-power NetEqualizer. This is especially true if you have a hidden node issue where some customer signals are getting crowded out.For more information on the NetEqualizer Lite, visit the links below or contact us at 800-918-2763 or sales@apconnections.net.

NetEqualizer Lite Data Sheet
NetEqualizer Hidden Node White Paper

*Limit four units total per customer (two at full price and two at 50 percent off). Offer applies only to NetEqualizer Lite units and ends August 15, 2010.

NetEqualizer To Be Featured At CCSA Conference

On September 27, APconnections and the NetEqualizer will be featured at the 2010 Canadian Cable Systems Alliance (CCSA) Conference“Click” Trade Show in Niagara-on-the-Lake, Ontario.The trade show will be part of the 17th annual CCSA Conference and Annual Meeting being held from September 26-28 at the White Oaks Conference Resort and Spa.If you’re planning to attend the conference, or will just be in the area, stop by to say hello and pick up some of our giveaways.

APconnections Is Coming To Ireland

Will you be in Ireland or the UK this October? If so, be sure not to miss the NetEqualizer Technical Seminar and Hands-On Workshop at Dublin’s Burlington Hotelon October 4-5.As part of our growing presence in the UK and Ireland, the two-day seminar will be of value to both existing and potential NetEqualizer users.Under the guidance of APconnections CTO Art Reisman, the seminar will cover:

The various tradeoffs regarding how to stem P2P and bandwidth abuse
How to use reporting to spot abuse and troubleshoot your network
Lots of customer Q&A and information sharing on how clients are using the NetEqualizer, including some hands-on probing of a live system

The seminar will be offered in partnership with authorized NetEqualizer distributer Ai Bridges.

In Ireland, to register for the seminar or for more information, contact Kevin Hayes at Ai Bridges via email at khayes@aibridges.ie or by phone at +353 65 6848768. In the UK, contact APconnections at sales.uk@apconnections.net or at +44-2070992104 (Toll free – 0-808-101-3487).

We Want Your Feedback

Would you like to have a smart Web page policy security feature integrated within your NetEqualizer? If so, please let us know. We will be doing some beta integration testing soon with eSoft’s Web ThreatPak.Also, please let us know if you are already using their product and what your thoughts are.While there will be licensing charges for the eSoft technology, early integrators will get significant price breaks.To provide your feedback, contact us at 303-997-1300, extension 103, or at sales@apconnections.net.

Best Of The Blog

Seven Points To Consider When Planning Internet Redundancy

The chances of being killed by a shark are 1 in 264 million. Despite those low odds, most people worry about sharks when they enter the ocean, and yet the same people do not think twice about getting into a car without a passenger-side airbag.

And so it is with networking redundancy solutions. Many equipment purchase decisions are enhanced by an irrational fear (created by vendors) and not on actual business-risk mitigation.

The solution to this problem is simple. It’s a matter of being informed and making decisions based on facts rather than fear or emotion. While every situation is different, here a few basic tips and questions to consider when it comes to planning Internet redundancy.

1) Where is your largest risk of losing Internet connectivity?
Vendors tend to push customers toward internal hardware solutions to reduce risk. It has been our experience that your Internet router’s chance of catastrophic failure is about 1 percent over a three-year period. On the other hand, your internet provider has an almost 100-percent chance of having a full-day outage during that same three-year period.

2) Do not turn on unneeded bells and whistles on your router and firewall equipment.

To keep reading, click here.

Contact Information admin@apconnections.net phone: 303-997-1300 web: http://www.netequalizer.com

Posted in Newsletters. Tags: NetEqualizer news, newsletter. Leave a Comment »

Top Five Causes For Disruption Of Internet Service

August 17, 2010 — netequalizer

Editor’s Note: We took a poll from our customer base consisting of thousands of NetEqualizer users. What follows are the top five most common causes for disruption of Internet connectivity.

1) Congestion: Congestion is the most common cause for short Internet outages. In general, a congestion outage is characterized by 10 seconds of uptime followed by approximately 30 seconds of chaos. During the chaotic episode, the circuit gridlocks to the point where you can’t load a Web page. Just when you think the problem has cleared, it comes back.

The cyclical nature of a congestion outage is due to the way browsers and humans retry on failed connections. During busy times usage surges and then backs off, but the relief is temporary. Congestion-related outages are especially acute at public libraries, hotels, residence halls and educational institutions. Congestion is also very common on wireless networks. (Have you ever tried to send a text message from a crowded stadium? It’s usually impossible.)

Fortunately for network administrators, this is one cause of disruption that can be managed and prevented (as you’ll see below, others aren’t that easy to control). So what’s the solution? The best option for preventing congestion is to use some form of bandwidth control. The next best option is to increase the size of your bandwidth link. However without some form of bandwidth control, bandwidth increases are often absorbed quickly and congestion returns. For more information on speeding up internet services using a bandwidth controller, check out this article.

2) Failed Link to Provider: If you have a business-critical Internet link, it’s a good idea to source service from multiple providers. Between construction work, thunderstorms, wind, and power problems, anything can happen to your link at almost any time. These types of outages are much more likely than internal equipment failures.

3) Service Provider Internet Speed Fluctuates: Not all DS3 lines are the same. We have seen many occasions where customers are just not getting their contracted rate 24/7 as promised.

4) Equipment Failure: Power surges are the most common cause for frying routers and switches. Therefore, make sure everything has surge and UPS protection. After power surges, the next most common failure is lockup from feature-overloaded equipment. Considering this, keep your configurations as simple as possible on your routers and firewalls or be ready to upgrade to equipment with faster newer processing power.

5) Operator Error: Duplicating IP addresses, plugging wires into the wrong jack, and setting bad firewall rules are the leading operator errors reported.

If you commonly encounter issues that aren’t discussed here, feel free to fill us in in the comments section. While these were the most common causes of disruptions for our customers, plenty of other problems can exist.

Posted in Educational Articles, Network Redundancy, Support. Tags: bandwidth control, internet congestion, internet service outage, Netequalizer. 15 Comments »

Google Verizon Net Neutrality Policy, is it sincere?

August 11, 2010 — netequalizer

With all the rumors circulating about the larger wireless providers trying to wall off competition or generate extra revenue through preferential treatment of traffic, they had to do something, hence Google and Verizon crafted a joint statement on Net Neutrality. Making a statement in denial of a rumor on such a scale is somewhat akin to admitting the rumor was true. It reminds me of a politician claiming he has no plans to raise taxes.

Yes, I believe that most people who work for Google and Verizon, executives included, believe in an open Neutral Internet. And yet, from experience, when push comes to shove, and profits are flat or dropping, the idea of leveraging your assets will be on the table. And what better way to leverage your assets than restrict competition to your captive audience. Walling off a captive audience to selected content will always be enticing to any service provider looking for low hanging fruit. Morals can easily be compromised or rationalized in the face of losing your house, and it only takes one over zealous leader to start a provider down the slope.

The checks and balances so far, in this case, are the consumers who have voiced outright disgust with anybody who dare toy with the idea of preferential treatment of Internet traffic for economic benefit.

For now this concept will have to wait, but it will be revisited again and hopefully consumers will rise up in disgust. It would be naive to think that today’s statement by Verizon and Google would be binding beyond the political moment.

Posted in Commentary and Editorials, Net Neutrality. Tags: Net Neutrality. Leave a Comment »

Natural Address Translation FAQ

August 8, 2010 — netequalizer

By Art Reisman

Editor’s note: Art Reisman is the CTO of APconnections. APconnections designs and manufactures the popular NetEqualizer bandwidth shaper.

Editors Note: The official term for one public IP address mapped to multiple private IP addresses is PAT. However, most IP people use the terms interchangeably.

I was doing some internal research on natural address translation (NAT) this past week, and as I looked for reliable sources, I became a bit frustrated with the information available. Yes, the information is out there and the Wikipedia article has some nice charts with all the details. But, if you’re looking for the rational reasons behind NAT, you might want to shoot yourself in the head by the time you read through all of the information and find what you’re looking for.

To preserve your sanity, as well as answer some key questions quickly, I’ve put together the following Q&A detailing some key points when it comes to NAT. We’ll start with the basics and go from there.

What is NAT?

In order to allow multiple users to share a single IP address, modern routers utilize NAT to find unused port numbers and map them to a set of local private IP addresses. So, for example, let’s say your Internet provider gives you a single IP address for your household. It could be something like 98.245.90.60, which is a public IP address owned by Comast.

All of the computers in your house must share the single IP address that Comcast provides. So, your local router — the Linksys wireless router you bought for $79 — will use NAT to tag traffic with port numbers and then create some additional IP addresses right where your house connects to the Internet.

Let’s say you contacted the Microsoft website to download the latest service pack. When Microsoft sends you the download, it’s going to send it to 98.245.90.60:5001. “5001” is the port number established for the FTP transfer and 98.245.90.60 is the Comcast-owned Internet address for your entire house. Using NAT, your router will then interpret the port number and change the IP address to a unique internal address (like 192.168.1.103:8700, for example) before it gets to your computer.

Why do we need NAT?

NAT is useful because home users often have more than one computer in their household and yet only have a single IP address from their provider. Since every computer that talks on the Internet requires an IP address, it would not be possible to have more than one computer in your house without NAT.

How does NAT map a single IP address to multiple computers without things like Web browsing getting mixed up?

First, here’s some background on the difference between a base IP address and a port number. Internet addresses have two parts: an IP address, such as 98.243.90.60, and a port number. The IP address is used to route data across the Internet and the port is used by the receiving device — your computer — to determine what service to provide. For example, port 80 is the default port address for Web browsing.

Before the invention of NAT, Internet routers mostly ignored the port part of the address as they did not need it to move IP packets across the Internet. When describing the function of a port number, I like to use the analogy of a large dormitory with individual room numbers for the people living there. The postal service ignores the room numbers as their service ends at the address of the dormitory. They do not sort the mail by room number. For internet routers, port numbers are like room numbers. They deliver the packet to the end user’s computer and the port number is then interpreted.

The range of possible port numbers are in the tens of thousands, which is more than enough interpreting services by a user’s computer. Think of a dorm with 1,000 residents in which they would only need 1,000 numbers for mailboxes, but still had 1,000,000 reserved.

What happens if there are no free ports to do the translation?

On small home networks this is not likely to happen, but you can get conflicts if, for example, you try to use NAT on a network with tens of thousands of users. The total number of unique ports available is 65,000 and most users will require more than one port at a time.

Does NAT slow down my Internet connection?

Not enough for you to notice.

Why does my provider only allocate one IP address for my residence?

Even though there are about 4,000,000,000 (four billion) possible Internet addresses, the actual addresses are given out in large blocks, and once given out, they are hard to get back. So, and this is purely an example, let’s say a large company was given a class B set of addresses (which used to be common in the early days). They would have 64,000 addresses in their control. Hence, even with 4,000,000,000 possible addresses, they are in short supply, and your provider cannot afford to give them out more than one at a time.

Can I have more than one IP address?

Yes, but you would likely need a business class Internet service, which is generally quite a bit more expensive than residential-type service.

When will the world run out of IP addresses?

Some say we already have and there is a big push to go to a new standard called IPV6. However, we don’t think that will ever happen.

Editors Note: The official term for one public IP address mapped to multiple private IP addresses is PAT. However, most IP people use the terms interchangeably.

Posted in Educational Articles. Tags: NAT, natural address translation, Netequalizer, PAT, router address translation. 3 Comments »

Does Lower cost bandwidth foretell a decline in Expensive Packet Shapers ?

July 19, 2010 — netequalizer

This excerpt is from a recent interview with Art Reisman and has some good insight into the future of bandwidth control appliances.

Are you seeing a drop off in layer 7 bandwidth shapers in the marketplace?

In the early stages of the Internet, up until the early 2000s, the application signatures were not that complex and they were fairly easy to classify. Plus the cost of bandwidth was in some cases 10 times more expensive than 2010 prices. These two factors made the layer 7 solution a cost-effective idea. But over time, as bandwidth costs dropped, speeds got faster and the hardware and processing power in the layer 7 shapers actually rose. So, now in 2010 with much cheaper bandwidth, the layer 7 shaper market is less effective and more expensive. IT people still like the idea, but slowly over time price and performance is winning out. I don’t think the idea of a layer 7 shaper will ever go away because there are always new IT people coming into the market and they go through the same learning curve. There are also many WAN type installations that combine layer 7 with compression for an effective boost in throughput. But, even the business ROI for those installations is losing some luster as bandwidth costs drop.

So, how is the NetEqualizer doing in this tight market where bandwidth costs are dropping? Are customers just opting to toss their NetEqualizer in favor of adding more bandwidth?

There are some that do not need shaping at all, but then there are many customers that are moving from $50,000 solutions to our $10,000 solution as they add more bandwidth. At the lower price points, bandwidth shapers still make sense with respect to ROI. Even with lower bandwidth costs users will almost always clog the network with new more aggressive applications. You still need a way to gracefully stop them from consuming everything, and the NetEqualizer at our price point is a much more attractive solution.

Related article on Packeteers recent Decline in Revenue

Related article Layer 7 becoming obsolete from SSL

Posted in Commentary and Editorials. Tags: bandwidth costs, bandwidth shaper, blue coat, expensive packet shapers, Netequalizer, packet shaper, packet shaping. 2 Comments »

The Inside Scoop on Where the Market for Bandwidth Control Is Going

July 14, 2010 — netequalizer

Editor’s Note: The modern traffic shaper appeared in the market in the late 1990s. Since then market dynamics have changed significantly. Below we discuss these changes with industry pioneer and APconnections CTO Art Reisman.

Editor: Tell us how you got started in the bandwidth control business?

Back in 2002, after starting up a small ISP, my partners and I were looking for a tool that we could plug-in and take care of the resource contention without spending too much time on it. At the time, we had a T1 to share among about 100 residential users and it was costing us $1200 per month, so we had to do something.

Editor: So what did you come up with?

I consulted with my friends at Cisco on what they had. Quite a few of my peers from Bell Labs had migrated to Cisco on the coat tails of Kevin Kennedy, who was also from Bell Labs. After consulting with them and confirming there was nothing exactly turnkey at Cisco, we built the Linux Bandwidth Arbitrator (LBA) for ourselves.

How was the Linux Bandwidth Arbitrator distributed and what was the industry response?

We put out an early version for download on a site called Freshmeat. Most of the popular stuff on that site are home-user based utilities and tools for Linux. Given that the LBA was not really a consumer tool, it rose like a rocket on that site. We were getting thousands of downloads a month, and about 10 percent of those were installing it someplace.

What did you learn from the LBA project?

We eventually bundled layer 7 shaping into the LBA. At the time that was the biggest request for a feature. We loosely partnered with the Layer 7 project and a group at the Computer Science Department at the University of Colorado to perfect our layer 7 patterns and filter. Myself and some of the other engineers soon realized that layer 7 filtering, although cool and cutting edge, was a losing game with respect to time spent and costs. It was not impossible but in reality it was akin to trying to conquer all software viruses and only getting half of them. The viruses that remain will multiply and take over because they are the ones running loose. At the same time we were doing layer 7, the core idea of Equalizing, the way we did fairness allocation on the LBA, was s getting rave reviews.

What did you do next ?

We bundled the LBA into a CD for install and put a fledgling GUI interface on it. Many of the commercial users were happy to pay for the convenience, and from there we started catering to the commercial market and now here we are with modern version of the NetEqualizer.

How do you perceive the layer 7 market going forward?

Customers will always want layer 7 filtering. It is the first thing they think of from the CIO on down. It appeals almost instinctively to people. The ability to choose traffic by type of application and then prioritize it by type is quite appealing. It is as natural as ordering from a restaurant menu.

We are not the only ones declaring a decline in Deep packet inspection we found this opinion on another popular blog regarding bandwidth control:

The end is that while Deep Packet Inspection presentations include nifty graphs and seemingly exciting possibilities; it is only effective in streamlining tiny, very predictable networks. The basic concept is fundamentally flawed. The problem with generous networks is not that bandwidth wants to be shifted from “terrible” protocols to “excellent” protocols. The problem is volume. Volume must be managed in a way that maintains the strategic goals of the arrangement administration. Nearly always this can be achieved with a macro approach of allocating an honest share to each entity that uses the arrangement. Any attempt to micro-manage generous networks ordinarily makes them of poorer quality; or at least simply results in shifting bottlenecks from one business to another.

So why did you get away from layer 7 support in the NetEqualizer back in 2007?

When trying to contain an open Internet connection it does not work very well. The costs to implement were going up and up. The final straw was when encrypted p2p hit the cloud. Encrypted p2p cannot be specifically classified. It essentially tunnels through $50,000 investments in layer 7 shapers, rendering them impotent. Just because you can easily sell a technology does not make it right.

We are here for the long haul to educate customers. Most of our NetEqualizers stay in service as originally intended for years without licensing upgrades. Most expensive layer 7 shapers are mothballed after about 12 months are just scaled back to do simple reporting. Most products are driven by channel sales and the channel does not like to work very hard to educate customers with alternative technology. They (the channel) are interested in margins just as a bank likes to collect fees to increase profit. We, on the other hand, sell for the long haul on value and not just what we can turn quickly to customers because customers like what they see at first glance.

Are you seeing a drop off in layer 7 bandwidth shapers in the marketplace?

In the early stages of the Internet up until the early 2000s, the application signatures were not that complex and they were fairly easy to classify. Plus the cost of bandwidth was in some cases 10 times more expensive than 2010 prices. These two factors made the layer 7 solution a cost-effective idea. But over time, as bandwidth costs dropped, speeds got faster and the hardware and processing power in the layer 7 shapers actually rose. So, now in 2010 with much cheaper bandwidth, the layer 7 shaper market is less effective and more expensive. IT people still like the idea, but slowly over time price and performance is winning out. I don’t think the idea of a layer 7 shaper will ever go away because there are always new IT people coming into the market and they go through the same learning curve. There are also many WAN type installations that combine layer 7 with compression for an effective boost in throughput. But, even the business ROI for those installations is losing some luster as bandwidth costs drop.

So, how is the NetEqualizer doing in this tight market where bandwidth costs are dropping? Are customers just opting to toss their NetEqualizer in favor of adding more bandwidth?

There are some that do not need shaping at all, but then there are many customers that are moving from $50,000 solutions to our $10,000 solution as they add more bandwidth. At the lower price points, bandwidth shapers still make sense with respect to ROI. Even with lower bandwidth costs, users will almost always clog the network with new more aggressive applications. You still need a way to gracefully stop them from consuming everything, and the NetEqualizer at our price point is a much more attractive solution.

Posted in Commentary and Editorials. Tags: bandwidth control, layer 7 shaping, Netequalizer, packet shaping. 1 Comment »

Seven Points to Consider When Planning Internet Redundancy

July 9, 2010 — netequalizer

By Art Reisman

Editor’s note: Art Reisman is the CTO of APconnections. APconnections designs and manufactures the popular NetEqualizer bandwidth shaper.

And so it is with networking redundancy solutions. Many equipment purchase decisions are enhanced by an irrational fear (created by vendors) and not on actual business-risk mitigation.

1) Where is your largest risk of losing Internet connectivity?

Vendors tend to push customers toward internal hardware solutions to reduce risk. For example, most customers want a circuit design within their servers that will allow traffic to pass should the equipment fail. Yet our polling data of our customers shows that your Internet router’s chance of catastrophic failure is about 1 percent over a three-year period. On the other hand, your internet provider has an almost 100-percent chance of having a full-day outage during that same three-year period.

Perhaps the cost of sourcing two independent providers is prohibitive, and there is no choice but to live with this risk. All well and good, but if you are truly worried about a connectivity failure into your business, you cannot meaningfully mitigate this risk by sourcing hot failover equipment at your site. You MUST source two separate paths to the Internet to have any significant reduction in risk. Requiring failover on individual pieces of equipment, without complete redundancy in your network from your provider down, with all due respect, is a mitigation of political and not actual risk.

2) Do not turn on unneeded bells and whistles on your router and firewall equipment.

Many router and device failures are not absolute. Equipment will get cranky, slow, or belligerent based on human error or system bugs. Although system bugs are rare when these devices are used in the default set-up, it seems turning on bells and whistles is often an irresistible enticement for a tech. The more features you turn on, the less standard your configuration becomes, and all too often the mission of the device is pushed well beyond its original intent. Routers doing billing systems, for example.

These “soft” failure situations are common, and the fail-over mechanism likely will not kick in, even though the device is sick and not passing traffic as intended. I have witnessed this type of failure first-hand at major customer installations. The failure itself is bad enough, but the real embarrassment comes from having to tell your customer that the fail-over investment they purchased is useless in a real-life situation. Fail-over systems are designed with the idea that the equipment they route around will die and go belly up like a pheasant shot point-blank with a 12-gauge shotgun. In reality, for every “hard” failure, there are 100 system-related lock ups where equipment sputters and chokes but does not completely die.

3) Start with a high-quality Internet line.

T1 lines, although somewhat expensive, are based on telephone technology that has long been hardened and paid for. While they do cost a bit more than other solutions, they are well-engineered to your doorstep.

4) If possible, source two Internet providers and use BGP to combine them.

Since Internet providers are the usually weakest link in your connection, critical operations should consider this option first before looking to optimize other aspects of your internal circuit.

5) Make sure all your devices have good UPS sources and surge protectors.

6) What is the cost of manually moving a wire to bypass a failed piece of equipment?

Look at this option before purchasing redundancy options on single point of failure. We often see customers asking for redundant fail-over embedded in their equipment. This tends to be a strategy of purchasing hardware such as routers, firewalls, bandwidth shapers, and access points that provide a “fail open” (meaning traffic will still pass through the device) should they catastrophically fail. At face value, this seems like a good idea to cover your bases. Most of these devices embed a failover switch internally to their hardware. The cost of this technology can add about $3,000 to the price of the unit.

7) If equipment is vital to your operation, you’ll need a spare unit on hand in case of failure. If the equipment is optional or used occasionally, then take it out of your network.

Again, these are just some basic tips, and your final Internet redundancy plan will ultimately depend on your specific circumstances. But, these tips and questions should put you on your way to a decision based on facts rather than one based on unnecessary fears and concerns.

Posted in Bandwidth Monitoring, Educational Articles, Network Redundancy. Tags: fail over router, internet redundancy, internet reliability, Netequalizer, network management, router. 4 Comments »

NetEqualizer News: July 2010

July 8, 2010 — netequalizer

July 2010

NetEqualizer News

New NetEqualizer Features Released; Flyaway Contest Winner Announced

Greetings!

Enjoy another issue of the NetEqualizer Newsletter. This month, we introduce NetEqualizer version 4.7 and announce the most recent Flyaway Contest winner. As always, feel free to pass this along to others who might be interested in NetEqualizer or AirEqualizer news.

In This Issue:

NetEqualizer Version 4.7 Released
NetEqualizer… Made in America
NetEqualizer’s Presence in the UK and Ireland Growing
Best of the Blog
And the Flyaway Contest Winner Is…

NetEqualizer Version 4.7 Released

We’re pleased to announce that version 4.7 of the NetEqualizer is now available. This latest release will include several exciting new features, including bursting capabilities and instant bandwidth reporting. With the new bursting feature, you can now set up a rate limit on a particular customer’s IP, which will allow you to define:

How long a customer’s burst will last
How long a customer must wait between bursts
How much additional bandwidth a customer will get during their burst

Furthermore, with our new instant bandwidth reporting, you can actually measure a customer’s current bandwidth utilization by IP, VLAN or Subnet, allowing you to actually see the burst in action. This instant bandwidth reporting feature is an industry first.

Finally, we’ve also added advanced tuning capabilities for better accuracy on large pools and VLANs.

This release will be available at no additional charge to customers with current NetEqualizer Software Subscriptions (NSS). For more information, contact us at admin@apconnections.net or 1-800-918-2763.

NetEqualizer…Made In America

Did you know that NetEqualizer is made in America? Since switching to ASA Computers for our semi-custom manufactured servers in 2005, NetEqualizer has been completely U.S.-based and manufactured.

ASA, who provides full solutions including ‘ISP packages’, servers for ‘.com’ companies, computing environments for schools and research organizations, RAID and NAS set-ups, network installation, help-desk support, and much more, also makes equipment for the likes of Cisco, Microsoft and many Fortune 500 companies.

NetEqualizer’s Presence In The UK And Ireland Growing

Over the next several months, you’re likely going to see various announcements regarding our NetEqualizer partnerships and users in the UK and Ireland.

Why are we doing this? Because the EU is currently blanketed with overpriced bandwidth shaping options. As the leader in the United States when it comes to low-cost quality bandwidth control, it only makes sense for us to further extend our product availability in the UK and Ireland. Our existing customers in the area love the NetEqualizer line and have been encouraging us to increase our presence for some time.

On that note, stay tuned for more information on our upcoming Technical Seminar in Dublin this September that will be co-sponsored by Ireland’s Ai Bridges. We’ll make an announcement with the final details on the date, time and location next month.

For more information, see our latest press release on NetEqualizer’s availability in the UK and Ireland and our partnership with Ai Bridges.

Best Of The Blog

White Paper: A Simple Guide to Network Capacity Planning

After many years of consulting and supporting the networking world with WAN optimization devices, we have sensed a lingering fear among Network Administrators who wonder if their capacity is within the normal range.

So, the question remains: How much bandwidth can you survive with before you impact morale or productivity?

The formal term we use to describe the number of users sharing a network link to the Internet is contention ratio. This term is defined as the size of an Internet trunk divided by the number of users. We normally think of Internet trunks in units of megabits. For example, 10 users sharing a one megabit trunk would have a 10-to- 1 contention ratio. If sharing the bandwidth on the trunk equally and simultaneously, each user could sustain a constant feed of 100kbs, which is exactly 1/10 of the overall bandwidth.

So what is an acceptable contention ratio?

To keep reading, click here.

And The Flyaway Contest Winner Is…

Every few months, we have a drawing to give away two roundtrip domestic airline tickets from Frontier Airlines to one lucky person who’s recently tried out our online NetEqualizer demo.

The time has come to announce this round’s winner.

And the winner is…Javier Toro from Hablando Todos.

Congratulations, Javier! Please contact us within 30 days at admin@apconnections.net or 303-997-1300 to claim your prize.

Contact Information

admin@apconnections.net

phone: 303-997-1300

web: http://www.netequalizer.com

APconnections Partners

AiBridges
Candela Technologies
Double Radius
Dynamic Broadband
ExNet
Extensive Networks
FISPA
Grupo Imaginación CibernéticaPacificNet
Telefonía Pública y Privada S.A.
Tranzeo Wireless Technologies
Vox Solutions
ZCorum

Posted in Newsletters. Tags: NetEqualizer news, newsletter. 1 Comment »

What to expect from Internet Bursting

July 6, 2010 — netequalizer

APconnections will be releasing ( version 4.7) a bursting feature on their NetEqualizer bandwidth controller this week. What follows is an explanation of the feature and also some facts and information about Internet Bursting that consumers will also find useful.

First an explanation on how the NetEqualizer bursting feature works.

– The NetEqualizer currently comes with a feature that lets you set a rate limit by IP address.

– Prior to the bursting feature, the top speed allowed for each user was fixed at a set rate limit.

– Now with bursting a user can be allowed a burst of bandwidth for 10 seconds with speeds multiples of two , three or four, or any multiple of their base rate limit.

So if for example a user has a base rate limit of 2 megabits a second, and a burst factor of 4, then their connection will be allowed to burst all the way up to 8 megabits for 10 seconds, at which time it will revert back to the original 2 megabits per second. This type of burst will be noticed when loading large Web pages loaded with graphics. They will essentially fly up in the browser at warp speed.

In order to make bursting a “special” feature it obviously can’t be on all the time. For this reason the NetEqualizer by default, will force a user to wait 80 seconds before they can burst again.

Will bursting show up in speed tests?

With the default settings of 10 second bursts and an 80 second time out before the next burst it is unlikely a user will be able to see their full burst speed accurately with a speed test site.

How do you set a bursting feature for an IP address ?

From the GUI

Select

Add Rules->set hard limit

The last field in the command specifies the burst factor. Set this field to the multiple of the default speed you wish to burst up to.

Note: Once bursting has been set-up, bursting on an IP address will start when that IP exceeds its rate limit (across all connections for that IP). The burst applies to all connections across the IP address.

How do you turn the burst feature off for an IP address.

You must remove the Hard Limit on the IP address and then recreate the Hard Limit by IP without bursting defined.

From the Web GUI Main Menu, Click on ->Remove/Deactivate Rules

Select the appropriate Hard Limit from the drop-down box. Click on ->Remove Rule

To re-add the rule without bursting, from the Web GUI Main Menu, Click on ->Add Rules->Hard Limit by IP and leave the last field set to 1.

Can you change the global bursting defaults for duration of burst and time between bursts ?

Yes, from the GUI screen you can select

misc->run command

In the space provided you would run the following command

/usr/sbin/brctl setburstparams my 40 30

The first parameter is the time,in seconds, an IP must wait before it can burst again. If an IP has done a burst cycle it will be forced to wait this long in seconds before it can burst again.

The second parameter is the time, in seconds, an IP will be allowed to burst before begin relegated back to its default rate cap.

The global burst parameters are not persistent, meaning you will need to put a command in the start up file if you want them to stick between reboots.

/usr/sbin/brctl

If speed tests are not a good way to measure a burst, then what do you recommend?

The easiest way would be to extend the burst time to minutes (instead of the default 10 seconds ) and then run the speed test.

With the default set at 10 seconds the best was to see a burst in action is to take a continuous snap shot of an IP’s consumption during an extended download.

Beware of the confusion that bursting might cause.

Posted in Bandwidth Management, Network Speed, New Features. Tags: bandwidth control, bandwidth hard limits, internet bursting, internet rate limits. Leave a Comment »

Instant Bandwidth Snapshot Feature: Is this an Industry First?

June 30, 2010 — netequalizer

One of the things that we have noticed with reporting tools lately, including ntop (the reporting tool we integrate), is that there is no easy way to show instant bandwidth for a user. Most reporting tools smooth out usage over some time period, a 5 minute average is the norm.

For example, this popular Netflow Analyzer touts a 10 minute average, right from the FAQ on their main page it states:

“Real-time Bandwidth Reports for each WAN link

As soon as Netflow data is received, graphs are generated showing details on incoming and outgoing traffic on the link for the last 10 minutes.”

No where can we find a reasonable bandwitdh monitoring tool that will show you instant, as of this second, bandwidth utilization. We are sure somebody will e-mail us to dispute this claim, and if so, we will gladly publish their link and give them credit on our BLOG.

When is an Instant Bandwidth Reporting Tool useful?

1) The five minute average reporting tool is of little use when a customer calls and tells you they are not getting their expected bandwidth on a speed test or video. In these cases it is best to see the instant report while they are consuming the bandwidth, not averaged into a 10 minute aggregate.

2) If a customer has a fixed rate cap, and calls and reports that their VOIP is not working well. The easiest and quickest way is to check what their consumption is during a VOIP call is to see it now. You don’t need a fancy protocol analyzer to tell them they are sucking up their full 1 megabit allocation with their YouTube video specifically. You just need to know that their line is clear and that they are consuming the full megabit at this instant, thus exonerating you (the ISP or support person) from getting drawn down in the dregs of culpability.

Here are some links to other reporting tools.

http://www.javvin.com/packet.html

Ip guard

Spiceworks

Here is a snapshot of our screen that allows you take an Instant Bandwidth Snapshot, showing the last second of utilization for a individual IP, Pool, or VLAN on your network.

Posted in Bandwidth Monitoring, New Features. Tags: bandwidth monitoring, bandwidth reporting, instant bandwidth snapshot. 2 Comments »

Nine Tips and Technologies for Network WAN Optimization

June 16, 2010 — netequalizer

By Art Reisman

Editor’s note: Art Reisman is the CTO of APconnections. APconnections designs and manufactures the popular NetEqualizer bandwidth shaper.

Although there is no way to actually make your true WAN speed faster, here are some tips for corporate IT professionals that can make better use of the bandwidth you already have, thus providing the illusion of a faster pipe.

1) Caching — How does it work and is it a good idea?

Caching servers have built-in intelligence to store the most recently and most frequently requested information, thus preventing future requests from traversing a WAN/Internet link unnecessarily.

Caching servers keep a time stamp of their last update to data. If the page time stamp has not changed since the last time a user has accessed the page, the caching server will present a local stored copy of the Web page, saving the time it would take to load the page from across the Internet.

Caching on your WAN link in some instances can reduce traffic by 50 percent or more. For example, if your employees are making a run on the latest PDF explaining their benefits, without caching each access would traverse the WAN link to a central server duplicating the data across the link many times over. With caching, they will receive a local copy from the caching server.

What is the downside of caching?

There are two main issues that can arise with caching:

a) Keeping the cache current –If you access a cache page that is not current you are at risk of getting old and incorrect information. Some things you may never want to be cached. For example, the results of a transactional database query. It’s not that these problems are insurmountable, but there is always the risk the data in cache will not be synchronized with changes. I personally have been misled by old data from my cache on several occasions.

b) Volume – There are some 300 million websites on the Internet. Each site contains upwards of several megabytes of public information. The amount of data is staggering and even the smartest caching scheme cannot account for the variation in usage patterns among users and the likelihood they will hit an uncached page.

We recommend Squid as a proxy solution.

2) Protocol Spoofing

Historically, there have been client server applications developed for an internal LAN. Many of these applications are considered chatty. For example, to complete a transaction between a client and server, tens of messages may be transmitted when perhaps one or two would suffice. Everything was fine until companies, for logistical and other reasons, extended their LANs across the globe using WAN links to tie different locations together.

To get a better visual on what goes on in a chatty application perhaps an analogy will help. It’s like sending family members your summer vacation pictures, and, for some insane reason, putting each picture in a separate envelope and mailing them individually on the same mail run. Obviously, this would be extremely inefficient, just as chatty applications can be.

What protocol spoofing accomplishes is to “fake out” the client or server side of the transaction and then send a more compact version of the transaction over the Internet (i.e., put all the pictures in one envelope and send it on your behalf, thus saving you postage).

For more information, visit the Protocol Spoofing page at WANOptimization.org.

3) Compression

At first glance, the term compression seems intuitively obvious. Most people have at one time or another extracted a compressed Windows ZIP file. If you examine the file sizes pre- and post-extraction, it reveals there is more data on the hard drive after the extraction. Well, WAN compression products use some of the same principles, only they compress the data on the WAN link and decompress it automatically once delivered, thus saving space on the link, making the network more efficient. Even though you likely understand compression on a Windows file conceptually, it would be wise to understand what is really going on under the hood during compression before making an investment to reduce network costs. Here are two questions to consider.

a) How Does it Work? — A good and easy way to visualize data compression is comparing it to the use of short hand when taking dictation. By using a single symbol for common words a scribe can take written dictation much faster than if he were to spell out each word. The basic principle behind compression techniques is to use shortcuts to represent common data.

Commercial compression algorithms, although similar in principle, can vary widely in practice. Each company offering a solution typically has its own trade secrets that they closely guard for a competitive advantage. However, there are a few general rules common to all strategies. One technique is to encode a repeated character within a data file. For a simple example, let’s suppose we were compressing this very document and as a format separator we had a row with a solid dash.

The data for this solid dash line is comprised of approximately 160 times the ASCII character “-�. When transporting the document across a WAN link without compression, this line of document would require 80 bytes of data, but with clever compression, we can encode this using a special notation “-� X 160.

The compression device at the front end would read the 160 character line and realize,”Duh, this is stupid. Why send the same character 160 times in a row?” So, it would incorporate a special code to depict the data more efficiently.

Perhaps that was obvious, but it is important know a little bit about compression techniques to understand the limits of their effectiveness. There are many types of data that cannot be efficiently compressed.

For example, many image and voice recordings are already optimized and there is very little improvement in data size that can be accomplished with compression techniques. The companies that sell compression based solutions should be able to provide you with profiles on what to expect based on the type of data sent on your WAN link.

b) What are the downsides? — Compression always requires equipment at both ends of the link and results can be sporadic depending on the traffic type.

If you’re looking for compression vendors, we recommend FatPipe, Juniper Networks

4) Requesting Text Only from Browsers on Remote Links

Editors note: Although this may seem a bit archaic and backwoods, it can be effective in a pinch to keep a remote office up and running.

If you are stuck with a dial-up or slower WAN connection, have your users set their browsers to text-only mode. However, while this will speed up general browsing and e-mail, it will do nothing to speed up more bandwidth intensive activities like video conferencing. The reason why text only can be effective is that most Web pages are loaded with graphics which take up the bulk of the load time. If you’re desperate, switching to text-only will eliminate the graphics and save you quite a bit of time.

5) Application Shaping on Your WAN Link

Editor’s Note: Application shaping is appropriate for corporate IT administrators and is generally not a practical solution for a home user. Makers of application shapers include Packeteer and Allot and are typically out of the price range for many smaller networks and home users.

One of the most popular and intuitive forms of optimizing bandwidth is a method called “application shaping,” with aliases of “traffic shaping,” “bandwidth control,” and perhaps a few others thrown in for good measure. For the IT manager that is held accountable for everything that can and will go wrong on a network, or the CIO that needs to manage network usage policies, this is a dream come true. If you can divvy up portions of your WAN/Internet link to various applications, then you can take control of your network and ensure that important traffic has sufficient bandwidth.

At the center of application shaping is the ability to identify traffic by type. For example, identifying between Citrix traffic, streaming audio, Kazaa peer-to-peer, or something else. However, this approach is not without its drawbacks.

Here are a few common questions potential users of application shaping generally ask.

a) Can you control applications with just a firewall or do you need a special product? — Many applications are expected to use Internet ports when communicating across the Web. An Internet port is part of an Internet address, and many firewall products can easily identify ports and block or limit them. For example, the “FTP” application commonly used for downloading files uses the well known “port 21.”

The fallacy with this scheme, as many operators soon find out, is that there are many applications that do not consistently use a fixed port for communication. Many application writers have no desire to be easily classified. In fact, they don’t want IT personnel to block them at all, so they deliberately design applications to not conform to any formal port assignment scheme. For this reason, any product that aims to block or alter application flows by port should be avoided if your primary mission is to control applications by type.

b) So, if standard firewalls are inadequate at blocking applications by port, what can help?

As you are likely aware, all traffic on the Internet travels around in what is called an IP packet. An IP packet can very simply be thought of as a string of characters moving from Computer A to Computer B. The string of characters is called the “payload,” much like the freight inside a railroad car. On the outside of this payload, or data, is the address where it is being sent. These two elements, the address and the payload, comprise the complete IP packet.

In the case of different applications on the Internet, we would expect to see different kinds of payloads. For example, let’s take the example of a skyscraper being transported from New York to Los Angeles. How could this be done using a freight train? Common sense suggests that one would disassemble the office tower, stuff it into as many freight cars as it takes to transport it, and then when the train arrived in Los Angeles hopefully the workers on the other end would have the instructions on how to reassemble the tower.

Well, this analogy works with almost anything that is sent across the Internet, only the payload is some form of data, not a physical hunk of bricks, metal and wires. If we were sending a Word document as an e-mail attachment, guess what, the contents of the document would be disassembled into a bunch of IP packets and sent to the receiving e-mail client where it would be re-assembled. If I looked at the payload of each Internet packet in transit, I could actually see snippets of the document in each packet and could quite easily read the words as they went by.

At the heart of all current application shaping products is special software that examines the content of Internet packets, and through various pattern matching techniques, determines what type of application a particular flow is. Once a flow is determined, then the application shaping tool can enforce the operators policies on that flow. Some examples of policy are:

Limit Citrix traffic to 100kbs
Reserve 500kbs for Shoretel voice traffic

The list of rules you can apply to traffic types and flow is unlimited. However, there is a downside to application shaping of which you should be aware. Here are a few:

The number of applications on the Internet is a moving target. The best application shaping tools do a very good job of identifying several thousand of them, and yet there will always be some traffic that is unknown (estimated at 10 percent by experts from the leading manufacturers). The unknown traffic is lumped into the unknown classification and an operator must make a blanket decision on how to shape this class. Is it important? Is it not? Suppose the important traffic was streaming audio for a Web cast and is not classified. Well, you get the picture. Although theory behind application shaping by type is a noble one, the cost for a company to stay up to date is large and there are cracks.
Even if the application spectrum could be completely classified, the spectrum of applications constantly changes. You must keep licenses current to ensure you have the latest in detection capabilities. And even then it can be quite a task to constantly analyze and change the mix of policies on your network. As bandwidth costs lessen, how much human time should be spent divvying up and creating ever more complex policies to optimize your WAN traffic?

6) Test Your WAN-Link Speed

A common issues with slow WAN link service is that your provider is not giving you what they have advertised.

For more information, see The Real Meaning of Comcast Generosity.

7) Make Sure There Is No Interference on Your Wireless Point-to-Point WAN Link

If the signal between locations served by a point to point link are weak, the wireless equipment will automatically downgrade its service to a slower speed. We have seen this many times where a customer believes they have perhaps a 40-megabit backhaul link and perhaps are only realizing five megabits.

8) Deploy a Fairness Device to Smooth Out Those Rough Patches During Contentious Busy Hours

Yes, this is the NetEqualizer News Blog, but with all bias aside, these things work great. If you are in an office sharing an Internet feed with various users, the NetEqualizer will keep aggressive bandwidth users from crowding others out. No, it cannot create additional bandwidth on your pipe, but it will eliminate the gridlock caused by your colleague in the next cubicle downloading a Microsoft service pack.

Yes, there are other devices on the market (like your fancy router), but the NetEqualizer was specifically designed for that mission.

9) Bonus Tip: Kill All of Those Security Devices and See What Happens

With recent out break of the H1N1 virus, it reminded me of how sometimes the symptoms and carnage from a vaccine are worse than the disease it claims to cure. Well, the same holds true for your security protection hardware on your network. From proxies to firewalls, underpowered equipment can be the biggest choke point on your network.

Created by APconnections, the NetEqualizer is a plug-and-play bandwidth control and WAN/Internet optimization appliance that is flexible and scalable. When the network is congested, NetEqualizer’s unique “behavior shaping” technology dynamically and automatically gives priority to latency sensitive applications, such as VoIP and email.

Click here for a full price list.

Links to other bandwidth control products on the market.

Packet Shaper by Blue Coat

Exinda

Riverbed

Exinda Packet Shaper and Riverbed tend to focus on the enterprise WAN optimization market.

Cymphonix

Cymphonix comes from a background of detailed reporting.

Emerging Technologies

Very solid product for bandwidth shaping.

Exinda

Exinda from Australia has really made a good run in the US market offering a good alternative to the incumbants.

Netlimiter

For those of you who are wed to Windows NetLimiter is your answer

Posted in Bandwidth Management, Educational Articles. Tags: application shaping, caching, compression, ftp, Netequalizer, protocol spoofing, wan acceleration, Wan Optimization. 2 Comments »

NetEqualizer Field Guide to Network Capacity Planning

June 10, 2010 — netequalizer

I recently reviewed an article that covered bandwidth allocations for various Internet applications. Although the information was accurate, it was very high level and did not cover the many variances that affect bandwidth consumption. Below, I’ll break many of these variances down, discussing not only how much bandwidth different applications consume, but the ranges of bandwidth consumption, including ping times and gaming, as well as how our own network optimization technology measures bandwidth consumption.

E-mail

Some bandwidth planning guides make simple assumptions and provide a single number for E-mail capacity planning, oftentimes overstating the average consumption. However, this usually doesn’t provide an accurate assessment. Let’s consider a couple of different types of E-mail.

E-mail — Text

Most E-mail text messages are at most a paragraph or two of text. On the scale of bandwidth consumption, this is negligible.

However, it is important to note that when we talk about the bandwidth consumption of different kinds of applications, there is an element of time to consider — How long will this application be running for? So, for example, you might send two kilobytes of E-mail over a link and it may roll out at the rate of one megabit. A 300-word, text-only E-mail can and will consume one megabit of bandwidth. The catch is that it generally lasts just a fraction of second at this rate. So, how would you capacity plan for heavy sustained E-mail usage on your network?

When computing bandwidth rates for classification with a commercial bandwidth controller such as a NetEqualizer, the industry practice is to average the bandwidth consumption for several seconds, and then calculate the rate in units of kilobytes per second (Kbs).

For example, when a two kilobyte file (a very small E-mail, for example) is sent over a link for a fraction of a second, you could say that this E-mail consumed two megabits of bandwidth. For the capacity planner, this would be a little misleading since the duration of the transaction was so short. If you take this transaction average over a couple of seconds, the transfer rate would be just one kbs, which for practical purposes, is equivalent to zero.

E-mail with Picture Attachments

A normal text E-mail of a few thousand bytes can quickly become 10 megabits of data with a few picture attachments. Although it may not look all the big on your screen, this type of E-mail can suck up some serious bandwidth when being transmitted. In fact, left unmolested, this type of transfer will take as much bandwidth as is available in transit. On a T1 circuit, a 10-megabit E-mail attachment may bring the line to a standstill for as long as six seconds or more. If you were talking on a Skype call while somebody at the same time shoots a picture E-mail to a friend, your Skype call is most likely going to break up for five seconds or so. It is for this reason that many network operators on shared networks deploy some form of bandwidth contorl or QoS as most would agree an E-mail attachment should not take priority over a live phone call.

E-mail with PDf Attachment

As a rule, PDF files are not as large as picture attachments when it comes to E-mail traffic. An average PDF file runs in the range of 200 thousand bytes whereas today’s higher resolution digital cameras create pictures of a few million bytes, or roughly 10 times larger. On a T1 circuit, the average bandwidth of the PDF file over a few seconds will be around 100kbs, which leaves plenty of room for other activities. The exception would be the 20-page manual which would be crashing your entire T1 for a few seconds just as the large picture attachments referred to above would do.

Gaming/World of Warcraft

There are quite a few blogs that talk about how well World of Warcraft runs on DSL, cable, etc., but most are missing the point about this game and games in general and their actual bandwidth requirements. Most gamers know that ping times are important, but what exactly is the correlation between network speed and ping time?

The problem with just measuring speed is that most speed tests start a stream of packets from a server of some kind to your home computer, perhaps a 20-megabit test file. The test starts (and a timer is started) and the file is sent. When the last byte arrives, a timer is stopped. The amount of data sent over the elapsed seconds yields the speed of the link. So far so good, but a fast speed in this type of test does not mean you have a fast ping time. Here is why.

Most people know that if you are talking to an astronaut on the moon there is a delay of several seconds with each transmission. So, even though the speed of the link is the speed of light for practical purposes, the data arrives several seconds later. Well, the same is true for the Internet. The data may be arriving at a rate of 10 megabits, but the time it takes in transit could be as high as 1 second. Hence, your ping time (your mouse click to fire your gun) does not show up at the controlling server until a full second has elapsed. In a quick draw gun battle, this could be fatal.

So, what affects ping times?

The most common cause would be a saturated network. This is when your network transmission rates of all data on your Internet link exceed the links rated capacity. Some links like a T1 just start dropping packets when full as there is no orderly line to send out waiting packets. In many cases, data that arrive to go out of your router when the link is filled just get tossed. This would be like killing off excess people waiting at a ticket window or something. Not very pleasant.

If your router is smart, it will try to buffer the excess packets and they will arrive late. Also, if the only thing running on your network is World of Warcraft, you can actually get by with 120kbs in many cases since the amount of data actually sent of over the network is not that large. Again, the ping time is more important and a 120kbs link unencumbered should have ping times faster than a human reflex.

There may also be some inherent delay in your Internet link beyond your control. For example, all satellite links, no matter how fast the data speed, have a minimum delay of around 300 milliseconds. Most urban operators do not need to use satellite links, but they all have some delay. Network delay will vary depending on the equipment your provider has in their network, and also how and where they connect up to other providers as well as the amount of hops your data will take. To test your current ping time, you can run a ping command from a standard Windows machine

Citrix

Applications vary widely in the amount of bandwidth consumed. Most mission critical applications using Citrix are fairly lightweight.

YouTube Video — Standard Video

A sustained YouTube video will consume about 500kbs on average over the video’s 10-minute duration. Most video players try to store the video up locally as fast as they can take it. This is important to know because if you are sizing a T1 to be shared by voice phones, theoretically, if a user was watching a YouTube video, you would have 1 -megabit left over for the voice traffic. Right? Well, in reality, your video player will most likely take the full T1, or close to it, if it can while buffering YouTube.

YouTube — HD Video

On average, YouTube HD consumes close to 1 megabit.

See these other Youtube articles for more specifics about YouTube consumption

Netflix – Movies On Demand

Netflix is moving aggressively to a model where customers download movies over the Internet, versus having a DVD sent to them in the mail. In a recent study, it was shown that 20% of bandwidth usage during peak in the U.S. is due to Netflix downloads. An average a two hour movie takes about 1.8 gigabits, if you want high-definition movies then its about 3 gigabits for two hours. Other estimates are as high as 3-5 gigabits per movie.

On a T1 circuit, the average bandwidth of a high-definition Netflix movie (conversatively 3 gigabits/2 hours) over one second will be around 400kbs, which consumes more than 25% of the total circuit.

Skype/VoIP Calls

The amount of bandwidth you need to plan for a VoIP network is a hot topic. The bottom line is that VoIP calls range from 8kbs to 64kbs. Normally, the higher the quality the transmission, the higher the bit rate. For example, at 64kbs you can also transmit with the quality that one might experience on an older style AM radio. At 8kbs, you can understand a voice if the speaker is clear and pronunciates their words clearly. However, it is not likely you could understand somebody speaking quickly or slurring their words slightly.

Real-Time Music, Streaming Audio and Internet Radio

Streaming audio ranges from about 64kbs to 128kbs for higher fidelity.

File Transfer Protocol (FTP)/Microsoft Servicepack Downloads

Updates such as Microsoft service packs use file transfer protocol. Generally, this protocol will use as much bandwidth as it can find. There are several limiting factors for the actual speed an FTP will attain, though.

The speed of your link — If the factors below (2 and 3) do not come into effect, an FTP transfer will take your entire link and crowd out VoIP calls and video.
The speed of the senders server — There is no guarantee that the sending serving is able to deliver data at the speed of your high speed link. Back in the days of dial-up 28.8kbs modems, this was never a factor. But, with some home internet links approaching 10 megabits, don’t be surprised if the sending server cannot keep up. During peak times, the sending server may be processing many requests at one time, and hence, even though it’s coming from a commercial site, it could actually be slower than your home network.
The speed of the local receiving machine — Yes, even the computer you are receiving the file on has an upper limit. If you are on a high speed university network, the line speed of the network can easily exceed your computers ability to take up data.

While every network will ultimately be different, this field guide should provide you with an idea of the bandwidth demands your network will experience. After all, it’s much better to plan ahead rather than risking a bandwidth overload that causes your entire network to come to a hault.

Related Article a must read for anybody upgrading their Internet Pipe is our article on Contention Ratios

Other products that classify bandwidth

White Paper: A Simple Guide to Network Capacity Planning

June 9, 2010 — netequalizer

So the question remains:

How much bandwidth can you survive with before you impact morale or productivity?

So what is an acceptable contention ratio?

From a business standpoint, it is whatever a customer will put up with and pay for without canceling their service. This definition may seem ethically suspect, but whether in the bygone days of telecommunications phone service or contemporary Internet bandwidth business, there are long-standing precedents for overselling. What do you think a circuit busy signal is caused by? Or a dropped cell phone call?

So, without pulling any punches, what exactly will a customer tolerate before pulling the plug?

Here are some basic observations about consumers and acceptable contention ratios:

Rural customers in the US and Canada: Contention ratios of 50 to 1 are common
International customers in remote areas of the world: Contention ratios of 80 to 1 are common
Internet providers in urban areas: Contention ratios of 15 to 1 are to be expected
Generic Business ratio 50 to 1 , and sometimes higher

Update Jan 2015, quite a bit has happened since these original numbers were published. Internet prices have plummeted, here is my updated observation.

Rural customers in the US and Canada: Contention ratios of 10 to 1 are common
International customers in remote areas of the world: Contention ratios of 20 to 1 are common
Internet providers in urban areas: Contention ratios of 2 to 1 are to be expected
Generic Business ratio 5 to 1 , and sometimes higher

As a rule Businesses can general get away with slightly higher contention ratios. Most business use does not create the same load as recreational use, such as YouTube and File Sharing. Obviously, many businesses will suffer the effects of recreational use and perhaps haphazardly turn their heads on enforcement of such use. The above ratio of 50 to 1 is a general guideline of what a business should be able to work with, assuming they are willing to police their network usage and enforce policy.

The numbers above are a good, rough starting point, but things are not as simple as they look. There is a statistical twist as bandwidth amounts get higher.

Contention ratios can actually increase as the overall Internet trunk size gets larger. For example, if 50 people can share one megabit without mutiny, it should follow that 100 people can share two megabits without mutiny as the ratio has not changed. It is still 50 to 1.

However, from observations of hundreds of ISPs, we can easily conclude that perhaps 110 people can share two megabits with the same tolerance as 50 people sharing one megabit. What this means is that the larger the ISP, the more bandwidth at a fixed cost per megabit, and thus the larger the contention ratios you can get away with.

Is this really true? And if so, what are its implications for your business?

This is simply an empirical observation, backed up by talking to literally thousands of ISPs over the course of four years and noticing how their oversubscription ratios increase with the size of their trunk.

A conservative estimate is that, starting with the baseline ratio listed above, you can safely add 10 percent more subscribers above and beyond the original contention ratio for each megabit of trunk they share.

Thus, to provide an illustration, 50 people sharing one megabit can safely be increased to 110 people sharing two megabits, and at four megabits you can easily handle 280 customers. With this understanding, getting more from your bandwidth becomes that much easier.

I also ran across this thread in a discussion group for Resnet Adminstrators around the country.

From Resnet Listserv

Brandon Enright at University of California San Diego breaks it down as follows:
Right now we’re at .2 Mbps per student. We could go as low as .1 right
now without much of any impact. Things would start to get really ugly
for us at .05 Mpbs / student.

So at 10k students I think our lower-bound is 500 Mbps.

I can’t disclose what we’re paying for bandwidth but even if we fully
saturated 2Gbps for the 95% percentile calculation it would come out to
be less than $5 per student per month. Those seem like reasonable
enough costs to let the students run wild.
Brandon

Editors note: I am not sure why a public institution can’t exactly disclose what they are paying for bandwidth ( Brian does give a good hint), as this would be useful to the world for comparison; however many Universities get lower than commercial rates through state infrastructure not available to private operators.

Simple Is Better with Bandwidth Monitoring and Traffic Shaping Equipment

June 7, 2010 — netequalizer

By Art Reisman

Editor’s note: Art Reisman is the CTO of APconnections. APconnections designs and manufactures the popular NetEqualizer bandwidth shaper.

For most IT administrators, bandwidth monitoring of some sort is an essential part of keeping track of, as well as justifying, network expenses. However, the question a typical CIO will want to know before approving any purchase is, “What is the return on investment for your equipment purchase?”. Putting a hard and fast number on bandwidth optimization equipment may seem straight forward. If you can quantify the cost of your bandwidth and project an approximate reduction in usage or increase in throughput, you can crunch the numbers. But, is that all you should consider when determining how much you should spend on a bandwidth optimization device?

The traditional way of looking at monitoring your Internet has two dimensions. First, the fixed cost of the monitoring tool used to identify traffic, and second, the labor associated with devising and implementing the remedy. In an ironic inverse correlation, we assert that your ROI will degrade with the complexity of the monitoring tool.

Obviously, the more detailed the reporting/shaping tool, the more expensive its initial price tag. Yet, the real kicker comes with part two. The more detailed data output generally leads to an increase in the time an administrator is likely to spend making adjustments and looking for optimal performance.

But, is it really fair to assume higher labor costs with more advanced monitoring and information?

Well, obviously it wouldn’t make sense to pay more for an advanced tool if there was no intention of doing anything with the detailed information it provides. But, typically, the more information an admin has about a network, the more inclined he or she might be to spend time making adjustments.

On a similar note, an oversight often made with labor costs is the belief that when the work needed to adjust the network comes to fruition, the associated adjustments can remain statically in place. However, in reality, network traffic changes constantly, and thus the tuning so meticulously performed on Monday may be obsolete by Friday.

Does this mean that the overall productivity of using a bandwidth tool is a loss? Not at all. Bandwidth monitoring and network adjusting can certainly result in a cost-effective solution. But, where is the tipping point? When does a monitoring solution create more costs than it saves?

A review of recent history reveals that technologies with a path similar to bandwidth monitoring have become commodities and shunned the overhead of most human intervention. For example, computer operators disappeared off the face of the earth with the invention of cheaper computing in the late 1980s. The function of a computer operator did not disappear completely, it just got automated and rolled into the computer itself. The point is, anytime the cost of a resource is falling, the attention and costs used to manage it should be revisited.

An effective compromise with many of our customers is that they are stepping down from expensive, complex reporting tools to a simpler approach. Instead of trying to determine every type of traffic on a network by type, time of day, etc., an admin can spot trouble by simply checking overall usage numbers once a week or so. With a basic bandwidth control solution in place (such as a NetEqualizer), the acute problems of a network locking up will go away, leaving what we would call only “chronic” problems, which may need to be addressed eventually, but do not require immediate action.

For example, with a simple reporting tool you can plot network usage by user. Such a report, although limited in detail, will often reveal a very distinct bell curve of usage behavior. Most users will be near the mean, and then there are perhaps one or two percent of users that will be well above the mean. You don’t need a fancy tool to see what they are doing. Abuse becomes obvious just looking at the usage (a simple report).

However, there is also the personal control factor, which often does not follow clear lines of ROI.

What we have experienced when proposing a more hands-off model to network management is that a customer’s comfort depends on their bias for needing to know, which is an unquantifiable personal preference. Even in a world where bandwidth is free, it is still human nature to want to know specifically what bandwidth is being used for, with detailed information regarding the type of traffic. There is nothing wrong with this desire, but we wonder how strong it might be if the savings obtained from using simpler monitoring tools were converted into, for example, a trip to Hawaii.

In our next article, we’ll put some real world numbers to the test for actual breakdowns, so stay tuned. In the mean time, here are some other articles on bandwidth monitoring that we recommend. And, don’t forget to take our poll.

List of monitoring tools compiled by Stanford

ROI tool , determine how much a bandwidth control device can save.

Great article on choosing a bandwidth controller

Planetmy
Linux Tips
How to set up a monitor for free

Good enough is better a lesson from the Digital Camera Revolution

Posted in Bandwidth Monitoring, Commentary and Editorials, ROI. Tags: bandwidth shaping return on investment, traffic shaping ROI. Leave a Comment »

« Older posts

Newer posts »

NetEqualizer News Blog

Follow NetEq Blog

Categories

Keyword Search

Pages

Recent Posts

QoS Over The Internet – Is it possible? Five Must-Know Facts

August 2010

NetEqualizer News

NetEqualizer Lite Sale; Ireland Tech Seminar Details

NetEqualizer Lite Overstock Sale

NetEqualizer To Be Featured At CCSA Conference

APconnections Is Coming To Ireland

We Want Your Feedback

Best Of The Blog

Top Five Causes For Disruption Of Internet Service

Google Verizon Net Neutrality Policy, is it sincere?

Natural Address Translation FAQ

Does Lower cost bandwidth foretell a decline in Expensive Packet Shapers ?

The Inside Scoop on Where the Market for Bandwidth Control Is Going

Seven Points to Consider When Planning Internet Redundancy

What to expect from Internet Bursting

Instant Bandwidth Snapshot Feature: Is this an Industry First?

Nine Tips and Technologies for Network WAN Optimization