URL-Based Shaping With Your NetEqualizer: A How To Guide


What is URL-based Shaping?

URL shaping is the ability to specify the URL, normally a popular site such as YouTube or NetFlix, and set up a fixed-rate limit for traffic to that specific URL.

Is URL shaping just a matter of using a reverse lookup on a URL to get the IP address and plugging it into a bandwidth controller?

In the simplest case, yes, but for sites such as YouTube, the URL of http://www.youtube.com will have many associated IP addresses used for downloading actual videos. Shaping exclusively on the base URL would not be effective.

Is URL shaping the same thing as application shaping?

No. Although similar in some ways, there are significant differences:

  1. URL shaping is essentially the same as shaping by a known IP address. The trick with URL shaping is to discover IP addresses associated with a well-known URL.
  2. Application shaping uses Deep Packet Inspection (DPI). URL shaping does not. It does not inspect or open customer data.

How to set up URL-based shaping on your NetEqualizer

The following specifications are necessary:

  1. NetEqualizer version 4.0 or later
  2. A separate Linux-based client such that the client must access the Internet through the NetEqualizer
  3. The Perl source code for client URL shaping (listed below) loaded onto a client
  4. You will also need to set up your client so that it has permissions to run RSH (remote Shell) commands on your NetEqualizer without requiring a password to be entered. If you do not do this, your Perl discovery routine will hang. The notes for setting up the RSH permissions are outlined below.

How it works…

Save the Perl source code into a .pl file we suggest urlfinder.pl

Make sure to make this file executable

chmod 777 urlfinder.pl

Run the perl command with the following syntax from the command line, where domain.com will be replaced with the specific URL you wish to shape:

./urlfinder.pl http://www.domain.com pool# downlimit uplimit x.x.x.x y.y.y.y

  • Pool# is an unused bandwidth pool on your NetEqualizer unit
  • Downlimit is the rate in bytes per second incoming for the URL
  • Uplimit is the rate bytes per second outgoing to the Internet for the URL
  • x.x.x.x is the IP address of your NetEqualizer
  • y.y.y.y is the IP address of the client

The script will attempt an http request using http://www.domain.com. It will then continue to do recursive Web accesses on subsequent links starting on the main domain URL. It will stop when there are no more links to follow or when 150 pages have been accessed. Any foreign IP’s found during the access session will be put into the given bandwidth pool as CLASS B masks, and will immediately be forever shaped until you remove the pool.

Notes:

In our beta testing, the script did well in finding YouTube subnets used for videos.  We did not confirm whether the main NetFlix home page URL shares IP subnets with their download sites.

Notes for setting up RSH

Begin Notes

These notes  assume you are either logged in on the Client as root or you use sudo -i and are acting as root.

192.168.1.143 is used in the example as the Server (NetEq) IP.

On your Client machine, do:

  • ssh-keygen -t rsa -b 4096
  • ssh-copy-id -i ~/.ssh/id_rsa.pub root@192.168.1.143
  • nano -w /etc/ssh/ssh_config

Make sure that these are as follows:

  • RhostsRSAAuthentication yes
  • RSAAuthentication yes
  • EnableSSHKeysign yes
  • HostbasedAuthentication yes

The next line is all one line to the ssh_known_hosts

  • scp /etc/ssh/ssh_host_rsa_key.pub root@192.168.1.143:/etc/ssh/ssh_known_hosts

The next line is all one line to the ssh_known_hosts2

  • scp /etc/ssh/ssh_host_rsa_key.pub root@192.168.1.143:/etc/ssh/ssh_known_hosts2

Now, find out your HOSTNAME on the Client:

  • echo $HOSTNAME

On the Server machine, do:

  • nano -w /etc/hosts.equiv
  • harry-lin root
  • my $HOSTNAME of the Client was harry-lin
  • nano -w /etc/ssh/sshd_config

Check the following:

  • PermitRootLogin yes
  • StrictModes yes
  • RSAAuthentication yes
  • PubkeyAuthentication yes
  • AuthorizedKeysFile %h/.ssh/authorized_keys
  • IgnoreRhosts no
  • RhostsRSAAuthentication no
  • HostbasedAuthentication yes

Now do:

  • chown root:root /root

Then:

  • /etc/init.d/ssh reload

Now you can try something like this from your Client:

  • ssh root@192.168.1.143

If it doesn’t work, then do the following, which gives you details if possible:

  • ssh -v root@192.168.1.143

Final Notes: While support for this utility is NOT currently included with your NetEqualizer, we will assist any customers with a current Network Software Subscription for up to one hour. For additional support, consulting fees may apply.

NetEqualizer Software Update 4.0 — Carrier-Class Shaping


Continuing the constant evolution of the NetEqualizer line, we’re pleased to offer the most recent NetEqualizer software release – Carrier Class 4.0. In addition to providing the features and quality found in past NetEqualizer releases, among its many enhancements Carrier Class 4.0 will now support three times as many shaping buffers, translating into smoother shaping for up to tens of thousands of users.

For example, you will be able to take a one-gigabit pipe and break off 1,000 users into a subnet mask to share 100 megabits (with smoother results than current versions). Most routers that break out chunks of bandwidth use harsh rate control methods such as dropping packets when the limit is reached. Although there will be a small margin of error, the optimizations and techniques used to break off larger chunks of bandwidth and shape them smoothly without dropping packets rivals that of carrier class shapers sold for 10 times our cost.

The following features and enhancements will also be available with Carrier Class 4.0:

  • Full one- to 32-bit mask fields for hard limits  — You can now take any IP address and specify a mask in x.x.x.x/y format where y is the number of bits you wish to mask. All IP addresses in the masked range will receive the specified hard limit (Hard limits are individual rate limits for an IP address).
  • Pools support masks – You can now add members to a bandwidth pool using a mask field of the form x.x.x.x/y, Y can range from one to 32. The NetEqualizer will automatically add members of the range specified as they become active and retire them if they become inactive. This optimization will allow users to specify large ranges without overwhelming the system.
  • Full one- to 32-bit masking for traffic masking – You can now use the NetEqualizer masking function with odd numbered mask specifications, prior to this release only /24 and /16 masks were allowed.
  • Pool number displayed in active connection table – You can now see if a connection is part of a pool, the pool number will be displayed in the last column of the connection table.
  • Release 1.0 of our URL-based blocking feature – Now you can block a list of URL’s. This feature is commonly used by libraries and private institutions where there is a mandate to block particular recreational sites. In the initial release, customers need only supply a config file with all URLs by name that they wish to block and then hit the start button. In future releases, we will be contracting with providers that supply updated lists on a regular basis. There will be no charge to enable our URL-blocking feature, however there will likely be subscription charges to use third party URL lists.
  • Connection limit masks now fully supported – You can specify a connection limit mask of the form x.x.x.x/y where y is an int from one to 32. Prior to this release, only /24 and /26 were supported.
  • New Automatic detection of license overruns – The NetEqualizer will now automatically report any new license overruns. Any time you log into the GUI, a message will be displayed indicating how many license overruns you may have incurred since your last reboot. If you do see a license overrun, you should call support and see about upgrading your license.
  • New license levels available for enforcement in kernel
  • URL-based shaping

For more information on the Carrier Class 4.0 update, contact us at admin@apconnections.net or 303-997-1300.

APconnections Announces NetEqualizer Lifetime Buyer Protection Policy


This week, we announced the launch of the NetEqualizer Lifetime Buyer Protection Policy. In the event of an un-repairable failure of a NetEqualizer unit at any time, or in the event that it is time to retire a unit, customers will have the option to purchase a replacement unit and apply a 50-percent credit of their original unit purchase price, toward the new unit.  For current pricing see register for our price list.  This includes units that are more than three years old (the expected useful life for hardware) and in service at the time of failure.

For example, if you purchased a unit in 2003 for $4000 and were looking to replace it or upgrade with a newer model, APconnections would kick in a $2000 credit toward the replacement purchase.

The Policy will be in addition to the existing optional yearly NetEqualizer Hardware Warranty (NHW), which offers customers cost-free repairs or replacement of any malfunctioning unit while NHW is in effect (read details on NHW).

Our decision to implement the policy was a matter of customer peace-of-mind rather than necessity. While the failure rate of any NetEqualizer unit is ultimately very low, we want customers to know that we stand behind our products – even if it’s several years down the line.

To qualify,

  • users must be the original owner of the NetEqualizer unit,
  • the customer must have maintained a support contract that has been current within last 18 months , lapses of support longer than 18 months will void our replacement policy
  • the unit must have been in use on your network at the time of failure.

Shipping is not included in the discounted price. Purchasers of the one-year NetEqualizer hardware warranty (NHW) will still qualify for full replacement at no charge while under hardware warranty.  Contact us for more details by emailing sales@apconnections.net, or calling 303.997.1300 x103 (International), or 1.888.287.2492 (US Toll Free).

Note: This Policy does not apply to the NetEqualizer Lite.

NetEqualizer Software Update Improves VLAN Shaping, NTOP


Editor’s Note: The following blog entry explains the newest NetEqualizer features available with our most recent software update. While minor bug fixes are often included in these updates, they will not always be detailed.

We recently released our newest NetEqualizer software update, further improving on our existing technology. The following fixes have been implemented from the the previous 2.43k version to the latest 3.32a.

  1. Upgraded internal disk memory caching. This feature remedied an issue with NTOP that was causing disk corruptions on the CF drive.
  2. Subnet masking was modified such that masked traffic will not count against your license level. Prior to this change, a customer with a 10-meg license who ran 100 meg local transfers across their NetEqualizer would experience a license violation. You can now mask that traffic (make it invisible to the NetEqualizer and hence not violate your license).
  3. A bug fix was put in for customers who run asymmetric pools. Bandwidth pools with different upload and download speeds were not working correctly prior to this fix.
  4. VLAN shaping fix. There was an issue on cold restarts.
  5. Support for multi-core CPU
  6. More efficient connection limit processing

This software update is available without charge for NetEqualizer customers with a current NetEqualizer Software Subscription (NSS). For more information on this update, or the NSS, contact us at admin@apconnections.net.


New Asymmetric Shaping Option Augments NetEqualizer-Lite


We currently have a new release in beta testing that allows for equalizing on an asymmetric link. As is the case with all of our equalizing products, this release will allow users to more efficiently utilize their bandwidth, thus optimizing network performance. This will be especially ideal for users of our recently released NetEqualizer-Lite.

Many wireless access points have a limit on the total amount of bandwidth they can transmit in both directions. This is because only one direction can be talking at a time. Unlike wired networks, where a 10-meg link typically means you can have 10 megs UP and 10 megs going the other direction simultaneously, in  a wireless network you can only have 10 megabits total at any one time.  So, if you had 7 megabits coming in, you could only have 3 megabits going out. These limits are a hard saturation point.

In the past, it was necessary to create separate settings for both the up and down stream. With the new NetEqualizer release, you can simply tell the NetEqualizer that you have an asymmetric 10-megabit link, and congestion control will automatically kick in for both streams,  alleviating bottlenecks more efficiently and keeping your network running smoothly.

For more information on APconnections’ equalizing technology, click here.

NetEqualizer-Lite Is Now Available!


Last month, we introduced our newest release, a Power-over-Ethernet NetEqualizer. Since then, with your help, we’ve titled the new release the NetEqualizer-Lite and are already getting positive feedback from users. Here’s a little background about what led us to release the NetEqualizer-Lite…Over the years, we’d had several customers express interest in placing a NetEqualizer as close as possible to their towers in order to relieve congestion. However, in many cases, this would require both a weatherproof and low-power NetEqualizer unit – two features that were not available up to this point. However, in the midst of a growing demand for this type of technology, we spent the last few months working to meet this need and thus developed the NetEqualizer-Lite.

Here’s what you can expect from the NetEqualizerLite:

  • Power over Ethernet
  • Up to 10 megabits of shaping
  • Up to 200 users
  • Comes complete with all standard NetEqualizer features

And, early feedback on the new release has been positive. Here’s what one user recently posted on DSLReports.com:

We’ve ordered 4 of these and deployed 2 so far. They work exactly like the 1U rackmount NE2000 that we have in our NOC, only the form factor is much smaller (about 6x6x1) and they use POE or a DC power supply. I amp clamped one of the units, and it draws about 7 watts….The Netequalizer has resulted in dramatically improved service to our customers. Most of the time, our customers are seeing their full bandwidth. The only time they don’t see it now is when they’re downloading big files. And, when they don’t see full performance, its only for the brief period that the AP is approaching saturation. The available bandwidth is re-evaulated every 2 seconds, so the throttling periods are often brief. Bottom line to this is that we can deliver significantly more data through the same AP. The customers hitting web pages, checking e-mail, etc. virtually always see full bandwidth, and the hogs don’t impact these customers. Even the hogs see better performance (although that wasn’t one of my priorities). (DSLReports.com)

Pricing for the new model will be $1,200 for existing NetEqualizer users and $1,550 for non-customers purchasing their first unit. However, the price for subsequent units will be $1,200 for users and nonusers alike.

For more information about the new release, contact us at admin@apconnections.net or 1-800-918-2763.

NetEqualizer Coming Attractions and Features


You heard it here first. Here is some of what is eminent in our pipeline. Due out in April 2009!

High end system improvements — The NE3000 – 1 gig  is currently undergoing load testing to validate 20,000 subscribers. For tier-2 ISPs and larger institutions, this is a welcome addition. There is nothing out there with this performance level that touches us on price. We have recently taken orders for several small national ISPs  in the Caribbean that  arelooking to cut their costs without sacrificing quality.

Top speed of these new high-end systems is 900 megabits up and 900 megabits down at the same time for a total of 1.8 megabits sustained.

More intelligence to effectively reign in rogue applications and P2P — Working with a couple of our large university customers, we have come up with some improvements to our behavior-based techniques used to battle P2P (without compromising privacy).  We now have a connection-limit feature that allows you to set a fixed connection limit for all IP’s on a particular subnet. We also have a smart script that will seek out your highest connection customers and set more stringent rate limits on them.

Faster start up time — For those users implementing pools, you are likely aware that it takes a few minutes for the NetEqualizer to fully come in on line. In our next release, we have reduced this time from minutes to seconds.

For more details on these features, please feel free to contact us.

NetEqualizer rolling out URL based traffic shaping.


February 10th, 2009

Lafayette Colorado

APconnections makers of the of the popular NetEqualizer line of bandwidth control and traffic shaping hardware appliances today announced a major feature enhancement to their product line. URL based shaping.

In our recent newsletter we asked our customers if they were in need of URL based shaping and the feedback was a resounding YES.

Using our current release, administrators  have the ability to shape their network traffic by, IP address , Mac Address, VLAN or subnet. With addition of URL shaping, our product line will meet the demands of Co-location operators.

A distinction we need to make clear, is that URL based shaping is not related to DPI or content based shaping. URLs are public information as they travel across the Internet, and are basically  a mapping into human readable  form of an IP address; therefore URL based shaping does not require opening private data for inspection.

If you are interested in details regarding this feature please contact APconnections directly.

NetEqualizer Network Access Control Module Helps Generate Revenue


Background: The NetEqualizer network access control module (NAC), which was released this past September, allows users to re-direct “unknown” or “unauthorized” traffic to a web server hosted on the NetEqualizer.  Once redirected, you can have the NetEqualizer perform a variety of actions, including:

1) Authenticating a user via login
2) Allowing the unknown user to create a paid account (using a credit card, for example)
3) Allowing the user to pass through to the Internet without logging in

Did you know that the NetEqualizer network access control module offers several options to generate revenue? One of the dilemmas many of our customers have mentioned is that in order to be competitive they don’t want to charge for their Internet service (hotels, etc.). Well, the cool thing about the NAC module is that you can offer multiple logins with different rate limits. For example, one could be your standard free service and another could be a paid service with higher bandwidth rates.

An additional revenue generating feature of the NAC module is the ability to run advertisements on the login screens. For example, if you’re a hotel operator, even if you’re not charging for Internet service, you could have your guests login on a screen with the logo and name of a local merchant, or anybody that is interested in cross marketing with your hotel.

The NAC module also has customizable splash screens on its default login page that you can edit, thus welcoming your users with whatever content you choose.

For more information about the NetEqualizer network access control module, visit our Web page at www.netequalizer.com or contact us at 1-888-287-2492 or via email at sales@netequalizer.com.

One Gigabit NetEqualizer Announced Today


Editors Note: We expect to go higher than 1 gigabit and 12,000 users in the near future. This is just a start.

APconnections Announces Fully Equipped One-Gigabit NetEqualizer Traffic Shaper for $8500

LAFAYETTE, Colo., Nov. 7/PRNewswire/ — APconnections, a leading supplier of plug-and-play bandwidth shaping products, today announced a one-gigabit enhancement to their NetEqualizer brand traffic shapers. The initial release will handle 12,000 users and sustained line speeds of one gigabit.

“Prior to this release, our largest model, the NE-3000 was rated for 350 megabits,” said Eli Riles, APconnections vice president of sales. “Many of our current customers liked our technology, but just needed a higher end machine.The other good news is that our current NE-3000 platform will be able to run this new version with just a software upgrade, no forklift required.”

Future releases are in the works for even higher speeds and more users, thus solidifying APConnections as the price-performance leader in the WAN optimization market place.

In its initial release, the one-gigabit model will start at $8,500 USD. For more information, contact APconnections at 1-800-918-2763 or via email at sales@netequalizer.com.

The NetEqualizer is a plug-and-play bandwidth control and WAN/Internet optimization appliance that is flexible and scalable. When the network is congested, NetEqualizer’s unique “behavior shaping” technology
gives priority to latency-sensitive applications, such as VoIP and email. Behavior based shaping is the industry alternative to Deep Packet Inspection (DPI). It does it all dynamically and automatically, improving on other bandwidth shaping technology available.

APconnections is a privately held company founded in 2003 and is based in Lafayette, Colorado.

Contact: APconnections, 1-800-918-2763

NetEqualizer Direct Sales and Restocking Fee


A customer recently stated that the restocking fee we charge gave the appearance of a lack of confidence in our product. I can appreciate that perception, especially with all the failed products many IT professionals have been burned with over the years.

However, here is the official reasoning behind why we charge a restocking fee.  As taken from my response to this customer:

The restocking fee has its roots based on a couple of factors

1) The restocking fee is designed to make sure we don’t get inundated with requests for free units from customers that are “just looking”. The other vendors  you mention charge much higher prices, sometimes four times as much,  and they typically use a channel that already purchases stock for the purposes of demo’s. All of this cost gets passed along to the customers that end up buying the product (basically covering the cost of dry wells).  We sell mostly direct and with  no local presence it is difficult to know a customer’s buying patterns.   You’d be surprised how many customers will trial something  without any intention to purchase.  But, many times it is not the immediate customers fault as the CIO might change the IT manager’s  budget, etc.

2) We are  not 100-percent certain that our unit will solve your issue. I’d  say we are closer to 80-percent certain based on what you described, but we will easily provide you with $200 of support helping you figure out what your issue is. You will have the chance to talk directly to our engineers who trouble shoot thousands of networks a year with similar problems. We do not want or pretend to be a consulting company, but we don’t want to consult without recouping some of our cost either — especially with our low margins which we are already passing along.

Network Access Control Features Redirection for Delinquent Customers


What should an Internet service provider do to customers who are late with their payment?   If you simply block the user completely  at your firewall, you will likely generate a costly call to  your support engineering. But, there are other options…

A better way to collect payment without creating support calls is to redirect the delinquent user to a splash screen  informing them they need to pay their bill.  Obviously a customer won’t call unless they are ready to pay, hence no costly call to support engineering. For the suggested redirection capability, you do not need a fully automated network access control system (requiring a login  and credit card payment), but you do need a way to redirect delinquent customers to a Web page telling them to pay their outstanding bill.

With our latest version that is available now, we can set up IP redirection to a custom web page for a nominal support charge.

Note:  We now host the redirection page right on your NetEqualizer, so there is no need for another third party server.

Call us if you are interested:  303-997-1300
www.netequalizer.com

NetEqualizer Evaluation Policy


Our official policy for customers requesting evaluation units is to require payment upfront.  However, we do honor a no-questions-asked  30-day return policy.

As you can imagine, we get a constant stream of requests for evaluation units. Obviously we’d love to provide everybody who asks with a demo unit. After all, the other brand name packet shapers will throw them at you. Especially if you are coming from an account they want to win over.

So, you may be wondering why we don’t do the same…

Some background:

APconnections  sells quite a few units under $3000 dollars. To put this in perspective, last year a CEO from a larger competitor selling similar equipment admitted that $4000 is their break-even point.

So, how do we offer units starting at $2000 and still turn a profit?

A big part of our model to is make sure that we do not drill dry wells. Dry well is industry speak for pursuing business that will never materialize. Yes, we love chatting with people, but in order to pay our engineers and stay in business, we must limit money spent supporting customers that are just “looking”.  The easiest way to do this is to enforce our evaluation policy.

Serious customers that are ready to buy something but need to see it work in their network usually have no problem with purchasing up front.  Some, but not all, customers that are not agreeable to purchasing up front may have cash flow problems of their own. In an economy where banks do not know how to qualify loans, we don’t want  to try to calculate this risk.

The result of our conservative policy translates to much lower prices , and to date nobody is arguing with that.

NetEqualizer Network Access Control Rollout



After several months in development, the NetEqualizer network access control module is now available for trial.

The basic features of the Module are what you would expect (think airport T Mobile daily access etc):

1) The ability to dynamically authenticate/restrict users through a login process.

2) Automated payment processing for users who do not have a login.

3) The ability to selectively exempt an IP range from authentication redirection server. For example well known IP addresses on your network will not be forced to login, while other open IP address ranges (guest network) will require a login to obtain access.

4) The ability to custom brand  login pages and redirection service

Plus a couple of new twists that go above and beyond normal Network Authentication products:

1) Class of Service Specification When Customers Sign Up

For example, you will be able to offer customers a free standard service at modem speed with an option to pay for an upgrade for a faster connection. We have been playing with this feature at the office with our test system and the option to upgrade is very compelling. This could be a great way to increase revenue for those who might otherwise not log on at all if asked to purchase up front. Of course, this is made possible because the network access control is hosted by our NetEqualizer platform with full access to rate limiting features.

2) Group Licenses

This was one of the main requests from Portola Plaza Hotel (our beta site). They wanted to know if we could offer a system that allowed them to sell a group license (limited by total simultaneous sessions) to conference attendees. The other part of the challenge was to maintain a pay as you go option for individual hotel guests at the same time. Well, we have all that in there and it really is pretty impressive!

3) Smooth Service

For those hotels and institutions that have not been able to pull the trigger on something to eliminate their busy hour congestion, they will get the full power of our automated bandwidth controller. Many hotels and institutions have too many irons in the fire to address this issue, and perhaps their management cannot quantify the cost of slow Internet service, so they just live with it. These same institutions love to implement pay for internet service because that feature is clear-cut revenue generating device. So, now they will get both — revenue and smoother service for their paying customers all in one swoop!

4) Reliable Engineering

As many of our customers already know, we are an engineering company that developed a product, not an investor with a business plan that hired an engineering company. What this means is that our roots are conservative and we are in no hurry to put a solution on the market that will cause customers headaches as a result of some greedy business plan and offshore engineering. We do the work here in the U.S. and pay our engineers salaries of U.S. quality. The end result is a smoother process from start to finish. In addition, we are very conservative with our roll out and will not sell more than we can support at one time.

Needless to say, we were very excited to see the control module in action. It’s been even better to see that the months of research and development have paid off. For more information about the NetEqualizer network access control module, please visit our Web site at www.netequalizer.com.

New NE3000-300 Now Available


APconnections today announced the release of the NetEqualizer NE2000-300. It has all the features of a standard NetEqualizer, CALEA probe, peer-to-peer throttling, priority for voice and optional priority for video.

This unit doubles the effective bandwidth pipe of the current NE2000-150.

For details, contact APconnections at admin@apconnections.net or 303-997-1300.