How Much YouTube Can the Internet Handle?


By Art Reisman, CTO, http://www.netequalizer.com 

Art Reisman CTO www.netequalizer.com

Art Reisman

 

As the Internet continues to grow and true speeds become higher,  video sites like YouTube are taking advantage of these fatter pipes. However, unlike the peer-to-peer traffic of several years ago (which seems to be abating), YouTube videos don’t face the veil of copyright scrutiny cast upon p2p which caused most users to back off.
 

In our experience, there are trade offs associated with the advancements in technology that have come with YouTube. From measurements done in our NetEqualizer laboratories, the typical normal quality YouTube video needs about 240kbs sustained over the 10 minute run time for the video. The newer higher definition videos run at a rate at least twice that. 

Many of the rural ISPs that we at NetEqualizer support with our bandwidth shaping and control equipment have contention ratios of about 300 users per 10-megabit link. This seems to be the ratio point where these small businesses can turn  a profit.  Given this contention ratio, if 40 customers simultaneously run YouTube, the link will be exhausted and all 300 customers will be wishing they had their dial-up back. At last check, YouTube traffic accounted for 10 percent of all Internet Traffic.  If left completely unregulated,  a typical rural  ISP could find itself on the brink of saturation from normal YouTube usage already. With tier-1 providers in major metro areas there is usually more bandwidth, but with that comes higher expectations of service and hence some saturation is inevitable. 

If you believe there is a conspiracy, or that ISPs are not supposed to profit as they take risk and operate in a market economy, you are entitled to your opinion, but we are dealing with reality. And there will always be tension between users and their providers, much the same as there is with government funds and highway congestion. 

The fact is all ISPs have a fixed amount of bandwidth they can deliver and when data flows exceed their current capacity, they are forced to implement some form of passive constraint. Without them many networks would lock up completely. This is no different than a city restricting water usage when reservoirs are low. Water restrictions are well understood by the populace and yet somehow bandwidth allocations and restrictions are perceived as evil. I believe this misconception is simply due to the fact that bandwidth is so dynamic, if there was a giant reservoir of bandwidth pooled up in the mountains where you could see this resource slowly become depleted , the problem could be more easily visualized. 

The best compromise offered, and the only comprise that is not intrusive is bandwidth rationing at peak hours when needed. Without rationing, a network will fall into gridlock, in which case not only do the YouTube videos come to halt , but  so does e-mail , chat , VOIP and other less intensive applications. 

There is some good news, alternative ways to watch YouTube videos. 

We noticed during out testing that YouTube videos attempt to play back video as a  real-time feed , like watching live TV.  When you go directly to YouTube to watch a video, the site and your PC immediately start the video and the quality becomes dependent on having that 240kbs. If your providers speed dips below this level your video will begin to stall, very annoying;  however if you are willing to wait a few seconds there are tools out there that will play back YouTube videos for you in non real-time. 

Buffering Tool 

They accomplish this by pre-buffering before the video starts playing.  We have not reviewed any of these tools so do your research. We suggest you google “YouTube buffering tools” to see what is out there. Not only do these tools smooth out the YouTube playback during peak times or on slower connections , but they also help balance the load on the network during peak times. 

Bio Art Reisman is a partner and co-founder of APconnections, a company that provides bandwidth control solutions (NetEqualizer) to ISPs, Universities, Libraries, Mining Camps and any organization where groups of users must share their Internet resources equitably. What follows is an objective educational journey on how consumers and ISPs can live in harmony with the explosion of YouTube video.

Deep packet Inspection a poison pill for NebuAd ?


Editors Note:

NebuAd had a great idea show ads to users based on content and share the revenue with ISPs that sign up for their service. What is wrong with this Idea ? I guess customers don’t like people looking at their private data using DPI hence the lawsuit detailed in the article below.  The funny thing is we are still hearing from customers that want DPI as part of their solution, this includes many Universities , ISPs and alike.  I think the message is clear: Don’t use Deep Packet Inspection unless you fully disclose this practice to your customers/employees or risk getting your head nailed to a table.

———————————————————————–

From Zdnet Nov 11, 2008

NebuAd, the controversial company that was trying to sell deep-packet inspection technology as a means of delivering more relevant ads, has already had most of the life sucked out of it. Now, a class action lawsuit filed in U.S. District Court in San Francisco today, could put the final nail in the coffin.

Full article

http://blogs.zdnet.com/BTL/?p=10774

One Gigabit NetEqualizer Announced Today


Editors Note: We expect to go higher than 1 gigabit and 12,000 users in the near future. This is just a start.

APconnections Announces Fully Equipped One-Gigabit NetEqualizer Traffic Shaper for $8500

LAFAYETTE, Colo., Nov. 7/PRNewswire/ — APconnections, a leading supplier of plug-and-play bandwidth shaping products, today announced a one-gigabit enhancement to their NetEqualizer brand traffic shapers. The initial release will handle 12,000 users and sustained line speeds of one gigabit.

“Prior to this release, our largest model, the NE-3000 was rated for 350 megabits,” said Eli Riles, APconnections vice president of sales. “Many of our current customers liked our technology, but just needed a higher end machine.The other good news is that our current NE-3000 platform will be able to run this new version with just a software upgrade, no forklift required.”

Future releases are in the works for even higher speeds and more users, thus solidifying APConnections as the price-performance leader in the WAN optimization market place.

In its initial release, the one-gigabit model will start at $8,500 USD. For more information, contact APconnections at 1-800-918-2763 or via email at sales@netequalizer.com.

The NetEqualizer is a plug-and-play bandwidth control and WAN/Internet optimization appliance that is flexible and scalable. When the network is congested, NetEqualizer’s unique “behavior shaping” technology
gives priority to latency-sensitive applications, such as VoIP and email. Behavior based shaping is the industry alternative to Deep Packet Inspection (DPI). It does it all dynamically and automatically, improving on other bandwidth shaping technology available.

APconnections is a privately held company founded in 2003 and is based in Lafayette, Colorado.

Contact: APconnections, 1-800-918-2763

Death to Deep Packet (layer 7 shaping) Inspection


Editors note: Deep packet inspection (layer 7 shaping) will likely be around for a while. It is very easy to explain this technology to customers, hence many IT resellers latch on to it as it makes a compelling elevator pitch.  We put out the press release below to formalize our position on this issue.

For detailed information on how the techniques of NetEqualizer differ from Deep Packet inspection, see the following link: http://www.netequalizer.com/Compare_NetEqualizer.php

LAFAYETTE, Colo., October 28, 2008 — APconnections, a leading supplier of plug-and-play bandwidth shaping products, today made a formal announcement to formally discontinue  deep packet inspection techniques in their NetEqualizer product line.

“Our behavior-based techniques worked so well that current customers stopped asking for the layer-7 techniques we had at one time implemented into our system,” said Art Reisman, CEO of APconnections. “So, we eventually just decided to phase the technique out completely.”

Although deep packet inspection, also known as layer-7 shaping, was unofficially discontinued nearly two years ago, the ongoing debates over user privacy spurred the official announcement.

“What prompted us to make a formal announcement was the continued industry lack of understanding that deep packet inspection not only does not work very well, but it also puts you are at risk of violating privacy laws if you use these techniques without customer consent,” said Reisman.

Although Reisman says most providers cross this line with the good intentions of controlling traffic congestion, the reality of it is that it’s no different than listening to a private phone conversation and terminating the call if you don’t like what you hear.

“It’s quite risky  that any public US based ISP would invest in  this technique, especially after the FCC slapped Comcast’s wrists in a recent decision” said Riesman.

For more information on the NetEqualizer technology, visit www.netequalizer.com or contact APconnections at 1-800-918-2763 or via email sales@netequalizer.com.

The NetEqualizer is a plug-and-play bandwidth control and WAN/Internet optimization appliance that is flexible and scalable. When the network is congested, NetEqualizer’s unique “behavior shaping” technology gives priority to latency sensitive applications, such as VoIP and email. It does it all dynamically and automatically, improving on other bandwidth shaping technology available.

APconnections is a privately held company founded in 2003 and is based in Lafayette, Colorado.

Proactive ISP Mikrotec, a Kentucky-based Internet Service Provider


An article by Jagan Jagannathan on TMCnet caught my attention today. For all of our NetEqualizer ISP customers competing with Tier 1 providers, you are likely aware that it is your superior customer service that retains your base.

Mikrotec has a very proactive service to pre-call customers when they spot a problem, a big hit with their subscribers.

Here is an excerpt from the article…

One of the service providers offering proactive support services is Mikrotec, a Kentucky-based Internet service provider that has branded this service Pro-Alert. Pro-Alert offers the broadband provider “live, dynamic visibility” into the performance and availability of every subscriber and application across the entire network ecosystem. This enables Mikrotec to proactively take action on an issue before one of their customers has even noticed that there might be a problem.

Here is a link to the full article: http://callcenterinfo.tmcnet.com/analysis/articles/43534-customer-king-proactive-customer-support.htm

The Pros and Cons of Technology Showdowns (for traffic shapers and more)


This is my second re-write of this quick post discussing the pros and cons of a tech showdown (i.e., pitting two products head-to-head in a trial demo duel). In my first edition, I put together a big list with very politically correct pros and cons.  And then I tossed it. Instead, I am just going to really speak my mind and shorten it a bit.  So here goes. Pulling no punches this time…

1) Assuming you have two or more very high-end solutions to choose from, are truly undecided, and are planning to purchase one of them, then by all means get those vendors in there an make them show them your stuff. But if for some reason you can eliminate one of them early, try to do so.  A showdown should be a final tie breaker, not the first thing you try.

2) Try to avoid a showdown (for political reasons) if  you are pretty sure you know what you want already. Do some background checks on the product.  With the information available on the Internet today, a bad product can’t hide.  Then,  if everything checks out, settle with your first choice and purchase it.  Obviously, make sure you can return it if for some reason it does not live up to expectations.

3) If you are doing a showdown because you have  tech time and no capital budget right now, then please have your techs do something else with their time.

Showdowns drive up product costs for everybody: the vendor, yourself, and future buyers in the industry.  Yes they are a valid tool and necessary in many situations, but please use some discretion.  The time, money, and resources saved by foregoing an uneeded showdown will help the tide come in and raise all ships.

Network Access Control Features Redirection for Delinquent Customers


What should an Internet service provider do to customers who are late with their payment?   If you simply block the user completely  at your firewall, you will likely generate a costly call to  your support engineering. But, there are other options…

A better way to collect payment without creating support calls is to redirect the delinquent user to a splash screen  informing them they need to pay their bill.  Obviously a customer won’t call unless they are ready to pay, hence no costly call to support engineering. For the suggested redirection capability, you do not need a fully automated network access control system (requiring a login  and credit card payment), but you do need a way to redirect delinquent customers to a Web page telling them to pay their outstanding bill.

With our latest version that is available now, we can set up IP redirection to a custom web page for a nominal support charge.

Note:  We now host the redirection page right on your NetEqualizer, so there is no need for another third party server.

Call us if you are interested:  303-997-1300
www.netequalizer.com

NetEqualizer the Safe Bet for Optimizing Internet Link During Economic Downturn


We just announced a record profit for the quarter ending September 2008. I have included a copy of that announcement below.

Although we do not believe (or want to see) our success come at the expense of other players in the market, there is a strong contrast if you compare our performance to the higher-cost publicly-traded players in this market (see charts below).

I suspect these high-end shapers with expensive sales channels  may have trouble in this slowing market as they come under price pressure. IT departments continue to cut costs and the main play  of optimization products, reducing  ROI,  will lose some luster as Internet costs slowly fall. At some point, a high-end piece of equipment will lose out to adding more bandwidth.

NetEqualizer, on the other hand, is priced so much lower than these other products that our window of value will extend out at least another 10 years — perhaps more.

Although we are private company, we would be happy to share financials under NDA with any customer that has concerns going forward.  We have plenty of operating cash on hand and will likely expand as we pull out of this downturn and customers continue to look to reduce costs.

Stock charts for major players in the Internet/WAN optimization market

http://finance.yahoo.com/q/bc?s=RVBD&t=1y

http://finance.yahoo.com/q/bc?s=ALLT&t=2y&l=on&z=m&q=l&c=

http://finance.yahoo.com/q/bc?s=BCSI&t=1y&l=on&z=m&q=l&c=

Now, here’s our latest press release reporting profits…

———————————————————-

APconnections Announces 50-percent Increase in Profits During Current Quarter

LAFAYETTE, Colo., Sept. 22, 2008 — APconnections, a leading supplier of plug-and-play bandwidth shaping products, today announced that sales revenues have increased by 50 percent during the current quarter.

Company officials report that APconnections is finding that a growing number of ISPs, businesses, libraries, and universities are looking to the NetEqualizer to solve their Internet bandwidth congestion issues, oftentimes switching from more expensive traffic shaping solutions.

As companies deal with the ongoing economic struggles that have hit the nation, the NetEqualizer’s rare combination of effectiveness and affordability has been a major factor fueling this growth.

Other factors driving the upturn are:

  1. Comcast has adopted a similar fairness-based strategy to solve Internet congestion issues, thus validating APconnections’ long-held belief that deep packet inspection is on its way out. (See APconnections’ previous announcements on net neutrality: http://www.netequalizer.com)
  2. Direct sales and support for 90 percent of their customers, thus reducing the overall cost of sales.
  3. Simple turnkey set-up allowing new customer installations to require only one hour of support.

The NetEqualizer is a plug-and-play bandwidth control and WAN/Internet optimization appliance that is flexible and scalable. When the network is congested, NetEqualizer’s unique “behavior shaping” technology gives priority to latency sensitive applications, such as VoIP and email. It does it all dynamically and automatically, improving on other bandwidth shaping technology out there. It controls network flow for the best WAN optimization.

APconnections is a privately held company founded in 2003 and is based in Lafayette, Colorado.

YouTube: The Unfunded Mandate


As some of you may know, I have chimed in several times on the debate on Internet access and the games ISP play to block certain types of traffic (Bittorrent).  I have leaned toward the side of Internet providers and defended some of their restrictive practices. I took quite a bit of heat for some of my previous positions. For example, this excerpt was posted in a discussion forum as a reply to an opinion piece I wrote recently for Extreme Tech magazine:

“So I was wondering why Extremetech would allow such blatant misinformation and FUD on their site…”

First off, please understand my point of reference before assuming I am an industry shill. I am an unbiased observer sitting on the sideline.

Secondly, you can villainize providers all you want, but they exist to make a profit. It is, after all, a business. And now they are facing a new threat with the explosion of YouTube and other video content. Here are some trends that we have seen.

Back in 2006, on a typical footprint of usage patterns on an ISP network, streams exceeding 200kbs (that is 200 kilo bits of data per second) averaged around 2 percent of the users at any one time. Almost all other streams were well under 50kbs. The 2006  ratio of big users to small users allowed  a typical Internet provider to serve approximately 500 people on a 10 megabit circuit without any serious issues. Today we are seeing 10 to 15 percent of the active streams exceeding 200 kbs. That is about a 700 percent increase in the last two years. This increase is mostly attributed to increased online video with  YouTube leading the way.

The ramification of YouTube and its impact on bandwidth demands is putting the squeeze on providers– like it or not they have not choice to but to implement some sort of quota system on bandwidth. Providers invested in certain sized networks and capacities based on the older usage model and smaller increases over time, not 700 percent in 2 years.  Some providers did build out higher capaciites with the hopes of reaping returns by supplying  their own video content, but as the caption says, running other people’s video content without sharing the revenue was not planned for.

Was that a mistake this lack of capacity an evil greed driven conspiracy? No, it was just all they could afford at that time. Video has always been out there, but several years ago it was just not in any form of original content that made it compelling to watch from a public content site . I am not predicting Armageddon caused by overburdened Internet access, however, in the next few years you will see things get ugly with finger pointing and most likely Congress getting involved, obviously to saber rattle and score brownie points with their constituents.

With all that said, we will do our best to stay net neutral and help everybody sort it out without playing sides.

See our recent article on net neutrality for more details.

NetEqualizer Gains Traction against Competition in Australia


In a recent discussion on how and where to deploy a NetEqualizer Stephan Wickham, Product Marketing Manager for KeyTrust (keytrust.com.au), had the following astounding revelation:

“My view is to try NetEqualizer and see how it works – I would then only apply a more expensive solution in instances that require special features or functionality not available with NetEqualizer. I believe this approach is the most practical. I also don’t believe that identifying and reporting on 100s of application types as performed by other products on the market serves much purpose. It would be like trying to manage freeway traffic flow by the identifying vehicle types and then reserving lanes per type. NetEqualizer works more like identifying a gang riding Harleys disrupting traffic and turns them into nice people riding Vespa scooters going with the flow.”

Failover and NetEqualizer: The Whys and Why Nots


Do you want failover on your NetEqualizer or wondered why it’s not available? Let me share a story with you that has developed our philosophy on failover.

A long time ago, back in 1993 or so, I was the Unix and operating system point person for the popular AT&T (i.e. Lucent and Avaya) voice messaging product called Audix. It was my job to make sure that the Unix operating system was bug free and to trouble shoot any issues.

At the time, Audix sales accounted for about $300 million in business and included many Fortune 500 companies around the world. One of the features which I investigated, tested, and certified was our RAID technology. The data on our systems consisted of the archives of all those saved messages that were so important, even more so before e-mail became the standard.

I had a lab setup with all sorts of disk arrays and would routinely yank one from the rack while an Audix system was running. The RAID software we’d integrated worked flawlessly in every test. We were one of the largest companies in the world and we spared no expense to ensure quality in our equipment, and we also charged a premium for everything we sold. If the RAID line item feature was included with an Audix system, it could run as high as $100,000.

Flash forward to the future. We get a call that a customer has lost all their data. A RAID system had failed. It was a well-known insurance company in the Northeast. Needless to say, they were not pleased that their 100 K insurance policy against disk failure did not pan out.

I had certified this mechanism and stood behind it. So, I called together the RAID manufacturer and several Unix kernel experts to do a postmortem. After several days locked in a room, we found was that the real world failure did not follow the lab testing where we had pulled live disk drives in our lab. In fact, it failed in such a way as to slowly corrupt the customer data on all disk drives rendering it useless.

I did some follow up research on failover strategies over the years and discovered that many people implement them for political reasons to cover their asses. I do not mean to demean people covering their asses, it is an important part of business, but the problem is the real cost of testing and validating failover is not practical for most manufacturers.

Many customers ask, “If a NetEqualizer fails, will the LAN cards still pass data?” The answer is, we could certainly engineer our product this way, but there is no guarantee for fail safe systems.

Here are the pros and cons of such a technology:

1) Just like my disk drive failure experience, a system can fail many different ways and the failover mechanism is likely not foolproof. So, I don’t want to recreate history for something we cannot (nor can anybody) reliably real-world test.

2) NetEqualizer’s failure rate is about two percent over two years, which is mostly attributed to harsh operating conditions. That means you have a 1 in 50 chance of having a failure over a two-year period. Put simply, the odds are against this happening.

3) If a NetEqualizer fails, it is usually a matter of moving a cable, which can be easily fixed. So, if you, or anyone with access to the NetEqualizer, are within an hour of your facility, that means you have a 1 in 50 chance of your network being down for one hour every two years because of a NetEqualizer.

4) Customers that really need a fully redundant failover for their operation duplicate their entire infrastructure and purchase two NetEqualizers. These customers are typically brokerage houses where large revenue could be lost. Since they already have a fully tested strategy at the macro level, a failover card on the NetEqualizer is not needed.

5) For customer that is just starting to dabble, they have gone to Cisco spanning tree protocol. Cisco has many years and billions of dollars invested in their switching technology and is rock solid.

6) Putting LAN failover cards in our product would likely raise our base price by about $1000. That would be a significant price increase for most customers, and one that would most likely not be worth paying for.

7) Most equipment failures are software or system related. We take pride in the fact that our boxes run forever and don’t lock up or need rebooting. A failover LAN card does not typically protect against system-type failures.

So, yes, we could sell our system as failsafe with a failover LAN card, but we would rather educate than exploit fears and misunderstandings. Hopefully we’ve accomplished that here.

Comcast Should Adopt Behavior-Based Shaping to Stay out of Trouble


Well it finally happened…

As reported by the NY times :

SAN FRANCISCO — Comcast, the country’s largest residential Internet provider, said on Thursday that it would take a more equitable approach toward managing the ever-expanding flow of Web traffic on its network.

The cable company, based in Philadelphia, has been under relentless pressure from the Federal Communications Commission and public interest groups after media reports last year that it was blocking some Internet traffic of customers who used online software based on the popular peer-to-peer BitTorrent protocol.

As many of our ISP customers already know, we have been proselytizing that using layer-7 packet shaping is a slippery slope for any provider and it was only a matter of time before a large provider such as Comcast would be forced to change their ways.

Note: Layer-7 shaping involves looking at data to determine what it is. A technique commonly used to identify bit torrent traffic.


The NetEqualizer methodology for application shaping has been agnostic with respect to type of data for quite some time. We have shown through our thousands of customers that you can effectively control and give priority to Internet traffic based on behavior. We did not feel comfortable with our layer-7 application shaping techniques and hence we ceased to support that methodology almost two years ago. We now manage traffic as a resource much the same way a municipality would/should ration water if there was a shortage.

Customers understand this. For example, if you simply tell somebody they must share a resource such as water, the Internet, or butter (as in WWII), and that they may periodically get a reduced amount, they will likely agree that sharing the resource is better than one person getting all of the resource while others suffer. Well, that is exactly what a NetEqualizer does with Internet resources, albeit in real time. Internet bandwidth is very spiky. It comes and goes in milliseconds and there is no time for a quorum.

We’ll keep an eye on this for you. If you are interested in learning more about how our technology differs from application-based shaping, the following link is very useful:

http://www.netequalizer.com/Compare_NetEqualizer.php

NetEqualizer Trivia, Famous Encounters with bandwidth shaping


What do Lance Armstrong, Barack Obama have in common with NetEqualizer? Read on to find out.

The engineers at APconnections, being the geeks that they are, like to play a little game of trying to make a valid case for famous people who may have used a NetEqualizer. Loosely defined this means have they ever logged into the Internet through an ISP provider that uses NetEqualizer for their bandwidth control.

Obviously most of this game based on p racticalspeculation, but there are some compelling cases.  In the case of Barack Obama it is a matter of timing. The diplomatic American Embassy in Kabul runs an unsecured  wireless internet service for employees and visitors. A few months ago they purchased and installed a NetEqualizer , seems there internet link was getting a bit overloaded.  We also know from our sources inside the embassy, that Diplomats, including US Congressmen and Senators, will often stop over, open their laptops and use the wireless network in the Embassy to check personal e-mail. So it is very likely that various US Senators and Congressmen have been logged into our system there, especially over the holidays when they are drumming up support by posing with the troops.  Unfortunately our research shows that Senator Obama’s recent world wide tour had him in Kabul on July 19th. The NetEqualizer did not arrive at the embassy until early October of this year.  A minor disappointment, but things are looking good for the next president.

Note: Sitting presidents do not use public Wi-fi systems when traveling.

Other likely famous users include Lance Armstrong. The Olympic Cycling training center in Colorado springs deploys a NetEqualizer going on two years now certainly Lance has stopped by once or twice over the years?  As for Sarah Palin,  we have quite a few units scattered around  regional ISPs in the state of Alaska.  If Sarah Palin gets out and about with her laptop, there is a good chance she has logged into the Internet through one of our units.

Since we first published this article back in November 2008, we added the Vancouver International Airport as well 100,000 additional users through ISPs throughout the world. We’ll keep searching for celebrity sitings as they come in.