More lies and deceit from your ISP

Note: We believe bandwidth shaping is a necessary and very valuable tool for both ISPs and the public. We also support open honest discussion about the need for this technology and encourage our customers to open and honest with their customers.    We do not like deception in the industry at any level and will continue to expose and write about it when we see it. 

Back in 2007, I wrote an article for PC magazine about all the shenanigans that ISPs use to throttle bandwidth.  The article set a record for on-line comments for the day, and the editor was happy.  At that time, I recall feeling like a lone wolf trying to point out these practices.  Finally some redemption came this morning. The FTC is flexing its muscles; they are now taking on AT&T for false claims with respect to unlimited data.

Federal officials on Tuesday sued AT&T, the nation’s second-largest cellular carrier, for allegedly deceiving millions of customers by selling them supposedly “unlimited” data plans that the company later “throttled” by slowing Internet speeds when customers surfed the Web too much.

It seems that you can have an unlimited data plan with AT&T, but if you try to use it all the time, they slow down your speed to the point where the amount of data you get approaches zero. You get unlimited data, as long as you don’t use it – huh?  Does that make sense?

Recently, I have been doing some experiments with Comcast and my live dropcam home video feed.  It seems that if I try to watch this video feed on my business class Comcast, (it comes down from the dropcam cloud), the video will time out within about minute or so. However, other people watching my feed do not have this problem. So, I am starting to suspect that Comcast is using some form of application shaper to cut off my feed (or slow it down to the point where it does not work).  My evidence is only anecdotal.  I am supposed to have unlimited 4 megabits up and 16 megabits down with my new business class service, but I am starting to think there may be some serious caveats hidden in this promise.

Internet User’s Bill of Rights

This is the second article in our series. Our first was a Bill of Rights dictating the etiquette of software updates. We continue with a proposed Bill of Rights for consumers with respect to their Internet service.

1) Providers must divulge the contention ratio of their service.

At the core of all Internet service is a balancing act between the number of people that are sharing a resource and how much of that resource is available.

For example, a typical provider starts out with a big pipe of Internet access that is shared via exchange points with other large providers. They then subdivide this access out to their customers in ever smaller chunks — perhaps starting with a gigabit exchange point and then narrowing down to a 10 megabit local pipe that is shared with customers across a subdivision or area of town.

The speed you, the customer, can attain is limited to how many people might be sharing that 10 megabit local pipe at any one time. If you are promised one megabit service, it is likely that your provider would have you share your trunk with more than 10 subscribers and take advantage of the natural usage behavior, which assumes that not all users are active at one time.

The exact contention ratio will vary widely from area to area, but from experience, your provider will want to maximize the number of subscribers who can share the pipe, while minimizing service complaints due to a slow network. In some cases, I have seen as many as 1,000 subscribers sharing 10 megabits. This is a bit extreme, but even with a ratio as high as this, subscribers will average much faster speeds when compared to dial up.

2) Service speeds should be based on the amount of bandwidth available at the providers exchange point and NOT the last mile.

Even if your neighborhood (last mile) link remains clear, your provider’s connection can become saturated at its exchange point. The Internet is made up of different provider networks and backbones. If you send an e-mail to a friend who receives service from a company other than your provider, then your ISP must send that data on to another network at an exchange point. The speed of an exchange point is not infinite, but is dictated by the type of switching equipment. If the exchange point traffic exceeds the capacity of the switch or receiving carrier, then traffic will slow.

3) No preferential treatment to speed test sites.

It is possible for an ISP to give preferential treatment to individual speed test sites. Providers have all sorts of tools at their disposal to allow and disallow certain kinds of traffic. There should never be any preferential treatment to a speed test site.

4) No deliberate re-routing of traffic.

Another common tactic to save resources at the exchange points of a provider is to re-route file-sharing requests to stay within their network. For example, if you were using a common file-sharing application such as BitTorrent, and you were looking some non-copyrighted material, it would be in your best interest to contact resources all over the world to ensure the fastest download.

However, if your provider can keep you on their network, they can avoid clogging their exchange points. Since companies keep tabs on how much traffic they exchange in a balance sheet, making up for surpluses with cash, it is in their interest to keep traffic confined to their network, if possible.

5) Clearly disclose any time of day bandwidth restrictions.

The ability to increase bandwidth for a short period of time and then slow you down if you persist at downloading is another trick ISPs can use. Sometimes they call this burst speed, which can mean speeds being increased up to five megabits, and they make this sort of behavior look like a consumer benefit. Perhaps Internet usage will seem a bit faster, but it is really a marketing tool that allows ISPs to advertise higher connection speeds – even though these speeds can be sporadic and short-lived.

For example, you may only be able to attain five megabits at 12:00 a.m. on Tuesdays, or some other random unknown times. Your provider is likely just letting users have access to higher speeds at times of low usage. On the other hand, during busier times of day, it is rare that these higher speeds will be available.

There is now a consortium called M-Lab which has put together a sophisticated speed test site designed to give specific details on what your ISP is doing to your connection. See the article below for more information.

Related article Ten things your internet provider does not want you to know.

Related article On line shoppers bill of rights

Editors Choice: The Best of Speeding up Your Internet

Edited by Art Reisman


Over the years we have written a variety of articles related to Internet Access Speed and all of the factors that can affect your service. Below, I have consolidated some of my favorites along with a quick convenient synopsis.

How to determine the true speed of video over your Internet connection: If you have ever wondered why you can sometimes watch a full-length movie without an issue while at other times you can’t get the shortest of YouTube videos to play without interruption, this article will shed some light on what is going on behind the scenes.

FCC is the latest dupe when it comes to Internet speeds: After the Wall Street Journal published an article on Internet provider speed claims, I decided to peel back the onion a bit. This article exposes anomalies between my speed tests and what I experienced when accessing real data.

How to speed up your Internet connection with a bandwidth controller: This is more of a technical article for Internet Service Providers. It details techniques used to eliminate congestion on their links and thus increase the perception of higher speeds to their end users.

You may be the victim of Internet congestion: An article aimed at consumer and business users to explain some of the variance in your network speeds when congestion rears its ugly head.

Just how fast is your 4g network?: When I wrote this article, I was a bit frustrated with all the amazing claims of speed coming with wireless 4G devices. There are some fundamental gating factors that will forever insure that your wired connection will likely always be a magnitude faster than any wireless data device.

How does your ISP enforce your Internet speed?: Goes into some of the techniques used on upstream routers to control the speed of Internet and data connections.

Burstable Internet connections, are they of any value?: Sheds light on the ambiguity of the term “burstable.”

Speeding up your Internet connection with an optimizing appliance: Breaks down the tradeoffs of various techniques.

Why caching alone will speed up your Internet: One of my favorite articles. Caching, although a good idea, often creates great unattainable expectations. Find out why.

QoS is a matter of sacrifice: Explains how quality of service is a “zero sum” game, and why somebody must lose when favoring one type of traffic.

Using QoS to speed up traffic: More on the pros and cons of using a QoS device.

Nine tips and tricks to speed up your Internet connection: A great collection of 15 tips, this article seems to be timeless and continually grows in popularity.

Network bottlenecks when your router drops packets: A simple, yet technical, explanation of how hitting your line speed limit on your router causes a domino effect.

Why is the Internet access in my hotel so slow: Okay I admit i , this was an attempt to draw some attention to our NetEqualizer which solves this problem about 99 percent of the time for the hotel industry. You can bring the horse to water but you cannot make them drink.

Speed test tools from M-labs: The most reliable speed test tool there is, uses techniques that cannot easily be fooled by special treatment from your provider.

Are hotels jamming 3g access?: They may not be jamming 3g but they are certainly in no hurry to make it better.

Five more tips in testing your Internet speed: More tips to test Internet speed.

How to Determine the True Speed of Video over Your Internet Connection

Art Reisman CTO

Editor’s note: Art Reisman is the CTO of APconnections. APconnections designs and manufactures the popular NetEqualizer bandwidth shaper.

More and more, Internet Service Providers are using caching techniques on a large scale to store local copies of Netflix Movies and YouTube videos. There is absolutely nothing wrong with this technology. In fact, without it, your video service would likely be very sporadic. When a video originates on your local provider’s caching server, it only has to make a single hop to get to your doorstep. Many cable operators now have dedicated wires from their office to your neighborhood, and hence very little variability in the speed of your service on this last mile.

So how fast can you receive video over the Internet? (Video that is not stored on your local providers caching servers.) I suppose this question would be moot if all video known to mankind was available from your ISP. In reality, they only store a tiny fraction of what is available on their caching servers. The reason why caching can be so effective is that, most consumers only watch a tiny fraction of what is available, and they tend to watch what is popular. To determine how fast you can receive video over the Internet you must by-pass your providers cache.

To insure that you are running a video from beyond your providers cache, google something really obscure. Like “Chinese language YouTube on preparing flowers.” Don’t use this search term if you are in a Chinese neighborhood, but you get the picture right? Search for something obscure that is likely never watched near you. Pick a video 10 minutes or longer, and then watch it. The video may get broken up, or more subtly you may notice the buffer bar falls behind or barely keeps up with the playing of the video. In any case, if you see a big difference watching an obscure video over a popular one, this will be one of the best ways to analyze your true Internet speed.

Note: Do not watch the same video twice in a row when doing this test. The second time you watch an obscure video from China, it will likely run from the your provider’s cache, thus skewing the experiment.

FCC is the Latest Dupe in Speed-Test Shenanigans

Shenanigans: is defined as the deception or tomfoolery on the part of carnival stand operators. In the case of Internet speed, claims made in the latest Wall Street Journal article, the tomfoolery is in the lack of details on how these tests were carried out.

According to the article, all the providers tested by the FCC delivered 50 megabits or more of bandwidth consistently for 24 hours straight. Fifty megabits should be enough for 50 people to continuously watch a YouTube stream at the same time. With my provider, in a large metro area, I often can’t even watch one 1 minute clip for more than a few seconds without that little time-out icon spinning in my face. By the time the video queues up enough content to play all the way through, I have long since forgotten about it and moved on. And then, when it finally starts playing again, I have to go back and frantically find it and kill the YouTube window that is barking at me from somewhere in the background.

So what gives here? Is there something wrong with my service?

I am supposed to have 10 megabit service. When I run a test I get 20 megabits of download enough to run 20 YouTube streams without issue, so far so good.

The problem with translating speed test claims to your actual Internet experience is that there are all kinds of potentially real problems once you get away from the simplicity of a speed test, and yes, plenty of deceptions as well.

First, lets look at the potentially honest problems with your actual speed when watching a YouTube video:

1) Remote server is slow: The YouTube server itself could actually be overwhelmed and you would have no way to know.

How to determine: Try various YouTube videos at once, you will likely hit different servers and see different speeds if this is the problem.

2) Local wireless problems: I have been the victim of this problem. Running two wireless access points and a couple of wireless cameras jammed one of my access points to the point where I could hardly connect to an Internet site at all.

How to determine: Plug your computer directly into your modem, thus bypassing the wireless router and test your speed.

3) Local provider link is congested: Providers have shared distribution points for your neighborhood or area, and these can become congested and slow.

How to determine: Run a speed test. If the local link to your provider is congested, it will show up on the speed test, and there cannot be any deception.


The Deceptions

1) Caching

I have done enough testing first hand to confirm that my provider caches heavily trafficked sites whenever they can. I would not really call this a true deception, as caching benefits both provider and consumer; however, if you end up hitting a YouTube video that is not currently in the cache, your speed will suffer at certain times during the day.

How to Determine: Watch a popular YouTube video, and then watch an obscure, seldom-watched YouTube.

Note: Do not watch the same YouTube twice in a row as it may end up in your local cache, or your providers local cache, after the first viewing.

2) Exchange Point Deceptions

The main congestion point between you and the open Internet is your providers exchange point. Most likely your cable company or DSL provider has a dedicated wire direct to your home. This wire, most likely has a clean path back to the NOC central location. The advertised speed of your service is most likely a declaration of the speed from your house to your providers NOC, hence one could argue this is your Internet speed. This would be fine except that most of the public Internet content lies beyond your provider through an exchange point.

The NOC exchange point is where you leave your local providers wires and go out to access information from data hosted on other provider networks. Providers pay extra costs when you leave their network, in both fees and in equipment costs. A few of things they can do to deceive you are:

– Give special priority to your speed tests through their site to insure the speed test runs as fast as possible.

– Re-route local traffic for certain applications back onto their network. Essentially limiting and preventing traffic from leaving their network.

– They can locally host the speed test themselves.
How to determine: Use a speed test tool that cannot be spoofed.

See also:

Is Your ISP Throttling your Bandwidth

NetEqualizer YouTube Caching

How to Speed Up Your Internet Connection with a Bandwidth Controller


It occurred to me today, that in all the years I have been posting about common ways to speed up your Internet, I have never really written a plain and simple consumer explanation dedicated to how a bandwidth controller can speed up your Internet. After all, it seems intuitive, that a bandwidth controller is something an ISP would use to slow down your Internet; but there can be a beneficial side to a bandwidth controller, even at the home-consumer level.

Quite a bit of slow Internet service problems are due to contention on your link to the Internet. Even if you are the only user on the Internet, a simple update to your virus software running in the background can dominate your Internet link. A large download often will cause everything else you try (email, browsing) to come to a crawl.

What causes slowness on a shared link?

Everything you do on your Internet creates a connection from inside your network to the Internet, and all these connections compete for the limited amount of bandwidth which your ISP provides.

Your router (cable modem) connection to the Internet provides first-come, first-serve service to all the applications trying to access the Internet. To make matters worse, the heavier users (the ones with the larger persistent downloads), tend to get more than their fair share of router cycles. Large downloads are like the school yard bully – they tend to butt in line, and not play fair.

So how can a bandwidth controller make my Internet faster?

A smart bandwidth controller will analyze all your Internet connections on the fly. It will then selectively take away some bandwidth from the bullies. Once the bullies are removed, other applications will get much needed cycles out to the Internet, thus speeding them up.

What application benefits most when a bandwidth controller is deployed on a network?

The most noticeable beneficiary will be your VoIP service. VoIP calls typically don’t use that much bandwidth, but they are incredibly sensitive to a congested link. Even small quarter-second gaps in a VoIP call can make a conversation unintelligible.

Can a bandwidth controller make my YouTube videos play without interruption?

In some cases yes, but generally no. A YouTube video will require anywhere from 500kbs to 1000kbs of your link, and is often the bully on the link; however in some instances there are bigger bullies crushing YouTube performance, and a bandwidth controller can help in those instances.

Can a home user or small business with a slow connection take advantage of a bandwidth controller?

Yes, but the choice is a time-cost-benefit decision. For about $1,600 there are some products out there that come with support that can solve this issue for you, but that price is hard to justify for the home user – even a business user sometimes.

Note: I am trying to keep this article objective and hence am not recommending anything in particular.

On a home-user network it might be easier just to police it yourself, shutting off background applications, and unplugging the kids’ computers when you really need to get something done. A bandwidth controller must sit between your modem/router and all the users on your network.

Related Article Ten Things to Consider When Choosing a Bandwidth Shaper.

You May Be the Victim of Internet Congestion

Have you ever had a mysterious medical malady? The kind where maybe you have strange spots on your tongue, pain in your left temple, or hallucinations of hermit crabs at inappropriate times – symptoms seemingly unknown to mankind?

But then, all of a sudden, you miraculously find an exact on-line medical diagnosis?

Well, we can’t help you with medical issues, but we can provide a similar oasis for diagnosing the cause of your slow network – and even better, give you something proactive to do about it.

Spotting classic congested network symptoms:

You are working from your hotel room late one night, and you notice it takes a long time to get connected. You manage to fire off a couple emails, and then log in to your banking website to pay some bills. You get the log-in prompt, hit return, and it just cranks for 30 seconds, until… “Page not found.” Well maybe the bank site is experiencing problems?

You decide to get caught up on Christmas shopping. Initially the Macy’s site is a bit a slow to come up, but nothing too out of the ordinary on a public connection. Your Internet connection seems stable, and you are able to browse through a few screens and pick out that shaving cream set you have been craving – shopping for yourself is more fun anyway. You proceed to checkout, enter in your payment information, hit submit, and once again the screen locks up. The payment verification page times out. You have already entered your credit card, and with no confirmation screen, you have no idea if your order was processed.

We call this scenario, “the cyclical rolling brown out,” and it is almost always a problem with your local Internet link having too many users at peak times. When the pressure on the link from all active users builds to capacity, it tends to spiral into a complete block of all access for 20 to 30 seconds, and then, service returns to normal for a short period of time – perhaps another 30 seconds to 1 minute. Like a bad case of Malaria, the respites are only temporary, making the symptoms all the more insidious.

What causes cyclical loss of Internet service?

When a shared link in something like a hotel, residential neighborhood, or library reaches capacity, there is a crescendo of compound gridlock. For example, when a web page times out the first time, your browser starts sending retries. Multiply this by all the users sharing the link, and nobody can complete their request. Think of it like an intersection where every car tries to proceed at the same time, they crash in the middle and nobody gets through. Additional cars keep coming and continue to pile on. Eventually the police come with wreckers and clear everything out of the way. On the Internet, eventually the browsers and users back off and quit trying – for a few minutes at least. Until late at night when the users finally give up, the gridlock is likely to build and repeat.

What can be done about gridlock on an Internet link?

The easiest way to prevent congestion is to purchase more bandwidth. However, sometimes even with more bandwidth, the congestion might overtake the link. Eventually most providers also put in some form of bandwidth control – like a NetEqualizer. The ideal solution is this layered approach – purchasing the right amount of bandwidth AND having arbitration in place. This creates a scenario where instead of having a busy four-way intersection with narrow streets and no stop signs, you now have an intersection with wider streets and traffic lights. The latter is more reliable and has improved quality of travel for everyone.

For some more ideas on controlling this issue, you can reference our previous article, Five Tips to Manage Internet Congestion.

%d bloggers like this: