What is Your True Internet Speed? Are those Speed Tests Telling the Truth?

When the consumer Internet came of age back in 1990, there was never any grand plan to insure a consistent speed from one point to another. Somewhere along the line, as the Internet went from an academic tool to an essential consumer device, providers in their effort to “out market” one another began to focus on speed as their primary differentiator. By definition, the Internet is a “best effort” corroboration between providers to move your data. No one provider can guarantee a consistent Internet speed for everything you do.  They only have control over their own physical lines, and even then, there are variables beyond their control (which I will address shortly).

Let’s take a look at the speed of wired networks common to most consumers, Cable and DSL.

The physical line into your house is generally what your cable or DSL provider is talking about when they advertise your Internet speed. Essentially, how fast is the link between the providers NOC and your house. Generally you will have a dedicated line for this, and so your speed on this last mile link does not vary.

The good news is that most consumers are more concerned with watching movies, video, listening to music, etc. than they are about pulling research data of some obscure server in Serbia. Given this reality, the Industry has gotten very smart, and popular content is not hosted at some distant server, but is usually distributed locally to each provider. The best example of this is Netflix. Your Netflix content is most likely coming from a server hosted a few miles from your house in your providers NOC, and not from some grand Netflix central location.

Why is Netflix data hosted locally ?

The dirty industry secret is that your provider pays a fee when you go off their network for data. There are also potential capacity problems when you go off their network.  Is this a bad thing? No not really, it is just a matter of efficiency. We see similar practices in other product distribution models. You don’t drive to New York to pick up a toaster, there is usually one waiting for you at your nearest discount store. For the some of the same reasons, that you don’t go to New York to pick up a toaster, your provider tries to host your digital data locally when possible.

What does this mean for your Internet Speed?

It means that when you retrieve content that your provider hosts locally you are likely going to get your advertised speed. This also holds true for some speed test sites, if they are hosted within your providers network they are going to register a constantly higher speed.

What happens to your Internet speed when you go off your providers network? 

There are several factors that will effect your speed.

The main governing factor affecting speed is the capacity the of your providers exchange point.  This is a switching point where your provider exchanges data with other networks.  Depending on how much investment your provider put into this infrastructure this switching point can back up when there is more data being moved than it has capacity to handle. When this happens you get gridlock at the exchange point, and  your Internet speed can plummet.  Gridlock is always a real possibility because your provider just cannot anticipate all the content you are retrieving and sometimes it is not hosted locally.

What does my provider to to alleviate gridlock not their exchange point?

Some providers will actually lower your Internet speed when you are crossing an exchange point.  Or if their circuits are overloaded in general. I experienced this effect which I described in detail a few months ago when I was updating my IPAD.

After the exchange point the speed at which you get your data external to your providers network depends on the whims of every provider and back bone along the route. That obscure research paper from that server in Serbia , may have to make multiple hops to get out of Serbia and then onto some international back bone, and finally to your providers exchange point. There is no way anyone can anticipate at what rate this data will arrive.

How can I run a speed test that better reflects my speed out to the real Internet, by passing locally hosted speed test servers?

A few years ago we ran into this tool set that deliberately tries to retrieve all kinds of remote data to measure your true internet speed. You can also search out files hosted on obscure servers and try to download them.  Perhaps I’ll run a follow up article documenting some of my experiences.

Five Bars Does not Always Mean Good Data Why ?

I have a remote get-away cabin in the middle of the Kansas Prairie where I sometimes escape to work for a couple of days.   I use my Verizon 4G data service as my Internet connection as this is my best option. Even though I usually have 3 or 4 bars of solid signal, my data service comes and goes. Sometimes it is unbelievably fast, and other times I can’t raise a simple web page before timing out. What gives?
The reason for this variability is the fact that the wireless providers actually have two different networks. One for their traditional phone service, and one for the Internet.  Basically what this means is that the tower sites that you are getting your cell signal from actually have two circuits coming in. One is for the traditional cell service, which is almost always available as long as you have a strong signal (5 bars) on your phone.  And the other carries the legacy phone connection. Each one taking a different path out from the cell tower.

Limited Data Line to towers. The data service to each tower is subject to local or regional congestion depending on where and how your provider connects you to the Internet.  In rural Kansas during the broadband initiative the cellular companies had no Internet presence in the area, so they contracted with the local Internet companies to back haul Internet links to their cell towers. Some of these back haul links to the Internet have very limited data capacity, and hence they can get congested when there are multiple data users competing for this limited resource.

A second reason for slow data service is the limited amount of wireless frequency between your phone and the tower. Even though you may have 4 bars and a good phone connection, it is likely that your wireless provider limits data usage during peak times so they are not forced to drop calls. Think of it like two lanes on a highway, one is the priority lane for phone service , and then there is the data lane which can get jammed with data.

So the next time you can’t find directions to your favorite restaurant, or Siri is having a fit, just remember not all is fair on the data circuit to your tower and beyond.

NetEqualizer DDoS Firewall: Simple and Effective without the Bloat

One of the challenges when creating a security tool is validating that it works when the S$%^ hits the fan.  We have heard (via anonymous sources) that many of the high-dollar solutions out there create bloated, rotting piles of information, whose only purpose is to look impressive due to their voluminous output.  A typical $100K buys you a CYA report. A tool that covers  everything, leaving the customer to decide what to do; which is usually nothing or some misguided “make work”. These non-specific tools are about as useful as a weather forecast that predicts everything all the time. Rain, Snow, Wind, Hot, Cold, for everyday of the year. If you predict everything you can’t be wrong?

On the other hand, the reports from the field coming in for our DDoS tool are:

Yes, it works.

Yes, it is simple to use.

Yes, it takes action when appropriate.

We have confirmation that our DDoS tool, combined with our shaping algorithms, has kept some very large institutions up and running while under very heavy, sophisticated DDoS attacks.   The reasons are simple. We look at the pattern of incoming packets in a normal situation.  When the pattern reaches a watermark that is clearly beyond normal, we block those incoming circuits. If needed, we can also take a softer approach, so the attacker is not aware we are throttling them.  This is needed because in some situations outright blocking will alert the attacker you are on to them and cause the attacker to double-down.

When under DDoS attack you don’t need reports; you need immediate action. If you would like to discuss our solution in more detail feel free to contact us.

A Cure for Electronic Theft?

What if we created  a new electronic currency a-la Bitcoin with a twist.   Let’s start by taking an idea from the Federal Government, and put a water mark on our personal funds , something unique that signifies who legally possesses the currency. Cattle ranchers do this with a brand so nobody steals their cattle.  This has worked pretty well for a few hundred years right ?

With our new personal watermark, suppose somebody breaks into your bank, and wires all your money to some idiot in Russia. In today’s world the only way to find that money is to follow the trail, and that takes a huge effort from a banking forensics person, working with International governments.  The money may travel so fast it may not be possible to recover. Now, suppose the funds had an electronic tag that could not be altered by a criminal.   For example currency in your possession  has  a public private encryption key, and only you can authorize a change in possession.

I am not going to spend any more effort on the mechanics of currency ownership, suffice to say it could be done in many different ways. The problem with my proposed solution is the resistance it will meet from all sides.

  • The privacy crowd, will beat the drum and scare ignorant people  into thinking that the government will know how much money they have. The flaw with this argument is , unless you are underground and dealing in cash now, every bank transaction you have ever made is visible to the government. In essence, there is no net change here in terms of privacy. I’d also be fine with an optional cash currency for those that want to opt out, I don’t really care. For tax paying citizens with nothing to hide there is no new privacy downside to watermarking your funds.
  • The security industry will backdoor fight this tooth and nail. As I alluded to in a previous article , the security business has grown to a magnitude of scale well beyond the assets they protect. In other words the security industry is extorting more funds than the actual threat they are protecting you against.
  • Mexico, a country that does 80 billion plus in the drug trade, has no interest in traceable funds. Someplace, some-where, they  will lobby against this change, under the guise of some legitimate reason.
  • Politicians and their donors. Despite the rhetoric, there is absolutely no incentive to make this process transparent.

IT Security Business Is Your Frenemy

Is there a security company out there working in conjunction with a hacker, possibly creating the demand for their services? The old Insurance protection shakedown turned high tech? And, if so, how would you know?  I try to make it clear to our customers  that we are not in the security business for this very reason, but for most IT equipment and consulting companies security is becoming their main business driver.

If the world’s largest automaker will commit fraud to gain an advantage, there must be a few security companies out there that might rationalize breaking into a companies network, while at the same time offering them security equipment in order to make a sale.  Perhaps they are not meeting their sales goals, or facing bankruptcy, or just trying to grow. The fact is, IT investment in security is big business.   The train is rolling down the tracks, and just like our war on drugs, increased spending and manpower seems to have no measurable results.  Who makes more money, companies that make bank vaults, or the criminals that attempt to rob banks? I bet, if you add up all the revenue gleaned from stolen credit cards or other electronic assets, that it is pennies on the dollar when compared to spending on IT security.

Yikes I Have Been Hacked

I had just opened up my network to outside requests ,thinking this will only take a few minutes.  The idea was to  attack my home network from the outside, blasting it  with endless loops of rapid queries from external servers in cyber space, thus simulating a DDOS attack  .    It turns out I was not alone in attacking my Network .

When I went to my monitor DDOS monitor screen to see my attack, I saw  the chart below.   All those Source Ports showing  22 are the result of a server on my network , randomly attempting to login to computers outside my network .  How ironic , while testing my own DDOS software from an outside attack , I find out that one of my servers has been hijacked to do the dirty work for some other hacker.  I am only showing about 46 attempts  in the table below, but all in all ,there were about 450 of them.  They  appeared all of a sudden out of nowhere.  And then, Comcast shut me down, when I hit their security circuit breaker.  Or so I surmised, because this is not the first time this has happened to me, and I usually get  a call from Comcast telling me to run my virus software.  You know how you are not supposed to talk to strangers ? Well I had been getting these calls out of the blue from somebody claiming to be “Comcast” security , and the sounds in the background during the scratchy call were like one of those Indian boiler plate call centers … so I had been ignoring them, just humoring these people.  But perhaps they really were Comcast ? Or perhaps this was just the coup do grace from the hacker pretending to be Comcast after orchestrating the attack, in order to gain my trust and get my bank account ?  Like a bad Mission Impossible plot I don’t know who to trust anymore.
Index     SRCP    DSTP    Wavg    Avg       IP1           IP2           Ptcl  Port  Pool  TOS
0     46762      22   203   336  TCP   1   2    1
1     54211      22    29    90  TCP   1   2    1
2     52734      22    15     0  TCP   1   2    1
3        22   33388    42     0  TCP   2   2    1
4        22   49398   238   277  TCP   2   2    1
5     49184      22    66   152  TCP   1   2    1
6        22   49184   163   374  TCP   2   2    1
7     51722      22   142   214  TCP   1   2    1
8     38133      22    11     0  TCP   1   2    1
9     55232      22    93   400  TCP   1   2    1
10     50373      22    20    40  TCP   1   2    1
11        22   40073    21    35  TCP   2   2    1
12        22   39950    11    40  TCP   2   2    1
13        22   51889     9     0  TCP   2   2    1
14        22   53866   204  1036  TCP   2   2    1
15     57596      22    93   236  TCP   1   2    1
16        22   51971   188   384  TCP   2   2    1
17        22   53617   328   580  TCP   2   2    1
18     52574      22   206   338  TCP   1   2    1
19        22   56081    23    93  TCP   2   2    1
20        22   41126   213   771  TCP   2   2    1
21        22   33853   209   384  TCP   2   2    1
22        22   52185   282  2369  TCP   2   2    1
23        22   54224   224  1032  TCP   2   2    1
24        22   52065   710   806  TCP   2   2    1
25     43568      22    28    88  TCP   1   2    1
26        22   39032   200   558  TCP   2   2    1
27     53968      22   148   265  TCP   1   2    1
28     39950      22    17    60  TCP   1   2    1
29        22   44785   320   464  TCP   2   2    1
30     41889      22    13     0  TCP   1   2    1
31        22   35743   233   368  TCP   2   2    1
32        22   48689   298   373  TCP   2   2    1
33     36165      22   226   293  TCP   1   2    1
34     44991      22    53   146  TCP   1   2    1
35     38500      22   180   345  TCP   1   2    1
36     50944      22     8     0  TCP   1   2    1
37     39511      22   168   319  TCP   1   2    1
38     53820      22    16    30  TCP   1   2    1
39     47030      22   225   261  TCP   1   2    1
40        22   38500   367   735  TCP   2   2    1
41     33165      22   119   248  TCP   1   2    1
42     51185      22    18    60  TCP   1   2    1
43     48472      22    18    60  TCP   1   2    1
44     32890      22    89   174  TCP   1   2    1
45     57725      22    75   180  TCP   1   2    1
46        22   55358  1072  1373  TCP   2   2    1

Dear Comcast, Please Stop Slowing my iOS Update

Last week I was forced to re-load my iPad from scratch. So I fired it up and went through the routine that wipes it clean and re-loads the entire OS from the Apple cloud.  As I watched the progress moniker it slowly climbed from 1 hour, then 2 hours, then all the way up to 23 hours –  and then it just stayed there. Now I know the iOS, or whatever they call it on the iPad, is big, but 23 hours big?  I double-checked the download throughput on my NetEqualizer status screen, and sure enough, it was only running at about 60 to 100kbs, no where near my advertised Business Class 20 megabits. So I did a little experiment. I turned on my VPN tunnel, unplugged my iPad for a minute, and then took some steps to hide my DNS (so Comcast had no way to see my DNS requests).  I then restarted my update and sure enough it sped up to about 10 megabits.

To make sure I was not imagining anything I repeated the test.

Without VPN  (slow)

With VPN (fast)

So what is going here, does the VPN make things go faster?   No not really, but it does prevent Comcast from recognizing my iOS update from Apple and singling it out for slower bandwidth.

Why does Comcast (allegedly) shape my download from Apple?

The long story behind this basically boils down to this: it is likely that Comcast really does not have a big enough switch going out to the Internet to support the deluge of bandwidth needed when a group of subscribers all try to update their devices at once.  Especially during peak hours!  Therefor, in order to keep basic services from becoming slow, they single out a few big hitters such as iOS updates.

%d bloggers like this: