Network Provider Outages and DDoS Attacks Dwarf Local Hardware Failure Problems


My Internet Service went down yesterday and I had to revert to my backup provider.

Network Outages due to upstream provider failure are endless…

Comcast Outage for North Denver Fiber cut

Comcast hit with massive Internet outage

Forum discussion about wide spread Internet outage Des Moines Iowa

Spokane Washington 10,000 customers without Internet service

Wide spread Internet outage London , Virgin Media

And even if your provider is not to blame, there are endless hackers out there instigating DDoS attacks , some with an ax to grind others just for random entertainment.

DDoS attack brings down Web Drive Client New Zealand

DDoS attack brings down dutch government

DDoS attack interrupts tournament.

Although this sampling of news stories is not very scientific, I could literally spend a month clipping these articles. There are new ones every day, and that is just the major ones that get reported. If I informally poll our customers, almost every single one of them has seen a DDoS attack of  some kind in the past year, and all have had some sort of upstream Internet outages within the last couple of years.

Now if I ask how many have had critical Network Equipment go down due to hardware failure, that list shrinks to maybe 1 or 2 percent of our customers. Basically, what this tells me is you have a 100 percent chance of a Network outage for some period of time every year due to a problem upstream with your provider. You have  a 2 percent chance due to a hardware failure with your local core Router/Firewall/Bandwidth/Switches.

To put that another way, for every 50 outages caused by external events at your provider beyond your control, you have 1 event due to internal hardware failure.

The solution is to have multiple distinct Internet Providers on hand at all times, so if one goes down you can switch over to the other. As I said there is nothing wrong with the idea of sourcing redundant local equipment, but statistically it is much more important to get a second Internet provider sourced before investing in redundant equipment.

Here is another article highlighting the prevalence network outages.

Note: Although DDoS attacks are provider Independent, your chances of stopping or mitigating the attack are enhanced by having multiple providers.

Other causes of failures:
Yes, wireless topologies are notoriously unstable, and so are applications running on Web Servers, both of which can cause service outages to local users. These types outages are usually not on the same order as catastrophic hardware failure problems or upstream failures. Outages with wireless equipment and service are usually related to these products getting into a bad state, and are not associated with a complete loss of communication to the outside world. You’ll still need to re-boot these systems to get them back into a good state.

Related Articles: 

The Top Five Causes of Disruption of Internet Service

Five Tips for Defending Against a DDoS Attack

 

 

Posted in DDoS. 1 Comment »

Firewall Recipe for DDoS Attack Prevention and Mitigation


Although you cannot “technically” stop a DDoS attack, there are ways to detect and automatically mitigate the debilitating effects on your public facing servers. Below, we shed some light on how to accomplish this without spending hundreds of thousands of dollars on a full service security solution that may be overkill for this situation.

Most of the damage done by a targeted DDoS attack is the result of the overhead incurred on your servers from large volume of  fake inquiries into your network. Often with these attacks, it is not the volume of raw bandwidth  that is the issue, but the reduced the slow response time due to the overhead on your servers. For a detailed discussion of how a DDoS attack is initiated please visit http://computer.howstuffworks.com/zombie-computer3.htm zombie-computer-3d

We assume in our recipe below, that you have some sort of firewall device on your edge that can actually count hits into your network from an outside IP, and also that you can program this device to take blocking action automatically.

Note: We provide this type of service with our NetGladiator line. As of our 8.2 software update, we also provide this in our NetEqualizer line of products.

Step 1
Calculate your base-line incoming activity. This should be a running average of unique hits per minute or perhaps per second. The important thing is that you have an idea of what is normal. Remember we are only concerned with Un-initiated hits into your network, meaning outside clients that contact you without being contacted first.

Step 2
Once you have your base hit rate of incoming queries, then set a flag to take action ( step 3 below), should this hit rate exceed more than 1.5 standard deviations above your base line.  In other words if your hit rate jumps by statistically large amount compared to your base line for no apparent reason i.e .you did not mail out a newsletter.

Step 3
You are at step 3 because you have noticed a much larger than average hit rate of un-initiated requested into your web site. Now you need to look for a hit count by external IP. We assume that the average human will only generate at most a hit every 10 seconds or so, maybe higher. And also on average they will like not generate more than 5 or 6 hits over a period of a few minutes.  Where as a hijacked client attacking your site as part of a DDOS attack is likely to hit you at a much higher rate.  Identify these incoming IP’s and go to Step 4.

Step 4
Block these IP’s on your firewall for a period of 24 hours. You don’t want to block them permanently because it is likely they are just hijacked clients ,and also if they are coming from behind a Nat’d community ( like a University) you will be blocking a larger number of users who had nothing to do with the attack.

If you follow these steps you should have a nice pro-active watch-dog on your firewall to mitigate the effects of any DDoS attack.

For further consulting on DDoS or other security related issues feel free to contact us at admin@apconnections.net.

Related Articles:

Defend your Web Server against DDoS Attacks – techrecipes.com

How DDoS Attacks Work, and Why They’re Hard to Stop

How to Launch a 65 gbps DDoS Attack – and How to Stop It

%d bloggers like this: