Editors note: Updated on Feb 1st, 2012. Our new product, NetGladiator, has been released. You can learn more about it on the NetGladiator website at www.netgladiator.net or calling us at 303.997.1300 x123.
In a few weeks we will be releasing a product to automatically detect and prevent a web application hacker from breaking into a private enterprise. What follows are the details of how this product was born. If you are currently seeking or researching intrusion detection & prevention technology, you will find the following quite useful.
Like many technology innovations, our solution resulted from the timely intersection of two technologies.
Technology 1: About one year ago we starting working with a consultant in our local tech community to do some programming work on a minor feature in our NetEqualizer product line. Fiddlerontheroot is the name of their company, and they specialize in ethical hacking. Ethical hacking is the process of deliberately hacking into a high-profile client company with the intention of exposing their weaknesses. The key expertise that they provided was a detailed knowledge of how to hack into a network or website.
Technology 2: Our NetEqualizer technology is well known for providing state-of-the-art bandwidth control. While working with Fiddler on the Root, we realized our toolset could be reconfigured to spot, and thwart, unwanted entry into a network. A key piece to the puzzle would be our long-forgotten Deep Packet Inspection technology. DPI is the frowned upon practice of looking inside data packets traversing the Internet.
An ironic twist to this new product journey was that, due to the privacy controversy, as well as finding a better way to shape bandwidth, we removed all of our DPI methodology from our core bandwidth shaping product four years ago. Just like with any weapon, there are appropriate uses for DPI. Over a lunch conversation one day, we realized that using DPI to prevent a hacker intrusion was a legitimate use of DPI technology. Preventing an attack is much different from a public ISP scanning and censoring customer data.
So how did we merge these technologies to create a unique heuristics-based IPS system?
Before I answer that question, perhaps you are thinking that revealing our techniques might provide a potential hacker or competitor with inside secrets? More on this later…
The key to using DPI to prevent an intrusion (hack) revolves around 3 key facts:
1) A hacker MUST try to enter your enterprise by exploiting weaknesses in your normal entry points.
2) One of the normal entry points is a web page, and everybody has them. After all, if you had no publicly available data there would be no reason to be attached to the Internet.
3) By using DPI technology to monitor incoming requests and looking for abnormalities, we can now reliably spot unwanted intrusion attempts.
When we met with Fiddler on the Root, we realized that a normal entry by a customer and a probing entry by a hacker are radically different. A hacker attempts things that no normal visitor could even possibly stumble into. In our new solution we have directed our DPI technology to watch for abnormal entry intrusion attempts. This involved months of observing a group of professional hackers and then developing a set of profiles which clearly distinguish them from a friendly user.
What other innovations are involved in a heuristics-based Intrusion Prevention System (IPS)?
Spotting the hacker pattern with DPI was only part of a complete system. We also had to make sure we did not get any false positives – this is the case where a normal activity might accidentally be flagged as an intruder, and this obviously would be unacceptable. In our test lab we have a series of computers that act like users searching the Internet, the only difference is we can ramp these robot users up to hyper-speed so that they access millions of pages over a short period of time. We then measure our “false positive” rate from our simulation and ensure that our false positive rate on intrusion detection is below 0.001 percent.
Our solution, NetGladiator, is different than other IPS appliances. We are not an “all-in-one solution”, which can be rendered useless by alerting you thousands of times a day, can block legitimate requests, and break web functionality. We do one thing very well – we catch & stop hackers during their information discovery process – keeping your web applications secure. NetGladiator is custom-configured for your environment, alerting you on meaningful attempts without false positive alerts.
We also had to dig into our expertise in real-time optimization. Although that sounds like marketing propaganda to impress somebody, we can break that statement down to mean something.
When doing DPI, you must look at and analyze every data stream and packet coming into your enterprise, skipping something might lead to a security breach. Looking at data and analyzing it requires quite a bit more CPU power than just moving it along a network. Many intrusion detection systems are afterthoughts to standard routers and switches. These devices were originally not designed to do computing-intensive heuristics on data. Doing so may slow your network down to a crawl, a common complaint with lower-end affordable security updates. We did not want to force our customers to make that trade-off. Our technology uses a series of processors embedded in our equipment all working in unison to analyze each packet of Internet data without causing any latency. Although we did not invent the idea of using parallel processing for analysis of data, we are the only product in our price range able to do this.
How did we validate and test our IPS solution?
1) We have been putting our systems in front of beta test sites and asking white knights to try to hack into them.
2) We have been running our technology in front of some of our own massive web crawlers. Our crawlers do not attempt anything abnormal but can push through millions of sites and web pages. This is how we test for false positives blocking a web crawler that is NOT attempting anything abnormal.
Back to the question, does divulging our methodology render it easier to breach?
The holes that hackers exploit are relatively consistent – in other words there really is only a finite number of exploitations that hackers use. They can either choose to exploit these holes or not, and if they attempt to exploit the hole they will be spotted by our DPI. Hence announcing that we are protecting these holes is more likely to discourage a hacker, who will then look for another target.
Like this:
Like Loading...
Will Bandwidth Shaping Ever Be Obsolete?
December 1, 2012 — netequalizerBy Art Reisman
CTO – www.netequalizer.com
I find public forums where universities openly share information about their bandwidth shaping policies an excellent source of information. Unlike commercial providers, these user groups have found technical collaboration is in their best interest, and they often openly discuss current trends in bandwidth control.
A recent university IT user group discussion thread kicked off with the following comment:
“We are in the process of trying to decide whether or not to upgrade or all together remove our packet shaper from our residence hall network. My network engineers are confident we can accomplish rate limiting/shaping through use of our core equipment, but I am not convinced removing the appliance will turn out well.”
Notice that he is not talking about removing rate limits completely, just backing off from an expensive extra piece of packet shaping equipment and using the simpler rate limits available on his router. The point of my reference to this discussion is not so much to discourse over the different approaches of rate limiting, but to emphasize, at this point in time, running wide-open without some sort of restriction is not even being considered.
Despite an 80 to 90 percent reduction in bulk bandwidth prices in the past few years, bandwidth is not quite yet cheap enough for an ISP to run wide-open. Will it ever be possible for an ISP to run wide-open without deliberately restricting their users?
The answer is not likely.
First of all, there seems to be no limit to the ways consumer devices and content providers will conspire to gobble bandwidth. The common assumption is that no matter what an ISP does to deliver higher speeds, consumer appetite will outstrip it.
Yes, an ISP can temporarily leap ahead of demand.
We do have a precedent from several years ago. In 2006, the University of Brighton in the UK was able to unplug our bandwidth shaper without issue. When I followed up with their IT director, he mentioned that their students’ total consumption was capped by the far end services of the Internet, and thus they did not hit their heads on the ceiling of the local pipes. Running without restriction, 10,000 students were not able to eat up their 1 gigabit pipe! I must caveat this experiment by saying that in the UK their university system had invested heavily in subsidized bandwidth and were far ahead of the average ISP curve for the times. Content services on the Internet for video were just not that widely used by students at the time. Such an experiment today would bring a pipe under a similar contention ratio to its knees in a few seconds. I suspect today one would need more or on the order of 15 to 25 gigabits to run wide open without contention-related problems.
It also seems that we are coming to the end of the line for bandwidth in the wireless world much more quickly than wired bandwidth.
It is unlikely consumers are going to carry cables around with their iPad’s and iPhones to plug into wall jacks any time soon. With the diminishing returns in investment for higher speeds on the wireless networks of the world, bandwidth control is the only way to keep order of some kind.
Lastly I do not expect bulk bandwidth prices to continue to fall at their present rate.
The last few years of falling prices are the result of a perfect storm of factors not likely to be repeated.
For these reasons, it is not likely that bandwidth control will be obsolete for at least another decade. I am sure we will be revisiting this issue in the next few years for an update.
Share this:
Like this: