Tracking Traffic by DNS


The video rental industry of the early 80’s was comprised of 1000’s of independent stores.  Corner video rental shops were as numerous as today’s Starbucks.  In the late 1990’s, consolidation took over.  Blockbuster with its bright blue canopy lighting up the night sky swallowed them up like doggy treats.   All the small retail outlets were gone. Blockbuster had changed everything, their economy of scale, and their chain store familiarity, had overrun the small operators.

In a similar fashion to the fledgling video rental industry, circa 1990’s Internet content was scattered across the spectrum of the web, ripe for consolidation.  I can still remember all of the geeks at my office creating and hosting their own personal websites. They used primitive tools and their own public IP’s to weave these sites together.  Movies  and music were bootlegged, and shared across a network of underground file-sharing sites.

Although we do not have one Internet “Blockbuster” today, there has been major consolidation.  Instead of all traffic coming from 100’s of thousands of personal or small niche content providers, most of it comes from the big content providers. Google, Amazon, Netflix, Facebook, Pinterest are all familiar names today.

So far I have reminisced about a nice bit of history, and I suspect you might be wondering how all of this prelude relates to tracking traffic by DNS?

Three years ago we added a DNS (domain name system) server lookup from our GUI interface, as more of a novelty than anything else. Tracking traffic by content was always a high priority for our customers, but most techniques had relied on a technology called “deep packet inspection” to identify traffic.  This technology was costly, and ineffective on its best day, but it was the only way to chase down nefarious content such as P2P.

Over the last couple of years I noticed again the world had changed. With the consolidation of content from a small number of large providers, you could now count on some consistency in the domain from which it originated.  I would often click on our DNS feature and notice a common name for my data.   For example, my YouTube videos resolved to one or two DNS names,  and I found the same to be true with my Facebook video.  We realized that this consolidation might make DNS tracking useful for our customers, and so we have now put DNS tracking into our current NetEqualizer 8.5 release.

Another benefit of tracking by domain is the fact that most encrypted data will report a valid domain.  This should help to identify traffic patterns on a network.

It will be interesting to get feedback on this feature as it hits the real world, stay tuned!

India IT a Limited Supply


Before founding my current company, I was on the technical staff for a large telecom provider.  In the early 1990’s about half of our tech team were hired on the H-1 visa’s  from India, all very sharp and good engineers.  As the tech economy heated up, the quality of our Engineers from India dropped off significantly, to the point where many were actually let go after trial periods, at a time when we desperately needed technical help.

The unlimited supply of offshore engineering talent evidently had its limits.  To illustrate I share the following experience.

Around the year 2000, in the height of the tech boom, my manager, also from India, sent me on a recruiting trip to look for grad students at a US job fair hosted for UCLA students.

In my pre-trip briefing we went over a list of ten technology universities in India, as he handed me the list he said,  “Don’t worry about a candidates technical ability, if they come from any one of these ten universities they are already vetted for competency, just make sure they have a good attitude, and can think out-of-the-box.”

He also said if they did not attend one of the 10 schools on the list then don’t even consider them, as there is a big drop off in talent at the second tier schools in India.

Upon some further conversations I learned that India’s top tech schools are on par with the  best US undergrad engineering schools.  In India there is extreme competition and vetting to get into these schools.  The dirty little secret was that there were only a limited number of graduates from these universities.  Initially, US companies were only seeing the cream of the Indian Education system.  As the tech demand grew, the second tier engineers were well-enough trained to “talk the talk” in an interview, but in the real world they often did not have that extra gear to do demanding engineering work and so projects suffered.

In the following years, many US-based engineers in the trenches saw some of this incompetence and were able to convince their management to put a halt to offshoring R&D projects when the warning signs were evident.  These companies seemed to be in the minority.  Since many large companies treated their IT staff, and to some extent their R&D staff, like commodities, they continued to offshore based on lower costs and the false stereotype that these Indian companies could perform on par with their in-house R&D teams.  The old adage you get what you pay for held true here once again.

This is not to say there were not some very successful cost savings made possible by Inidan engineers,  but the companies that benefited were the ones that got in early and had strong local Indian management, like my boss, who knew the limits of Indian engineering resources.

How I Survived a Ransomware Attack


By Art Reisman

About six months ago, I was trying to access a web site when I got the infamous message: “Your Flash Player is out-of-date”.  I was provided with a link to a site to update my Adobe Flash Player.  At the time, I thought nothing of updating my Flash Player, as this had happened perhaps 100 times already. That begs the question as to why my perfectly fine and happy Adobe Flash Player constantly needs to be updated?  Another story for another day.

In my haste, I clicked the link and promptly received the Adobe Flash update for my Mac and installed it. For all intents and purposes, that was the end of my Mac.  This thing just took it over, destroying it.  It would insidiously let me get started with my daily work and then within a few minutes I would receive a barrage of almost constant messages popping up telling me I had a virus and to call some number for help.  Classic Ransomware.  At the time I did not think Macs were vulnerable to this type of thing, as the only viruses I had contracted prior were on my Windows machines, which I tossed in the scrap pile several years ago for that very reason.

My solution to this dilemma was simply to re-load my Mac from scratch.  I was up and running again in about one hour.   A hassle yes, the end of the world – no.

Now you might be wondering what about all my data programs and files I store on my Mac?  And to that I answer what data files?  Everything I do is in the Cloud, nothing is stored on my Mac, as I believe that there is no reason to store anything locally.

Gmail, Quickbooks, WordPress, photos, documents, and everything else that I use are all stored in the Cloud!

For backup purposes, I periodically e-mail a list of all my important Cloud links to myself.  Since they are stored in Gmail, they are always accessible and I can access them from any computer.  Data recovery amounts to nothing more than finding my most recent backup list e-mail and clicking on my Cloud links as needed.

NetEqualizer News: May 2017


We hope you enjoy this month’s NetEqualizer Newsletter. Highlights include an update on the 8.5 Release, a reminder about our NetEqualizer Leasing program, a customer story about how RTR helped them identify a virus in their network, a refreshed NetEqualizer white paper, and more!

 

May 2017

 

8.5 Release – Development is Complete!
Greetings! Enjoy another issue of NetEqualizer News.

Our 8.5 Release development has completed, five happy stars to the development team! This month we talk about Enhanced DNS resolution reporting, and preview the Read-Only Administrative Access screens. If you would like to get your hands on the 8.5 release as a beta tester, please contact us to participate. Look for the 8.5 Release to be generally available early summer 2017.  

We continue to work with you to solve some of your most pressing network problems – so if you have one that you would like to discuss with us, please call or email me anytime at 303.997.1300 x103 or art@apconnections.net.

And remember we are now on Twitter. You can follow us @NetEqualizer.

– Art Reisman (CTO)

In this Issue:

:: First Impressions on our 8.5 Release

:: NetEqualizer Leasing always great option!

:: RTR Traffic History Reports Capture Unknown Virus Activity for WNPL

:: Updated Executive White Paper

:: Best of Blog: How to Create and Send an Encrypted File With No NSA Backdoor

First Impressions on our 8.5 Release

8.5 Release Development is Complete!

I finally got a chance to kick the tires on our 8.5 release, and it kicked me back! I really like so much in 8.5, it is difficult to determine where to start!

The biggest wow factor to hit me was our enhanced DNS reporting. For years our customers have been asking, “Can you tell me how much traffic went to YouTube or Netflix?” Well, I am thrilled to say that finally the answer is yes.

With our DNS reporting you can now track all the data to any well-known public domain. The screen shot below of our Active Connections Report says it all. You can see the domains in the DST (destination) and SRC (source) columns for all live traffic flowing through the network.

In addition to being able to see public domains for all active traffic, you can now also view Traffic History for any host names that you have set up to track. You can see history for a selected public domain in the screenshot below.

I also love our expanded login capabilities. Many of our customers have a support Help Desk team on site. These are the first responders for network problems in your organization. Just like their counterparts in the medical field (EMTs), they may not be trained in advanced surgical techniques. For example, you would not want them making policy changes on your bandwidth controller, but you definitely need your help desk personnel to do some initial triage, gather some data, and keep the patient alive, until the doctor arrives on scene.

Our NEW Read Only login will give them access to RTR and all its advanced reporting screens, without the possibility of any life-threatening policy changes to the network.

You can see that this person is logged in as Read Only by looking in the top right portion of the menu bar for the RTR icon, or looking for the “report” login next to the new Logout feature.

And finally, we continue to expand our Preferences capabilities, so that you can customize your experience with the NetEqualizer.

“Remember one hundred and twenty eight thousand bytes per second is a Megabit.” I was getting tired of uttering this phrase and being an apologist for our units displayed on our connection tracking (Active Connections) screen. In fairness to our engineering team, the connection tracking table started out as a simple troubleshooting tool for internal use, and has now morphed into one of the more important real-time reporting screens for our customers.

With 8.5 we bring you units in Megabits (or whatever unit your preference may be)! See the screenshot below. Once you select your preference, these units will be on your Active Connections Report.

With our 8.5 release entering the system test phase, we will soon be looking for Beta Testers (June/July timeframe). If you are interested in participating, please contact us.

NetEqualizer leasing always a great option!

Check out our leasing program!

Do you have a short term need for a bandwidth shaper? Perhaps a conference event, or something to tide you over while waiting for a bandwidth upgrade? Maybe your business model requires you to spread out expenses over time instead of an up front purchase? If you answered yes to one or more of these questions, now is the time to contact us about leasing a NetEqualizer. We offer leases starting at 2 months with no long term contract.

RTR Traffic History Reports Capture Unknown Virus Activity for WNPL

RTR has more uses than you might think…

The following is an email we received from Kevin Getty – Head of Information Technology for Warren-Newport Public Library. Kevin found an interesting use for our RTR interface and associated data.

Thanks Kevin!
————————————————————————————————

Having some spare time one day I decided to see what I could do with the data that’s provided by the four week traffic history report. First I downloaded the data and imported it into a spreadsheet to see what was available and formatted it into a more user friendly display. Once formatted I grouped the data by IP address and started to analyze the upload throughputs by user per day. I quickly realized what “normal” activity was looking like for our network PC’s.

Once the IP’s were resolved to DNS names, I was surprised to see such high bandwidth from a specific PC. Bandwidth that was large and during closed office hours spreading across multiple days. I approached the user to see if they had any experience with slow or intermittent internet access and sure enough they did. Their experience of slowness was the NetEQ doing it’s wonderful job of penalizing them and normally it should, but the user experienced slowness due to a bug on the computer. They also stated they left their PC on overnight because they didn’t want to lose what they had been working on, so this explained why the traffic showed during closed hours. When asked if they knew when they started to experience slowness, their answer matched what the data showed in the four week report.

The next step was to investigate to see if the PC had any bugs or malware that hadn’t been caught yet. Using several antivirus and malware removal utilities that are known to be successful at finding the little buggers, all came up empty. So the next thing was to restore the user’s Windows profile from backup prior to the high traffic use. After restoring the profile and monitoring for a day, the reports showed normal traffic.

Since then I’ve written a windows application that will download the reports and analyze the traffic. Granted, not all high usage is necessarily bad traffic, but the uploading was what caught my eye and is analyzed in the application to signal an alarm. The first screenshot shows the overall bandwidth usage of the report.

The second shows the selected PC that was showing high usage. You’ll notice the consistent upload and download across several days.

I do have suggestions for your reports. Selectable date ranges and resolved DNS names would be nice.

I’ve been using NetEQ for over a decade now and I’m not sure what I’d do without it. Thanks for a great product and keep up the great work!

Updated Executive White Paper

Take a look at our refreshed white paper…

Our Executive White Paper has been updated to highlight a key capability of the NetEqualizer – that we are able to shape both encrypted and unencrypted traffic.

This is an important advantage of behavior-based shaping. Application traffic shapers are only able to shape unencrypted traffic, as they need to classify it to work with it. We have also updated the Comparison Table on Page 2 to better highlight how the NetEqualizer compares to Other Traffic Shapers.

The Executive White Paper is a good summary for management to read to get a quick feel of the NetEqualizer’s core capabilities, and also to understand how it differs from traditional application shapers.

Read our updated Executive White Paper here

Best Of Blog

How to Create and Send an Encrypted File with No NSA Backdoor

By Art Reisman

Note: Believe it or not in a previous life, before I settled on computer science, I was a math major. Not much stuck from those days but I did remember one lesson very well, and that was that there are a plethora of ways to implement a mathematic encryption of data. In the following blog article I share with you an easy to use a program for personal encryption.

Below is a little routine I wrote to encrypt and decrypt a file.

This script is meant for encrypting text files and sending them privately through e-mail as an attachment.

Note: The author makes no claims about whether this encryption technique can be broken. It would not be easy.

Here is what you need to use this program.
1. Mac or other computer that can run a perl script from the command line
2. very basic knowledge of how to create and edit a file from the command line

Photo of the Month
Kansas Prairie Home from Days Gone By…

One of our staff members was visiting western Kansas recently, and decided to take a bike tour.  This house is one of many that remain from the Dust Bowl days, when many farmers abandoned their homes due to the prolonged severe drought. These homes dot the land, much of which was given away by the Homestead Acts.

APconnections, home of the NetEqualizer | (303) 997-1300 | Email | Website 

How to Survive High Contention Ratios and Prevent Network Congestion


Is there a way to raise contention ratios without creating network congestion, thus allowing your network to service more users?

Yes there is.

First a little background on the terminology.

Congestion occurs when a shared network attempts to deliver more bandwidth to its users than is available. We typically think of an oversold/contended network with respect to ISPs and residential customers; but this condition also occurs within businesses, schools and any organization where more users are vying for bandwidth than is available.

 The term, contention ratio, is used in the industry as a way of determining just how oversold your network is.  A contention ratio is simply the size of an Internet trunk divided by the number of users. We normally think of Internet trunks in units of megabits. For example, 10 users sharing a one megabit trunk would have a 10-to- 1 contention ratio.
 A decade ago, a 10-to-1 contention ratio was common. Today, bandwidth is much less expensive and the average contention ratios have come down.  Unfortunately, as bandwidth costs have dropped, pressure on trunks has risen, as today’s applications require increasing amounts of bandwidth. The most common congestion symptom is  slow network response times.
Now back to our original question…
Is there a way to raise contention ratios without creating congestion, thus allowing your network to service more users?
This is where a smart bandwidth controller can help.  Back in the “old” days before encryption was king, most solutions involved classifying types of traffic, and restricting less important traffic based on customer preferences.   Classifying by type went away with encryption, which prevents traffic classifiers from seeing the specifics of what is traversing a network.  A modern bandwidth controller uses dynamic rules to restrict  traffic based on aberrant behavior.  Although this might seem less intuitive than specifically restricting traffic by type, it turns out to be just as reliable, not to mention simpler and more cost-effective to implement.
We have seen results where a customer can increase their user base by as much as 50 percent and still have decent response times for interactive  cloud applications.
To learn more, contact us, our engineering team is more than happy to go over your specific situation, to see if we can help you.

Three Myths About QoS and Your Internet Speed


Myth #1:  A QoS device will somehow make your traffic go faster across the Internet.

The Internet does not care about your local QoS device.  In fact, QoS means nothing to the Internet.  The only way your traffic can get special treatment across the Internet would be for you to buy a private dedicated link – which is really not practical for general Internet usage, as it would only be a point-to-point link.

Myth #2:  QoS will enhance the speed of your internal network.

The speed of your local internal links are a fixed rate, they always run at maximum speed.  The only way applying QoS can make something “appear” to go faster is by restricting some traffic in favor of other traffic.  I constantly get asked by our customers  if we can make important traffic get through faster, and my follow on questions are always the same.

  1. Do you have a congestion problem now?
    If not, than there is no need for any form of QoS, because your data already moving as fast as possible.
  2. If you do have congestion, what traffic do you want me to degrade so that other traffic can run without congestion?

Myth #3:  There is nothing you can do to give priority to incoming traffic on your Internet.  

Wrong! Okay, so this sounds like it may be a contradiction to Myth #1, but there is a difference in how you ask this question.   Yes, it is true that the Internet does not care about your QoS desires and will never give preferential treatment to your traffic.  But, the sending service DOES care about whether the data being transmitting is being sent at the appropriate speed for the link you get, and you can take advantage of this.

All senders of data into your network are constantly monitoring the speed at which that traffic is getting to you.  Now, if you recall the very definition of QoS is restricting one type of traffic over another.  Let’s say for example that you have a very congested Internet link with many incoming downloads.  Let’s say one download is a iOS update, and the other one is your favorite streaming Netflix movie.  By delaying the iOS update packets at the edge of your network, the sender will sense this delay, and back off on the download. The result is that there is more bandwidth left over for your favorite Netflix , and hence you have attained a higher quality of service for your Netflix over the iOS download.  How this delay is implemented is another story.

If you are interested in learning more, please feel free to contact us.

NetEqualizer News: March 2017


We hope you enjoy this month’s NetEqualizer Newsletter. Highlights include an overview of more 8.5 Release features, a preview of our new website, and more!

 

  March 2017

 

8.5 Release – More Features!
Greetings! Enjoy another issue of NetEqualizer News.

Our 8.5 Release development is almost complete! This month we preview some of the new features for you, and also show
some of the new screens that our development team has been willing to share. Look for 8.5 to be available in early summer 2017!

Our wireless Internet Provider customers may be interested in our newly released Hidden Node White Paper. And we are experimenting with a new website design. We would love your feedback!

We continue to work with you to solve some of your most pressing network problems – so if you have one that you would like to discuss with us, please call or email me anytime at 303.997.1300 x103 or art@apconnections.net.

And remember we are now on Twitter. You can follow us @NetEqualizer.

– Art Reisman (CTO)

In this Issue:

:: 8.5 Release – Features Preview

:: NetEqualizer is a Hidden Node Solution

:: Under Construction – New Website?

:: Best of Blog: The Best Monitoring Tool for Your Network – May Not Be What You Think

8.5 Release – Features Preview

8.5 Release Additions – Continued from February…

In February, we talked about adding Real-Time Penalties to the RTR Dashboard, and adding Host Name from NSLookup to RTR Reports. This month we introduce several more features planned for 8.5:

1) Configuration Validation for Traffic Limits & P2P Limits

In order to make it easier for you to setup and configure your NetEqualizer, in 8.5 we are adding automated configuration validation to our toolset. In the first offering, we will automate the rules around defining traffic limits and P2P limits. As part of the installation process, when you send your diagnostic to Support, we will then run our configuration validation on your rule set. This will be particularly useful for customers that set up hundreds of traffic limiting rules.

2) Add Units to Active Connections Report

You can now select the units that you wish to see on the Active Connections Report. We currently show Active Connections in bytes/second (Bps), as this was aligned with how we used to show units in the configuration. However, in 8.5 we added the ability to select Configuration Units – the traditional Bytes per Second (Bps), or Megabits per second (Mbps), or Kilobits per second (Kbps). Now we are aligning Active Connections with those changes, by expanding our units selection to include Active Connections. See below for screenshots of this new feature.

In this example, as Megabits per second (Mbps) are selected, you can see that both Wavg (column 4) and Avg (column 5) are now shown in Mbps. Hopefully this will make it easier for you, as you can see your reports in Units that are meaningful to you:

As always, the 8.5 Release will be free to our customers with valid NetEqualizer Software and Support (NSS) plans.

NetEqualizer is a Hidden Node Solution

 Read our Hidden Node White Paper

If you are an Wireless Internet Provider, and are challenged with Hidden Nodes in your network infrastructure, you may want to read our newly released Hidden Node White Paper, to see how the NetEqualizer resolves this issue.Of the numerous growing pains that can accompany the expansion of a wireless network, the issue of hidden nodes is one of the most difficult problems to solve. Despite best efforts, the communication breakdown between nodes can wreak havoc on a network, often leading to sub par performance and unhappy users. Many times, the cost of potential solutions appears to outweigh the benefits of expansion, which in some cases may not be a choice, but a necessity. Yet, hidden nodes are a problem that must be addressed and ultimately solved if a wireless network is to achieve successful growth and development.

To continue reading, view the full white paper here. Check it out!

Under Construction – New Website?

Website Design Alternative – Tell Us What You Think!

We are working with a design agency to put together new web pages. Our initial set of pages are ready. We picked a dark background, and aimed for an interface that aligned with today’s mobile platforms, as it is more scrollable, and trend towards less text/more pictures.

Before we switch over to use these pages across our entire platform (we are using for our Google Adwords leads now), we would love to get your feedback.

Please take a minute to look at the new pages, and then click on the feedback button to email us your thoughts. 
Click the above picture or this link to view the new design: http://netequalizer.com/fast/
What do you like? Dislike? Any recommendations for what we should change? And the big question – should we keep our current website or move to this?
Best Of Blog

The Best Monitoring Tool for Your Network – May Not Be What You Think

By Art Reisman

A common assumption in the IT world is that the starting point for any network congestion solution begins with a monitoring tool.  “We must first figure out what specific type of traffic is dominating our network, and then we’ll decide on the solution”.  This is a reasonable and rational approach for a one time problem. However, the source of network congestion can change daily, and it can be a different type of traffic or different user dominating your bandwidth each day…

Photo of the Month
Pipeline Swallowtail

This is a picture of a Pipevine Swallowtail butterfly taken in Arizona in the high desert grasslands area over a recent spring break getaway. This butterfly can be found in a variety of habitats, but is most commonly found in forests.

APconnections, home of the NetEqualizer | (303) 997-1300 | Email | Website 
%d bloggers like this: