The video rental industry of the early 80’s was comprised of 1000’s of independent stores. Corner video rental shops were as numerous as today’s Starbucks. In the late 1990’s, consolidation took over. Blockbuster with its bright blue canopy lighting up the night sky swallowed them up like doggy treats. All the small retail outlets were gone. Blockbuster had changed everything, their economy of scale, and their chain store familiarity, had overrun the small operators.
In a similar fashion to the fledgling video rental industry, circa 1990’s Internet content was scattered across the spectrum of the web, ripe for consolidation. I can still remember all of the geeks at my office creating and hosting their own personal websites. They used primitive tools and their own public IP’s to weave these sites together. Movies and music were bootlegged, and shared across a network of underground file-sharing sites.
Although we do not have one Internet “Blockbuster” today, there has been major consolidation. Instead of all traffic coming from 100’s of thousands of personal or small niche content providers, most of it comes from the big content providers. Google, Amazon, Netflix, Facebook, Pinterest are all familiar names today.
So far I have reminisced about a nice bit of history, and I suspect you might be wondering how all of this prelude relates to tracking traffic by DNS?
Three years ago we added a DNS (domain name system) server lookup from our GUI interface, as more of a novelty than anything else. Tracking traffic by content was always a high priority for our customers, but most techniques had relied on a technology called “deep packet inspection” to identify traffic. This technology was costly, and ineffective on its best day, but it was the only way to chase down nefarious content such as P2P.
Over the last couple of years I noticed again the world had changed. With the consolidation of content from a small number of large providers, you could now count on some consistency in the domain from which it originated. I would often click on our DNS feature and notice a common name for my data. For example, my YouTube videos resolved to one or two DNS names, and I found the same to be true with my Facebook video. We realized that this consolidation might make DNS tracking useful for our customers, and so we have now put DNS tracking into our current NetEqualizer 8.5 release.
Another benefit of tracking by domain is the fact that most encrypted data will report a valid domain. This should help to identify traffic patterns on a network.
It will be interesting to get feedback on this feature as it hits the real world, stay tuned!