alphaspirit - Fotolia
Recent security breaches are so egregious and so truly detrimental to the bottom lines of some really well-known brands that they're -- ironically -- becoming decreasingly effective cautionary tales about security. The spin machine is in full swing, and any time lawyers and PR get involved, takeaways for network administrators are lost.
Far better is to search out personal examples of network security misadventures we encounter during the course of our everyday lives. And this holiday season, while on the road visiting family for Christmas, I found something really juicy. It's also unfortunately something all too common in our enterprise networks, especially when we're standing up branch office infrastructure.
What, me hack a network?
I'm not saying I make even a half-hearted attempt to probe every hotel network I encounter. I do admit, however, that I travel on business quite a bit, I find hotel rooms really boring and that you can kill a lot of time with an RJ45 jack. Wait, that sounds untoward. Well, you get the gist.
Let's just say I had a couple of days over the holidays to experiment with getting the most from a hotel network and to determine how much that experimentation might compromise hotel Wi-Fi security. I won't say in which inn I resided, but it was your garden variety, smaller business hotel with thin walls, decent decor and a better-than-average complimentary breakfast.
Now, I don't want my media access control (MAC) address to end up on a hospitality no-fly list, so I'm not one to hack a network for grins and glory. I do, however, have a networking toolset just sitting here on my laptop; discovering networks is pretty trivial. So, on occasion I've been known to see if I can get out when access is non-complimentary, and once achieved, pay for access and delight in sending the details of the exploit to the network owner. Admin karma +1, right?
Generally, access holes fall into one of three categories:
- We don't care, we're just keeping honest people honest. These networks tend to block a few ports like HTTP, Simple Mail Transfer Protocol (SMTP) and a couple others with the goal of being annoying enough to get you to pay for access even as they keep enforcement costs at a minimum.
- We did our best, but overlooked something. These are generally found in larger pay-to-play networks like convention centers, where they really did a pretty good job with log-on redirection and deny-all MAC filtering, but untended network configuration changes leave invisible routes open. In other cases, temporary convenience access -- ahem, secure shell -- is open and tunneling gets you on the interwebs in seconds.
- Intelligent, but not experienced design. This is the scary one. I'll spare the exact quote from Star Trek II, but Khan was undone by two-dimensional thinking, and in the case of my hotel this week, they weren't thinking in the third dimension of IP -- IPv6. How I found this was the result of a network device performance problem, which led me to the dark side of 128-bit addressing in the first place.
It started innocently enough: I couldn't log into the paid wireless because the log-on redirect was failing. But it wasn't failing in a consistent way, and it felt like I wasn't getting an IP address. Debugging this is pretty trivial and I always try to have details before I call the helpdesk.
To that end, I enabled Dynamic Host Configuration Protocol (DHCP) client logging for the Windows Event Viewer and bounced the WLAN port. After a couple of minutes, a pattern emerged. I was getting two DHCP OFFER messages almost exactly 1000 milliseconds apart. The MACs told me they were different servers and the addresses in the follow-on DHCPACK messages flapping my laptop back and forth told me something else. The DHCP scopes had an address range overlap.
I loved it: classic DHCP split scope misconfiguration exposed by primary scope provider exhaustion or performance issues. I was all set to triumphantly call the hotel network central helpdesk when for some reason I got the idea to disable IPv4 on my laptop. Maybe they served IPv6 -- which, of course, would have no overlap. Seconds later, I had a healthy network status icon and refreshed my browser to get the log-on redirect. Instead, I got Fark.com. Something was wrong.
There's nothing wrong with a daily dose of snarkiness -- on the contrary, it makes meetings tolerable, and Fark.com actually is a great test site. It gets enough traffic to keep domain name system (DNS) servers caching out to most common subnets, but it's not so huge that it has geo redundancy. It's always 126.96.36.199 or similar.
Anyway, I launched my VPN to the office and that connected. I did quick command line FTP and SMTP and both succeeded. Finally, I got out my toolset and realized the scope of the access hole. I could ping-sweep the entire local network, and a port sweep revealed nothing was blocked. I was looking at a special, increasingly common and serious issue -- unknown, wide-open, IPv6 dual-stack.
We all hear antidotes to surprise workstation and server IPv6 DHCP assignments, but today a startling number of network boxes come with IPv6-enabled, dual-stack by default. Couple that with default dual-stack adaptive security appliances and ISPs supplying both protocols and it's quite possible you might never know unless you're monitoring your DHCP at the scope level for both IPv4 and v6. Of course, you should be doing DDI -- that is, a combination of DHCP, DNS and IP address management -- or at least basic IPAM automation if only to detect the IP address conflict that sent me down the v6 path in the first place -- especially if you're a major hotel chain.
Timely security resolutions
At any rate, I pulled my logs together along with a brief bug report and sent it to the hotel's security report email address. I felt good. Three days later, there was still no reply and I continued "testing" the IPv6 connection for them, which apparently also wasn't throttled at 5 Mbps like the Wi-Fi for the other
suckers guests. That was pretty good, too. Maybe the hotel will fix it one day. In the meantime, ping me and I'll recommend a great place to stay.
About the author:
Patrick Hubbard is a head geek and senior technical product marketing manager at SolarWinds. With 20 years of technical expertise and IT customer perspective, his networking management experience includes work with campus, data center, storage networks, VoIP and virtualization, with a focus on application and service delivery in both Fortune 500 companies and startups in high tech, transportation, financial services and telecom industries. He can be reached at Patrick.Hubbard@solarwinds.com