If you recall, some time ago I had problems with my AT&T U-verse DSL service until a small village's worth of AT&T folk got involved and resolved the issues. Since then, an executive director from the "Office of the President at AT&T," who I'll call "Bob," has followed up with me once a week to make sure things are still OK.
Earlier this month Bob emailed me to schedule a telephone call, saying, "I need to discuss something with you ... Has to do with a change we are making that affects some customers ...painless and non-service affecting, but I wanted to personally cover it with you."
MORE: Broadband infrastructure: Time for real policy
We had the call and the thing he wanted to tell me was I would have to change my network subnet address and that there was a tech support document to explain what was required. The document, "Changing the private 10.x IP range on your AT&T U-verse Modem/Gateway," explains:
"As part of AT&T's efforts to enhance our network to accommodate future growth, we will be making a firmware upgrade to your AT&T U-verse Gateway. Customers who have configured their network to use the 10.0.0.1 - 10.255.255.255 private Internet Protocol (IP) ranges within their AT&T U-verse Gateway will need to change to an alternate IP range. AT&T recommends changing to a 192.168.1.x IP range. Customers who don't update their network by July 6, 2012 may potentially encounter a disruption in service."
Hummm. Curious. What could "disruption of service" mean? I'm guessing dead as a bag of hammers. But why would AT&T want me or anyone else to make this change? What reason could AT&T have for caring about what my subnet addressing might be?
In case you're not aware, RFC1918 states:
The Internet Assigned Numbers Authority (IANA) has reserved the following three blocks of the IP address space for private internets:
10.0.0.0 - 10.255.255.255 (10/8 prefix)
172.16.0.0 - 172.31.255.255 (172.16/12 prefix)
192.168.0.0 - 192.168.255.255 (192.168/16 prefix)
For reasons lost in the mists of time I've used the 10.0.0.x subnet in my network for years and, as a consequence, I have servers, NAS devices, printers, VoIP gateways, and so on, that have static address assignments. We'll come back to the issue of configuring equipment in a minute.
I wasn't surprised that Bob knew what my subnet addressing scheme was as my network setup had come up in one of the tech support calls while they were trying to figure out what was wrong with my service. One of the chaps on the call said at one point he could see I had X number of devices on my network (whatever the count was at that moment) and that I was using the subnet 10.0.0.x ... we'll also come back to this in a second.
Anyway, Bob said he wasn't sure why AT&T required the change and said he'd check, but thought it might be "proprietary in nature" which I understood to mean, "We aren't going to tell you." Two weeks later I got a formal answer:
"With all Internet service providers facing a shortage of IPv4 addresses, we are upgrading U-verse Internet customers' equipment to help us maximize the use of these addresses. This will be a seamless process for virtually all of our customers. However, the less than 1 percent of U-Verse customers who have altered their network settings will need to reconfigure their network back to its default settings. For more information, visit www.att.com/lansettings."
And that was all Bob would say.
FAIL: IPv6 on home routers and DSL/cable modems
Let's think about this: The opening assertion is true, IPv4 addresses are getting scarce (see the article "Sales of unused IPv4 addresses gathering steam"), and with IP address swapping hands for more than $11 a piece, no ISP wants to have to go looking for large chunks of address real estate. Even so, what's that got to do with my network?
A theory put forth on an AT&T customer forum pointed the way. It suggested the company is deploying carrier-grade NAT on their network to ease IPv4 address exhaustion.
As you're all aware, Network Address Translation (NAT), as Wikipedia explains, allows you to "hide an entire IP address space, usually consisting of private IP addresses, behind a single IP address (or in some cases a small group of IP addresses) in another (usually public) address space."
Wikipedia also notes that "In the mid-1990s NAT became a popular tool for alleviating the consequences of IPv4 address exhaustion. It has become a common, indispensable feature in routers for home and small-office Internet connections. Most systems using NAT do so in order to enable multiple hosts on a private network to access the Internet using a single public IP address."
As a friend pointed out, a posting on the SANS Storm Center, [Reserved IP Address Space Reminder," explains:
"As we are running out of IPv4 address space, many networks, instead of embracing IPv6, stretch existing IPv4 space via multiple levels of NAT. NAT then uses 'reserved' IP address space. However, there are more address ranges reserved then listed in RFC1918, and not all of them should be used in internal networks. Here is a (probably incomplete) list of address ranges that are reserved, and which once are usable inside your network behind a NAT gateway."
A table of reserved IPv4 address ranges is given and the article concludes (the underlining is mine):
"Most interesting in this context is RFC6598 (100.64.0.0/10), which was recently assigned to provide ISPs with a range for NAT that is not going to conflict with their customers NAT networks. It has been a more and more common problem that NAT'ed networks once connected with each other via, for example, a VPN tunnel, have conflicting assignments."
A search of the Network World archives revealed more support for the carrier grade NAT theory. In the article titled "AT&T lagging while others lead on IPv6," published in October 2010(!), says:
"An AT&T executive did speak at Google's IPv6 Implementers Conference in June, explaining that the carrier will transition its broadband network to IPv6 using 6rd, a technique for tunneling IPv6 traffic over an IPv4 network that was pioneered by French ISP Free. / 'With our high-speed Internet access -- the U-verse and DSL product sets -- our plan is to go carrier-grade NAT to reduce IPv4 consumption and 6rd for IPv6 end content over our network ... We're not doing a trial yet. Not until 2011."
Looks like 2011 was optimistic and it appears this is the year that AT&T is going to cut loose with a carrier grade NAT service to interface their U-verse service to the Internet.
And, while Bob might be, for whatever strange reason, unwilling to explain the reasons for the change, it seems likely that AT&T plans to deploy 10.x.x.x for their U-verse WAN services.
Why AT&T isn't going to use the recommendations of RFC6598 and deploy 100.64.0,0/10 is totally obscure at this point but, hopefully, one day they'll explain and we'll all go "Oh yeah, that makes sense." Alas, AT&T being what it is ... a huge, lumbering, bureaucratic, anti-competitive, customer indifferent machine for making money ... there's a good chance the reasons may never become clear.
What is clear are several things ...
First, AT&T is, in effect, demanding the right to look into our private networks. Remember when I mentioned the techs could see the device table in my DSL modem? It would appear that for AT&T supplied DSL modems such as the wretched Motorola NVG510 I have, the company has remote management abilities that aren't fully explained anywhere.
Bob's response to my query about this was as follows:
"We have the ability to collect data generated by modems or other equipment used to access our network in order to protect our network and to serve our customers. We disclose this capability in a number of places including in our AT&T High Speed Internet Terms of Service and "The Information We Collect, How We Collect It, and How We Use It" section of the AT&T Privacy Policy ... section 7 of our TOS. In your case, in an our effort to troubleshoot the latency problem you were experiencing, our technicians collected the MAC addresses of the devices you have connected via your modem and using publicly available information, translated that information to determine the manufacturer of those devices, all in an effort to better serve you."
Section 7 of the ToS reads: "Regardless of whether the equipment used to access your Service (modem, gateway, etc.) is owned by you or AT&T, AT&T reserves the right to manage such equipment for the duration of your Service, and retains exclusive rights to data generated by the equipment. Neither you nor a third party may change, interfere with, or block access to equipment data or settings."
My problem with this is, how is AT&T's access into my DSL modem secured? If I can't grant and revoke remote management rights then I hope AT&T has some kind of serious digital certificate system that controls access because I, in common with many other people, do most of my financial transactions online and if AT&T is relying on what would be, in effect, security through obscurity, I would be seriously concerned. The problem is that I don't know whether to be concerned or not. Which makes me even more concerned.
Second, from this table AT&T can see all of the devices that are or have been connected, how they are connected (wired or wireless), their current status (alive or dead), and their MAC and IP addresses. This is how AT&T knows that some 1,600 customers need to make the change to a new subnet addressing scheme. Of course, those customers using non-AT&T supplied DSL modems aren't visible, but I guess the number is so low AT&T is good with that problem.
Third, the contention, as Bob wrote in his original email, that the change would be "painless" for end users is simply wrong. For some people, such as me, having to go to each of a dozen or more devices and reconfigure them, restart them, and verify they work will take, conservatively, a couple of hours. That's a couple of hours that I wouldn't need to burn if AT&T had got their act together. I might be able to forgive them if I knew the what and why behind what they're doing but I don't and most likely never will.
And what about the network of the 75-year-old widow in Ventura whose son, who worked for a computer company and now lives 3,000 miles away in Boston, set up for her a year ago using 10.0.0.x? On July 6, her Internet connection will stop working and she'll call AT&T tech support and after an hour of someone telling her to reboot her PC and restart her gateway and log on to her DSL modem ("What?") and sacrifice a chicken and, and, and ... she will give up and her son, 3,000 miles away will have to figure out what to do.
Fourth and lastly, there's the issue that, as far as I can tell, come July 6, everyone on the AT&T U-verse network who finds themselves front-ended by AT&T's NAT service, will also find services such as their Dynamic DNS (DDNS) will not work. Their DDNS updater will find that their Internet facing address is 10.x.x.x and, as far as I can figure it, that will not be resolvable back to the user's network because it will be part of a private address space.
In fact, given that the architecture of AT&T U-verse will be a NAT service front-ending a NAT service, you have to wonder what else will break ... VPNs? Remote desktop sessions?
So, disclaimer: I have no idea whether AT&T is really going to implement carrier grade NAT and what its implications will be because AT&T isn't talking. If the end result is a big nothing other than a few of us wasting our time, then I guess it won't be much more than an annoyance (for which AT&T should compensate us).
If, on the other hand, lot of stuff breaks then the results will be huge and demonstrate very clearly what I've contended for a very long time: Without a truly competitive business environment, there's a high probability that products and services will suck.
Thanks to Morris Tabush of Tabush Consulting for his insights into the implications of AT&T's use of NAT.
Gibbs is waiting for July 6 with bated breath in Ventura, Calif. Tell him your fears at gearhead@gibbs.com and follow him on Twitter (@quistuipater) and on Facebook (quistuipater).