July 19, 2006
OpenDNS - Technical Review
I have serious doubts about the new OpenDNS service.
The idea is simple – point your DNS resolver to their name servers. What you get in return is faster cached results and phishing site blocking. They make money by advertising on the pages returned when you make a spelling mistake in the domain name. OpenDNS also claims to have “caches at the major intersections of the Internet” though I have not been able to get to anywhere but their California Verio site (tried tracerouting from Australia, US east coast and west coast.) So, I’m not sure if they are using anycast.
Having done some preliminary technical evaluation of the service, here are my thoughts (feel free to disprove me):
1. It would be pretty difficult to scale a service like this and make it resilient to DDoS attacks. In order to support typo-correction, they reply to queries for non-existent domains with an A record to their web server with a 1-second TTL value. This effectively tells the client not to cache the record, and because it is not an NXDOMAIN status no negative caching is possible either. What this means is that their name servers will have to answer the same query multiple times within a short time frame for non-existent domains (even from the same client).
2. They have a preference page that allows users to turn on/off the typo-correction and phishing-blocking features by the user’s IP address. I don’t know how they are going to solve the potential problem of clients behind NAT changing the settings thinking that they’re the only one using it. I tried turning off both the features, waited, and tried again. This time, non-existent domains are replied with an NXDOMAIN answer. However, this differs from the behavior of most modern recursive name servers in that it does not return the SOA record of the parent domain with the TTL set to the SOA “minimum” field value (used for negative caching.)
3. Shifting DNS resolution responsibility from the ISPs to OpenDNS is going to be a big problem. This essentially shifts the point of failure out of the hands of ISPs and into a single vendor. ISPs will no doubt loath them for it because of the possible increase in support calls because people start messing with the DNS settings on their machines and broadband routers. Just imagine a user starts using OpenDNS now and it works fine. Three months down the road, somehow he is not able to connect to OpenDNS’s name servers (maybe because of some routing fault between his ISP and one of OpenDNS’s). He’s not going to remember that he had configured his home router to ues the OpenDNS name servers and may think that his Internet is down. So he rings up his ISP, and the support person on the other line would have to spend at least 30 minutes trying to diagnose a problem that wasn’t even theirs to begin with. If an ISP had enough of it, it might decide to block udp/53 access to OpenDNS’s name servers. Worse, they may even block DNS request to anywhere but their own name servers.
4. OpenDNS’s FAQ has an entry that clarifies why its service is different from Verisign’s SiteFinder. Those statements are true. However, Verisign running SiteFinder affected only the zones that they operate (COM and NET at that point). Now, users of OpenDNS will have the “feature” for ALL TLDs including .GOV, .MIL and .CN, to name a few that would raise controversy.
5. Running an open recursive servers also opens them up to the risk of DNS cache poisoning, and having a large user base would make them an attractive target for the baddies. Instead of poisoning the caches of some small ISP shop, they can target a much larger audience.
6. They are using the same order for the first and second name server IP addresses on their web page. So, users will follow the instructions and configure their system with those IPs in that order. The way I understand most resolvers work is that unlike NS records which sometimes get returned in random order, the first name server is always hit first, and if no response is received only will it consult the next name server. This would mean a much higher load on the first IP than the second IP.
7. At the recent ICANN at Marrakech, I have seen trademark holders getting very defensive over typo-domains that resemble their company / product names going to an ad-laden page (which may eventually brings the user back to the intended site but it could also lead the visitor to a competitor.) At the Domain Name Marketplace Workshop, fellow panelists Sarah Deutsche (Verizon representative) and Paul Stahura from eNom were arguing over some “variations of [their] Verizon and SuperPages trademars” registered by eNom – on stage (transcripts can be found here)!
My point is that – running an ad-sponsored service that intercepts all typos across all TLDs has the word LAWSUIT written all over it. So, good luck to OpenDNS.
Don’t get me wrong, I think it’s a noble and novel idea. If they can address the above concerns and pull it off, I would use it especially if they do deliver the faster response times. To be fair, it is pretty darn fast for me already considering the packets have to travel across the pacific ocean to get to the server and back.
I also have great respect for David Ulevitch for running the excellent EveryDNS service (which hosts the DNS for dready.org.)
Full Disclosure: I work for NeuStar – the registry operator for .BIZ, .US and .TRAVEL. This post is in NO WAY representative of my employer’s position. This is just my personal opinion.
Posted 1 year, 11 months ago on July 19, 2006
The trackback url for this post is http://dready.org/blog-v1/bblog/trackback.php/70/
Thanks for the thoughtful post. I can address many of your points, but I think I'll wait and have David dive in, since he can also cover the more technical details, too.
John Roberts
OpenDNS
I don't think your assertion in #1 is correct.
Just because the OpenDNS server is replying with non-cacheable A records for non-existent domains, doesn't mean that they aren't doing caching internally in the OpenDNS server.
OpenDNS could be doing a domain lookup for the first non-existent domain that comes in, and then cache that result for some period of time (fulfilling requests from its cache), until it invalidates the cache and tries to resolve it again.
I agree that these guys are a nice DDoS target, but I don't think that these A records with a TTL of 1 is how they will get hurt. To hurt OpenDNS, you would have to repeatedly request a non-existent domain, and then do an HTTP get on the result, which causes them to serve up an HTML page. That puts more drain on them, than just pounding them with DNS lookups.
-Andy Reitz.
I agree, and as David said, that the part that would be hit harder is the HTTP server rather than tiny udp packets.
I'm just saying that a caching resolver that forwards all requests to OpenDNS wouldn't be able to cache:
1) non-cacheable A records due to non-existent domains;
2) NXDOMAIN responses because they don't reply with the parent domain SOA record. (This is the case when you turn off both features on the preferences page.)
Suppose I'm an organization who runs a caching nameserver that forwards queries to OpenDNS. Somebody forwards an email that contains a bad link and everybody tries to go to the site around the same time. Each request by my user will cause the caching nameserver to query OpenDNS' nameservers.
Of course, this wouldn't hurt OpenDNS but other scenarios may amplify the traffic.
Here it comes, I hope it'll be formatted okay... Took me a while to write. :-)
-david
Your views (and they're clueful) are your own. I'll just try to clarify things you've said about our service and give some technical info where I can. I do like the technical discussion, though. :-)
Some of these issues are more just caveats or like one techie saying to another: "you should be aware of this." I appreciate those, I think in all the cases below, we're aware and sensitive to them. No policy is fixed though, discussion is the best way to help figure out the best policies to have.
1. It would be pretty difficult to scale a service like this and make it resilient to DDoS attacks. In order to support typo-correction, they reply to queries for non-existent domains with an A record to their web server with a 1-second TTL value. This effectively tells the client not to cache the record, and because it is not an NXDOMAIN status no negative caching is possible either. What this means is that their name servers will have to answer the same query multiple times within a short time frame for non-existent domains (even from the same client).
This doesn't impact our nameserver as much as it would impact our webserver. Both the DNS servers and the the webservers are perfect candidates for anycasting. We are anycasting the DNS right now and will anycast the HTTP side of things down the road. We're currently finishing up our configurations and deployments as I write this. As you noted Verio is not sending things to all our locations. That's being resolved.
2. They have a preference page that allows users to turn on/off the typo-correction and phishing-blocking features by the user’s IP address. I don’t know how they are going to solve the potential problem of clients behind NAT changing the settings thinking that they’re the only one using it. I tried turning off both the features, waited, and tried again. This time, non-existent domains are replied with an NXDOMAIN answer. However, this differs from the behavior of most modern recursive name servers in that it does not return the SOA record of the parent domain with the TTL set to the SOA “minimum” field value (used for negative caching.)
NAT is a problem that's tough to solve, you're right. As to who can change it, well that just requires an account management page and a verified point of contact. The page you see up there mentions it's just a taste of what we're working on in the background. It does prove that a highly dynamic recursive DNS server can exist and be fast. I'm glad you did find it useful.
I think the way we are replying in the SOA is more appropriate for the level of service we're offering. As the first intelligent recursive DNS service we have had to make decisions how we interoperate with authoritative servers. In this case, it provides the flexibility we need for dynamically changing network preferences and settings without impacting the authoritative server. If we find ourselves retrying authoritative servers too often, we'll pull back.
3. Shifting DNS resolution responsibility from the ISPs to OpenDNS is going to be a big problem. This essentially shifts the point of failure out of the hands of ISPs and into a single vendor. ISPs will no doubt loath them for it because of the possible increase in support calls because people start messing with the DNS settings on their machines and broadband routers. Just imagine a user starts using OpenDNS now and it works fine. Three months down the road, somehow he is not able to connect to OpenDNS’s name servers (maybe because of some routing fault between his ISP and one of OpenDNS’s). He’s not going to remember that he had configured his home router to ues the OpenDNS name servers and may think that his Internet is down. So he rings up his ISP, and the support person on the other line would have to spend at least 30 minutes trying to diagnose a problem that wasn’t even theirs to begin with. If an ISP had enough of it, it might decide to block udp/53 access to OpenDNS’s name servers. Worse, they may even block DNS request to anywhere but their own name servers.
This is not quite true and I think this is a bit unrealistic. Your company just acquired a company which proved this to be false (and it can be proven to be false even more so). Most companies outsource a lot of things, this is just one more. As we grow and enhance our feature set and reporting, it'll make our service distinction clear. Reliability and Control for users and network administrators is what drives our platform, everything we do is tied back to that. People want a managed service, and they know what that means for them, so do we.
4. OpenDNS’s FAQ has an entry that clarifies why its service is different from Verisign’s SiteFinder. Those statements are true. However, Verisign running SiteFinder affected only the zones that they operate (COM and NET at that point). Now, users of OpenDNS will have the “feature” for ALL TLDs including .GOV, .MIL and .CN, to name a few that would raise controversy.
The real issue here is about the ability to allow individuals and networks to control their own network. I believe in this strongly. We have firewalls and proxies and all other kinds of things users can have to better secure and operate their network. DNS is like a firehose that is all or nothing. That makes very little sense. In the nicest and most serious way possible I'm curious if you oppose DNSBLs and spam filtering for the same reasons? Like IP addresses, a domain doesn't ensure reachability. Most networks don't want their computers connecting up to massive DNS based botnets.
5. Running an open recursive servers also opens them up to the risk of DNS cache poisoning, and having a large user base would make them an attractive target for the baddies. Instead of poisoning the caches of some small ISP shop, they can target a much larger audience.
DNS is what I do. While there are many ways someone can be poisoned, the one I fear most is social engineering, not technical cache inconsistencies. Like most of these concerns, they apply to Neustar and UltraDNS just as well. Just a different link in the chain.
6. They are using the same order for the first and second name server IP addresses on their web page. So, users will follow the instructions and configure their system with those IPs in that order. The way I understand most resolvers work is that unlike NS records which sometimes get returned in random order, the first name server is always hit first, and if no response is received only will it consult the next name server. This would mean a much higher load on the first IP than the second IP.
This is true. Our load at this point isn't such that it's a problem. Big customers will get their own dst_addr and we have other techniques we use to balance load.
7. At the recent ICANN at Marrakech, I have seen trademark holders getting very defensive over typo-domains that resemble their company / product names going to an ad-laden page (which may eventually brings the user back to the intended site but it could also lead the visitor to a competitor.) At the Domain Name Marketplace Workshop, fellow panelists Sarah Deutsche (Verizon representative) and Paul Stahura from eNom were arguing over some “variations of [their] Verizon and SuperPages trademars” registered by eNom – on stage (transcripts can be found here)!
My point is that – running an ad-sponsored service that intercepts all typos across all TLDs has the word LAWSUIT written all over it. So, good luck to OpenDNS.
I don't think you haven't looked close enough to what we're doing and what I've said. When someone types in craigslist.og they go directly to craigslist.org That makes craigslist happy. They'd be even happier if craigslists.org went to Craigslist. That should clue you in on what it is we want to be doing on the other side of the dot. The trademark holders should be fans of what we're doing and hopefully they are. I'd expect them to defend us, actually.
Don’t get me wrong, I think it’s a noble and novel idea. If they can address the above concerns and pull it off, I would use it especially if they do deliver the faster response times. To be fair, it is pretty darn fast for me already considering the packets have to travel across the pacific ocean to get to the server and back.
We're going to get even faster too. We didn't launch because we are 100% finished. We launched because we have a platform to show people and get feedback on. We're stable, but we're no where near where we want to be.
Reliability and Control is what this is about. Not about us being your central point of control or failure. Hopefully as we move forward that'll be more and more clear.
I also have great respect for David Ulevitch for running the excellent EveryDNS service (which hosts the DNS for dready.org.)
Awesome! Thanks!
Full Disclosure: I work for NeuStar – the registry operator for .BIZ, .US and .TRAVEL. This post is in NO WAY representative of my employer’s position. This is just my personal opinion.
Email me privately, would love to chat.
Comments have now been turned off for this post