2011 San Francisco Workshop Agenda

2011 San Francisco Workshop Agenda

We are still working to finalize the agenda but the list of presenters includes:

Eric Ziegast (ISC) - DNS Amplification DDOS
Eric Ziegast (ISC) - CC-based Conflicker Sinkhole Collection
Jot Powers (PayPal) - DNSSEC side effects - A Commercial Example
Joseph Gersch (Secure64) - CERTs in the DNS protected by DNSSEC
Andrzej Bartosiewicz (Yonita) - Automatic Detection of Vulnerabilities in Core Internet Systems
Stephane Bortzmeyer (AFNIC) - Monitoring DNSSEC Zones, How and What ?
George Michaelson (APNIC) - AS112bis, Taking Account of V6 and IN-ADDR re-delegation update
Paul Hoffman (VPN Consortium) - New DNS Records for certificates: Sizes and Request Patterns

Submitted by on Mon, 2011-02-07 16:20. categories [ ]

2010 DSC Workshop (Denver)

In recent months I have been talking to some of you about what the future holds for DSC. I am not really in a position to give it a significant amount of time (and have not given it much attention for at least a year).

In October there is a DNS-OARC meeting in Denver, Colorado, USA and I think that quite a few DSC users may be there. I am proposing to have a day (or perhaps 1/2 day) meeting of interested parties to discuss the future of DSC. Such a meeting would have an agenda like this:

- How do we use DSC now and what do we like
- What asspects of DSC should be updated/fixed/improved/removed?
- Can we put a plan in place to make it happen?

Note that I am assuming the DSC community generally feels that the software should be evolved for our future needs. If this assumption is incorrect, now would be a good time to let me know :-)

To be clear: assuming we get to the final point in the above list, I will be asking you (your companies) to make financial contributions so that a programmer can be employed to implement the necessary changes. Details (such as amounts and where such money would be collected) are not known at this time. That will be part of the discussion.

If you are planning on attending the DNS-OARC meeting and wish to attend the DSC meeting, please be sure to indicate so on the registration page.

If you are unable to attend the meeting but wish to contribute, please drop me an email with your comments.

Submitted by on Fri, 2010-09-03 13:16. categories [ ]

ODVR Update

* * * Important notice to ODVR users * * *

On September 3, 2010, ICANN decommissioned the IANA DNSSEC testbed (announcement here). As an interim measure, we continue to resolve DNS queries sent to the IP addresses that that were used for access to the testbed, namely: and 2001:4f8:3:2bc:1::64:22. However, we will discontinue this service on October 1, 2010. We will continue to offer the BIND and Unbound open DNSSEC-validating resolvers on the following IP addresses:

Submitted by on Wed, 2010-09-01 09:35. categories [ ]

Job Description: DNS-OARC Executive Director


The Domain Name System Operations Analysis and Research Center (DNS-OARC) is a non-profit, membership organization that seeks to improve the security, stability, and understanding of the Internet's DNS infrastructure.

DNS-OARC's mission is: to build relationships among its community of members and facilitate an environment where information can be shared confidentially; to enable knowledge transfer by organizing workshops; to promote research with operational relevance through data collection and analysis; to increase awareness of the DNS's significance; and to offer useful, publicly available tools and services.


Provide technical, outreach, and governance leadership to ensure the smooth operation of DNS-OARC as a center of co-ordination, innovation and excellence in the Internet Domain Name System operator community. Manage engineering and research projects required to maintain and develop DNS-OARC's infrastructure and data collection. The setting and raising of high standards of availability and support for DNS-OARC activities. Play a key role in building DNS-OARC as a growing, autonomous, neutral organization.

Specific tasks:

  1. All aspects of Management of DNS-OARC, including but not limited to:

    • Liaison and outreach to existing and new DNS-OARC Members.
    • Implementation of the policies and instructions of the OARC Inc. Board.
    • Administration of twice-yearly DNS-OARC public/member meetings.
    • Presentation and promotion of DNS-OARC projects, activities and analysis to international meetings.
    • Development and evolution of DNS-OARC as a neutral membership organization.
    • Creation and management of DNS-OARC documents, website content, and mailing lists.
    • Liaison with ISC DNS-OARC Secretariat for administration of OARC Inc. as an autonomous nonprofit legal entity.
    • Generation of regular monthly and quarterly reports to DNS-OARC's Board and Members.

  2. Management, support and development of DNS-OARC's infrastructure, including but not limited to:

    • Ongoing system administration of DNS-OARC's multi-server collaboration and data gathering platform.
    • Ensure availability and connectivity of DNS-OARC network infrastructure to Internet via ISC's backbone.
    • Primary point of contact and assistance for DNS-OARC members and other operators/researchers requiring support.
    • Liaison with ISC Operations for hosting and remote-hands services for DNS-OARC infrastructure.
    • Enable and ensure effective communication and information sharing within DNS-OARC community.

  3. Contribute to research on topics relevant to DNS-OARC, including but not limited to:

    • Development of DNS-OARC toolset for supporting member community.
    • Large-scale gathering of data from DNS servers for projects such as DITL and Root Zone scaling.
    • Development of the Domain Statistics Collector and other software.
    • Characterization of traffic to key name servers, including the root and top-level domains.
    • Analysis of abuse and anomalies in DNS traffic.
    • Other operational collaboration and research projects as they emerge.

  4. Other duties as required from time to time as directed by the OARC Inc. Board in support of DNS-OARC activities and projects.

Minimum job requirements:

  • Education: Bachelors in Computer Science or related engineering discipline.
  • Experience: At least 5 years experience of Internet Engineering and Unix System Administration in an operational environment.
  • Specific Skills:

    • Proactive in making decisions, takes initiative and has autonomy.
    • Ability to work unsupervised as part of a geographically-distributed team in an international environment.
    • Good communications skills both in person and via electronic media, including excellent written and spoken English abilities.
    • Presentation skills to provide updates on work to member and industry meetings.
    • Unix system adminstration, preferably including FreeBSD, Linux and Solaris.
    • Server installation, management, and content generation, preferably including Apache, Postfix, Mailman, Drupal, Jabber.
    • Experience of Internet protocol software development projects.
    • TCP/IP routing and applications. IP router and ethernet switch configuration and management desirable.

  • Specialized Knowledge:

    • Understanding of Internet Domain Name System governance, architecture, protocols and current development issues. Familiarity with current best practice and issues in ensuring security and integrity of systems connected to the public Internet in a "hostile" environment. Clue. Candidates already recognized by and active within the Internet technical community are preferred.

Working conditions

Reports to: OARC Inc. Board

Supervises: DNS-OARC Engineering staff

DNS-OARC does not currently have physical offices, so this role will be performed remotely, preferably from a suitably equipped office within easy reach of an international airport.

Travel at least once per quarter to DNS-OARC and other industry meetings in the US and internationally, and to visit DNS-OARC's infrastructure in the ISC data center in Redwood City, California, will be required.


DNS-OARC has assigned a budget of US$80-100k for the fulfilment of the above role.


Candidates should submit a Resume/CV and any supporting documents by-email to by 23:59 UTC on 8th Jan 2010.

Prepared by: Keith Mitchell
Last updated: 18-Dec-09 v1.0

Submitted by admin on Fri, 2009-12-18 17:27. categories [ ]

DO=1 Queries and small reply size limits at the Root

Recently, the U.S. Department of Commerece, ICANN, and Verisign announced their cooperation to get the DNS Root zone signed by the end of 2009.

Anyone who has had the pleasure of signing a DNS zone knows that the DNSSEC keys and signatures are much larger than most DNS resource records (and not particularly pretty, either). There is some concern among DNS operators that a signed root zone will lead to truncated responses and queries arriving over TCP, rather than UDP. RFC 4035 has something to say about DNSSEC and message sizes:

A security-aware name server MUST support the EDNS0 ([RFC2671]) message size extension, MUST support a message size of at least 1220 octets, and SHOULD support a message size of 4000 octets.

DNS-OARC looked at EDNS data last year and found about 65% of queries used EDNS and a message size of 4096, a little less than 35% did not use EDNS at all, and a very small percentage used EDNS with an advertized message size of 512 bytes. More recently we've been asked to look at EDNS message sizes again and see how often a root server receives queries with the DO bit set and a message size of 512 bytes. The results of our analysis of the DITL 2009 data are below:

The first graph (above) shows distributions of queries received from three classes of clients: No EDNS, EDNS with the DNSSEC OK bit clear, and EDNS with the DNSSEC OK bit set. For example, in this plot we can see that "DO=1" queries were received from about one million different clients, the busiest of that group sent more than 10 million queries in this 72 hour period. The "No EDNS" group sent fewer queries overall, but there are more clients in that group.

The second graph shows, again, that about 65% of queries arrive with DO=1 and a buffer size of 4096. These DNS clients meet the requirements of RFC 4035 should be fine with respect to large DNSSEC responses. The blue area represents the approximately 31% of queries that lack EDNS. These should be fine also, since they are not requesting to be sent any DNSSEC records. However, the green area beneath the blue shows that about 2% of queries arrive with DO=1 and a message size of 512 bytes. If these clients switch to TCP, it could present a significant load to the root server system.

The final graph shows a distribution, similar to the first, for the clients in this "DO=1 512 bytes" category. In this 72 hour period, the root server received such queries from about 75,000 unique sources.

Submitted by admin on Tue, 2009-06-09 21:10. categories [ ]

Corporate Documents and Structure


OARC, Inc. is a nonprofit corporation formed under the laws of the State of Delaware. The corporation was created June 30, 2008 (file number 4569769).


Submitted by admin on Wed, 2009-05-13 22:44. categories [ ]

Upward Referrals Considered Harmful

Note: you can skip ahead to BIND fixes.

Recently the hosting company ISPrime became the victim of a DNS-based DDoS attack using spoofed source addresses. Some are calling it an amplification attack because the query ". IN NS" is quite small (47 octets) while an upward referral response is a bit larger (256 octets). For some additional information on this attack, see ISC SANS story #5713. OARC members can track this as incident #10780.

One interesting aspect of this attack is that the queries are apparently sent to authoritative nameservers only. In the past we have seen DNS-based attacks bounce queries through open resolvers. Its not clear why this attack is using authoritative nameservers. Perhaps because it is easy to get a list of nameservers if you already have a list of domain names and you can be reasonably sure that an authoritative nameserver will give some kind of answer.

The attack brings an old debate back into the light: What is an authoriative nameserver's appropriate response to a query that cannot be answered?

The configuration and/or implementation of some authoritative nameservers causes them to return an upward referral to the root zone. We recommend against this behavior for a number of reasons:

  1. Upward referrals are generally useless. The resolver that is iterating through the space already knows where to start.
  2. A proper iterative resolver should consider the upward referral "out of bailiwick" and ignore the data anyway.
  3. The authoritative nameserver's root zone "hints" may become stale over time if not properly maintained, causing delivery of queries to decommissioned root server addresses.
  4. Upward referrals are known to cause "referral loops" that result in hundreds of useless queries.

We feel that a REFUSED response code is better than an upward referral.

To find out if your authoritative nameservers return an upward referral, send them a query with the dig program:

$ dig . NS

If you see a long list of ROOT-SERVERS.NET names and addresses, you have an upward referral. Click here to see an example.

Eliminating Upward Referrals


  • Disable recursion on authoritative nameservers with this global option:
    recursion no;

  • If your BIND nameserver is a master for some zones, it needs the root hints to correctly send NOTIFY messages to the slave nameservers. To prevent upward referral responses, you can add this line to the global options:
    additional-from-cache no;

    Then, a query such as ". IN NS" should result in a REFUSED response.

    Alternatively, you can use access controls to accomplish the same thing by denying all queries globally and then allowing queries for each zone. Click here to see an example.

  • If your nameserver is authoritative and acting only as a slave, it doesn't really need the root hints because it won't send NOTIFY messages. However, simply removing the root hints from your configuration does not solve this problem because BIND comes with the root hints built into the source code. Therefore, you should still use one of the above techniques.
  • If your nameserver is both authoritative and caching, you really should separate the two functions. Caching nameservers are susceptible to poisoning and other types of attacks. You don't necessarily need separate hardware for each. You might need to use separate IP addresses, or possibly configure the authoritative nameserver to use an external address while the caching nameserver uses an internal address.
  • If you have a caching-only nameserver, make sure that it has proper access controls in place to prevent its abuse by external users.
Submitted by admin on Thu, 2009-01-22 00:06. categories [ ]

dig +bufsiz=2048 XN--9T4B11YI5A RRSIG

While working on the TLDmon plugins a couple of weeks ago, I noticed that a certain query to was consistenly failing:

$ dig +bufsiz=2048 XN--9T4B11YI5A RRSIG

; <<>> DiG 9.3.5-P2 <<>> +bufsiz=2048 XN--9T4B11YI5A RRSIG
; (1 server found)
;; global options:  printcmd
;; connection timed out; no servers could be reached

It was strange because queries to the same server for all of the other TLD hosted there work just fine:

$ dig +short +bufsiz=2048 XN--KGBECHTV RRSIG | wc
       6      78    1794
$ dig +short +bufsiz=2048 XN--HGBK6AJ7F53BBA RRSIG | wc
       6      78    1831

It was also strange because the problematic TLD works fine from hosts outside of ISC's network (which is where the OARC servers are located), and it works if the query is sent to or

A tcpdump shows that the DNS reply message is fragmented and we only get the first fragment.

That this problem happens only (?) when querying from ISC's network seems to imply it is caused by something on ISC's network. But then why does it work when querying Why would the second fragment from c arrive, but not the fragment from b? Here's a tcpdump showing the correctly received second fragment from c.

I think it safe to assume that the fragment leaving b is the same, except with different values some TCP header fields (ip_sum, ip_id, ip_ttl, ip_src). Here's another tcpdump showing the fragment from b received outside ISC's network.

Note that both of these fragments are smaller (ip_len=31) than the minimum Ethernet payload size so they are padded out to 46 bytes.

Submitted by on Tue, 2008-12-09 21:22. categories [ ]

OARC's Open DNSSEC Validating Resolver

7 June 2011 UPDATE: The .de zone is now fully signed and the corresponding DS Resource Record has been added to the root zone, so the testbed redirection has been removed from both resolvers.

4 October 2010 UPDATE: We have now added the .de DNSSEC Testbed to both resolvers.

How To Use ODVR

OARC is pleased to offer open DNSSEC-validating resolvers ("ODVR") that anyone can use to experiment with DNSSEC. The IP addresses for ODVR nameservers are:

Instance   IPv4   IPv6
BIND 9   2001:4f8:3:2bc:1::64:20
Unbound   2001:4f8:3:2bc:1::64:21

You might like to manually query the ODVR nameservers with a tool such as dig. Be sure to add the +dnssec option:

$ dig +dnssec @ | less

The AD bit in the response flags tells you that the reply data has been validated:

;; flags: qr rd ra ad; ...

Another way to use ODVR is to place the following lines in your Unix /etc/resolv.conf file:


Windows users can manually set DNS servers in the Internet Protocol Properties dialog of a network connection.

Trust Anchors

ODVR has been configured with the following list of trust anchors:

ZoneKey VerifiedPGP Sig Verified

ODVR also validates against ISC's DLV registry.

Data Collection

OARC collects data from the ODVR nameservers and makes this data available to our members for research purposes.

Traffic Graphs

These graphs, updated nightly, show the number of queries received with and without the "DO" bit set, and the number of responses sent with and without the "AD" bit set.

Configuration Files


Frequently Anticipated Questions

Q: Does it mean all my DNS lookups are secure if I use OARC's validating resolvers?

A: No, probably not, for the following reasons:

  1. Most zones are not yet signed. Chances are that, for most of your DNS queries, there will not be any DNSSEC signatures. However, we expect this to improve over time as more and more zones take advantage of DNSSEC.
  2. Most end-user applications (think Web browser) and stub resolvers (a part your computer's operating system) do not yet perform DNSSEC validation. This means that the channel between you and the OARC nameserver is still vulnerable to attack. In other words, security of the DNSSEC transaction is only guaranteed up to the point where the validation has been performed.

Q: Then why are you doing this?

A: A few reasons:

  1. So that you can play with DNSSEC without changing the configuration of your own nameserver.
  2. To convince you that a DNSSEC-validating resolver works almost exactly like a non-validating resolver and that you should go ahead and enable DNSSEC on your own resolvers.
  3. To collect and publish data on adoption of DNSSEC over time.

Q: Can I use ODVR nameservers provide protection from Kaminsky-style spoofing attacks?

A: The answer is complicated and depends on a number of other factors. Generally, this should not be your motivation for using ODVR. If you are stuck using a DNS resolver with poor source port randomization then ODVR may make you more secure. However, a determined attacker could probably spoof answers that appear to come from the ODVR nameservers and give you bad answers.

Q: I thought open resolvers were a bad thing?

A: It's true that open resolvers are usually considered to be a problem and have been used — in combination with source address spoofing — to conduct large-scale DDoS attacks. Such attacks are made possible because (1) there are hundreds of thousands, if not millions, of open resolvers, and (2) their owners/operators are unaware of the openness. The ODVR nameservers are rate-limited and closely monitored. If we have reason to suspect abuse of the ODVR nameservers, we will act quickly to stop it. Please contact the OARC Admin if you have abuse concerns.

Q: Can DNS-OARC members have non-rate-limited access?

A: Absolutely. Write to the OARC Admin to find out how.

Submitted by admin on Tue, 2008-10-28 21:13. categories [ ]

OARC Services, Projects, and Activities

This page catalogs OARC's services to members and the public, its projects and other activities.

Submitted by wessels on Mon, 2008-08-11 21:19. categories [ ]