idnits 2.17.00 (12 Aug 2021) /tmp/idnits15851/draft-ietf-dnsop-serve-stale-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC1034], [RFC1035], [RFC2181]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. == There are 1 instance of lines with non-RFC2606-compliant FQDNs in the document. -- The draft header indicates that this document updates RFC1034, but the abstract doesn't seem to directly say this. It does mention RFC1034 though, so this could be OK. -- The draft header indicates that this document updates RFC1035, but the abstract doesn't seem to directly say this. It does mention RFC1035 though, so this could be OK. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (February 23, 2019) is 1182 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Obsolete informational reference (is this intentional?): RFC 7719 (Obsoleted by RFC 8499) Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DNSOP Working Group D. Lawrence 3 Internet-Draft Oracle 4 Updates: 1034, 1035 (if approved) W. Kumari 5 Intended status: Standards Track P. Sood 6 Expires: August 27, 2019 Google 7 February 23, 2019 9 Serving Stale Data to Improve DNS Resiliency 10 draft-ietf-dnsop-serve-stale-03 12 Abstract 14 This draft defines a method for recursive resolvers to use stale DNS 15 data to avoid outages when authoritative nameservers cannot be 16 reached to refresh expired data. It updates the definition of TTL 17 from [RFC1034], [RFC1035], and [RFC2181] to make it clear that data 18 can be kept in the cache beyond the TTL expiry and used for responses 19 when a refreshed answer is not readily available. One of the 20 motivations for serve-stale is to make the DNS more resilient to DoS 21 attacks, and thereby make them less attractive as an attack vector. 23 Ed note 25 Text inside square brackets ([]) is additional background 26 information, answers to frequently asked questions, general musings, 27 etc. They will be removed before publication. This document is 28 being collaborated on in GitHub at . The most recent version of the document, open issues, etc 30 should all be available here. The authors gratefully accept pull 31 requests. 33 Status of This Memo 35 This Internet-Draft is submitted in full conformance with the 36 provisions of BCP 78 and BCP 79. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF). Note that other groups may also distribute 40 working documents as Internet-Drafts. The list of current Internet- 41 Drafts is at https://datatracker.ietf.org/drafts/current/. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on August 27, 2019. 50 Copyright Notice 52 Copyright (c) 2019 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents 57 (https://trustee.ietf.org/license-info) in effect on the date of 58 publication of this document. Please review these documents 59 carefully, as they describe your rights and restrictions with respect 60 to this document. Code Components extracted from this document must 61 include Simplified BSD License text as described in Section 4.e of 62 the Trust Legal Provisions and are provided without warranty as 63 described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 68 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 69 3. Background . . . . . . . . . . . . . . . . . . . . . . . . . 3 70 4. Standards Action . . . . . . . . . . . . . . . . . . . . . . 4 71 5. Example Method . . . . . . . . . . . . . . . . . . . . . . . 4 72 6. Implementation Caveats . . . . . . . . . . . . . . . . . . . 6 73 6.1. Implementation Considerations . . . . . . . . . . . . . . 7 74 7. Implementation Status . . . . . . . . . . . . . . . . . . . . 8 75 8. EDNS Option . . . . . . . . . . . . . . . . . . . . . . . . . 8 76 9. Security Considerations . . . . . . . . . . . . . . . . . . . 9 77 10. Privacy Considerations . . . . . . . . . . . . . . . . . . . 9 78 11. NAT Considerations . . . . . . . . . . . . . . . . . . . . . 9 79 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 80 13. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 10 81 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 82 14.1. Normative References . . . . . . . . . . . . . . . . . . 10 83 14.2. Informative References . . . . . . . . . . . . . . . . . 10 84 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11 86 1. Introduction 88 Traditionally the Time To Live (TTL) of a DNS resource record has 89 been understood to represent the maximum number of seconds that a 90 record can be used before it must be discarded, based on its 91 description and usage in [RFC1035] and clarifications in [RFC2181]. 93 This document proposes that the definition of the TTL be explicitly 94 expanded to allow for expired data to be used in the exceptional 95 circumstance that a recursive resolver is unable to refresh the 96 information. It is predicated on the observation that authoritative 97 server unavailability can cause outages even when the underlying data 98 those servers would return is typically unchanged. 100 We describe a method below for this use of stale data, balancing the 101 competing needs of resiliency and freshness. 103 2. Terminology 105 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 106 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 107 "OPTIONAL" in this document are to be interpreted as described in BCP 108 14 [RFC2119] [RFC8174] when, and only when, they appear in all 109 capitals, as shown here. 111 For a comprehensive treatment of DNS terms, please see [RFC7719]. 113 3. Background 115 There are a number of reasons why an authoritative server may become 116 unreachable, including Denial of Service (DoS) attacks, network 117 issues, and so on. If the recursive server is unable to contact the 118 authoritative servers for a query but still has relevant data that 119 has aged past its TTL, that information can still be useful for 120 generating an answer under the metaphorical assumption that "stale 121 bread is better than no bread." 123 [RFC1035] Section 3.2.1 says that the TTL "specifies the time 124 interval that the resource record may be cached before the source of 125 the information should again be consulted", and Section 4.1.3 further 126 says the TTL, "specifies the time interval (in seconds) that the 127 resource record may be cached before it should be discarded." 129 A natural English interpretation of these remarks would seem to be 130 clear enough that records past their TTL expiration must not be used. 131 However, [RFC1035] predates the more rigorous terminology of 132 [RFC2119] which softened the interpretation of "may" and "should". 134 [RFC2181] aimed to provide "the precise definition of the Time to 135 Live", but in Section 8 was mostly concerned with the numeric range 136 of values and the possibility that very large values should be 137 capped. (It also has the curious suggestion that a value in the 138 range 2147483648 to 4294967295 should be treated as zero.) It closes 139 that section by noting, "The TTL specifies a maximum time to live, 140 not a mandatory time to live." This is again not [RFC2119]-normative 141 language, but does convey the natural language connotation that data 142 becomes unusable past TTL expiry. 144 Several major recursive resolver operators currently use stale data 145 for answers in some way, including Akamai (in three different 146 resolver implementations), BIND, Knot, OpenDNS, and Unbound. Apple 147 can also use stale data as part of the Happy Eyeballs algorithms in 148 mDNSResponder. The collective operational experience is that it 149 provides significant benefit with minimal downside. 151 4. Standards Action 153 The definition of TTL in [RFC1035] Sections 3.2.1 and 4.1.3 is 154 amended to read: 156 TTL a 32-bit unsigned integer number of seconds that specifies the 157 duration that the resource record MAY be cached before the source 158 of the information MUST again be consulted. Zero values are 159 interpreted to mean that the RR can only be used for the 160 transaction in progress, and should not be cached. Values SHOULD 161 be capped on the orders of days to weeks, with a recommended cap 162 of 604,800 seconds. If the authority for the data is unavailable 163 when attempting to refresh, the record MAY be used as though it is 164 unexpired. 166 Interpreting values which have the high order bit set as being 167 positive, rather than 0, is a change from [RFC2181]. Suggesting a 168 cap of seven days, rather than the 68 years allowed by [RFC2181], 169 reflects the current practice of major modern DNS resolvers. 171 5. Example Method 173 There is conceivably more than one way a recursive resolver could 174 responsibly implement this resiliency feature while still respecting 175 the intent of the TTL as a signal for when data is to be refreshed. 177 In this example method four notable timers drive considerations for 178 the use of stale data, as follows: 180 o A client response timer, which is the maximum amount of time a 181 recursive resolver should allow between the receipt of a 182 resolution request and sending its response. 184 o A query resolution timer, which caps the total amount of time a 185 recursive resolver spends processing the query. 187 o A resolution recheck timer, which limits the frequency at which a 188 failed lookup will be attempted. 190 o A maximum stale timer, which caps the amount of time that records 191 will be kept past their expiration. 193 Most recursive resolvers already have the query resolution timer, and 194 effectively some kind of resolution recheck timer. The client 195 response timer and maximum stale timer are new concepts for this 196 mechanism. 198 When a request is received by the recursive resolver, it SHOULD start 199 the client response timer. This timer is used to avoid client 200 timeouts. It SHOULD be configurable, with a recommended value of 1.8 201 seconds as being just under a common timeout value of 2 seconds while 202 still giving the resolver a fair shot at resolving the name. 204 The resolver then checks its cache for any unexpired data that 205 satisfies the request and of course returns them if available. If it 206 finds no relevant unexpired data and the Recursion Desired flag is 207 not set in the request, it SHOULD immediately return the response 208 without consulting the cache for expired records. 210 If iterative lookups will be, done then the resolution recheck timer 211 is consulted. Attempts to refresh from the authorities are 212 recommended to be done no more frequently than every 30 seconds. If 213 this request was received within this period, the cache may be 214 immediately consulted for stale data to satisfy the request. 216 Outside the period of the resolution recheck timer, the resolver 217 SHOULD start the query resolution timer and begin the iterative 218 resolution process. This timer bounds the work done by the resolver 219 when contacting external authorities, and is commonly around 10 to 30 220 seconds. 222 If the answer has not been completely determined by the time the 223 client response timer has elapsed, the resolver SHOULD then check its 224 cache to see whether there is expired data that would satisfy the 225 request. If so, it adds that data to the response message; it MUST 226 set the TTL of each expired record in the message greater than 0, 227 with 30 seconds recommended. The response is then sent to the client 228 while the resolver continues its attempt to refresh the data. 230 When no authorities are able to be reached during a resolution 231 attempt, the resolver SHOULD attempt to refresh the delegation. 233 Outside the resolution process, the maximum stale timer is used for 234 cache management and is independent of the query resolution process. 235 This timer is conceptually different from the maximum cache TTL that 236 exists in many resolvers, the latter being a clamp on the value of 237 TTLs as received from authoritative servers and recommended to be 7 238 days in the TTL definition above. The maximum stale timer SHOULD be 239 configurable, and defines the length of time after a record expires 240 that it SHOULD be retained in the cache. The suggested value is 7 241 days, which gives time for monitoring to notice the resolution 242 problem and for human intervention to fix it. 244 6. Implementation Caveats 246 Answers from authoritative servers that have a DNS Response Code of 247 either 0 (NOERROR) or 3 (NXDOMAIN) MUST be considered to have 248 refreshed the data at the resolver. In particular, this means that 249 this method is not meant to protect against operator error at the 250 authoritative server that turns a name that is intended to be valid 251 into one that is non-existent, because there is no way for a resolver 252 to know intent. 254 Stale data is used only when refreshing has failed, in order to 255 adhere to the original intent of the design of the DNS and the 256 behaviour expected by operators. If stale data were to always be 257 used immediately and then a cache refresh attempted after the client 258 response has been sent, the resolver would frequently be sending data 259 that it would have had no trouble refreshing. As modern resolvers 260 use techniques like pre-fetching and request coalescing for 261 efficiency, it is not necessary that every client request needs to 262 trigger a new lookup flow in the presence of stale data, but rather 263 that a good-faith effort has been recently made to refresh the stale 264 data before it is delivered to any client. The recommended period 265 between attempting refreshes is 30 seconds. 267 It is important to continue the resolution attempt after the stale 268 response has been sent, until the query resolution timeout, because 269 some pathological resolutions can take many seconds to succeed as 270 they cope with unavailable servers, bad networks, and other problems. 271 Stopping the resolution attempt when the response with expired data 272 has been sent would mean that answers in these pathological cases 273 would never be refreshed. 275 Canonical Name (CNAME) records mingled in the expired cache with 276 other records at the same owner name can cause surprising results. 277 This was observed with an initial implementation in BIND when a 278 hostname changed from having an IPv4 Address (A) record to a CNAME. 279 The version of BIND being used did not evict other types in the cache 280 when a CNAME was received, which in normal operations is not a 281 significant issue. However, after both records expired and the 282 authorities became unavailable, the fallback to stale answers 283 returned the older A instead of the newer CNAME. 285 6.1. Implementation Considerations 287 This document mainly describes the issues behind serving stale data 288 and intentionally does not provide a formal algorithm. The concept 289 is not overly complex, and the details are best left to resolver 290 authors to implement in their codebases. The processing of serve- 291 stale is a local operation, and consistent variables between 292 deployments are not needed for interoperability. However, we would 293 like to highlight the impact of various variables. 295 The most obvious of these is the maximum stale timer. If this 296 variable is too large it could cause excessive cache memory usage, 297 but if it is too small, the serve-stale technique becomes less 298 effective, as the record may not be in the cache to be used if 299 needed. Memory consumption could be mitigated by prioritizing 300 removal of stale records over non-expired records during cache 301 exhaustion. Implementations may also wish to consider whether to 302 track the names in requests for their last time of use or their 303 popularity, using that as an additional factor when considering cache 304 eviction. A feature to manually flush only stale records could also 305 be useful. 307 The client response timer is another variable which deserves 308 consideration. If this value is too short, there exists the risk 309 that stale answers may be used even when the authoritative server is 310 actually reachable but slow; this may result in sub-optimal answers 311 being returned. Conversely, waiting too long will negatively impact 312 user experience. 314 The balance for the resolution recheck timer is responsiveness in 315 detecting the renewed availability of authorities versus the extra 316 resource use of resolution. If this variable is set too large, stale 317 answers may continue to be returned even after the authoritative 318 server is reachable. If this variable is too small, authoritative 319 servers may be rapidly hit with a significant amount of traffic when 320 they become reachable again. 322 Regarding the TTL to set on stale records in the response, 323 historically TTLs of zero seconds have been problematic for some 324 implementations, and negative values can't effectively be 325 communicated to existing software. Other very short TTLs could lead 326 to congestive collapse as TTL-respecting clients rapidly try to 327 refresh. The recommended 30 seconds not only sidesteps those 328 potential problems with no practical negative consequences, it also 329 rate limits further queries from any client that honors the TTL, such 330 as a forwarding resolver. 332 Apart from timers, one more implementation consideration is the use 333 of stale nameserver addresses for lookups. This is mentioned 334 explicitly because, in some resolvers, getting the addresses for 335 nameservers is a separate path from a normal cache lookup. If 336 authoritative server addresses are not able to be refreshed, 337 resolution can possibly still be successful if the authoritative 338 servers themselves are up. For instance, consider an attack on a 339 toplevel domain that takes its nameservers offline; serve-stale 340 resolvers that had expired glue addresses for subdomains within that 341 TLD would still be able to resolve names within those subdomains, 342 even those it had not previously looked up. 344 7. Implementation Status 346 [RFC Editor: per RFC 6982 this section should be removed prior to 347 publication.] 349 The algorithm described in the Section 5 section was originally 350 implemented as a patch to BIND 9.7.0. It has been in production on 351 Akamai's production network since 2011, and effectively smoothed over 352 transient failures and longer outages that would have resulted in 353 major incidents. The patch was contributed to Internet Systems 354 Consortium and the functionality is now available in BIND 9.12 via 355 the options stale-answer-enable, stale-answer-ttl, and max-stale-ttl. 357 Unbound has a similar feature for serving stale answers, but will 358 respond with stale data immediately if it has recently tried and 359 failed to refresh the answer by pre-fetching. 361 Knot Resolver has a demo module here: https://knot- 362 resolver.readthedocs.io/en/stable/modules.html#serve-stale 364 Details of Apple's implementation are not currently known. 366 In the research paper "When the Dike Breaks: Dissecting DNS Defenses 367 During DDoS" [DikeBreaks], the authors detected some use of stale 368 answers by resolvers when authorities came under attack. Their 369 research results suggest that more widespread adoption of the 370 technique would significantly improve resiliency for the large number 371 of requests that fail or experience abnormally long resolution times 372 during an attack. 374 8. EDNS Option 376 During the discussion of serve-stale in the IETF dnsop working group, 377 it was suggested that an EDNS option should be available to either 378 explicitly opt-in to getting data that is possibly stale, or at least 379 as a debugging tool to indicate when stale data has been used for a 380 response. 382 The opt-in use case was rejected as the technique was meant to be 383 immediately useful in improving DNS resiliency for all clients. 385 The reporting case was ultimately also rejected as working group 386 participants determined that even the simpler version of a proposed 387 option was still too much bother to implement for too little 388 perceived value. 390 9. Security Considerations 392 The most obvious security issue is the increased likelihood of DNSSEC 393 validation failures when using stale data because signatures could be 394 returned outside their validity period. This would only be an issue 395 if the authoritative servers are unreachable, the only time the 396 techniques in this document are used, and thus does not introduce a 397 new failure in place of what would have otherwise been success. 399 Additionally, bad actors have been known to use DNS caches to keep 400 records alive even after their authorities have gone away. This 401 potentially makes that easier, although without introducing a new 402 risk. 404 In [CloudStrife] it was demonstrated how stale DNS data, namely 405 hostnames pointing to addresses that are no longer in use by the 406 owner of the name, can be used to co-opt security such as to get 407 domain-validated certificates fraudulently issued to an attacker. 408 While this RFC does not create a new vulnerability in this area, it 409 does potentially enlarge the window in which such an attack could be 410 made. An obvious mitigation is that not only should a certificate 411 authority not use a resolver that has this feature enabled, it should 412 probably not use a caching resolver at all and instead fully look up 413 each name freshly from the root. 415 10. Privacy Considerations 417 This document does not add any practical new privacy issues. 419 11. NAT Considerations 421 The method described here is not affected by the use of NAT devices. 423 12. IANA Considerations 425 There are no IANA considerations. 427 13. Acknowledgements 429 The authors wish to thank Robert Edmonds, Tony Finch, Bob Harold, 430 Matti Klock, Jason Moreau, Giovane Moura, Jean Roy, Mukund Sivaraman, 431 Davey Song, Paul Vixie, Ralf Weber and Paul Wouters for their review 432 and feedback. 434 14. References 436 14.1. Normative References 438 [RFC1034] Mockapetris, P., "Domain names - concepts and facilities", 439 STD 13, RFC 1034, DOI 10.17487/RFC1034, November 1987, 440 . 442 [RFC1035] Mockapetris, P., "Domain names - implementation and 443 specification", STD 13, RFC 1035, DOI 10.17487/RFC1035, 444 November 1987, . 446 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 447 Requirement Levels", BCP 14, RFC 2119, 448 DOI 10.17487/RFC2119, March 1997, 449 . 451 [RFC2181] Elz, R. and R. Bush, "Clarifications to the DNS 452 Specification", RFC 2181, DOI 10.17487/RFC2181, July 1997, 453 . 455 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 456 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 457 May 2017, . 459 14.2. Informative References 461 [CloudStrife] 462 Borgolte, K., Fiebig, T., Hao, S., Kruegel, C., and G. 463 Vigna, "Cloud Strife: Mitigating the Security Risks of 464 Domain-Validated Certificates", ACM 2018 Applied 465 Networking Research Workshop, DOI 10.1145/3232755.3232859, 466 July 2018, . 470 [DikeBreaks] 471 Moura, G., Heidemann, J., Mueller, M., Schmidt, R., and M. 472 Davids, "When the Dike Breaks: Dissecting DNS Defenses 473 During DDos", ACM 2018 Internet Measurement Conference, 474 DOI 10.1145/3278532.3278534, October 2018, 475 . 477 [RFC7719] Hoffman, P., Sullivan, A., and K. Fujiwara, "DNS 478 Terminology", RFC 7719, DOI 10.17487/RFC7719, December 479 2015, . 481 Authors' Addresses 483 David C Lawrence 484 Oracle 486 Email: tale@dd.org 488 Warren "Ace" Kumari 489 Google 490 1600 Amphitheatre Parkway 491 Mountain View CA 94043 492 USA 494 Email: warren@kumari.net 496 Puneet Sood 497 Google 499 Email: puneets@google.com