idnits 2.17.00 (12 Aug 2021) /tmp/idnits34319/draft-ietf-grow-as-path-prepending-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (October 30, 2020) is 567 days in the past. Is this intentional? Checking references for intended status: Best Current Practice ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1' on line 406 -- Looks like a reference, but probably isn't: '2' on line 408 -- Looks like a reference, but probably isn't: '3' on line 411 ** Downref: Normative reference to an Informational RFC: RFC 5398 ** Downref: Normative reference to an Informational RFC: RFC 5737 ** Downref: Normative reference to an Informational RFC: RFC 8195 Summary: 3 errors (**), 0 flaws (~~), 2 warnings (==), 5 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. McBride 3 Internet-Draft Futurewei 4 Intended status: Best Current Practice D. Madory 5 Expires: May 3, 2021 Oracle 6 J. Tantsura 7 Apstra 8 R. Raszuk 9 Bloomberg LP 10 H. Li 11 HPE 12 October 30, 2020 14 AS Path Prepending 15 draft-ietf-grow-as-path-prepending-01 17 Abstract 19 AS Path Prepending provides a tool to manipulate the BGP AS_Path 20 attribute through prepending multiple entries of an AS. AS Path 21 Prepending is used to deprioritize a route or alternate path. By 22 prepending the local ASN multiple times, ASs can make advertised AS 23 paths appear artificially longer. Excessive AS Path Prepending has 24 caused routing issues in the internet. This document provides 25 guidance,to the internet community, with how best to utilize AS Path 26 Prepending in order to avoid negatively affecting the internet. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at https://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on May 3, 2021. 45 Copyright Notice 47 Copyright (c) 2020 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (https://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 63 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 64 2. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 3 65 3. Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 4 66 3.1. Excessive Prepending . . . . . . . . . . . . . . . . . . 4 67 3.2. Prepending during a routing leak . . . . . . . . . . . . 5 68 3.3. Prepending to All . . . . . . . . . . . . . . . . . . . . 6 69 3.4. Memory . . . . . . . . . . . . . . . . . . . . . . . . . 6 70 3.5. Errant announcement . . . . . . . . . . . . . . . . . . . 7 71 4. Alternatives to AS Path Prepend . . . . . . . . . . . . . . . 7 72 5. Best Practices . . . . . . . . . . . . . . . . . . . . . . . 7 73 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 74 7. Security Considerations . . . . . . . . . . . . . . . . . . . 9 75 8. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 9 76 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 9 77 9.1. Normative References . . . . . . . . . . . . . . . . . . 9 78 9.2. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 9 79 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 10 81 1. Introduction 83 The Border Gateway Protocol (BGP) [RFC4271] specifies the AS_PATH 84 attribute which enumerates ASs a route update has traversed. If the 85 UPDATE message is propagated over an external link, then the local AS 86 number is prepended to the AS_PATH attribute, and the NEXT_HOP 87 attribute is updated with an IP address of the router that should be 88 used as a next hop to the network. If the UPDATE message is 89 propagated over an internal link, then the AS_PATH attribute and the 90 NEXT_HOP attribute are passed unmodified. 92 A common practice among operators is to prepend multiple entries of 93 an AS (known as AS Path Prepending) in order to deprioritize a route 94 or a path. This has worked well in practice but the practice is 95 increasing, with both IPv4 and IPv6, and there are inherit risks to 96 the global internet especially with excessive AS Path Prepending. 97 Prepending is frequently employed in an excessive manner such that it 98 renders routes vulnerable to disruption or misdirection. AS Path 99 Prepending is discussed in Use of BGP Large Communities [RFC8195] and 100 this document provides additional, and specific, guidance to 101 operators on how to be a good internet citizen with the proper use of 102 AS Path Prepending. 104 1.1. Requirements Language 106 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 107 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 108 document are to be interpreted as described in RFC 2119 [RFC2119]. 110 2. Use Cases 112 There are various reasons that AS Path Prepending is in use today 113 including: 115 o Preferring one ISP over another ISP on the same ASBR or across 116 different ASBRs 118 o Preferring one ASBR over another ASBR in the same site 120 o Utilize one path exclusively and another path solely as a backup 122 o Signal to indicate that one path may have a different amount of 123 capacity than another where the lower capacity link still takes 124 traffic 126 o An ISP doesn't accept traffic engineering using BGP communities. 127 Prepending is the only option. 129 The following illustration, from Geoff Hustons Path Prepending in BGP 130 [1], shows how AS Prepending is typically used: 132 +---+ +---+ 133 +---| D |----| F | 134 | +---+ +---+ 135 +---+ +---+ | 136 | A |---| B | | 137 +---+ +---+ | 138 | +---+ +---+ 139 +---| C |----| E | 140 +---+ +---+ 142 B will normally prefer the path via C to send traffic to E, as this 143 represents the shorter AS path for B. If E were to prepend a further 144 two instances of its own AS number when advertising its routes to C, 145 then B will now see a different situation, where the AS Path via D 146 represents the shorter path. Through the use of selective prepending 147 E is able to alter the routing decision of B, even though B is not an 148 adjacent neighbour of E. The result is that traffic from A and B 149 will be passed via D and F to reach E, rather than via C. In this 150 way prepending implements action at a distance where the routing 151 decisions made by non-adjacent ASs can be influenced by selective AS 152 Path prepending. 154 To illustrate, in August 2020 a large ISP had a network outage that 155 affected their customers and other ISPs. One major problem was that 156 the ISP wasn't withdrawing BGP routes, the stale routes were 157 continuing to be announced as legitimate by the down ISP. This 158 caused blackholing of traffic even when customers had backup ISPs. 159 What could customers do in this situation? They could change local 160 preference to help send traffic to the backup ISP. They could send 161 more specifics to the backup ISP. They could also pre-provision the 162 use of AS Path Prepend to prepend the same AS amount to both primary 163 and backup ISPs before failure. Customers could then, during a 164 failure, remove one prepend to the backup ISP to make it more 165 preferred over the down ISP. This is one, of several, scenarios 166 where using AS Path Prepend can be beneficial. 168 3. Problems 170 Since it is so commonly used, what is the problem with the excessive 171 use of AS Path Prepending? Here are a few examples: 173 3.1. Excessive Prepending 175 The risk of excessive use of AS Path Prepending can be illustrated 176 with real-world examples that have been anonymized using documention 177 prefixes [RFC5737] and ASs [RFC5398] . Consider the prefix 178 198.51.100.0/24 which is normally announced with an inordinate amount 179 of prepending. A recent analysis revealed that 198.51.100.0/24 is 180 announced to the world along the following AS path: 182 64496 64511 64511 64511 64511 64511 64511 64511 64511 64511 64511 183 64511 64511 64511 64511 64511 64511 64511 64511 64511 64511 64511 184 64511 64511 186 In this example, the origin AS64511 appears 23 consecutive times 187 before being passed on to a single upstream (AS64496), which passes 188 it on to the global internet, prepended-to-all. An attacker, wanting 189 to intercept or manipulate traffic to this prefix, could enlist a 190 datacenter to allow announcements of the same prefix with a 191 fabricated AS path such as 999999 64496 64511. Here the fictional 192 AS999999 represents the shady datacenter. This malicious route would 193 be preferred due to the shortened AS path length and might go 194 unnoticed by the true origin, even if route-monitoring had been 195 implemented. Standard BGP route monitoring checks a route's origin 196 and upstream and both would be intact in this scenario. The length 197 of the prepending gives the attacker room to craft an AS path that 198 would appear plausible to the casual observer, comply with origin 199 validation mechanisms, and not be detected by off-the-shelf route 200 monitoring. 202 3.2. Prepending during a routing leak 204 In April 2010, a service provider experienced a routing leak. While 205 analyzing the leak something peculiar was noticed. When we ranked 206 the approximately 50,000 prefixes involved in the leak based on how 207 many ASs accepted the leaked routes, most of the impact was 208 constrained to Country A routes. However, two of the top five most- 209 propagated leaked routes (listed in the table below) were Country B 210 routes. 212 During the routing leak, nearly all of the ASs of the internet 213 preferred the Country A leaked routes for 192.0.2.0/21 and 214 198.51.100.0/22 because, at the time, these two Country B prefixes 215 were being announced to the entire internet along the following 216 excessively prepended AS path: 64496 64500 64511 64511 64511 64511 217 64511 64511. Virtually any illegitimate route would be preferred 218 over the legitimate route. In this case, the victim is all but 219 ensuring their victimhood. 221 There was only a single upstream seen in the prepending example from 222 above, so the prepending was achieving nothing except incurring risk. 223 You would think such mistakes would be relatively rare, especially 224 now, 10 years later. As it turns out, there is quite a lot of 225 prepending-to-all going on right now and during leaks, it doesn't go 226 well for those who make this mistake. While one can debate the 227 merits of prepending to a subset of multiple transit providers, it is 228 difficult to see the utility in prepending to every provider. In 229 this configuration, the prepending is no longer shaping route 230 propagation. It is simply incentivizing ASs to choose another origin 231 if one were to suddenly appear whether by mistake or otherwise. 233 3.3. Prepending to All 235 Based on analysis done in 2019, Excessive AS Path Prepending [2], out 236 of approximately 750,000 routes in the IPv4 global routing table, 237 nearly 60,000 BGP routes are prepended to 95% or more of hundreds of 238 BGP sources. About 8% of the global routing table, or 1 out of every 239 12 BGP routes, is configured with prepends to virtually the entire 240 internet. The 60,000 routes include entities of every stripe: 241 governments, financial institutions, even important parts of internet 242 infrastructure. 244 Much of the worst propagation of leaked routes during big leak events 245 have been due to routes being prepended-to-all. AS64505 leak of 246 April 2014 (>320,000 prefixes) was prepended-to-all. And the AS64506 247 leak of June 2015 (>260,000 prefixes) was also prepended-to-all. 248 Prepended-to-all prefixes are those seen as prepended by all (or 249 nearly all) of the ASs of the internet. In this configuration, 250 prepending is no longer shaping route propagation but is simply 251 incentivizing ASs to choose another origin if one were to suddenly 252 appear whether by mistake or otherwise. The percentage of the IPv4 253 table that is prepended-to-all is growing at 0.5% per year. The IPv6 254 table is growing slower at 0.2% per year. The reasons for using 255 prepend-to-all appears to be due to 1) the AS forgetting to remove 256 the prepending for one of its transit providers when it is no longer 257 needed and 2) the AS attempting to de-prioritize traffic from transit 258 providers over settlement-free peers and 3) there are simply a lot of 259 errors in BGP routing. Consider the prepended AS path below: 261 64496 64501 64501 64510 64510 64501 64510 64501 64501 64510 64510 262 64501 64501 64510 264 The prepending here involves a mix of two distinct ASNs (64501 and 265 64510) with the last two digits transposed. 267 3.4. Memory 269 BGP attribute sets are shared among stored routes, ie, if two stored 270 routes have the same attribute sets, the attribute set is stored only 271 once. AS Paths are shared among attribute sets so that if two stored 272 attribute sets have the same AS Path, then the AS Path is stored only 273 once. Storing them in the control plane is not a big problem. 275 However, AS Paths can be sent in Netflow which is generated in the 276 forwarding plane. AS Paths are not stored in expensive fast memory 277 on the forwarding plane, but still, using memory on the forwarding 278 plane has greater impact than on the control plane. An AS Path 279 consists of AS_SEQUENCE (and other elements). An AS_SEQUENCE can 280 contain a maximum of 255 ASNs. If the AS Path is longer, then 281 multiple AS_SEQUENCE's are required. The code to parse them and 282 create them is not often exercised and is a potential for bugs in 283 fresh code. The older implementations have these bugs well and truly 284 shaken out of them. Some BGP implementations have had memory 285 corruption/fragmentation problems with long AS Paths. 287 3.5. Errant announcement 289 There was an Internet-wide outage caused by a single errant routing 290 announcement. In this incident, AS64496 announced its one prefix 291 with an extremely long AS path. Someone entered their ASN instead of 292 the prepend count 64496 modulo 256 = 252 prepends and when a path 293 lengths exceeded 255, routers crashed 295 4. Alternatives to AS Path Prepend 297 There are various options to provide path preference without needing 298 to use AS Path Prepend: 300 o Use predefined communities that are mapped to a particular 301 behavior when propagated. 303 o Announce more specific routes on the preferred path. 305 o The BGP Origin Code is an attribute that is used for path 306 selection and can be used as a high order tie-breaker. The three 307 origin codes are IGP, EGP and INCOMPLETE. When AS Paths are of 308 equivalent length, users could advertise paths, with IGP or EGP 309 origin, over the preferred path while the other ASBRs (which would 310 otherwise need to prepend N times) advertises with an INCOMPLETE 311 origin code. 313 5. Best Practices 315 Many of the best practices, or lack thereof, can be illustrated from 316 the preceeding examples. Here's a summary of the best current 317 practices when using AS Path Prepending: 319 o Network operators should ensure prepending is absolutely necessary 320 as many networks have excessive prepending 322 o There is no need to prepend more than 5 ASs. The following 323 diagram shows that, according to Excessive AS Path Prepending [3], 324 90% of AS path lengths are 5 ASNs or fewer in length. 326 +------------------------------------+ 327 90| | 328 | X | 329 80| X X | 330 | X X | 331 70| X X | 332 | X X | 333 60| X X | 334 | X X | 335 50| X X | 336 | X X | 337 40| X X | 338 | X X | 339 30| X X | 340 | X X | 341 20| X XX | 342 | XX XX | 343 10| XX XXXX | 344 |XX XXXXXXXXXXXXXXXXX| 345 +------------------------------------+ 346 5 10 15 347 AS Path Length in IPv4 349 X Axis = unique AS Paths in millions 351 o Don't prepend ASNs that you don't own. 353 o Prepending-to-all is a self-inflicted and needless risk that 354 serves little purpose. Those excessively prepending their routes 355 should consider this risk and adjust their routing configuration. 357 o The Internet is typically around 5 ASs deep with the largest 358 AS_PATH being 16-20 ASNs. Some have added 100 or more AS Path 359 Prepends and operators should therefore consider limiting the 360 maximum AS-path length being accepted through aggressive filter 361 policies. 363 6. IANA Considerations 364 7. Security Considerations 366 Long prepending may make a network more vulernable to route hijacking 367 which will exist whenever there is a well connected peer that is 368 willing to forge their AS_PATH or allows announcements with a 369 fabricated AS path. 371 8. Acknowledgement 373 The authors would like to thank Greg Skinner, Randy Bush, Dave 374 Farmer, Nick Hilliard, Martijn Schmidt, Jakob Heitz, Michael Still, 375 Geoff Huston and Jeffrey Haas for contributing to this document. 377 9. References 379 9.1. Normative References 381 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 382 Requirement Levels", BCP 14, RFC 2119, 383 DOI 10.17487/RFC2119, March 1997, 384 . 386 [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 387 Border Gateway Protocol 4 (BGP-4)", RFC 4271, 388 DOI 10.17487/RFC4271, January 2006, 389 . 391 [RFC5398] Huston, G., "Autonomous System (AS) Number Reservation for 392 Documentation Use", RFC 5398, DOI 10.17487/RFC5398, 393 December 2008, . 395 [RFC5737] Arkko, J., Cotton, M., and L. Vegoda, "IPv4 Address Blocks 396 Reserved for Documentation", RFC 5737, 397 DOI 10.17487/RFC5737, January 2010, 398 . 400 [RFC8195] Snijders, J., Heasley, J., and M. Schmidt, "Use of BGP 401 Large Communities", RFC 8195, DOI 10.17487/RFC8195, June 402 2017, . 404 9.2. URIs 406 [1] https://labs.apnic.net/?p=1264 408 [2] https://blogs.oracle.com/internetintelligence/excessive-as-path- 409 prepending-is-a-self-inflicted-vulnerability 411 [3] https://blogs.oracle.com/internetintelligence/excessive-as-path- 412 prepending-is-a-self-inflicted-vulnerability 414 Authors' Addresses 416 Mike McBride 417 Futurewei 419 Email: michael.mcbride@futurewei.com 421 Doug Madory 422 Oracle 424 Email: douglas.madory@oracle.com 426 Jeff Tantsura 427 Apstra 429 Email: jefftant.ietf@gmail.com 431 Robert Raszuk 432 Bloomberg LP 434 Email: robert@raszuk.net 436 Hongwei Li 437 HPE 439 Email: flycoolman@gmail.com