idnits 2.17.00 (12 Aug 2021) /tmp/idnits61871/draft-ietf-grow-bgp-gshut-13.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year ** The document contains RFC2119-like boilerplate, but doesn't seem to mention RFC 2119. The boilerplate contains a reference [BCP14], but that reference does not seem to mention RFC 2119 either. -- The document date (December 14, 2017) is 1612 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'BCP14' is mentioned on line 124, but not defined ** Downref: Normative reference to an Informational RFC: RFC 6198 -- Obsolete informational reference (is this intentional?): RFC 8203 (Obsoleted by RFC 9003) Summary: 2 errors (**), 0 flaws (~~), 2 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group P. Francois, Ed. 3 Internet-Draft Individual Contributor 4 Intended status: Standards Track B. Decraene, Ed. 5 Expires: June 17, 2018 Orange 6 C. Pelsser 7 Strasbourg University 8 K. Patel 9 Arrcus, Inc. 10 C. Filsfils 11 Cisco Systems 12 December 14, 2017 14 Graceful BGP session shutdown 15 draft-ietf-grow-bgp-gshut-13 17 Abstract 19 This draft standardizes a new well-known BGP community 20 GRACEFUL_SHUTDOWN to signal the graceful shutdown of paths. This 21 draft also describes operational procedures which use this community 22 to reduce the amount of traffic lost when BGP peering sessions are 23 about to be shut down deliberately, e.g. for planned maintenance. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at https://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on June 17, 2018. 42 Copyright Notice 44 Copyright (c) 2017 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (https://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 60 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 61 3. Packet loss upon manual EBGP session shutdown . . . . . . . . 3 62 4. EBGP graceful shutdown procedure . . . . . . . . . . . . . . 4 63 4.1. Pre-configuration . . . . . . . . . . . . . . . . . . . . 4 64 4.2. Operations at maintenance time . . . . . . . . . . . . . 4 65 4.3. BGP implementation support for graceful shutdown . . . . 5 66 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 5 67 6. Security Considerations . . . . . . . . . . . . . . . . . . . 5 68 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 6 69 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 6 70 8.1. Normative References . . . . . . . . . . . . . . . . . . 6 71 8.2. Informative References . . . . . . . . . . . . . . . . . 6 72 Appendix A. Alternative techniques with limited applicability . 7 73 A.1. Multi Exit Discriminator tweaking . . . . . . . . . . . . 7 74 A.2. IGP distance Poisoning . . . . . . . . . . . . . . . . . 7 75 Appendix B. Configuration Examples . . . . . . . . . . . . . . . 7 76 B.1. Cisco IOS XR . . . . . . . . . . . . . . . . . . . . . . 7 77 B.2. BIRD . . . . . . . . . . . . . . . . . . . . . . . . . . 8 78 B.3. OpenBGPD . . . . . . . . . . . . . . . . . . . . . . . . 8 79 Appendix C. Beyond EBGP graceful shutdown . . . . . . . . . . . 9 80 C.1. IBGP graceful shutdown . . . . . . . . . . . . . . . . . 9 81 C.2. EBGP session establishment . . . . . . . . . . . . . . . 9 82 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 10 84 1. Introduction 86 Routing changes in BGP can be caused by planned maintenance 87 operations. This document defines a well-known community [RFC1997], 88 called GRACEFUL_SHUTDOWN, for the purpose of reducing the management 89 overhead of gracefully shutting down BGP sessions. The well-known 90 community allows implementers to provide an automated graceful 91 shutdown mechanism that does not require any router reconfiguration 92 at maintenance time. 94 This document discusses operational procedures to be applied in order 95 to reduce or eliminate loss of packets during a maintenance 96 operation. Loss comes from transient lack of reachability during BGP 97 convergence which follows the shutdown of an EBGP peering session 98 between two Autonomous System Border Routers (ASBR). 100 This document presents procedures for the cases where the forwarding 101 plane is impacted by the maintenance, hence when the use of Graceful 102 Restart does not apply. 104 The procedures described in this document can be applied to reduce or 105 avoid packet loss for outbound and inbound traffic flows initially 106 forwarded along the peering link to be shut down. These procedures 107 trigger, in both Autonomous Sytems (AS), rerouting to alternate paths 108 if they exist within the AS, while allowing the use of the old path 109 until alternate ones are learned. This ensures that routers always 110 have a valid route available during the convergence process. 112 The goal of the document is to meet the requirements described in 113 [RFC6198] at best, without changing the BGP protocol. 115 Other maintenance cases, such as the shutdown of an IBGP session or 116 the establishement of an EBGP session, are out of scope of this 117 document. For information, they are briefly discussed in Appendix C. 119 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 120 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 121 "OPTIONAL" in this document are to be interpreted as described in 122 [BCP14] [RFC2119] [RFC8174] when, and only when, they appear in all 123 capitals, as shown here. 125 2. Terminology 127 graceful shutdown initiator: a router on which the session shutdown 128 is performed for the maintenance. 130 graceful shutdown receiver: a router that has a BGP session, to be 131 shutdown, with the graceful shutdown initiator. 133 3. Packet loss upon manual EBGP session shutdown 135 Packets can be lost during the BGP convergence following a manual 136 shutdown of an EBGP session for two reasons. 138 First, some routers can have no path toward an affected prefix, and 139 drop traffic destined to this prefix. This is because alternate 140 paths can be hidden by nodes of an AS. This happens when [RFC7911] 141 is not used and the paths are not selected as best by the ASBR that 142 receive them on an EBGP session, or by Route Reflectors that do not 143 propagate them further in the IBGP topology because they do not 144 select them as best. 146 Second, the FIB can be inconsistent between routers within the AS, 147 and packets toward affected prefixes can loop and be dropped unless 148 encapsulation is used within the AS. 150 This document only addresses the first reason. 152 4. EBGP graceful shutdown procedure 154 This section describes configurations and actions to be performed for 155 the graceful shutdown of EBGP peering links. 157 The goal of this procedure is to retain the paths to be shutdown 158 between the peers, but with a lower LOCAL_PREF value, allowing the 159 paths to remain in use while alternate paths are selected and 160 propagated, rather than simply withdrawing the paths. The LOCAL_PREF 161 value SHOULD be lower than any of the alternative paths. The 162 RECOMMENDED value is 0. 164 Note that some alternative techniques with limited applicability are 165 discussed for information in Appendix A. 167 4.1. Pre-configuration 169 On each ASBR supporting the graceful shutdown receiver procedure, an 170 inbound BGP route policy is applied on all EBGP sessions of the ASBR, 171 that: 173 o matches the GRACEFUL_SHUTDOWN community. 175 o sets the LOCAL_PREF attribute of the paths tagged with the 176 GRACEFUL_SHUTDOWN community to a low value. 178 For information purpose, example of configurations are provided in 179 Appendix B. 181 4.2. Operations at maintenance time 183 On the graceful shutdown initiator, at maintenance time, the 184 operator: 186 o applies an outbound BGP route policy on the EBGP session to be 187 shutdown. This policy tags the paths propagated over the session 188 with the GRACEFUL_SHUTDOWN community. This will trigger the BGP 189 implementation to re-advertise all active routes previously 190 advertised, and tag them with the GRACEFUL_SHUTDOWN community. 192 o applies an inbound BGP route policy on the EBGP session to be 193 shutdown. This policy tags the paths received over the session 194 with the GRACEFUL_SHUTDOWN community and sets LOCAL_PREF to a low 195 value. 197 o wait for route readvertisement over the EBGP session, and BGP 198 routing convergence on both ASBRs. 200 o shutdown the EBGP session, optionally using [RFC8203] to 201 communicate the reason of the shutdown. 203 In the case of a shutdown of the whole router, in addition to the 204 graceful shutdown of all EBGP sessions, there is a need to gracefully 205 shutdown the routes originated by this router (e.g, BGP aggregates 206 redistributed from other protocols, including static routes). This 207 can be performed by tagging these routes with the GRACEFUL_SHUTDOWN 208 community and setting LOCAL_PREF to a low value. 210 4.3. BGP implementation support for graceful shutdown 212 BGP Implementers SHOULD provide configuration knobs that utilize the 213 GRACEFUL_SHUTDOWN community to drain BGP neighbors in preparation of 214 impending neighbor shutdown. Implementation details are outside the 215 scope of this document. 217 5. IANA Considerations 219 The IANA has assigned the community value 0xFFFF0000 to the planned- 220 shut community in the "BGP Well-known Communities" registry. IANA is 221 requested to change the name planned-shut to GRACEFUL_SHUTDOWN and 222 set this document as the reference. 224 6. Security Considerations 226 By providing the graceful shutdown service to a neighboring AS, an 227 ISP provides means to this neighbor and possibly its downstream ASes 228 to lower the LOCAL_PREF value assigned to the paths received from 229 this neighbor. 231 The neighbor could abuse the technique and do inbound traffic 232 engineering by declaring some prefixes as undergoing a maintenance so 233 as to switch traffic to another peering link. 235 If this behavior is not tolerated by the ISP, it SHOULD monitor the 236 use of the graceful shutdown community. 238 7. Acknowledgments 240 The authors wish to thank Olivier Bonaventure, Pradosh Mohapatra, Job 241 Snijders, John Heasley, and Christopher Morrow for their useful 242 comments. 244 8. References 246 8.1. Normative References 248 [RFC1997] Chandra, R., Traina, P., and T. Li, "BGP Communities 249 Attribute", RFC 1997, DOI 10.17487/RFC1997, August 1996, 250 . 252 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 253 Requirement Levels", BCP 14, RFC 2119, 254 DOI 10.17487/RFC2119, March 1997, 255 . 257 [RFC6198] Decraene, B., Francois, P., Pelsser, C., Ahmad, Z., 258 Elizondo Armengol, A., and T. Takeda, "Requirements for 259 the Graceful Shutdown of BGP Sessions", RFC 6198, 260 DOI 10.17487/RFC6198, April 2011, 261 . 263 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 264 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 265 May 2017, . 267 8.2. Informative References 269 [I-D.ietf-idr-best-external] 270 Marques, P., Fernando, R., Chen, E., Mohapatra, P., and H. 271 Gredler, "Advertisement of the best external route in 272 BGP", draft-ietf-idr-best-external-05 (work in progress), 273 January 2012. 275 [RFC7911] Walton, D., Retana, A., Chen, E., and J. Scudder, 276 "Advertisement of Multiple Paths in BGP", RFC 7911, 277 DOI 10.17487/RFC7911, July 2016, 278 . 280 [RFC8203] Snijders, J., Heitz, J., and J. Scudder, "BGP 281 Administrative Shutdown Communication", RFC 8203, 282 DOI 10.17487/RFC8203, July 2017, 283 . 285 Appendix A. Alternative techniques with limited applicability 287 A few alternative techniques have been considered to provide graceful 288 shutdown capabilities but have been rejected due to their limited 289 applicability. This section describes them for possible reference. 291 A.1. Multi Exit Discriminator tweaking 293 The MED attribute of the paths to be avoided can be increased so as 294 to force the routers in the neighboring AS to select other paths. 296 The solution only works if the alternate paths are as good as the 297 initial ones with respect to the LOCAL_PREF value and the AS Path 298 Length value. In the other cases, increasing the MED value will not 299 have an impact on the decision process of the routers in the 300 neighboring AS. 302 A.2. IGP distance Poisoning 304 The distance to the BGP NEXT_HOP corresponding to the maintained 305 session can be increased in the IGP so that the old paths will be 306 less preferred during the application of the IGP distance tie-break 307 rule. However, this solution only works for the paths whose 308 alternates are as good as the old paths with respect to their 309 LOCAL_PREF value, their AS Path length, and their MED value. 311 Also, this poisoning cannot be applied when BGP NEXT_HOP self is used 312 as there is no BGP NEXT_HOP specific to the maintained session to 313 poison in the IGP. 315 Appendix B. Configuration Examples 317 This appendix is non-normative. 319 Example routing policy configurations to honor the GRACEFUL_SHUTDOWN 320 well-known BGP community. 322 B.1. Cisco IOS XR 323 community-set comm-graceful-shutdown 324 65535:0 325 end-set 326 ! 327 route-policy AS64497-ebgp-inbound 328 ! normally this policy would contain much more 329 if community matches-any comm-graceful-shutdown then 330 set local-preference 0 331 endif 332 end-policy 333 ! 334 router bgp 64496 335 neighbor 2001:db8:1:2::1 336 remote-as 64497 337 address-family ipv6 unicast 338 send-community-ebgp 339 route-policy AS64497-ebgp-inbound in 341 ! 342 ! 343 ! 345 B.2. BIRD 347 function honor_graceful_shutdown() { 348 if (65535, 0) ~ bgp_community then { 349 bgp_local_pref = 0; 350 } 351 } 352 filter AS64497_ebgp_inbound 353 { 354 # normally this policy would contain much more 355 honor_graceful_shutdown(); 356 } 357 protocol bgp peer_64497_1 { 358 neighbor 2001:db8:1:2::1 as 64497; 359 local as 64496; 360 import keep filtered; 361 import filter AS64497_ebgp_inbound; 362 } 364 B.3. OpenBGPD 365 AS 64496 366 router-id 192.0.2.1 367 neighbor 2001:db8:1:2::1 { 368 remote-as 64497 369 } 370 # normally this policy would contain much more 371 match from any community GRACEFUL_SHUTDOWN set { localpref 0 } 373 Appendix C. Beyond EBGP graceful shutdown 375 C.1. IBGP graceful shutdown 377 For the shutdown of an IBGP session, provided the IBGP topology is 378 viable after the maintenance of the session, i.e, if all BGP speakers 379 of the AS have an IBGP signaling path for all prefixes advertised on 380 this graceful shutdown IBGP session, then the shutdown of an IBGP 381 session does not lead to transient unreachability. As a consequence, 382 no specific graceful shutdown action is required. 384 C.2. EBGP session establishment 386 We identify two potential causes for transient packet losses upon the 387 establishment of an EBGP session. The first one is local to the 388 startup initiator, the second one is due to the BGP convergence 389 following the injection of new best paths within the IBGP topology. 391 C.2.1. Unreachability local to the ASBR 393 An ASBR that selects as best a path received over a newly established 394 EBGP session may transiently drop traffic. This can typically happen 395 when the NEXT_HOP attribute differs from the IP address of the EBGP 396 peer, and the receiving ASBR has not yet resolved the MAC address 397 associated with the IP address of that "third party" NEXT_HOP. 399 A BGP speaker implementation MAY avoid such losses by ensuring that 400 "third party" NEXT_HOPs are resolved before installing paths using 401 these in the RIB. 403 Alternatively, the operator (script) MAY ping third party NEXT_HOPs 404 that are expected to be used before establishing the session. By 405 proceeding like this, the MAC addresses associated with these third 406 party NEXT_HOPs are resolved by the startup initiator. 408 C.2.2. IBGP convergence 410 During the establishment of an EBGP session, in some corner cases a 411 router may have no path toward an affected prefix, leading to loss of 412 connectivity. 414 A typical example for such transient unreachability for a given 415 prefix is the following: 417 Let's consider three Route Reflectors (RR): RR1, RR2, RR3. There is 418 a full mesh of IBGP sessions between them. 420 1. RR1 is initially advertising the current best path to the 421 members of its IBGP RR full-mesh. It propagated that path within 422 its RR full-mesh. RR2 knows only that path toward the prefix. 424 2. RR3 receives a new best path originated by the startup 425 initiator, being one of its RR clients. RR3 selects it as best, 426 and propagates an UPDATE within its RR full-mesh, i.e., to RR1 and 427 RR2. 429 3. RR1 receives that path, reruns its decision process, and picks 430 this new path as best. As a result, RR1 withdraws its previously 431 announced best-path on the IBGP sessions of its RR full-mesh. 433 4. If, for any reason, RR3 processes the withdraw generated in 434 step 3, before processing the update generated in step 2, RR3 435 transiently suffers from unreachability for the affected prefix. 437 The use of [RFC7911] or [I-D.ietf-idr-best-external] among the RR of 438 the IBGP full-mesh can solve these corner cases by ensuring that 439 within an AS, the advertisement of a new route is not translated into 440 the withdraw of a former route. 442 Indeed, "best-external" ensures that an ASBR does not withdraw a 443 previously advertised (EBGP) path when it receives an additional, 444 preferred path over an IBGP session. Also, "best-intra-cluster" 445 ensures that a RR does not withdraw a previously advertised (IBGP) 446 path to its non clients (e.g. other RRs in a mesh of RR) when it 447 receives a new, preferred path over an IBGP session. 449 Authors' Addresses 451 Pierre Francois (editor) 452 Individual Contributor 454 Email: pfrpfr@gmail.com 456 Bruno Decraene (editor) 457 Orange 459 Email: bruno.decraene@orange.com 460 Cristel Pelsser 461 Strasbourg University 463 Email: pelsser@unistra.fr 465 Keyur Patel 466 Arrcus, Inc. 468 Email: keyur@arrcus.com 470 Clarence Filsfils 471 Cisco Systems 473 Email: cfilsfil@cisco.com