idnits 2.17.00 (12 Aug 2021) /tmp/idnits16107/draft-ycai-mboned-mvpn-pim-deploy-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 534. -- Found old boilerplate from RFC 3978, Section 5.5, updated by RFC 4748 on line 527. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 543. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 550. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 556. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** An RFC 3978, Section 5.1 paragraph was found, but not on the first page, as required. == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (February 2008) is 5202 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: draft-ietf-l3vpn-2547bis-mcast has been published as RFC 6513 == Outdated reference: A later version (-03) exists of draft-sdry-bmwg-mvpnscale-02 -- Obsolete informational reference (is this intentional?): RFC 4601 (ref. 'Fenner') (Obsoleted by RFC 7761) Summary: 2 errors (**), 0 flaws (~~), 4 warnings (==), 8 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group Yiqun Cai 2 Internet Draft Mike McBride 3 Intented Status: Informational Cisco 4 Expires: August 2008 5 Chris Hall 6 Sprint 8 Maria Napierala 9 AT&T 11 Wim Henderickx 12 Alcatel-Lucent 14 February 2008 16 PIM Based MVPN Deployment Recommendations 18 draft-ycai-mboned-mvpn-pim-deploy-02.txt 20 Status of this Memo 22 By submitting this Internet-Draft, each author represents that any 23 applicable patent or other IPR claims of which he or she is aware 24 have been or will be disclosed, and any of which he or she becomes 25 aware will be disclosed, in accordance with Section 6 of BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF), its areas, and its working groups. Note that 29 other groups may also distribute working documents as Internet- 30 Drafts. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 The list of current Internet-Drafts can be accessed at 38 http://www.ietf.org/ietf/1id-abstracts.txt. 40 The list of Internet-Draft Shadow Directories can be accessed at 41 http://www.ietf.org/shadow.html. 43 Copyright Notice 45 Copyright (C) The IETF Trust (2008). 47 Abstract 49 Multicast VPN, based on pre-standard drafts, has been in operation in 50 production networks for many years. This document describes some of 51 the practices and experiences gained from implementation and 52 deployment of MVPN using PIM with GRE tunnels. It is informational 53 only. 55 Table of Contents 57 1 Introduction ....................................... 3 58 2 Implementation ..................................... 3 59 2.1 RPF ................................................ 3 60 2.2 MTU ................................................ 4 61 2.3 EIBGP Load Balancing ............................... 4 62 2.4 MTRACE ............................................. 5 63 3 Operational Experience ............................. 5 64 3.1 Multicast VPN Design Considerations ................ 5 65 3.2 PIM Modes For MI-PMSI .............................. 6 66 3.2.1 PIM-SSM for MI-PMSI ................................ 6 67 3.2.2 ASM for MI-PMSI .................................... 6 68 3.3 PIM Modes For S-PMSI ............................... 7 69 3.4 CE to PE PIM Modes ................................. 7 70 3.5 Timer Alignment .................................... 8 71 3.6 Addressing ......................................... 8 72 3.7 Filtering .......................................... 8 73 3.8 Scalability ........................................ 9 74 3.9 QOS ................................................ 11 75 4 Security Considerations ............................ 11 76 5 Iana Considerations ................................ 11 77 6 Acknowledgments .................................... 11 78 7 Normative References ............................... 12 79 8 Informative References ............................. 12 80 9 Authors' Addresses ................................. 12 81 10 Full Copyright Statement ........................... 13 82 11 Intellectual Property .............................. 13 84 1. Introduction 86 Multicast support for L3VPN based on RFC2547 [2547bis] was first 87 presented in San Diego IETF, 2000. It had not been included in the 88 charter of L3VPN (formerly PPVPN) working group until San Diego IETF 89 in 2004 and stayed on as an individual submission. The mvpn draft 90 continued to evolve as pre-standards work. Several vendors provided 91 implementations based on this solution. Service providers began 92 deploying this mvpn solution in production networks. 94 Since the working group officially accepted the challenge to define a 95 solution or solutions to support multicast, several proposals have 96 been proposed. They are now captured in [MVPN] which forms a base 97 for future standards work. 99 This document provides MVPN deployment experience based solely on the 100 original PIM and GRE based MVPN solution. This solution is now 101 outlined as one of the options in the standards track [MVPN] 102 document. 104 In this document, we describe some of the lessons learned from 105 implementing and deploying MVPN. We hope it will benefit 106 implementors as well network operators looking to deploy MVPN 107 services. Throughout the document, where the term "MVPN" is used, the 108 reference is to the original MVPN deployment based upon PIM and GRE 109 tunnels. 111 2. Implementation 113 There are three known MVPN implementations: IOS from Cisco, JunOS 114 from Juniper Networks and TimOS from Alcatel-Lucent. Contact these 115 vendors for implementation details beyond what is provided in this 116 draft. The following sections describe common mvpn deployment 117 considerations. 119 2.1. RPF 121 [MVPN] specifies that the source address of any PIM packets that a PE 122 router generates over the MDT tunnel must be the same as the BGP 123 nexthop for updates originated by the PE router for all multicast 124 traffic sources existing in the site. Otherwise, a PE router will not 125 resolve the RPF neighbour towards the source connected to a remote PE 126 router. 128 A PE needs to have a particular IP address which it uses in both the 129 IP source address field of the PIM packet and the next hop field of 130 the BGP updates. If this requirement is overlooked, RPF 131 determination may fail. This has caused interoperability problems in 132 the past and implementors should be careful about it in the future. 134 2.2. MTU 136 When GRE encapsulation is used in the core, 24 bytes are added to the 137 IP packets generated in the VPNs. Due to the lack of a path MTU 138 discovery mechanism for multicast, a PE router may have to fragment 139 the incoming packets. 141 The best practice is to fragment the packets before performing any 142 GRE encapsulation. This spares the egress PE routers from 143 reassembling the fragments, and leaves that for the end-systems. 144 This doesn't work if the "DF" bit is set in the original packet since 145 the packet will be dropped. Its best to ensure that the backbone does 146 not have any links with a 1500 byte MTU. 148 It is further recommended to read Section 5.1 of [Worster] for a more 149 detailed look at preventing fragmentation and reassembly. 151 2.3. EIBGP Load Balancing 153 External and Internal Border Gateway Protocol (eiBGP) load sharing is 154 an enhancement to BGP that enables load sharing over parallel links 155 between CE and PE routers. EIBGP enables service providers to share 156 customer traffic loads over parallel paths within an MPLS core 157 network. 159 When EIBGP load balancing is enabled on all PE routers, we have seen 160 that multicast RPF check, inside the VRF, may be affected if the path 161 towards the source resolves via iBGP. The best practice is to ensure 162 that, when both eBGP and iBGP routes are present, multicast RPF 163 selects eBGP paths only. 165 Example: 166 +---+ 167 ------ |PE1| 168 +--+ +---+ 169 source -- |CE| 170 +--+ +---+ 171 ------ |PE2| 172 +---+ 174 With EIBGP configured, PE1 will have two paths towards the source, 175 one directly via the CE using eBGP, and one via PE2 using iBGP. 177 By default PIM will pick the neighbor with the highest IP address. If 178 this happens to be PE2, the RPF check will fail as it will use the 179 global table. 181 A workaround would be either a static mroute on PE1 pointing towards 182 the CE router, or make sure that CE has a higher ip address than PE2. 183 The better solution is additional logic in the RPF code that where 184 EIBGP is used the EBGP link is preferred. 186 2.4. MTRACE 188 MTRACE is a tool that allows a network operator to obtain multicast 189 routing information from routers and to explore a path to the source 190 of the traffic or the RP. 192 Since there is no security mechanism embedded in MTRACE, some service 193 providers express concern when the mtrace packet has to traverse the 194 PE routers in order to obtain the full information. 196 Vendors have their own mechanisms to remove, or hide, certain fields 197 in the MTRACE packets in order to satisfy the needs of their 198 customers. We need to define a better mechanism for MTRACE in an MVPN 199 environment. 201 3. Operational Experience 203 3.1. Multicast VPN Design Considerations 205 When deploying a multicast VPN service, providers try to optimize 206 multicast traffic distribution and delays while reducing the amount 207 of state. The following considerations have given MVPN providers 208 direction in their MVPN deployment: 210 + Core multicast routing states should typically be kept to a 211 minimum 213 + MVPN packet delays should typically be the same as unicast 214 traffic 216 + Data should typically be sent only to PEs with interested 217 receivers 219 3.2. PIM Modes For MI-PMSI 221 The MI-PMSI is used to build an overlay network connecting all PE 222 routers attaching to the same MVPN. 224 Service providers have implemented PIM-SM and PIM-SSM instantiated 225 MI-PMSI in production networks. The majority of MI-PMSI deployments 226 are using PIM-SM using static Anycast RP with MSDP assignment. But a 227 dynamic RP discovery protocol, such as BSR, is also being used. 229 The decision to deploy either PIM-SM or PIM-SSM is based on the 230 following concerns, 232 + the number of multicast routing states 234 + the overhead of managing the RP if PIM-SM is used 236 + the difference of forwarding delay between shared tree and source 237 trees 239 3.2.1. PIM-SSM for MI-PMSI 241 Optimal MVPN forwarding is most easily achievable when there is a 242 single multicast tree per MVPN per PE. Such trees are naturally built 243 with PIM-SSM since it permits the PE to directly join a source tree 244 for an MDT. With PIM-SSM, no Rendezvous Points are required. With 245 SSM, however, all PEs on an MVPN tree need to maintain source state. 246 Each PE, which is participating in MVPN, is a source. Unless VPN 247 customers locate their multicast sources within a constrained set of 248 sites, SSM may become a scalability concern in the service providers 249 network. 251 3.2.2. ASM for MI-PMSI 253 One solution to minimize the amount of multicast state in an MVPN 254 environment is to configure PIM-SM or BIDIR PIM to stay on the shared 255 tree. With shared trees, multicast state scalability is no longer a 256 function of the number of PE's but rather of the number of VPNs. 258 The scale benefit of shared trees comes at the cost of less efficient 259 multicast distribution. MVPN providers use the MI-PMSI to achieve 260 bandwidth optimality. MVPN providers may address the sub-optimality 261 of shared tree forwarding by deploying an RP at the best location for 262 each VPN. Such an assignment would be based on the VPN source 263 locations, something which may be difficult to maintain. 265 3.3. PIM Modes For S-PMSI 267 The S-PMSI has also been widely deployed by service providers. While 268 both PIM-SM and PIM-SSM are used, PIM-SSM is the more widely 269 deployed, and recommended, S-PMSI tree building model. The majority 270 of S-PMSI deployments today are using SSM since the source address is 271 included in the PIM Hello packet sent from the source PE. 273 MVPN providers deploy the S-PMSI to achieve optimal bandwidth usage, 274 especially when SSM is deployed as well. S-PMSIs are optimized for 275 active sources and receivers and triggered per (S,G) for a subset of 276 (S,G) of a given VPN. Since S-PMSIs are triggered by (S,G) states in 277 a VPN, they could increase the amount of multicast states in an MVPN 278 network. 280 The decision to switch from MI-PMSI to S-PMSI is always made by the 281 ingress PE based upon the traffic load exceeding a configurable 282 threshold. 284 3.4. CE to PE PIM Modes 286 The PIM protocols that are deployed within the customer VPN are 287 independent of the PIM Protocols in use within the Provider core. 288 Customers can choose to deploy PIM-DM, PIM-SM, Bidir, or SSM. 290 With SM or Bidir, customers may choose to deploy the RP on either a 291 PE or CE router. It is generally recommended to have a CE router 292 serve as the RP. This is done primarily to avoid an increase in 293 customer/provider interaction on matters such as the integration of 294 the PE/RP into the customer chosen RP discovery mechanism and to 295 avoid any additional burden on a busy PE router. While RP deployment 296 is most commonly performed on CE routers, we have seen RPs deployed 297 successfully on PE as well as CE routers. 299 If a customer desires to have a provider managed RP, they should 300 consider requesting the service provider manage a CE and have it 301 serve as the RP. To avoid managing an RP altogether, SSM should be 302 deployed. 304 3.5. Timer Alignment 306 When PIM-SM is used in an SP's core MVPN environment, some 307 interesting observations were made. When BSR, for example, is used in 308 the service provider network, to discover RPs in the provider tunnel, 309 it takes more than 3 minutes to detect the failure of the RP if the 310 default timer is used. During the window, PIM Hellos originated by 311 C-PIM instances will be dropped, which cause PIM adjacencies to be 312 torn down. But since the default PIM Hello timer is 30 seconds, C- 313 PIM instance on a PE router detects an outage much faster than the 314 P-PIM instance on the same PE router. 316 This is a factor to be considered when choosing the protocol for RP 317 redundancy and fast failover. One option, for fast failover, is to 318 use BSR only for RP discovery and then utilize Anycast-RP for RP 319 redundancy. 321 3.6. Addressing 323 It has become general practice to use 239/8 private address space 324 when assigning address space to mvpn's. This helps to prevent vpn 325 traffic from being sent outside the mvpn core. When SSM is used, 326 239.232/16 addressing is the common practice according to [Meyer], 327 Administratively Scoped IP Multicast. Operators typically deploy an 328 addressing tool to manage their addresses. 330 This addressing practice can also be used to prevent non-VPN traffic, 331 originating outside the SP boundaries, from entering a VPN. 333 The reader should also reference section 11.5.4 of the [MVPN] draft 334 entitled "Avoiding Conflict with Internet Multicast". 336 3.7. Filtering 338 Filtering at the SP boundaries is needed to prevent VPN security 339 violations. It may be necessary to modify these deployed filters to 340 permit GRE and possibly UDP port 3232. UDP port 3232 is the UDP port 341 used for the S-PMSI Join messages. 343 3.8. Scalability 345 PIM retransmission overhead on a given MI-PMSI increases in linear 346 proportion to any increase in the number of PEs that join the MI- 347 PMSI. The overhead also increases in linear proportion with an 348 increase in the number of J/P messages received from the CEs. 350 There have been no scaling issues with current deployments of MVPN. 351 Current MVPN deployments consist of up to a few hundred sites per 352 MVPN. The number of PE's participating in a MI-PMSI continues to 353 increase as customers extend the multicast group participation to 354 additional VPN sites. There are unicast VPN customers with several 355 thousand sites. These sites are gradually becoming multicast enabled. 356 The number of J/P messages received from CEs will also increase over 357 time. 359 At some level of scaling of the MI-PMSI, PIM Hello's and J/P messages 360 will become a scaling issue. The scaling point at which these 361 messages become a real operational problem is not clear. Empirical 362 field data shows they do not affect the broad range of MVPN 363 deployments today. MVPN is scalable as specified across a wide range 364 of deployments. 366 Some analysis is needed to clarify at what operational level PIM 367 messages do become a problem. The L3VPN WG has gathered requirements 368 information in [MORIN]. A benchmarking draft [DRY] has been submitted 369 to the BMWG to provide consistent MVPN test methodology. The PIM WG 370 is evaluating methods to decrease PIM messages when this becomes of 371 operational value. Extensions to PIM such as PIM J/P Acks and TCP 372 based approaches are being evaluated by the working group. 374 Increasing the Hello timer and increasing the periodic join/prune 375 timer may also help in future MVPN scaling. Doing so, however, may 376 affect join and leave latency in times when control messages are 377 lost. OAM, to verify the health of the data and control paths, would 378 also be affected if the Hello timer were increased or removed 379 altogether. 381 The following analysis compares different pim modes and their 382 resulting mvpn state using the following values: 384 Number of P interfaces 5 386 Number of PE P-PIM interfaces 2 388 Number of PE C-PIM interfaces 1 390 Number of PE 20 391 Number of M-VPN 100 393 S-PMSI/VPN 2 395 PIM adjacencies C-PIM 100 397 PIM adjacencies P-PIM 1900 399 PIM adjacencies PE 2 401 PIM adjacencies Total 2002 403 MIPMSI/PE SPMSI/PE MDT/PE MIPMSI/RP(P) SPMSI/RP(P) MDT/P 405 PIM SSM PIM SSM PIM SSM 407 (S,G) state 2000 4000 6000 2000 4000 6000 408 (*,G) state 0 0 0 0 0 0 409 total state 2000 4000 6000 2000 4000 6000 410 MIPMSI nbrs 1900 1900 NA NA NA 411 MIPMSI 100 100 NA NA NA 412 Inband MDT 3800 3800 NA NA NA 413 Outband MDT 200 200 NA NA NA 415 PIM SM PIM SSM MIPMSI:SM 416 SPMSI:SSM 418 (S,G) state 2000 4000 6000 2000 4000 6000 419 (*,G) state 100 0 100 100 0 100 420 total state 2100 4000 6100 2100 4000 6100 421 MIPMSI nbrs 1900 1900 NA NA NA 422 MIPMSI 100 100 NA NA NA 423 Inband MDT 3800 3800 NA NA NA 424 Outband MDT 200 200 NA NA NA 426 SM(no spt) SSM MIPMSI:SM(no spt) 427 SPMSI:SSM 429 (S,G) state 100 4000 4100 2000 4000 6000 430 (*,G) state 100 0 100 100 0 100 431 total state 200 4000 4200 2100 4000 6100 432 MIPMSI nbrs 1900 1900 NA NA NA 433 MIPMSI 100 100 NA NA NA 434 Inband MDT 3800 3800 NA NA NA 435 Outband MDT 200 200 NA NA NA 437 3.9. QOS 439 Deployments of MVPN, that have deployed QOS, typically use the same 440 QOS mechanisms for the MVPN GRE header that they would use for their 441 other data traffic. VPN customers may want to separate the queuing of 442 multicast data from unicast data. Service Providers are extending 443 their QOS portfolio to support more classes of service to allow for 444 better separation of multicast and unicast traffic. Enhanced QOS 445 mechanisms support applications with short bursts but which require 446 bounded delay (such as video streaming). Since multicast (UDP) 447 traffic might not be subject to the same drop behavior as TCP 448 traffic, QOS profiles support Weighted Random Early Detection (WRED) 449 treatment. 451 4. Security Considerations 453 The use of GRE encapsulations and IP Multicast has certain security 454 implications. As discussed in [Farinacci], security in a network 455 using GRE should be relatively similar to security in a normal IPv4 456 network. And Section 6 of [Fenner] clearly outlines the various 457 security concerns related to PIM and how to use IPsec to secure the 458 protocol. 460 5. Iana Considerations 462 This document does not require any action on the part of IANA. 464 6. Acknowledgments 466 We'd like to thank Dino Farinacci, Yuji Kamite, Hitoshi Fukuda and 467 Eric Rosen for their feedback on this draft. 469 7. Normative References 471 [2547bis] "BGP/MPLS VPNs", Rosen, Rekhter, et. al., February 2006, 472 RFC 4364 474 [MVPN] "Multicast in MPLS/BGP IP VPNs", Rosen, Aggarwal, July 2007, 475 draft-ietf-l3vpn-2547bis-mcast-05.txt 477 8. Informative References 479 [MORIN] T. Morin, "Requirements for Multicast in L3 Provider- 480 Provisioned VPNs", RFC 4834 482 [DRY] S. Dry, "Multicast VPN Scalability Benchmarking", draft-sdry- 483 bmwg-mvpnscale-02.txt 485 [Meyer] D. Meyer, "Administratively Scoped IP Multicast". RFC 2365 487 [Farinacci] D. Farinacci, "Generic Routing Encapsulation (GRE)". RFC 488 2784 490 [Fenner] B. Fenner, "Protocol Independent Multicast - Sparse Mode 491 (PIM-SM)". RFC 4601. 493 [Worster] T. Worster, "Encapsulating MPLS in IP or Generic Routing 494 Encapsulation (GRE)" RFC 4023. 496 9. Authors' Addresses 498 Yiqun Cai 499 ycai@cisco.com 501 Mike McBride 502 mmcbride@cisco.com 504 Chris Hall 505 chall@sprint.net 507 Maria Napierala 508 mnapierala@att.com 510 Wim Henderickx 511 wim.henderickx@alcatel-lucent.be 513 10. Full Copyright Statement 515 Copyright (C) The IETF Trust (2008). 517 This document is subject to the rights, licenses and restrictions 518 contained in BCP 78, and except as set forth therein, the authors 519 retain all their rights. 521 This document and the information contained herein are provided on an 522 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 523 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND 524 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS 525 OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 526 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 527 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 529 11. Intellectual Property 531 By submitting this Internet-Draft, each author represents that any 532 applicable patent or other IPR claims of which he or she is aware 533 have been or will be disclosed, and any of which he or she becomes 534 aware will be disclosed, in accordance with Section 6 of BCP 79. 536 The IETF takes no position regarding the validity or scope of any 537 Intellectual Property Rights or other rights that might be claimed to 538 pertain to the implementation or use of the technology described in 539 this document or the extent to which any license under such rights 540 might or might not be available; nor does it represent that it has 541 made any independent effort to identify any such rights. Information 542 on the procedures with respect to rights in RFC documents can be 543 found in BCP 78 and BCP 79. 545 Copies of IPR disclosures made to the IETF Secretariat and any 546 assurances of licenses to be made available, or the result of an 547 attempt made to obtain a general license or permission for the use of 548 such proprietary rights by implementers or users of this 549 specification can be obtained from the IETF on-line IPR repository at 550 http://www.ietf.org/ipr. 552 The IETF invites any interested party to bring to its attention any 553 copyrights, patents or patent applications, or other proprietary 554 rights that may cover technology that may be required to implement 555 this standard. Please address the information to the IETF at ietf- 556 ipr@ietf.org.