idnits 2.17.00 (12 Aug 2021) /tmp/idnits15670/draft-ietf-l2vpn-pbb-evpn-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 20, 2012) is 3622 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Unexpected draft version: The latest known version of draft-ietf-l2vpn-vpls-pbb-interop is -00, but you're referring to -02. == Outdated reference: draft-ietf-l2vpn-evpn has been published as RFC 7432 Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Working Group Ali Sajassi 3 Internet Draft Samer Salam 4 Category: Standards Track Sami Boutros 5 Cisco 7 Florin Balus Nabil Bitar 8 Wim Henderickx Verizon 9 Alcatel-Lucent 10 Aldrin Isaac 11 Clarence Filsfils Bloomberg 12 Dennis Cai 13 Cisco Lizhong Jin 14 ZTE 16 Expires: December 20, 2012 June 20, 2012 18 PBB-EVPN 19 draft-ietf-l2vpn-pbb-evpn-03 21 Status of this Memo 23 This Internet-Draft is submitted to IETF in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF), its areas, and its working groups. Note that 28 other groups may also distribute working documents as 29 Internet-Drafts. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 The list of current Internet-Drafts can be accessed at 37 http://www.ietf.org/1id-abstracts.html 39 The list of Internet-Draft Shadow Directories can be accessed at 40 http://www.ietf.org/shadow.html 42 Copyright and License Notice 44 Copyright (c) 2012 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Abstract 59 This document discusses how Ethernet Provider Backbone Bridging 60 [802.1ah] can be combined with E-VPN in order to reduce the number of 61 BGP MAC advertisement routes by aggregating Customer/Client MAC (C- 62 MAC) addresses via Provider Backbone MAC address (B-MAC), provide 63 client MAC address mobility using C-MAC aggregation and B-MAC sub- 64 netting, confine the scope of C-MAC learning to only active flows, 65 offer per site policies and avoid C-MAC address flushing on topology 66 changes. The combined solution is referred to as PBB-EVPN. 68 Conventions 70 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 71 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 72 document are to be interpreted as described in RFC 2119. 74 Table of Contents 76 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 77 2. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 4 78 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 79 4. Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 4 80 4.1. MAC Advertisement Route Scalability . . . . . . . . . . . 5 81 4.2. C-MAC Mobility with MAC Summarization . . . . . . . . . . 5 82 4.3. C-MAC Address Learning and Confinement . . . . . . . . . . 5 83 4.4. Per Site Policy Support . . . . . . . . . . . . . . . . . 6 84 4.5. Avoiding C-MAC Address Flushing . . . . . . . . . . . . . 6 85 5. Solution Overview . . . . . . . . . . . . . . . . . . . . . . 6 86 6. BGP Encoding . . . . . . . . . . . . . . . . . . . . . . . . . 7 87 6.1. BGP MAC Advertisement Route . . . . . . . . . . . . . . . 7 88 6.2. Ethernet Auto-Discovery Route . . . . . . . . . . . . . . 7 89 6.3. Per VPN Route Targets . . . . . . . . . . . . . . . . . . 7 90 6.4. MAC Mobility Extended Community . . . . . . . . . . . . . 7 91 7. Operation . . . . . . . . . . . . . . . . . . . . . . . . . . 8 92 7.1. MAC Address Distribution over Core . . . . . . . . . . . . 8 93 7.2. Device Multi-homing . . . . . . . . . . . . . . . . . . . 8 94 7.2.1 Flow-based Load-balancing . . . . . . . . . . . . . . . 8 95 7.2.1.1 MES B-MAC Address Assignment . . . . . . . . . . . 8 96 7.2.1.2. Automating B-MAC Address Assignment . . . . . . . 10 97 7.2.1.3 Split Horizon and Designated Forwarder Election . . 11 98 7.2.2 I-SID Based Load-balancing . . . . . . . . . . . . . . . 11 99 7.2.2.1 MES B-MAC Address Assignment . . . . . . . . . . . . 11 100 7.2.2.2 Split Horizon and Designated Forwarder Election . . 12 101 7.3. Network Multi-homing . . . . . . . . . . . . . . . . . . . 12 102 7.4. Frame Forwarding . . . . . . . . . . . . . . . . . . . . . 12 103 7.4.1. Unicast . . . . . . . . . . . . . . . . . . . . . . . 12 104 7.4.2. Multicast/Broadcast . . . . . . . . . . . . . . . . . 13 105 8. Minimizing ARP Broadcast . . . . . . . . . . . . . . . . . . . 13 106 9. Seamless Interworking with IEEE 802.1aq/802.1Qbp . . . . . . . 13 107 9.1 B-MAC Address Assignment . . . . . . . . . . . . . . . . . . 14 108 9.2 IEEE 802.1aq / 802.1Qbp B-MAC Advertisement Route . . . . . 14 109 9.3 Operation: . . . . . . . . . . . . . . . . . . . . . . . . . 15 110 10. Solution Advantages . . . . . . . . . . . . . . . . . . . . . 15 111 10.1. MAC Advertisement Route Scalability . . . . . . . . . . . 15 112 10.2. C-MAC Mobility with MAC Sub-netting . . . . . . . . . . . 16 113 10.3. C-MAC Address Learning and Confinement . . . . . . . . . 16 114 10.4. Seamless Interworking with TRILL and 802.1aq Access 115 Networks . . . . . . . . . . . . . . . . . . . . . . . . 16 116 10.5. Per Site Policy Support . . . . . . . . . . . . . . . . . 17 117 10.6. Avoiding C-MAC Address Flushing . . . . . . . . . . . . . 17 118 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 18 119 12. Security Considerations . . . . . . . . . . . . . . . . . . . 18 120 13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 121 14. Intellectual Property Considerations . . . . . . . . . . . . 18 122 15. Normative References . . . . . . . . . . . . . . . . . . . . 18 123 16. Informative References . . . . . . . . . . . . . . . . . . . 18 124 17. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 18 126 1. Introduction 128 [E-VPN] introduces a solution for multipoint L2VPN services, with 129 advanced multi-homing capabilities, using BGP for distributing 130 customer/client MAC address reach-ability information over the core 131 MPLS/IP network. [802.1ah] defines an architecture for Ethernet 132 Provider Backbone Bridging (PBB), where MAC tunneling is employed to 133 improve service instance and MAC address scalability in Ethernet as 134 well as VPLS networks [PBB-VPLS]. 136 In this document, we discuss how PBB can be combined with E-VPN in 137 order to: reduce the number of BGP MAC advertisement routes by 138 aggregating Customer/Client MAC (C-MAC) addresses via Provider 139 Backbone MAC address (B-MAC), provide client MAC address mobility 140 using C-MAC aggregation and B-MAC sub-netting, confine the scope of 141 C-MAC learning to only active flows, offer per site policies and 142 avoid C-MAC address flushing on topology changes. The combined 143 solution is referred to as PBB-EVPN. 145 2. Contributors 147 In addition to the authors listed above, the following individuals 148 also contributed to this document. 150 Keyur Patel, Cisco 151 Sam Aldrin, Huawei 153 3. Terminology 155 BEB: Backbone Edge Bridge 156 B-MAC: Backbone MAC Address 157 CE: Customer Edge 158 C-MAC: Customer/Client MAC Address 159 DHD: Dual-homed Device 160 DHN: Dual-homed Network 161 LACP: Link Aggregation Control Protocol 162 LSM: Label Switched Multicast 163 MDT: Multicast Delivery Tree 164 MES: MPLS Edge Switch 165 MP2MP: Multipoint to Multipoint 166 P2MP: Point to Multipoint 167 P2P: Point to Point 168 PoA: Point of Attachment 169 PW: Pseudowire 170 E-VPN: Ethernet VPN 172 4. Requirements 173 The requirements for PBB-EVPN include all the requirements for E-VPN 174 that were described in [EVPN-REQ], in addition to the following: 176 4.1. MAC Advertisement Route Scalability 178 In typical operation, an [E-VPN] MES sends a BGP MAC Advertisement 179 Route per customer/client MAC (C-MAC) address. In certain 180 applications, this poses scalability challenges, as is the case in 181 virtualized data center environments where the number of virtual 182 machines (VMs), and hence the number of C-MAC addresses, can be in 183 the millions. In such scenarios, it is required to reduce the number 184 of BGP MAC Advertisement routes by relying on a 'MAC summarization' 185 scheme, as is provided by PBB. Note that the MAC summarization 186 capability already built into E-VPN is not sufficient in those 187 environments, as will be discussed next. 189 4.2. C-MAC Mobility with MAC Summarization 191 Certain applications, such as virtual machine mobility, require 192 support for fast C-MAC address mobility. For these applications, it 193 is not possible to use MAC address summarization in E-VPN, i.e. 194 advertise reach-ability to a MAC address prefix. Rather, the exact 195 virtual machine MAC address needs to be transmitted in BGP MAC 196 Advertisement route. Otherwise, traffic would be forwarded to the 197 wrong segment when a virtual machine moves from one Ethernet segment 198 to another. This hinders the scalability benefits of summarization. 200 It is required to support C-MAC address mobility, while retaining the 201 scalability benefits of MAC summarization. This can be achieved by 202 leveraging PBB technology, which defines a Backbone MAC (B-MAC) 203 address space that is independent of the C-MAC address space, and 204 aggregate C-MAC addresses via a B-MAC address and then apply 205 summarization to B-MAC addresses. 207 4.3. C-MAC Address Learning and Confinement 209 In E-VPN, all the MES nodes participating in the same E-VPN instance 210 are exposed to all the C-MAC addresses learnt by any one of these MES 211 nodes because a C-MAC learned by one of the MES nodes is advertise in 212 BGP to other MES nodes in that E-VPN instance. This is the case even 213 if some of the MES nodes for that E-VPN instance are not involved in 214 forwarding traffic to, or from, these C-MAC addresses. Even if an 215 implementation does not install hardware forwarding entries for C-MAC 216 addresses that are not part of active traffic flows on that MES, the 217 device memory is still consumed by keeping record of the C-MAC 218 addresses in the routing table (RIB). In network applications with 219 millions of C-MAC addresses, this introduces a non-trivial waste of 220 MES resources. As such, it is required to confine the scope of 221 visibility of C-MAC addresses only to those MES nodes that are 222 actively involved in forwarding traffic to, or from, these addresses. 224 4.4. Per Site Policy Support 226 In many applications, it is required to be able to enforce 227 connectivity policy rules at the granularity of a site (or segment). 228 This includes the ability to control which MES nodes in the network 229 can forward traffic to, or from, a given site. PBB-EVPN is capable of 230 providing this granularity of policy control. In the case where per 231 C-MAC address granularity is required, the EVI can always continue to 232 operate in E-VPN mode. 234 4.5. Avoiding C-MAC Address Flushing 236 It is required to avoid C-MAC address flushing upon link, port or 237 node failure for multi-homed devices and networks. This is in order 238 to speed up re-convergence upon failure. 240 5. Solution Overview 242 The solution involves incorporating IEEE 802.1ah Backbone Edge Bridge 243 (BEB) functionality on the E-VPN MES nodes similar to PBB-VPLS, where 244 BEB functionality is incorporated in the VPLS PE nodes. The MES 245 devices would then receive 802.1Q Ethernet frames from their 246 attachment circuits, encapsulate them in the PBB header and forward 247 the frames over the IP/MPLS core. On the egress E-VPN MES, the PBB 248 header is removed following the MPLS disposition, and the original 249 802.1Q Ethernet frame is delivered to the customer equipment. 250 BEB +--------------+ BEB 251 || | | || 252 \/ | | \/ 253 +----+ AC1 +----+ | | +----+ +----+ 254 | CE1|-----| | | | | |---| CE2| 255 +----+\ |MES1| | IP/MPLS | |MES3| +----+ 256 \ +----+ | Network | +----+ 257 \ | | 258 AC2\ +----+ | | 259 \| | | | 260 |MES2| | | 261 +----+ | | 262 /\ +--------------+ 263 || 264 BEB 265 <-802.1Q-> <------PBB over MPLS------> <-802.1Q-> 267 Figure 1: PBB-EVPN Network 268 The MES nodes perform the following functions:- Learn customer/client 269 MAC addresses (C-MACs) over the attachment circuits in the data- 270 plane, per normal bridge operation. 272 - Learn remote C-MAC to B-MAC bindings in the data-plane from traffic 273 ingress from the core per [802.1ah] bridging operation. 275 - Advertise local B-MAC address reach-ability information in BGP to 276 all other MES nodes in the same set of service instances. Note that 277 every MES has a set of local B-MAC addresses that uniquely identify 278 the device. More on the MES addressing in section 5. 280 - Build a forwarding table from remote BGP advertisements received 281 associating remote B-MAC addresses with remote MES IP addresses and 282 the associated MPLS label(s). 284 6. BGP Encoding 286 PBB-EVPN leverages the same BGP Routes and Attributes defined in [E- 287 VPN], adapted as follows: 289 6.1. BGP MAC Advertisement Route 291 The E-VPN MAC Advertisement Route is used to distribute B-MAC 292 addresses of the MES nodes instead of the C-MAC addresses of end- 293 stations/hosts. This is because the C-MAC addresses are learnt in the 294 data-plane for traffic arriving from the core. The MAC Advertisement 295 Route is encoded as follows: 297 - The MAC address field contains the B-MAC address. 298 - The Ethernet Tag field is set to 0. 300 The route is tagged with the RT corresponding to the EVI associated 301 with the B-MAC address. 303 All other fields are set as defined in [E-VPN]. 305 6.2. Ethernet Auto-Discovery Route 307 This route and all of its associated modes are not needed in PBB- 308 EVPN. 310 6.3. Per VPN Route Targets 312 PBB-EVPN uses the same set of route targets defined in [E-VPN]. The 313 future revision of this document will describe new RT types. 315 6.4. MAC Mobility Extended Community 316 This extended community is a new transitive extended community. It 317 may be advertised along with the MAC Advertisement route. When used 318 in PBB-EVPN, it indicates that the C-MAC forwarding tables for the I- 319 SIDs associated with the RT tagging the MAC Advertisement route must 320 be flushed. This extended community is encoded in 8-bytes as follows: 322 - Type (1 byte) = Pending IANA assignment. 323 - Sub-Type (1 byte) = Pending IANA assignment. 324 - Reserved (2 bytes) 325 - Counter (4 bytes) 327 Note that all other BGP messages and/or attributes are used as 328 defined in [E-VPN]. 330 7. Operation 332 This section discusses the operation of PBB-EVPN, specifically in 333 areas where it differs from [E-VPN]. 335 7.1. MAC Address Distribution over Core 337 In PBB-EVPN, host MAC addresses (i.e. C-MAC addresses) need not be 338 distributed in BGP. Rather, every MES independently learns the C-MAC 339 addresses in the data-plane via normal bridging operation. Every MES 340 has a set of one or more unicast B-MAC addresses associated with it, 341 and those are the addresses distributed over the core in MAC 342 Advertisement routes. 344 7.2. Device Multi-homing 346 7.2.1 Flow-based Load-balancing 348 This section describes the procedures for supporting device multi- 349 homing in an all-active redundancy model with flow-based load- 350 balancing. 352 7.2.1.1 MES B-MAC Address Assignment 354 In [802.1ah] every BEB is uniquely identified by one or more B-MAC 355 addresses. These addresses are usually locally administered by the 356 Service Provider. For PBB-EVPN, the choice of B-MAC address(es) for 357 the MES nodes must be examined carefully as it has implications on 358 the proper operation of multi-homing. In particular, for the scenario 359 where a CE is multi-homed to a number of MES nodes with all-active 360 redundancy and flow-based load-balancing, a given C-MAC address would 361 be reachable via multiple MES nodes concurrently. Given that any 362 given remote MES will bind the C-MAC address to a single B-MAC 363 address, then the various MES nodes connected to the same CE must 364 share the same B-MAC address. Otherwise, the MAC address table of the 365 remote MES nodes will keep oscillating between the B-MAC addresses of 366 the various MES devices. For example, consider the network of Figure 367 1, and assume that MES1 has B-MAC BM1 and MES2 has B-MAC BM2. Also, 368 assume that both links from CE1 to the MES nodes are part of an all- 369 active multi-chassis Ethernet link aggregation group. If BM1 is not 370 equal to BM2, the consequence is that the MAC address table on MES3 371 will keep oscillating such that the C-MAC address CM of CE1 would 372 flip-flop between BM1 or BM2, depending on the load-balancing 373 decision on CE1 for traffic destined to the core. 375 Considering that there could be multiple sites (e.g. CEs) that are 376 multi-homed to the same set of MES nodes, then it is required for all 377 the MES devices in a Redundancy Group to have a unique B-MAC address 378 per site. This way, it is possible to achieve fast convergence in the 379 case where a link or port failure impacts the attachment circuit 380 connecting a single site to a given MES. 382 +---------+ 383 +-------+ MES1 | IP/MPLS | 384 / | | 385 CE1 | Network | MESr 386 M1 \ | | 387 +-------+ MES2 | | 388 /-------+ | | 389 / | | 390 CE2 | | 391 M2 \ | | 392 \ | | 393 +------+ MES3 +---------+ 395 Figure 2: B-MAC Address Assignment 397 In the example network shown in Figure 2 above, two sites 398 corresponding to CE1 and CE2 are dual-homed to MES1/MES2 and 399 MES2/MES3, respectively. Assume that BM1 is the B-MAC used for the 400 site corresponding to CE1. Similarly, BM2 is the B-MAC used for the 401 site corresponding to CE2. On MES1, a single B-MAC address (BM1) is 402 required for the site corresponding to CE1. On MES2, two B-MAC 403 addresses (BM1 and BM2) are required, one per site. Whereas on MES3, 404 a single B-MAC address (BM2) is required for the site corresponding 405 to CE2. All three MES nodes would advertise their respective B-MAC 406 addresses in BGP using the MAC Advertisement routes defined in [E- 407 VPN]. The remote MES, MESr, would learn via BGP that BM1 is reachable 408 via MES1 and MES2, whereas BM2 is reachable via both MES2 and MES3. 409 Furthermore, MESr establishes via the normal bridge learning that C- 410 MAC M1 is reachable via BM1, and C-MAC M2 is reachable via BM2. As a 411 result, MESr can load-balance traffic destined to M1 between MES1 and 412 MES2, as well as traffic destined to M2 between both MES2 and MES3. 413 In the case of a failure that causes, for example, CE1 to be isolated 414 from MES1, the latter can withdraw the route it has advertised for 415 BM1. This way, MESr would update its path list for BM1, and will send 416 all traffic destined to M1 over to MES2 only. 418 For single-homed sites, it is possible to assign a unique B-MAC 419 address per site, or have all the single-homed sites connected to a 420 given MES share a single B-MAC address. The advantage of the first 421 model over the second model is the ability to avoid C-MAC destination 422 address lookup on the disposition PE (even though source C-MAC 423 learning is still required in the data-plane). Also, by assigning the 424 B-MAC addresses from a contiguous range, it is possible to advertise 425 a single B-MAC subnet for all single-homed sites, thereby rendering 426 the number of MAC advertisement routes required at par with the 427 second model. 429 In summary, every MES may use a unicast B-MAC address shared by all 430 single-homed CEs or a unicast B-MAC address per single-homed CE and, 431 in addition, a unicast B-MAC address per dual-homed CE. In the latter 432 case, the B-MAC address MUST be the same for all MES nodes in a 433 Redundancy Group connected to the same CE. 435 7.2.1.2. Automating B-MAC Address Assignment 437 The MES B-MAC address used for single-homed sites can be 438 automatically derived from the hardware (using for e.g. the 439 backplane's address). However, the B-MAC address used for multi-homed 440 sites must be coordinated among the RG members. To automate the 441 assignment of this latter address, the MES can derive this B-MAC 442 address from the MAC Address portion of the CE's LACP System 443 Identifier by flipping the 'Locally Administered' bit of the CE's 444 address. This guarantees the uniqueness of the B-MAC address within 445 the network, and ensures that all MES nodes connected to the same 446 multi-homed CE use the same value for the B-MAC address. 448 Note that with this automatic provisioning of the B-MAC address 449 associated with multi-homed CEs, it is not possible to support the 450 uncommon scenario where a CE has multiple bundles towards the MES 451 nodes, and the service involves hair-pinning traffic from one bundle 452 to another. This is because the split-horizon filtering relies on B- 453 MAC addresses rather than Site-ID Labels (as will be described in the 454 next section). The operator must explicitly configure the B-MAC 455 address for this fairly uncommon service scenario. 457 Whenever a B-MAC address is provisioned on the MES, either manually 458 or automatically (as an outcome of CE auto-discovery), the MES MUST 459 transmit an MAC Advertisement Route for the B-MAC address with a 460 downstream assigned MPLS label that uniquely identifies that address 461 on the advertising MES. The route is tagged with the RTs of the 462 associated EVIs as described above. 464 7.2.1.3 Split Horizon and Designated Forwarder Election 466 [E-VPN] relies on access split horizon, where the Ethernet Segment 467 Label is used for egress filtering on the attachment circuit in order 468 to prevent forwarding loops. In PBB-EVPN, the B-MAC source address 469 can be used for the same purpose, as it uniquely identifies the 470 originating site of a given frame. As such, Segment Labels are not 471 used in PBB-EVPN, and the egress split-horizon filtering is done 472 based on the B-MAC source address. It is worth noting here that 473 [802.1ah] defines this B-MAC address based filtering function as part 474 of the I-Component options, hence no new functions are required to 475 support split-horizon beyond what is already defined in [802.1ah]. 476 Given that the Segment label is not used in PBB-EVPN, the MES sets 477 the Label field in the Ethernet Segment Route to 0. 479 The Designated Forwarder election procedures are defined in [I-D- 480 Segment-Route]. 482 7.2.2 I-SID Based Load-balancing 484 This section describes the procedures for supporting device multi- 485 homing in an all-active redundancy model with per-ISID load- 486 balancing. 488 7.2.2.1 MES B-MAC Address Assignment 490 In the case where per-ISID load-balancing is desired among the MES 491 nodes in a given redundancy group, multiple unicast B-MAC addresses 492 are allocated per multi-homed Ethernet Segment: Each MES connected to 493 the multi-homed segment is assigned a unique B-MAC. Every MES then 494 advertises its B-MAC address using the BGP MAC advertisement route. 496 A remote MES initially floods traffic to a destination C-MAC address, 497 located in a given multi-homed Ethernet Segment, to all the MES nodes 498 connected to that segment. Then, when reply traffic arrives at the 499 remote MES, it learns (in the data-path) the B-MAC address and 500 associated next-hop MES to use for said C-MAC address. When a MES 501 connected to a multi-homed Ethernet Segment loses connectivity to the 502 segment, due to link or port failure, it withdraws the B-MAC route 503 previously advertised for that segment. This causes the remote MES 504 nodes to flush all C-MAC addresses associated with the B-MAC in 505 question. This is done across all I-SIDs that are mapped to the EVI 506 of the withdrawn MAC route. 508 7.2.2.2 Split Horizon and Designated Forwarder Election The procedures 509 are similar to the flow-based load-balancing case, with the only 510 difference being that the DF filtering must be applied to unicast as 511 well as multicast traffic, and in both core-to-segment as well as 512 segment-to-core directions. 514 7.3. Network Multi-homing 516 When an Ethernet network is multi-homed to a set of MES nodes running 517 PBB-EVPN, an all-active redundancy model can be supported with per 518 service instance (i.e. I-SID) load-balancing. In this model, DF 519 election is performed to ensure that a single MES node in the 520 redundancy group is responsible for forwarding traffic associated 521 with a given I-SID. This guarantees that no forwarding loops are 522 created. Filtering based on DF state applies to both unicast and 523 multicast traffic, and in both access-to-core as well as core-to- 524 access directions (unlike the multi-homed device scenario where DF 525 filtering is limited to multi-destination frames in the core-to- 526 access direction). Similar to the multi-homed device scenario, with 527 I-SID based load-balancing, a unique B-MAC address is assigned to 528 each of the MES nodes connected to the multi-homed network (Segment). 530 7.4. Frame Forwarding 532 The frame forwarding functions are divided in between the Bridge 533 Module, which hosts the [802.1ah] Backbone Edge Bridge (BEB) 534 functionality, and the MPLS Forwarder which handles the MPLS 535 imposition/disposition. The details of frame forwarding for unicast 536 and multi-destination frames are discussed next. 538 7.4.1. Unicast 540 Known unicast traffic received from the AC will be PBB-encapsulated 541 by the MES using the B-MAC source address corresponding to the 542 originating site. The unicast B-MAC destination address is determined 543 based on a lookup of the C-MAC destination address (the binding of 544 the two is done via transparent learning of reverse traffic). The 545 resulting frame is then encapsulated with an LSP tunnel label and the 546 MPLS label which uniquely identifies the B-MAC destination address on 547 the egress MES. If per flow load-balancing over ECMPs in the MPLS 548 core is required, then a flow label is added as the end of stack 549 label. 551 For unknown unicast traffic, the MES forwards these frames over MPLS 552 core. When these frames are to be forwarded, then the same set of 553 options used for forwarding multicast/broadcast frames (as described 554 in next section) are used. 556 7.4.2. Multicast/Broadcast 558 Multi-destination frames received from the AC will be PBB- 559 encapsulated by the MES using the B-MAC source address corresponding 560 to the originating site. The multicast B-MAC destination address is 561 selected based on the value of the I-SID as defined in [802.1ah]. The 562 resulting frame is then forwarded over the MPLS core using one out of 563 the following two options: 565 Option 1: the MPLS Forwarder can perform ingress replication over a 566 set of MP2P tunnel LSPs. The frame is encapsulated with a tunnel LSP 567 label and the E-VPN ingress replication label advertised in the 568 Inclusive Multicast Route. 570 Option 2: the MPLS Forwarder can use P2MP tunnel LSP per the 571 procedures defined in [E-VPN]. This includes either the use of 572 Inclusive or Aggregate Inclusive trees. 574 Note that the same procedures for advertising and handling the 575 Inclusive Multicast Route defined in [E-VPN] apply here. 577 8. Minimizing ARP Broadcast 579 The MES nodes implement an ARP-proxy function in order to minimize 580 the volume of ARP traffic that is broadcasted over the MPLS network. 581 This is achieved by having each MES node snoop on ARP request and 582 response messages received over the access interfaces or the MPLS 583 core. The MES builds a cache of IP / MAC address bindings from these 584 snooped messages. The MES then uses this cache to respond to ARP 585 requests ingress on access ports and targeting hosts that are in 586 remote sites. If the MES finds a match for the IP address in its ARP 587 cache, it responds back to the requesting host and drops the request. 588 Otherwise, if it does not find a match, then the request is flooded 589 over the MPLS network using either ingress replication or LSM. 591 9. Seamless Interworking with IEEE 802.1aq/802.1Qbp 592 +--------------+ 593 | | 594 +---------+ | MPLS | +---------+ 595 +----+ | | +----+ +----+ | | +----+ 596 |SW1 |--| | |MES1| |MES2| | |--| SW3| 597 +----+ | 802.1aq |---| | | |--| 802.1aq | +----+ 598 +----+ | .1Qbp | +----+ +----+ | .1Qbp | +----+ 599 |SW2 |--| | | Backbone | | |--| SW4| 600 +----+ +---------+ +--------------+ +---------+ +----+ 602 |<------ IS-IS -------->|<-----BGP----->|<------ IS-IS ------>| CP 604 |<------------------------- PBB -------------------------->| DP 605 |<----MPLS----->| 607 Legend: CP = Control Plane View 608 DP = Data Plane View 610 Figure 7: Interconnecting 802.1aq/802.1Qbp Networks with PBB-EVPN 612 9.1 B-MAC Address Assignment 614 For the same reasons cited in the TRILL section, the B-MAC addresses 615 need to be globally unique across all the IEEE 802.1aq / 802.1Qbp 616 networks. The same hierarchical address assignment scheme depicted 617 above is proposed for B-MAC addresses as well. 619 9.2 IEEE 802.1aq / 802.1Qbp B-MAC Advertisement Route 621 B-MAC addresses associated with 802.1aq / 802.1Qbp switches are 622 advertised using the BGP MAC Advertisement route already defined in 623 [E-VPN]. 625 The encapsulation for the transport of PBB frames over MPLS is 626 similar to that of classical Ethernet, albeit with the additional PBB 627 header, as shown in the figure below: 629 +------------------+ 630 | IP/MPLS Header | 631 +------------------+ 632 | PBB Header | 633 +------------------+ 634 | Ethernet Header | 635 +------------------+ 636 | Ethernet Payload | 637 +------------------+ 638 | Ethernet FCS | 639 +------------------+ 641 Figure 8: PBB over MPLS Encapsulation 643 9.3 Operation: 645 When a MES receives a PBB-encapsulated Ethernet frame from the access 646 side, it performs a lookup on the B-MAC destination address to 647 identify the next hop. If the lookup yields that the next hop is a 648 remote MES, the local MES would then encapsulate the PBB frame in 649 MPLS. The label stack comprises of the VPN label (advertised by the 650 remote PE), followed by an LSP/IGP label. From that point onwards, 651 regular MPLS forwarding is applied. 653 On the disposition MES, assuming penultimate-hop-popping is employed, 654 the MES receives the MPLS-encapsulated PBB frame with a single label: 655 the VPN label. The value of the label indicates to the disposition 656 MES that this is a PBB frame, so the label is popped, the TTL field 657 (in the 802.1Qbp F-Tag) is reinitialized and normal PBB processing is 658 employed from this point onwards. 660 10. Solution Advantages 662 In this section, we discuss the advantages of the PBB-EVPN solution 663 in the context of the requirements set forth in section 3 above. 665 10.1. MAC Advertisement Route Scalability 667 In PBB-EVPN the number of MAC Advertisement Routes is a function of 668 the number of segments (sites), rather than the number of 669 hosts/servers. This is because the B-MAC addresses of the MESes, 670 rather than C-MAC addresses (of hosts/servers) are being advertised 671 in BGP. And, as discussed above, there's a one-to-one mapping between 672 multi-homed segments and B-MAC addresses, whereas there's a one-to- 673 one or many-to-one mapping between single-homed segments and B-MAC 674 addresses for a given MES. As a result, the volume of MAC 675 Advertisement Routes in PBB-EVPN is multiple orders of magnitude less 676 than E-VPN. 678 10.2. C-MAC Mobility with MAC Sub-netting 680 In PBB-EVPN, if a MES allocates its B-MAC addresses from a contiguous 681 range, then it can advertise a MAC prefix rather than individual 48- 682 bit addresses. It should be noted that B-MAC addresses can easily be 683 assigned from a contiguous range because MES nodes are within the 684 provider administrative domain; however, CE devices and hosts are 685 typically not within the provider administrative domain. The 686 advantage of such MAC address sub-netting can be maintained even as 687 C-MAC addresses move from one Ethernet segment to another. This is 688 because the C-MAC address to B-MAC address association is learnt in 689 the data-plane and C-MAC addresses are not advertised in BGP. To 690 illustrate how this compares to E-VPN, consider the following 691 example: 693 If a MES running E-VPN advertises reachability for a MAC subnet that 694 spans N addresses via a particular segment, and then 50% of the MAC 695 addresses in that subnet move to other segments (e.g. due to virtual 696 machine mobility), then in the worst case, N/2 additional MAC 697 Advertisement routes need to be sent for the MAC addresses that have 698 moved. This defeats the purpose of the sub-netting. With PBB-EVPN, on 699 the other hand, the sub-netting applies to the B-MAC addresses which 700 are statically associated with MES nodes and are not subject to 701 mobility. As C-MAC addresses move from one segment to another, the 702 binding of C-MAC to B-MAC addresses is updated via data-plane 703 learning. 705 10.3. C-MAC Address Learning and Confinement 707 In PBB-EVPN, C-MAC address reachability information is built via 708 data-plane learning. As such, MES nodes not participating in active 709 conversations involving a particular C-MAC address will purge that 710 address from their forwarding tables. Furthermore, since C-MAC 711 addresses are not distributed in BGP, MES nodes will not maintain any 712 record of them in control-plane routing table. 714 10.4. Seamless Interworking with TRILL and 802.1aq Access Networks 716 Consider the scenario where two access networks, one running MPLS and 717 the other running 802.1aq, are interconnected via an MPLS backbone 718 network. The figure below shows such an example network. 720 +--------------+ 721 | | 722 +---------+ | MPLS | +---------+ 723 +----+ | | +----+ +----+ | | +----+ 724 | CE |--| | |MES1| |MES2| | |--| CE | 725 +----+ | 802.1aq |---| | | |--| MPLS | +----+ 726 +----+ | | +----+ +----+ | | +----+ 727 | CE |--| | | Backbone | | |--| CE | 728 +----+ +---------+ +--------------+ +---------+ +----+ 730 Figure 9: Interoperability with 802.1aq 732 If the MPLS backbone network employs E-VPN, then the 802.1aq data- 733 plane encapsulation must be terminated on MES1 or the edge device 734 connecting to MES1. Either way, all the MES nodes that are part of 735 the associated service instances will be exposed to all the C-MAC 736 addresses of all hosts/servers connected to the access networks. 737 However, if the MPLS backbone network employs PBB-EVPN, then the 738 802.1aq encapsulation can be extended over the MPLS backbone, thereby 739 maintaining C-MAC address transparency on MES1. If PBB-EVPN is also 740 extended over the MPLS access network on the right, then C-MAC 741 addresses would be transparent to MES2 as well. 743 Interoperability with TRILL access network will be described in 744 future revision of this draft. 746 10.5. Per Site Policy Support 748 In PBB-EVPN, a unique B-MAC address can be associated with every site 749 (single-homed or multi-homed). Given that the B-MAC addresses are 750 sent in BGP MAC Advertisement routes, it is possible to define per 751 site (i.e. B-MAC) forwarding policies including policies for E-TREE 752 service. 754 10.6. Avoiding C-MAC Address Flushing 756 With PBB-EVPN, it is possible to avoid C-MAC address flushing upon 757 topology change affecting a multi-homed device. To illustrate this, 758 consider the example network of Figure 1. Both MES1 and MES2 759 advertize the same B-MAC address (BM1) to MES3. MES3 then learns the 760 C-MAC addresses of the servers/hosts behind CE1 via data-plane 761 learning. If AC1 fails, then MES3 does not need to flush any of the 762 C-MAC addresses learnt and associated with BM1. This is because MES1 763 will withdraw the MAC Advertisement routes associated with BM1, 764 thereby leading MES3 to have a single adjacency (to MES2) for this B- 765 MAC address. Therefore, the topology change is communicated to MES3 766 and no C-MAC address flushing is required. 768 11. Acknowledgements 770 TBD. 772 12. Security Considerations 774 There are no additional security aspects beyond those of VPLS/H-VPLS 775 that need to be discussed here. 777 13. IANA Considerations 779 This document requires IANA to assign a new SAFI value for L2VPN_MAC 780 SAFI. 782 14. Intellectual Property Considerations 784 This document is being submitted for use in IETF standards 785 discussions. 787 15. Normative References 789 [802.1ah] "Virtual Bridged Local Area Networks Amendment 7: Provider 790 Backbone Bridges", IEEE Std. 802.1ah-2008, August 2008. 792 16. Informative References 794 [PBB-VPLS] Sajassi et al., "VPLS Interoperability with Provider 795 Backbone Bridges", draft-ietf-l2vpn-vpls-pbb-interop- 796 02.txt, work in progress, July, 2011. 798 [EVPN-REQ] Sajassi et al., "Requirements for Ethernet VPN (E-VPN)", 799 draft-sajassi-raggarwa-l2vpn-evpn-req-01.txt, work in 800 progress, July, 2011. 802 [E-VPN] Aggarwal et al., "BGP MPLS Based Ethernet VPN", draft-ietf- 803 l2vpn-evpn-00.txt, work in progress, February, 2012. 805 17. Authors' Addresses 807 Ali Sajassi 808 Cisco 809 170 West Tasman Drive 810 San Jose, CA 95134, US 811 Email: sajassi@cisco.com 812 Samer Salam 813 Cisco 814 595 Burrard Street, Suite 2123 815 Vancouver, BC V7X 1J1, Canada 816 Email: ssalam@cisco.com 818 Sami Boutros 819 Cisco 820 170 West Tasman Drive 821 San Jose, CA 95134, US 822 Email: sboutros@cisco.com 824 Nabil Bitar 825 Verizon Communications 826 Email : nabil.n.bitar@verizon.com 828 Aldrin Isaac 829 Bloomberg 830 Email: aisaac71@bloomberg.net 832 Florin Balus 833 Alcatel-Lucent 834 701 E. Middlefield Road 835 Mountain View, CA, USA 94043 836 Email: florin.balus@alcatel-lucent.com 838 Wim Henderickx 839 Alcatel-Lucent 840 Email: wim.henderickx@alcatel-lucent.be 842 Clarence Filsfils 843 Cisco 844 Email: cfilsfil@cisco.com 846 Dennis Cai 847 Cisco 848 Email: dcai@cisco.com 850 Lizhong Jin 851 ZTE Corporation 852 889, Bibo Road 853 Shanghai, 201203, China 854 Email: lizhong.jin@zte.com.cn