idnits 2.17.00 (12 Aug 2021) /tmp/idnits22628/draft-rosen-bess-secure-l3vpn-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 11, 2018) is 1440 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force E. Rosen, Ed. 3 Internet-Draft R. Bonica 4 Intended status: Informational Juniper Networks, Inc. 5 Expires: December 13, 2018 June 11, 2018 7 Augmenting RFC 4364 Technology to 8 Provide Secure Layer L3VPNs over Public Infrastructure 9 draft-rosen-bess-secure-l3vpn-00 11 Abstract 13 The Layer 3 Virtual Private Network (VPN) technology described in RFC 14 4364 is focused on the scenario in which a network Service Provider 15 (SP) maintains a secure backbone network and offers VPN service over 16 that network to its customers. Customers access the SP's network by 17 attaching "Customer Edge" (CE) routers to "Provider Edge" (PE) 18 routers, and exchanging cleartext IP packets. PE routers generally 19 serve multiple customers, and prevent unauthorized communication 20 among customers. Customer data sent across the backbone (from one PE 21 to another) is encapsulated in MPLS, using an MPLS label to associate 22 a given packet with a given customer. The labeled packets are then 23 sent across the backbone network in the clear, using MPLS transport. 24 However, many customers want a VPN service that is secure enough to 25 run over the public Internet, and which does not require them to send 26 cleartext IP packets to a service provider. Often they want to 27 connect directly to edge nodes of the public Internet, which does not 28 provide MPLS support. Each customer may itself have multiple tenants 29 who are not allowed to intercommunicate with each other freely. In 30 this case, the customer many need to provide a VPN service for the 31 tenants. This document describes a way in which this can be achieved 32 using the technology of RFC 4364. The functionality assigned therein 33 to a PE router can be placed instead in Customer Premises Equipment. 34 This functionality can be augmented by transmitting MPLS packets 35 through IPsec Security Associations. The BGP control plane sessions 36 can also be protected by IPsec. This allows a customer to use RFC 37 4364 technology to provide VPN service to its internal departments, 38 while sending only IPsec-protected packets to the Internet or other 39 backbone network, and eliminating the need for MPLS transport in the 40 backbone. 42 Status of This Memo 44 This Internet-Draft is submitted in full conformance with the 45 provisions of BCP 78 and BCP 79. 47 Internet-Drafts are working documents of the Internet Engineering 48 Task Force (IETF). Note that other groups may also distribute 49 working documents as Internet-Drafts. The list of current Internet- 50 Drafts is at https://datatracker.ietf.org/drafts/current/. 52 Internet-Drafts are draft documents valid for a maximum of six months 53 and may be updated, replaced, or obsoleted by other documents at any 54 time. It is inappropriate to use Internet-Drafts as reference 55 material or to cite them other than as "work in progress." 57 This Internet-Draft will expire on December 13, 2018. 59 Copyright Notice 61 Copyright (c) 2018 IETF Trust and the persons identified as the 62 document authors. All rights reserved. 64 This document is subject to BCP 78 and the IETF Trust's Legal 65 Provisions Relating to IETF Documents 66 (https://trustee.ietf.org/license-info) in effect on the date of 67 publication of this document. Please review these documents 68 carefully, as they describe your rights and restrictions with respect 69 to this document. Code Components extracted from this document must 70 include Simplified BSD License text as described in Section 4.e of 71 the Trust Legal Provisions and are provided without warranty as 72 described in the Simplified BSD License. 74 Table of Contents 76 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 77 1.1. Review of L3VPN Concepts and Terminology . . . . . . . . 3 78 1.2. Secured L3VPN . . . . . . . . . . . . . . . . . . . . . . 4 79 1.3. Terminology . . . . . . . . . . . . . . . . . . . . . . . 5 80 2. Model of Operation . . . . . . . . . . . . . . . . . . . . . 6 81 3. How the C-PEs Advertise Red Routes . . . . . . . . . . . . . 9 82 3.1. Red and Black C-PE Loopback Addresses . . . . . . . . . . 9 83 3.2. Setting Up Red BGP Sessions Between C-PEs and RRs . . . . 10 84 3.3. Routes Transmited by the C-PE on Red BGP Sessions . . . . 12 85 3.4. Propagating Red Routes . . . . . . . . . . . . . . . . . 13 86 4. VPN-IP Routes and Recursive Route Resolution . . . . . . . . 13 87 5. MPLS-in-IPsec . . . . . . . . . . . . . . . . . . . . . . . . 15 88 6. Security Handle . . . . . . . . . . . . . . . . . . . . . . . 16 89 7. Data Plane Security Procedures . . . . . . . . . . . . . . . 16 90 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 17 91 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 92 10. Implementation Challenges . . . . . . . . . . . . . . . . . . 17 93 11. Security Considerations . . . . . . . . . . . . . . . . . . . 17 94 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 17 95 12.1. Normative References . . . . . . . . . . . . . . . . . . 17 96 12.2. Informational References . . . . . . . . . . . . . . . . 19 98 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 19 100 1. Introduction 102 1.1. Review of L3VPN Concepts and Terminology 104 In conventional Virtual Private Networks (L3VPNs) based on the 105 technology of [RFC4364], a Service Provider (SP) maintains a secure 106 private network (known as the "SP backbone"). An SP maintains a 107 number of "Provider Edge" (PE) routers to which customers may attach. 108 A customer router that attaches to a PE router is known as a 109 "Customer Edge" (CE) router. 111 Multiple customers may connect to a single PE router. Within a given 112 PE, each customer is associated with a routing context of its own 113 (known as a Virtual Routing and Forwarding table, or VRF). A 114 particular customer attaches to the PE via a set of one or more 115 interfaces or Virtual LANs (VLANs) that are not shared with other 116 customers. (In the subsequent text, the term "interface" will 117 include VLANs and other "virtual" interfaces.) Each such interface 118 is associated with a particular customer's VRF; thus such interfaces 119 are known as "VRF interfaces". These are the PE's customer-facing 120 interfaces. The VRF interfaces carry IP datagrams, either IPv4 or 121 IPv6 or both. 123 A given customer's VRF is automatically populated with, and only 124 with, routes that lead out the local VRF interfaces, and routes that 125 lead to remote VRF interfaces of the same customer. Routes leading 126 outside a customer's VPN are excluded from that customer's VRF unless 127 explicitly allowed by policy. Thus two customers can attach to the 128 same PE even if they are not allow to communicate with each other 129 through that PE. 131 The PE at which a customer data packet enters the SP backbone network 132 is known as the packet's "ingress PE". The PE at which a customer 133 data packet leaves the SP backbone is known as the packet's "egress 134 PE". Generally, the ingress PE pushes two MPLS labels onto each data 135 packet. The top label (sometimes known as the "transport label") 136 directs the packet to its egress PE. The second label (sometimes 137 known as the "VPN label") is used at the egress PE to associate a 138 given customer's packets with that customer's VRF at the egress PE. 140 These labeled packets travel across the SP backbone "in the clear" 141 (i.e., with no cryptographic protection to provide privacy, 142 authentication, or integrity), as the SP backbone is presumed to be 143 adequately secure. 145 The control plane protocol for this type of VPN is BGP. A given 146 customer's routes are distributed among the PEs to which that 147 customer attaches by means of a BGP address family known as "VPN-IP" 148 (either VPN-IPv4 or VPN-IPv6). Distribution of these routes is 149 controlled in such a way as to ensure that a given customer's routes, 150 exported from one of that customer's VRFs, are imported only by other 151 VRFs associated with the same customer. 153 1.2. Secured L3VPN 155 For security reasons, the L3VPN technology summarized in Section 1.1 156 is not generally used in the following scenarios: 158 o Some or all of the customer sites need to be reached over the 159 public Internet, rather than over a secure SP backbone network. 161 o The customer does not want to expose any of his data in cleartext 162 to the SP, even if the SP backbone network is secure. 164 o The customer does not want to expose any of his routing control 165 information in cleartext to the SP, and/or wishes to hide his 166 internal IP addressing structure from the SP. 168 In such situations, the customer needs to use cryptographic methods 169 in order to ensure privacy, integrity, and authentication for the IP 170 datagrams he sends over the backbone network; the cryptography must 171 be applied before the datagrams are sent to the SP backbone network 172 or Internet. (It is presumed of course that the customer's own sites 173 and systems have been secured to his satisfaction; how that is 174 achieved is outside the scope of this document.) 176 In these use cases, the customer may still want the benefits of the 177 L3VPN service, e.g.: 179 o The customer may itself be providing a VPN service to multiple 180 "tenants". E.g., 182 * The customer may be an enterprise or governmental agency that 183 consists of multiple internal departments or organizations that 184 are not allowed to communicate freely with each other, and that 185 may even have independent IP address spaces. We will use the 186 term "tenant" to refer to such a department or organization. 188 * The customer may be a Data Center operator that is providing a 189 virtual network to each of multiple Data Center tenants. 191 o In L3VPN, a CE router at one customer site does not have to be 192 provisioned with the addresses of CE routers at other sites. 194 Rather, these are auto-discovered via BGP. This sort of auto- 195 discovery is just as valuable when the customer needs more 196 security than is provided by conventional L3VPN. Auto-discovery 197 also allows some or all of the CE routers to be mobile, changing 198 their IP addresses from time to time. 200 It is possible to adapt the L3VPN technology to handle use cases 201 where cryptographic methods must be applied before a packet is sent 202 to an SP or to a backbone network . This document describes a way in 203 which this may be done. We will refer to this adaptation as a 204 "Secured L3VPN". Section 2 outlines the way this adaptation works. 205 Subsequent sections of this document specify the necessary procedures 206 in more detail. 208 Secured L3VPN makes use of IPsec technology. This document does not 209 discuss the details of IPsec. A roadmap through the set of RFCs 210 describing IPsec can be found in [RFC6071]. Of particular importance 211 are [RFC4303] (IPsec Encapsulating Security Payload), [RFC7296] 212 (Internet Key Exchange Protocol version 2), and [RFC8221] 213 (Cryptographic Algorithm Implementation Requirements and Usage 214 Guidance). 216 1.3. Terminology 218 In this document we shall use the following terminology: 220 SP 222 A network service provider (possibly an Internet service 223 provider). 225 customer 227 An organization or other entity that obtains network service 228 (private network service or Internet service) from an SP. 230 tenant 232 An organization or other entity that obtains VPN service from the 233 customer. For example, if the customer is a governmental agency, 234 its tenants might be the various departments of the agency. If 235 the customer is an enterprise, its tenants might be the various 236 organizations within the enterprise. If the customer is a Data 237 Center provider, its tenants might be organizations to which it 238 sells Data Center services. 240 C-PE router: 242 A router that performs the functions of an L3VPN PE router 243 ([RFC4364]), but that is operated and managed not by a network 244 service provider, but rather by a customer of the network service 245 provider. The customer may use the C-PE to provide a Secured 246 L3VPN service to one or more of its tenants. 248 Red interface: 250 A tenant-facing interface of a C-PE device, where the tenant in 251 question is receiving Secured L3VPN service. 253 Black interface: 255 A C-PE interface that is not a red interface. This may be a 256 network-facing interface, or a tenant-facing interface to a tenant 257 that is not receiving any L3VPN service. 259 Red BGP session: 261 A BGP session, protected by IPsec, between two C-PEs, or between a 262 C-PE and a BGP Route Reflector. 264 Black BGP session: 266 A BGP session other than a red BGP session. 268 Local red route: 270 A route whose next hop interface is a local red interface. 272 Remote red route: 274 A route received, as a VPN-IPv4 or VPN-IPv6 route, over a red BGP 275 session. 277 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 278 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 279 "OPTIONAL" in this document are to be interpreted as described in BCP 280 14 [RFC2119] [RFC8174] when, and only when, they appear in all 281 capitals, as shown here. 283 2. Model of Operation 285 In a Secured L3VPN, the functions conventionally performed by an 286 L3VPN PE router (as detailed in [RFC4364]) are instead performed by a 287 router that is operated and managed by the customer, rather than by 288 the SP. Since such a router is part of the customer's network, but 289 has the functionality of an L3VPN PE router, we will refer to it as a 290 "C-PE router". The customer is responsible for ensuring that the 291 C-PE itself is properly secured. The C-PE provides L3VPN 292 functionality to the customer's tenants. 294 Each interface of a C-PE is either a "red interface" or a "black 295 interface": 297 o A red interface is a tenant-facing interface that attaches to a 298 tenant who is receiving Secured L3VPN service from the customer. 300 o A black interface is any interface that is not a red interface. 301 Black interfaces may be network-facing interfaces (attached to an 302 SP backbone), or may be tenant-facing interfaces attached to 303 tenants that are not receiving any L3VPN from the customer. We 304 assume in this document that a C-PE that provides the Secured 305 L3VPN service to one or more tenants does not provide a 306 conventional (unsecured) L3VPN service to any of the tenants. 308 A C-PE has one or more VRFs, one per tenant. Each VRF is associated 309 with a distinct set of red interfaces, the ones that lead to the 310 network(s), VLAN(s), or virtual network(s) that is (are) specific to 311 the given tenant. Standard L3VPN techniques then prevent 312 communication among the different tenants unless explicitly allowed 313 by policy. In simpler scenarios, the customer may have sites with 314 only a single tenant. The C-PEs at those sites require only a single 315 VRF, and all the red interfaces will be associated with that VRF. 317 The black interfaces of a C-PE can attach to an access router of the 318 public internet, or to a conventional L3VPN PE router belonging to an 319 SP, or to any other router that provides IP connectivity among the 320 customer's C-PE routers. (If a C-PE attaches to a conventional L3VPN 321 PE router, then the C-PE appears to the conventional PE to be a CE 322 router.) 324 As in any L3VPN, the VRFs are populated with a combination of local 325 routes and remote routes: 327 o The local routes in a given VRF are those routes whose "next hop 328 interface" is a local red interface associated with that VRF. 330 o The remote routes in a given VRF are those routes learned via BGP 331 from the customer's other C-PEs. These routes may be learned 332 directly via BGP sessions to those other C-PEs, or indirectly via 333 one or more BGP Route Reflectors (RRs). 335 In this document, we will use the term "red routes" to refer to 336 routes within a VRF. These are distinguished from the "black routes" 337 that exist in a C-PE's global routing table. 339 A conventional PE router sends and receives MPLS packets over its 340 network facing interfaces. A C-PE, on the other hand, sends "MPLS- 341 in-IPsec" packets (see [RFC4023] and Section 5 of this document) over 342 its black interfaces. Since an MPLS-in-IPsec packet is an IP 343 datagram, there is no need for the backbone network to support MPLS 344 transport. IPsec is used to provide privacy, integrity and 345 authentication for the packets sent by the C-PE to the backbone 346 network. 348 In a Secured L3VPN, protection of the control plane is just as 349 important as is protection of the data plane. It is therefore 350 necessary to ensure that the BGP messages used to disseminate the red 351 routes also have privacy, integrity, and authentication. In order to 352 ensure this, the BGP sessions used to disseminate information about 353 red routes will be protected by IPsec. We will refer to such BGP 354 sessions as "red BGP sessions". It is recommended to use IPsec 355 Transport mode to protect these BGP sessions. This means that a C-PE 356 MUST NOT send or receive VPN-IP routes over any BGP session that is 357 not protected by IPsec. (A VPN-IP route is a route whose BGP Address 358 Family (AFI) is 1 (IPv4) or 2 (IPv6) and whose Subsequent Address 359 Family (SAFI) is 128, 129, or 5.) 361 Note that if RRs are used, the RRs must be as secure as the C-PEs. 362 This likely means that they are managed by the customer and located 363 at sites regarded by the customer as adequately secure. 365 Thus in a Secured L3VPN, red routes are propagated only among trusted 366 systems, and always in red BGP sessions. The propagation of red 367 routes on red BGP sessions is controlled by attaching Route Targets 368 to those routes, as with any [RFC4364]-based technology. 370 As in any L3VPN, BGP uses the VPN-IPv4 and/or VPN-IPv6 address 371 families when disseminating information about VPN routes from one VRF 372 to another. Each such route carries an MPLS label that is to be 373 pushed on the label stack of any tenant packet for which the address 374 prefix in the route's NLRI is the best match to the packet's IP 375 destination address. 377 In Secured L3VPNs, these routes MUST also carry a Tunnel 378 Encapsulation attribute ([tunnel_encaps]) specifying an "MPLS-in- 379 IPsec" tunnel type (see Section 5, as well as Sections 3 and 8.1 of 380 [RFC4023]). This indicates that before a tenant's MPLS packet is 381 sent to the backbone network, it must be encapsulated in IP and then 382 sent on an IPsec Transport Mode Security Association (SA). 384 A C-PE may have unprotected (black) BGP sessions, e.g., to gather 385 public Internet routes. However, the black BGP sessions MUST NOT be 386 enabled for the VPN-IP AFI/SAFIs. This prevents any routes learned 387 over the black BGP sessions from being imported into the VRFs. 389 As we shall see in Section 3.3, there are also some IP routes (as 390 distinguished from VPN-IP routes) that MUST NOT be transmitted on 391 black BGP sessions, and that MUST be ignored if received on black BGP 392 sessions. These are known as the "red loopback routes". 394 The procedures of this document result in a network overlay whose 395 control plane consists of red BGP sessions, and whose data plane 396 consists of MPLS-in-IPsec Security Associations. This allows an SP's 397 customer to provide Secured L3VPN service to its tenants. 399 When RRs are used, C-PEs "register" with the RRs by setting up BGP 400 sessions to them, running the BGP sessions through IPsec SAs. The 401 necessary authentication of the C-PE is provided during the course of 402 setting up the IPsec SA. One C-PE learns of another other C-PE's 403 presence when the RR propagates routes from the latter C-PE to the 404 former. 406 The procedures specified in this document result in one MPLS-in-IPsec 407 SA between a given pair of C-PEs. This one SA will carry the traffic 408 of all the tenants that are attached to both C-PEs. That should 409 provide adequate security, as the tenants' data is already exposed to 410 the C-PEs. If for some reason it is desired to have a distinct SA 411 for each tenant, a method of doing so is mentioned in Section 4. 413 3. How the C-PEs Advertise Red Routes 415 3.1. Red and Black C-PE Loopback Addresses 417 To support the Secured L3VPN control plane, each C-PE MUST have two 418 loopback addresses. One of these will be known as its "red 419 loopback", the other as its "black loopback". 421 The black loopbacks MUST be addresses that are globally routable. 422 That is, they are public addresses. (Strictly speaking, the black 423 loopback only needs to be routable in any network that might be used 424 to carry traffic between two C-PEs. But we will assume that traffic 425 between two C-PEs might need to traverse the public Internet.) 426 Typically a C-PE's black loopback will be in the address space 427 administered by the network service provider to which the C-PE 428 attaches. The service provider may assign it dynamically, or it may 429 be assigned statically and configured in the C-PE by the customer. 431 In addition to having a globally routable black loopback, a C-PE will 432 of course have globally routable interface addresses for each of its 433 black interfaces. 435 Interface addresses of the red interfaces SHOULD NOT be globally 436 routable. 438 If the C-PE attaches to multiple service providers, the black 439 loopback is likely to be be a provider-independent address. However, 440 it MUST be routable in the backbone network of both providers, and 441 most likely will need to be globally routable. 443 The C-PE may have one or more (black) BGP sessions with service 444 provider peers, in which case it may advertise the black loopback; 445 the next hop field of such an advertisement would be the interface 446 address of the interface over which that BGP session runs. 448 Each C-PE of a given customer MUST be provisioned with a red loopback 449 that is unique among the set of C-PEs of that customer. The red 450 loopback MUST NOT be a routable address in the public Internet or in 451 the backbone networks of any service provider to which any of the 452 C-PEs is attached. If a C-PE has a (black) BGP session with a 453 service provider peer, it MUST NOT advertise a route to its red 454 loopback over that session. That is, any IP route to a red loopback 455 is considered to be a red route, and MUST NOT be advertised or 456 received on a black BGP session. These "red loopback routes" can 457 thus be considered to be "red routes", even though they are IP rather 458 than VPN-IP routes. 460 3.2. Setting Up Red BGP Sessions Between C-PEs and RRs 462 The customer is expected to have two or more BGP Route Reflectors 463 (red RRs). The red RRs are presumed to be secure; making them so is 464 the responsibility of the customer. As with the C-PEs, each red RR 465 has a black loopback and a red loopback. If the RR is not also a 466 C-PE, it will have only black interfaces, each of course with a 467 globally routable interface address. 469 A customer's red RRs will form BGP sessions with that customer's 470 C-PEs. These BGP sessions MUST be protected by IPsec. The use of 471 IPsec transport mode is RECOMMENDED. If the RR's red loopback is an 472 IPv4 address, it may be used as the RR's BGP Identifier (see 473 [RFC4271]). 475 When a C-PE device comes up, it attempts to set up an IPsec-protected 476 BGP session with the red RRs. This requires first setting up an 477 IPsec SA with each red RR, and then using IPsec Transport Mode to 478 protect the BGP session. 480 If the C-PE's red loopback is an IPv4 address, the C-PE's BGP 481 Identifier (see [RFC4271]) may be the red loopback. 483 The endpoint addresses of the IPsec SA are the black loopbacks of the 484 endpoint systems. 486 Therefore, in order to initiate a BGP session to a red RR, a C-PE 487 must be provisioned to know a publicly routable address (i.e., the 488 black loopback) of the RR. A C-PE must also be provisioned with 489 whatever additional information is needed in order to set up an IPsec 490 SA with each of the red RRs. Each C-PE will attempt to continuously 491 maintain live BGP sessions (protected by IPsec) with each red RR. 492 Note that the source and destination IP address fields of the IP 493 datagrams carrying the IPsec-encapsulated BGP messages will be 494 publicly routable adresses. 496 In some scenarios, it may be desirable to provision each red RR with 497 the publicly routable address and pre-shared secret of every C-PE. 498 This makes it easy for the C-PEs to authenticate themselves to the 499 RR, but requires each RR to be reprovisioned every time a new C-PE is 500 added to the network. 502 In other scenarios, it may be considered desirable to allow the RRs 503 to auto-discover the C-PEs, without the need for any per-C-PE pre- 504 provisioning of the RRs. In this case, a certificate-based 505 authentication method [reference?] can be used when setting up the 506 IPsec SAs that carry the BGP sessions. 508 In either type of scenario, the C-PE SHOULD NOT be assumed to have a 509 fixed black loopback address or fixed black interface addresses; 510 rather, it SHOULD be assumed that a C-PE might be a mobile device 511 whose globally routable addresses change from time to time. 513 If a customer's C-PEs support multiple VPNs (for multiple tenants), 514 that customer's red RRs will receive and disseminate the VPN-IP 515 routes of all those VPNs. 517 Note that according to the above procedures, the C-PEs will only have 518 red BGP sessions to the red RRs; the C-PEs will not have BGP sessions 519 to each other. Thus it is not necessary for the C-PEs to know of 520 each other in advance. Of course, if a particular customer deems it 521 desirable for the C-PEs to have red BGP sessions to each other, each 522 C-PE can be provisioned with a publicly routable address of each 523 other C-PE, along with any additional information needed to set up an 524 IPsec SA to each other C-PE. 526 It is RECOMMENDED that, for the purpose of setting up the red BGP 527 sessions, all the RRs and C-PEs be considered to be in the same 528 Autonomous System (AS). Then the red BGP sessions will all be IBGP 529 sessions, and the next hop field of a red route will not be modified 530 as the route is propagated. Note that if an implementation allows a 531 given router to be attached to two different ASes, this does not 532 require that all the C-PEs and red RRs attach to the Internet via the 533 same AS. The "red overlay" may appear to be within a single AS, but 534 the "black underlay" need not be within a single AS. 536 If it is necessary to use an EBGP session between a C-PE and an RR, 537 the RR SHOULD have a configured policy to leave the next hop 538 unchanged when propagating red VPN-IP routes on an EBGP session. See 539 Section 3.4. 541 In some scenarios, the C-PEs may set up red BGP sessions to 542 Autonomous System Border Routers (ASBRs), rather than to RRs. This 543 is transparent to the C-PE. 545 3.3. Routes Transmited by the C-PE on Red BGP Sessions 547 A C-PE MUST transmit its local VPN-IP routes on the red BGP sessions, 548 and only on the red BGP sessions. The next hop of each local VPN-IP 549 route MUST be set to the red loopback of the C-PE. The choice to 550 transmit a particular VPN-IP route on a particular session may of 551 course be influenced by the route's Route Targets. 553 A C-PE MUST NOT transmit its local VPN-IP routes on black BGP 554 sessions. VPN-IP routes MUST NOT be accepted from black BGP 555 sessions. 557 In all other respects, the handling of VPN-IP routes is done by 558 normal L3VPN procedures. 560 Each C-PE MUST also transmit the following IP (IPv4 or IPv6) route on 561 the red BGP sessions. We refer to this route as the C-PE's "red 562 loopback route": 564 o The address prefix field of the route's Network Layer Reachability 565 Information (NLRI) contains the C-PE's red loopback as a host 566 address. 568 o The Next Hop of the route is the C-PE's black loopback. 570 o The route carries a Tunnel Encapsulation Attribute [tunnel_encaps] 571 with the the following parameters: 573 * Tunnel Type = "MPLS-in-IPsec" (see Section 5.) 575 * Remote Endpoint = the C-PE's black loopback 576 * A "Security Handle" (see Section 6). This provides any 577 information needed by another C-PE to set up an MPLS-in-IPsec 578 Security Association with the advertising C-PE. 580 A C-PE MUST NOT transmit, on any black BGP session, an IP route whose 581 NLRI contains its red loopback. 583 A given C-PE's red loopback route must be propagated to all other the 584 C-PEs belonging to the same customer. Therefore, such routes SHOULD 585 NOT carry Route Targets. 587 3.4. Propagating Red Routes 589 A route that is received over a red BGP session may need to be 590 propagated to other red BGP sessions. A route that is received over 591 a red BGP session MUST NOT be propagated over a black BGP session. 592 Similarly, a route that is received over a black BGP session MUST NOT 593 be propagated over a red BGP session. 595 When a route is propagated from one red BGP session to another, its 596 next hop SHOULD be left unchanged. As specified in Section 4, this 597 will ensure that a data packet sent on the path advertised by that 598 route are sent on an IPsec SA between its ingress C-PE and its egress 599 C-PE. Changing the next hop will change the IPsec SA endpoint. 601 This may be useful in certain deployments. For instance, the path 602 from an ingress C-PE to an egress C-PE may traverse several ASBRS. 603 If these ASBRs are secure, it may be desirable to set up a sequence 604 of IPsec SAs, (e.g., C-PE1--ASBR1, ASBR1--ASBR2, ASBR2--C-PE2) 605 instead of using a single IPsec SA between C-PE1 and C-PE2. If this 606 is not the intention, the red BGP sessions MUST leave the next hop 607 unchanged, even if those sessions are EBGP sessions. 609 In all other respects, propagation of red routes is governed by the 610 normal procedures for propagating routes. If the route carries one 611 or more Route Targets, these may affect its propagation. However, 612 note that propagation of a route between a red BGP session and a 613 black BGP session MUST NOT be done, irrespective of the Route 614 Targets. 616 4. VPN-IP Routes and Recursive Route Resolution 618 Suppose a C-PE, say C-PE1, receives a packet, say packet P, on one of 619 its local red interfaces. Suppose that packet P is addressed to a 620 system that is reachable via one of the red interfaces at another 621 C-PE, say C-PE2. C-PE1 looks up packet P's destination address in 622 the VRF associated with P's incoming interface. The matching route 623 will be a "Labeled VPN-IP route" [RFC4364] originated by C-PE2, and 624 disseminated to C-PE1 over a red BGP session. Per Section 3.3, the 625 next hop of that route will be C-PE2's red loopback. 627 The labeled VPN-IP route matched by packet P's destination address 628 will contain an MPLS label, the "VPN label". C-PE1 pushes the VPN 629 label onto packet P's MPLS label stack. Then C-PE1 needs to 630 determine how to transmit the resulting MPLS packet to the next hop 631 of the VPN-IP route. The next hop of the labeled VPN-IP route will 632 be the red loopback address C-PE2. So C-PE1 looks for the route to 633 that red loopback address. This will be the red loopback route 634 (i.e., the red IP route, see Section 3.3) originated by C-PE2. 636 C-PE2's red loopback will then be recursively resolved by means of 637 C-PE2's red loopback route. By virtue of the Tunnel Encapsulation 638 attribute carried by that route, C-PE1 will realize that to send 639 packet P, it must set up an MPLS-in-IPsec SA (see Section 5, as well 640 as Sections 3 and 8.1 of [RFC4023]) with C-PE2. Per the route's 641 Tunnel Encapsulation attribute, the remote endpoint of this IPsec SA 642 will be C-PE2's black loopback, and the Security Handle in the Tunnel 643 Encapsulation attribute will carry any other information needed to 644 set up the Security Association. 646 Note that the remote endpoint of the IPsec SA is determined by the 647 Tunnel Encapsulation attribute of the red loopback route, rather than 648 by the next hop field of that route. This ensures that the SA is 649 made to the proper endpoint, even if the next hop field of the red 650 loopback route was modified while the route was propagated. 652 IMPORTANT: A VPN-IP route MUST NOT be recursively resolved by an IP 653 route that was received over a black BGP session. If a VPN-IP 654 route's next hop resolves to a route received over a black BGP 655 session, the existence of the latter route MUST be regarded as the 656 result of an attempt to spoof the location of the egress C-PE. That 657 is, the latter route MUST be considered to be a spoofed route. Note 658 that it may not be possible to detect this spoofing attack until the 659 attempt is made to recursively resolve the VPN-IP route. 660 Implementors should take special care to ensure that their 661 implementations are not vulnerable to this sort of spoofing attack. 662 IF an implementation cannot detect this sort of attack during the 663 recursive route resolution process, then the C-PE MUST NOT have any 664 black BGP sessions. 666 When packet P is transmitted, it is transmitted through an MPLS-in- 667 IPsec SA. Thus the only information that appears in the clear is the 668 IP header needed to get the packet across the network. The IP source 669 and destination addresses of that packet will be the black loopbacks 670 of C-PE1 and C-PE2 respectively. The red loopback addresses do not 671 appear in the packets at all, and no part of the payload packet 672 (neither the VPN label nor the IP datagram following the VPN label) 673 appears in the clear. 675 The MPLS-in-IPsec SA between C-PE1 and C-PE2 may be initiated by 676 C-PE1 as soon as it receives a red loopback route originated by 677 C-PE2. Alternatively, the initiation of the setup of the Security 678 Association may be delayed until the SA is actually needed for 679 transmitting packets. 681 These procedures will result in a single IPsec SA between a pair of 682 C-PEs, with the data of multiple tenants carried on that single SA. 683 If for some reason it is considered preferable to have an SA per 684 tenant, the following procedures can be used: 686 o On each C-PE, provision a distinct red loopback for each tenant. 688 o Each C-PE will originate a red loopback route for each red 689 loopback. 691 o Each red loopback route will have its own Tunnel Encapsulation 692 attribute. The respective Security Handle sub-TLVs (if present) 693 MUST be distinct. 695 5. MPLS-in-IPsec 697 Packets traveling from one C-PE to another travel through "MPLS-in- 698 IPsec" tunnels. To transmit an MPLS packet through an MPLS-in-IPsec 699 tunnel, one does the following: 701 o Encapsulate the MPLS packet in IP, as specified in Section 3 of 702 [RFC4023]. 704 o Use an IPsec transport mode Security Association to send the MPLS- 705 in-IP packet from one C-PE to the other. This is specified in 706 Section 8.1 of [RFC4023]. 708 The result of encapsulating MPLS in IP and then transmitting the 709 MPLS-in-IP packet on an IPsec transport mode Security Association is 710 known as an MPLS-in-IPsec packet. 712 On the wire, an MPLS-in-IPsec packet consists of a cleartext IP 713 header followed by a payload. The IP source and destination 714 addresses of an MPLS-in-IPsec packet will be the black loopbacks of 715 the source and destination C-PEs. The payload will be an MPLS 716 packet. If the IPsec Security Association is providing privacy, 717 authentication, and integrity, the payload is protected from 718 inspection or alteration. 720 When the packet arrives at the destination C-PE, any necessary 721 decryption is done, and packet appears to be an MPLS-in-IP packet 722 addressed to the black address of the destination C-PE. The IP 723 encapsulation is removed, yielding an MPLS packet. Per the usual 724 L3VPN procedures, the label at the top of the MPLS label stack will 725 be used to govern the further disposition of the packet. However, if 726 a packet received over a black interface was not received though an 727 IPsec SA, the packet MUST NOT be sent out any VRF interface. 729 MPLS-in-IP packets received in the clear (i.e., not received over an 730 IPsec SA) MUST be discarded. 732 Note that this section is not intended to describe an implementation 733 strategy. 735 6. Security Handle 737 This document defines a new BGP Tunnel Encapsulation attribute sub- 738 TLV, the "Security Handle". This sub-TLV has a one-octet length 739 field. It is intended for use in the Tunnel Encapsulation attribute 740 carried by the red loopback routes. Its use is deployment specific. 742 As an example, in some deployments, this sub-TLV might be used to 743 carry the IPsec Security Parameters Index (SPI). When setting up an 744 SA to the originator of a particular Tunnel Encapsulation attribute, 745 the SPI would be used as part of the SA setup procedure. 747 In deployments where the C-PEs auto-discover each other through RRs, 748 and authenticate via certificate-based mechanisms, the Security 749 Handle may not be needed at all. If a given deployment does not make 750 use of the Security Handle, the sub-TLV SHOULD be omitted from the 751 Tunnel Encapsulation attribue. 753 7. Data Plane Security Procedures 755 If a C-PE receives data over one of its local red interfaces, it may 756 forward the data out another of its local red interfaces, as long as 757 those two interfaces are associated with the same VRF, or if there is 758 policy allowing communication ("extranet") between those two 759 interfaces. 761 However, data received by a C-PE over one of its red interfaces MUST 762 NOT be forwarded out a black interface, unless that data is being 763 sent over the black interface through an IPsec SA. 765 Similarly, data received by a C-PE over one of its black interfaces 766 MUST NOT be forwarded out a red interface unless the data arrived 767 through an IPsec SA. 769 Typically an IPsec implementation has procedures to prevent 770 unauthorized red-to-black or black-to-red forwarding. However, the 771 conventional procedures are based on filtering of IP addresses, and 772 hence do not apply directly if MPLS-in-IPsec is used. Implementors 773 should take care to ensure that unauthorized red-to-black or black- 774 to-red forwarding is prohibited. 776 8. Acknowledgments 778 The authors wish to thank John Scudder for his ideas and 779 contributions to this work. 781 9. IANA Considerations 783 IANA is requested to create a new entry in the "BGP Tunnel 784 Encapsulation Attribute Sub-TLVs" registry, "Security Handle". This 785 sub-TLV is defined in Section 6 to have a one-octet length field. 786 Thus it needs to be assigned a codepoint in the range 0-127 787 inclusive. 789 10. Implementation Challenges 791 This document specifies an architecture for Secured L3VPNs, but a 792 successful implementation faces a number of challenges. 794 This document specifies a recursive route resolution process that 795 makes use of the Tunnel Encapsulation attribute. This is a new 796 feature. 798 This document specifies that during recurisve route resolution, 799 resolution of a red route via a route received over a black BGP 800 session must be prohibited. This is a new feature that may present 801 challenges. 803 Ultimately, success will require a highly scalable IPsec 804 implementation, that can set up SAs dynamically based on information 805 disseminated by BGP. This presents a number of implementation 806 challenges. 808 11. Security Considerations 810 12. References 812 12.1. Normative References 814 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 815 Requirement Levels", BCP 14, RFC 2119, 816 DOI 10.17487/RFC2119, March 1997, 817 . 819 [RFC4023] Worster, T., Rekhter, Y., and E. Rosen, Ed., 820 "Encapsulating MPLS in IP or Generic Routing Encapsulation 821 (GRE)", RFC 4023, DOI 10.17487/RFC4023, March 2005, 822 . 824 [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 825 Border Gateway Protocol 4 (BGP-4)", RFC 4271, 826 DOI 10.17487/RFC4271, January 2006, 827 . 829 [RFC4303] Kent, S., "IP Encapsulating Security Payload (ESP)", 830 RFC 4303, DOI 10.17487/RFC4303, December 2005, 831 . 833 [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private 834 Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February 835 2006, . 837 [RFC7296] Kaufman, C., Hoffman, P., Nir, Y., Eronen, P., and T. 838 Kivinen, "Internet Key Exchange Protocol Version 2 839 (IKEv2)", STD 79, RFC 7296, DOI 10.17487/RFC7296, October 840 2014, . 842 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 843 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 844 May 2017, . 846 [RFC8221] Wouters, P., Migault, D., Mattsson, J., Nir, Y., and T. 847 Kivinen, "Cryptographic Algorithm Implementation 848 Requirements and Usage Guidance for Encapsulating Security 849 Payload (ESP) and Authentication Header (AH)", RFC 8221, 850 DOI 10.17487/RFC8221, October 2017, 851 . 853 [tunnel_encaps] 854 Rosen, E., Patel, K., and G. Van de Velde, "The BGP Tunnel 855 Encapsulation Attribute VPN", internet-draft draft-ietf- 856 idr-tunnel-encaps-09, February 2018. 858 12.2. Informational References 860 [RFC6071] Frankel, S. and S. Krishnan, "IP Security (IPsec) and 861 Internet Key Exchange (IKE) Document Roadmap", RFC 6071, 862 DOI 10.17487/RFC6071, February 2011, 863 . 865 Authors' Addresses 867 Eric C. Rosen (editor) 868 Juniper Networks, Inc. 869 10 Technology Park Drive 870 Westford, Massachusetts 01886 871 United States 873 Email: erosen@juniper.net 875 Ron Bonica 876 Juniper Networks, Inc. 877 2251 Corporate Park Drive 878 Herndon, Virginia 20171 879 United States 881 Email: rbonica@juniper.net