idnits 2.17.00 (12 Aug 2021) /tmp/idnits63498/draft-ietf-bess-bgp-sdwan-usage-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 10 instances of too long lines in the document, the longest one being 4 characters in excess of 72. == There are 11 instances of lines with non-RFC6890-compliant IPv4 addresses in the document. If these are example addresses, they should be changed. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (April 18, 2022) is 26 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'RFC 4364' is mentioned on line 162, but not defined == Missing Reference: 'RFC4456' is mentioned on line 329, but not defined == Missing Reference: 'RFC4684' is mentioned on line 785, but not defined == Missing Reference: 'RFC4023' is mentioned on line 972, but not defined == Missing Reference: 'RFC3715' is mentioned on line 992, but not defined == Missing Reference: 'RFC3948' is mentioned on line 994, but not defined == Missing Reference: 'RFC3947' is mentioned on line 997, but not defined == Unused Reference: 'RFC2119' is defined on line 1044, but no explicit reference was found in the text == Unused Reference: 'RFC4364' is defined on line 1047, but no explicit reference was found in the text == Unused Reference: 'RFC7296' is defined on line 1050, but no explicit reference was found in the text == Unused Reference: 'RFC7432' is defined on line 1053, but no explicit reference was found in the text == Unused Reference: 'RFC8365' is defined on line 1056, but no explicit reference was found in the text == Unused Reference: 'RFC8192' is defined on line 1064, but no explicit reference was found in the text == Unused Reference: 'RFC5521' is defined on line 1067, but no explicit reference was found in the text == Unused Reference: 'RFC8388' is defined on line 1071, but no explicit reference was found in the text == Unused Reference: 'Net2Cloud-Gap' is defined on line 1074, but no explicit reference was found in the text == Unused Reference: 'VPN-over-Internet' is defined on line 1082, but no explicit reference was found in the text == Unused Reference: 'DMVPN' is defined on line 1086, but no explicit reference was found in the text == Unused Reference: 'DSVPN' is defined on line 1090, but no explicit reference was found in the text == Unused Reference: 'ITU-T-X1036' is defined on line 1101, but no explicit reference was found in the text == Unused Reference: 'Net2Cloud-gap' is defined on line 1109, but no explicit reference was found in the text == Outdated reference: A later version (-04) exists of draft-dm-net2cloud-gap-analysis-02 == Outdated reference: A later version (-04) exists of draft-dunbar-idr-sdwan-edge-discovery-01 == Outdated reference: A later version (-01) exists of draft-rosen-bess-secure-l3vpn-00 == Outdated reference: A later version (-05) exists of draft-sajassi-bess-secure-evpn-01 == Outdated reference: A later version (-01) exists of draft-rosen-bess-secure-l3vpn-00 -- Duplicate reference: draft-rosen-bess-secure-l3vpn, mentioned in 'SECURE-L3VPN', was also mentioned in 'VPN-over-Internet'. -- No information found for draft-rtgwg-net2cloud-problem-statement - is the name correct? -- No information found for draft-rtgwg-net2cloud-gap-analysis - is the name correct? Summary: 1 error (**), 0 flaws (~~), 28 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft J. Guichard 3 Intended status: Informational Futurewei 4 Expires: October 18, 2022 Ali Sajassi 5 Cisco 6 J. Drake 7 Juniper 8 B. Najem 9 Bell Canada 10 Ayan Barnerjee 11 D. Carrel 12 IPsec Research 13 April 18, 2022 15 BGP Usage for SDWAN Overlay Networks 16 draft-ietf-bess-bgp-sdwan-usage-05 18 Abstract 20 The document discusses the usage and applicability of BGP as the 21 control plane for multiple SDWAN scenarios. The document aims to 22 demonstrate how the BGP-based control plane is used for large-scale 23 SDWAN overlay networks with little manual intervention. 25 SDWAN edge nodes are commonly interconnected by multiple types of 26 underlay networks owned and managed by different network providers. 28 Status of this Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 This Internet-Draft is submitted in full conformance with the 34 provisions of BCP 78 and BCP 79. This document may not be modified, 35 and derivative works of it may not be created, except to publish it 36 as an RFC and to translate it into languages other than English. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF), its areas, and its working groups. Note that 40 other groups may also distribute working documents as Internet- 41 Drafts. 43 Internet-Drafts are draft documents valid for a maximum of six 44 months and may be updated, replaced, or obsoleted by other documents 45 at any time. It is inappropriate to use Internet-Drafts as 46 reference material or to cite them other than as "work in progress." 48 The list of current Internet-Drafts can be accessed at 49 http://www.ietf.org/ietf/1id-abstracts.txt 51 The list of Internet-Draft Shadow Directories can be accessed at 52 http://www.ietf.org/shadow.html 54 This Internet-Draft will expire on April 18, 2009. 56 Copyright Notice 58 Copyright (c) 2021 IETF Trust and the persons identified as the 59 document authors. All rights reserved. 61 This document is subject to BCP 78 and the IETF Trust's Legal 62 Provisions Relating to IETF Documents 63 (http://trustee.ietf.org/license-info) in effect on the date of 64 publication of this document. Please review these documents 65 carefully, as they describe your rights and restrictions with 66 respect to this document. Code Components extracted from this 67 document must include Simplified BSD License text as described in 68 Section 4.e of the Trust Legal Provisions and are provided without 69 warranty as described in the Simplified BSD License. 71 Table of Contents 73 1. Introduction...................................................3 74 2. Conventions used in this document..............................4 75 3. Use Case Scenario Description and Requirements.................5 76 3.1. Requirements..............................................6 77 3.1.1. Supporting SDWAN Segmentation........................6 78 3.1.2. Client Service Requirement...........................6 79 3.1.3. SDWAN Traffic Segmentation...........................6 80 3.1.4. Zero Touch Provisioning..............................7 81 3.1.5. Constrained Propagation of SDWAN Edge Properties.....8 82 3.2. Scenario #1: Homogeneous WAN..............................9 83 3.3. Scenario #2: Hybrid WAN Underlay.........................10 84 3.4. Scenario #3: Private VPN PE based SDWAN..................12 86 4. Provisioning Model............................................13 87 4.1. Client Service Provisioning Model........................13 88 4.2. Policy Configuration.....................................14 89 4.3. IPsec related parameters Provisioning....................14 90 5. BGP Controlled SDWAN..........................................14 91 5.1. BGP Walk Through for Homogeneous SDWAN...................14 92 5.2. BGP Walk Through for Hybrid WAN Underlay.................16 93 5.3. BGP Walk Through for Application Flow Based Segmentation.17 94 5.4. Benefit of Using Recursive Next Hop Resolution...........19 95 5.5. Why BGP as Control Plane for SDWAN?......................19 96 6. SDWAN Forwarding Model........................................20 97 6.1. Forwarding Model for Homogeneous SDWAN...................20 98 6.1.1. Network and Service Startup Procedures..............20 99 6.1.2. Packet Walk-Through.................................21 100 6.2. Forwarding Model for Hybrid Underlay SDWAN...............22 101 6.2.1. Network and Service Startup Procedures..............22 102 6.2.2. Packet Walk-Through.................................22 103 6.3. Forwarding Model for PE based SDWAN......................23 104 6.3.1. Network and Service Startup Procedures..............23 105 6.3.2. Packet Walk-Through.................................23 106 7. Manageability Considerations..................................24 107 8. Security Considerations.......................................25 108 9. IANA Considerations...........................................25 109 10. References...................................................25 110 10.1. Normative References....................................25 111 10.2. Informative References..................................26 112 11. Acknowledgments..............................................27 114 1. Introduction 116 SDWAN optimizes transport of IP Packets over multiple underlay 117 connectivity services. Here are some of the main characteristics of 118 "SDWAN" networks: 120 - Augment of transport, which refers to utilizing paths over 121 different underlay networks. Very often, there are multiple 122 parallel overlay paths between any two SDWAN edges; some of them 123 are private networks over which traffic can traverse with or 124 without encryption; others require encryption, e.g., over 125 untrusted public networks. 126 - Direct Internet breakout from remote branch offices is allowed 127 instead of all traffic hauled to Corporate HQ for centralized 128 policy control. 129 - Some traffic can be forwarded based on their application 130 identifiers instead of based on destination IP addresses by the 131 edge nodes placing the traffic onto specific overlay paths based 132 on the application-specific policies. 133 - The traffic forwarding can also be based on specific performance 134 criteria (e.g., packets delay, packet loss, jitter) to provide 135 better application performance by choosing the underlay that 136 meets or exceeds the specified policies. 138 [Net2Cloud-Problem] describes the network-related problems to 139 connect enterprises' branch offices to dynamic workloads in 140 different Cloud Data Centers (DC). SDWAN has been positioned as a 141 flexible way to reach dynamic workloads in third-party Cloud DCs. 142 However, scaling becomes a significant issue when hundreds or 143 thousands of nodes need to be interconnected by SDWAN overlay 144 networks. 146 This document describes using BGP as the control plane to scale 147 large SDWAN overlay networks. 149 2. Conventions used in this document 151 Cloud DC: Third party data centers that host applications and 152 workloads owned by different organizations or tenants. 154 Controller: Used interchangeably with SDWAN controller to manage 155 SDWAN overlay path creation/deletion and monitor the 156 path conditions between sites. 158 CPE: Customer Premise Equipment 160 CPE-Based VPN: Virtual Private Secure network formed among CPEs. 161 This is to differentiate from more commonly used PE- 162 based VPNs [RFC 4364]. 164 Homogeneous SDWAN: A SDWAN network in which all traffic to/from the 165 SDWAN edge nodes are carried by IPsec tunnels regardless 166 of underlay networks. I.e., the client traffic is 167 carried by IPsec tunnel even over MPLS private networks. 169 ISP: Internet Service Provider 171 NSP: Network Service Provider. NSP usually provides more 172 advanced network services, such as MPLS VPN, private 173 leased lines, or managed Secure WAN connections, many 174 times within a private trusted domain, whereas an ISP 175 usually provides plain Internet services over public 176 untrusted domains. 178 PE: Provider Edge 180 SDWAN Edge Node: an edge node, which can be physical or virtual, 181 maps the attached clients' traffic to the wide area 182 network (WAN) overlay tunnels. 184 SDWAN: Software Defined Wide Area Network. A connectivity 185 service, offered by a Service Provider, that optimizes 186 transport of IP Packets over multiple underlay 187 connectivity services by recognizing applications at 188 Ingress and determining forwarding behavior by applying 189 policies to them. 191 SDWAN IPsec SA: IPsec Security Association between two SDWAN ports 192 or nodes. 194 SDWAN over Hybrid Networks: SDWAN over Hybrid Networks typically 195 have edge nodes utilizing bandwidth resources from 196 different types of underlay networks, some being private 197 networks and others being public Internet. 199 WAN Port: A Port or Interface facing an ISP or Network Service 200 Provider (NSP), with address allocated by the ISP or the 201 NSP. 203 C-PE: SDWAN Edge node, which can be CPE for customer managed 204 SDWAN, or PE for provider managed SDWAN services. 206 ZTP: Zero Touch Provisioning 208 3. Use Case Scenario Description and Requirements 210 This section describes some essential requirements for SDWAN 211 networks and several SDWAN scenarios used by the subsequent sections 212 to explain how the BGP control plane is applied. 214 3.1. Requirements 216 3.1.1. Supporting SDWAN Segmentation 218 "SDWAN Segmentation" is a frequently used term in SDWAN deployment, 219 referring to policy-driven network partitioning. An SDWAN segment is 220 a virtual private network (SDWAN VPN) consisting of a set of edge 221 nodes interconnected by the tunnels, such as IPsec tunnels and MPLS 222 VPN tunnels. 224 This document assumes that an SDWAN VPN configuration on a PE 225 follows the same way as MPLS VPN, i.e., via VRFs. One SDWAN VPN can 226 be mapped to one or multiple SD-WAN virtual topologies, governed by 227 the SDWAN controller's policies. 229 When using BGP for SDWAN, the Client Route UPDATE is the same as 230 MPLS VPN. Route Target in the BGP Extended Community can be used to 231 differentiate routes belonging to different SDWAN VPNs. 233 As SDWAN is an overlay network arching over multiple types of 234 networks, MPLS L2VPN/L3VPN or pure L2 underlay can continue using 235 the VPN ID, VN-ID, or VLAN in the data plane to differentiate 236 packets belonging to different SDWAN VPNs. For packets carried by an 237 IPsec tunnel, the IPsec's inner encapsulation header can have the 238 SDWAN VPN Identifier to distinguish the packets belonging to 239 different SDWAN VPNs. 241 3.1.2. Client Service Requirement 243 The Client interface of SDWAN nodes can be IP or Ethernet-based. 245 For Ethernet-based client interfaces, SDWAN edge should support 246 VLAN-based service interfaces (EVI100), VLAN bundle service 247 interfaces (EVI200), or VLAN-Aware bundling service interfaces. EVPN 248 service requirements apply to the Client traffic, as described in 249 Section 3.1 of RFC8388. 251 For IP-based client interfaces, L3VPN service requirements are 252 applicable. 254 3.1.3. SDWAN Traffic Segmentation 256 SD-WAN Traffic Segmentation enables the separation of the traffic 257 based on the business and the security needs of different users' 258 groups and/or application requirements. Each user group and/or 259 application may need different isolated topologies and/or policies 260 to fulfill the business requirements. The SD-WAN Traffic 261 Segmentation is enabled on a single SD-WAN Service to a single 262 subscriber. 264 For example, a retail business requires the point-of-sales (PoS) 265 application to be on a different topology from other applications. 266 The PoS application is routed only to the payment processing entity 267 at a hub site; other applications can be routed to all other sites. 269 The traffic from the PoS application follows a Tree topology in the 270 figure below, whereas other traffic can be multipoint-to-multipoint 271 topology. 273 +--------+ 274 Payment traffic |Payment | 275 +------+----+-+gateway +------+----+-----+ 276 / / | +----+---+ | \ \ 277 / / | | | \ \ 278 +-+--+ +-+--+ +-+--+ | +-+--+ +-+--+ +-+--+ 279 |Site| |Site| |Site| | |Site| |Site| |Site| 280 | 1 | | 2 | | 3 | | |4 | | 5 | | 6 | 281 +--+-+ +--+-+ +--|-+ | +--|-+ +--|-+ +--|-+ 282 | | | | | | | 283 ==+=======+=======+====+======+=======+=======+=== 284 Multi-point connection for Other traffic 286 Another example is an enterprise that wants to isolate the traffic 287 from different departments, with each department having its unique 288 topology and policy. The HR department may need to access specific 289 applications that are NOT accessible by the engineering department. 290 Also, the contractors may have limited access to the enterprise 291 resources. 293 3.1.4. Zero Touch Provisioning 295 SDWAN zero-tour provisioning (ZTP) allows devices to be configured 296 and provisioned centrally. When an SDWAN edge is installed at a 297 remote location, ZTP automates follow-up steps, including updates to 298 the OS, software version, and configuration before client traffic 299 being forwarded. The ZTP can bootstrap a remote SDWAN node and 300 establish a secure connection to the local SDWAN Controller, making 301 it convenient to add or delete an SDWAN edge node (virtual or 302 physical). From the network control perspective, ZTP includes the 303 following: 305 - Upon power-up, an SDWAN node can establish the transport layer 306 secure connection (such as TLS, SSL, etc.) to its controller, 307 whose address can be burned or preconfigured on the device. 309 - The SDWAN Controller can designate a local network controller 310 in the proximity of the SDWAN node. Like the Route-Reflector (RR) 311 for BGP controlled SDWAN, the local network controller manages and 312 monitors the communication policies for traffic to/from the edge 313 node. 315 3.1.5. Constrained Propagation of SDWAN Edge Properties 317 One SDWAN edge node may only be authorized to communicate with a 318 small number of other SDWAN edge nodes. Under this circumstance, the 319 property of the SDWAN edge node cannot be propagated to other nodes 320 that are not authorized to communicate. But a remote SDWAN edge 321 node, upon powering up, might not have the right policies to know 322 which peers are authorized to communicate. Therefore, SDWAN 323 deployment needs to have a central point to distribute the 324 properties of an SDWAN edge node to its authorized peers. 326 BGP is well suited for this purpose. RFC4684 has specified the 327 procedure to constrain the distribution of BGP UPDATE to only a 328 subset of nodes. Each edge node informs the Route-Reflector (RR) 329 [RFC4456] on its interested SDWAN VPNs. The RR only propagates the 330 BGP UPDATE for the relevant SDWAN VPNs to the edge. 332 The connection between an SDWAN edge node and its RR can be over an 333 insecure network. Therefore, an SDWAN node needs to establish a 334 secure transport layer connection (TLS, SSL, etc.) to its designated 335 RR upon power-up. The BGP UPDATE messages need to be sent over the 336 secure channel (TLS, SSL, etc.) to the RR. 338 +---+ 339 Peer Group 1 |RR | Peer Group 2 340 +======+====+=+ +======+====+=====+ 341 / / | +---+ | \ \ 342 / / | | \ \ 343 +-+--+ +-+--+ +-+--+ +-+--+ +-+--+ +-+--+ 344 |C-PE| |C-PE| |C-PE| |C-PE| |C-PE| |C-PE| 345 | 1 | | 2 | | 3 | |4 | | 5 | | 6 | 346 +----+ +----+ +----+ +----+ +----+ +----+ 347 Tenant 1 Tenant 2 348 Figure 1: Peer Groups managed by RR 350 Tenant separation is achieved by the SDWAN VPN identifiers 351 represented in the control plane and data plane, respectively. 353 3.2. Scenario #1: Homogeneous WAN 355 Homogeneous WAN refers to a type of SDWAN network with edge nodes 356 encrypting all traffic over WAN to other edge nodes, regardless of 357 whether the underlay is private or public. For lack of better 358 terminology, we call this Homogeneous SDWAN throughout this 359 document. 361 Some typical scenarios for the use of a Homogeneous SDWAN network 362 are as follows: 364 - A small branch office to connect to its HQ offices via the 365 Internet. All sensitive traffic to/from this small branch office 366 must be encrypted, usually achieved by IPsec Tunnels. 368 - A store in a shopping mall may need to securely connect to its 369 applications in one or more Cloud DCs via the Internet. A common way 370 of achieving this is to establish IPsec SAs to the Cloud DC gateway 371 to carry the sensitive data to/from the store. 373 As described in [SECURE-EVPN], the granularity of the IPsec SAs for 374 Homogeneous SDWAN can be per site, per subnet, per tenant, or 375 address. Once the IPsec SA is established for a specific 376 subnet/tenant/site, all traffic to/from the subnets/tenants/site is 377 encrypted. 379 +---+ 380 +--------------|RR |------------+ 381 / Untrusted +-+-+ \ 382 / \ 383 / \ 384 +----+ +---------+ +------+ +----+ 385 | CN3|--| P1-----+ -------------+------ P1 |--| CN3| 386 +----+ | C-PE1 P2-----+ | | C-PE2| +----+ 387 +----+ | P3-----+ Wide +------ P2 | +----+ 388 | CN2|--| | | area +------ P3 |--| CN1| 389 +-+--+ +---------+ | network | +------+ +-+--+ 390 \ | | / 391 \ +---------+ | all packets | +------+ / 392 +--| P1-----+ encrypted +------ P1 |-+ 393 | C-PE3 P2-----+ by | | C-PE4| 394 +----+ | P3-----+ IPsec SAs +------ P2 | +----+ 395 | CN1|--| P4-----+--------------+ | |--| CN2| 396 +----+ +---------+ +------+ +----+ 398 CN: Client Networks, which is same as Tenant Networks used by NVo3 400 Figure 2: Homogeneous SDWAN 402 One of the properties of a homogeneous SDWAN is that the SDWAN Local 403 Network Controller (RR)might be connected to C-PEs via an untrusted 404 public network, therefore, requiring a secure connection between RR 405 and C-PEs (TLS, DTLS, etc.). 407 Homogeneous SDWAN has some similarity to commonly deployed IPsec 408 VPN, albeit the IPsec VPN is usually point-to-point among a small 409 number of nodes and with heavy manual configuration for IPsec 410 between nodes. In contrast, an SDWAN network can have many edge 411 nodes and has a central controller to manage the configurations on 412 the edge nodes. 414 Existing Private VPNs (e.g., MPLS based) can use homogeneous SDWAN 415 to extend over the public network to remote sites to which the VPN 416 operator does not own or lease infrastructural connectivity, as 417 described in [SECURE-EVPN] and [SECURE-L3VPN] 419 3.3. Scenario #2: Hybrid WAN Underlay 421 Since IPsec requires additional processing power and the encrypted 422 traffic over the Internet does not have the premium SLA commonly 423 offered by Private VPNs, especially over a long distance, it is more 424 desired for traffic over private VPN to be forwarded without 425 encryption. The Hybrid WAN Underlay scenario refers to an SDWAN 426 network in which traffic over IP VPN is forwarded natively without 427 IPsec protection. IPsec tunnels protect only the traffic sent over 428 the public Internet. 430 One C-PE might have the Internet-facing WAN ports managed by 431 different ISPs/NSPs with the WAN ports' addresses assigned by the 432 corresponding ISPs/NSPs. Clients might have policies to specify: 434 1) Some flows can only be forwarded over private VPNs. 435 2) Some flows can be forwarded over either private VPNs or public 436 Inter. The packets over the public Internet are encrypted. 437 3) Some flows, especially Internet-bound browsing ones, can be handed 438 off to the Internet without any encryption. 440 Suppose a flow, traversing multiple segments such as A<->B<->C<->D, 441 has the Policy 2) above. The flow can cross different underlays in 442 different segments, such as over Private underlay between A<->B 443 without encryption or over the public Internet between B<->C 444 protected by an IPsec SA. 446 As shown in the figure below, C-PE-1 has two different types of 447 interfaces (A1 to Internet and A2 & A3 to VPN). The C-PE's loopback 448 address and the C-PE attached client addresses may or may not be 449 visible to the ISPs/NSPs. The WAN ports' addresses can be allocated 450 by the service providers or dynamically assigned (e.g., by DHCP). 452 +---+ 453 +--------------|RR |----------+ 454 / Untrusted +-+-+ \ 455 / \ 456 / \ 457 +----+ +---------+ packets encrypted over +------+ +----+ 458 | CN3|--| A1-----+ Untrusted +------ B1 |--| CN1| 459 +----+ | C-PE1 A2-\ | C-PE2| +----+ 460 +----+ | A3--+--+ +---+---B2 | +----+ 461 | CN2|--| | |PE+--------------+PE |---B3 |--| CN3| 462 +----+ +---------+ +--+ trusted +---+ +------+ +----+ 463 | WAN | 464 +----+ +---------+ +--+ packets +---+ +------+ +----+ 465 | CN1|--| C1--|PE| go natively |PE |-- D1 |--| CN1| 466 +----+ | C-PE3 C2--+--+ without encry+---+ | C-PE4| +----+ 467 | | +--------------+ | | 468 | | | | 469 +----+ | | without encrypt over | | +----+ 470 | CN2|--| C3--+---- Untrusted --+------D2 |--| CN2| 471 +----+ +---------+ +------+ +----+ 473 CN: Client Network 474 Figure 3: Hybrid SDWAN 476 Also, the connection between C-PEs and their Controller (RR) might 477 be via the untrusted public network. It is necessary to encrypt 478 the communication between RR and C-PEs, by TLS, DTLS, etc. 480 There could be multiple SDWAN edges (C-PEs) sharing common 481 property, such as a geographic location. Some applications over 482 SDWAN may need to traverse specific geographic areas for various 483 reasons, such as to comply with regulatory rules, to utilize 484 specific value-added services, or others. 486 Services may not be congruent, i.e., the packets from A-> B may 487 traverse one underlay network, and the packets from B -> A may go 488 over a different underlay. 490 3.4. Scenario #3: Private VPN PE based SDWAN 492 This scenario refers to the existing VPN (e.g., EVPN or IPVPN) being 493 expanded by adding extra ports facing the untrusted Internet for PEs 494 to offload low-priority traffic when the VPN paths are congested. 496 Throughout this document, this scenario is also called Internet 497 Offload for Private VPN, or PE-based SDWAN. 499 Here are some differences from the Hybrid Underlay scenario (Section 500 3.3): 502 - For MPLS-based VPN, PEs would have MPLS as payload 503 encapsulated within the IPsec tunnel egressing the Internet 504 WAN ports, MPLS-in-IP/GRE-in-IPsec. 505 - The BGP RR is connected to PEs in the same way as VPN, i.e., 506 via the trusted network. 508 The PE-based SDWAN can be used by VPN service providers to 509 temporarily increase bandwidth between sites when not sure if the 510 demand will sustain for an extended period or as a temporary 511 solution before the permanent infrastructure is built or leased. 513 +---+ 514 +======>|PE2| 515 // +---+ 516 // ^ 517 // || VPN 518 // VPN v 519 ++--+ ++-+ +---+ 520 |PE1| <====> |RR| <===> |PE3| 521 +-+-+ +--+ +-+-+ 522 | | 523 +--- Public Internet -- + 524 Offload 526 Figure 4: Additional Internet paths added to the VPN 528 4. Provisioning Model 530 4.1. Client Service Provisioning Model 532 Client service provisioning can follow the same approach as MPLS 533 VRFs. A client VPN can establish the communication policy by 534 specifying the Route Targets to be imported and exported. 535 Alternatively, traditional Match and Action ACLs can identify the 536 specific routes allowed or denied to or from the client VPN. 538 For an SDWAN edge node dedicated to one subscriber with one virtual 539 network, provisioning can be automated. All the prefixes attached to 540 the client port(s) of the edge node can be considered in one VRF, 541 and the RR can manage the policies for import/export of the VRF. 543 4.2. Policy Configuration 545 One of the characteristics of an SDWAN service is that packets can 546 be forwarded over multiple types of underlays. Policies are needed 547 to govern which underlay paths can carry an application flow, as 548 described by Section 8 of MEF70.1. An Application Flow consists of 549 packets that match specific criteria. For example, client-prefix-x 550 can only be mapped to MPLS topology. 552 4.3. IPsec related parameters Provisioning 554 For the IPsec tunnel to be established, the SDWAN edge nodes need to 555 support the common IPsec encryption algorithms (DES, 3DES, or AES), 556 the hash algorithm (SHA or MD5), and the DH groups. Each SDWAN edge 557 node can have the default supported values for those attributes or 558 get the attributes from its controller to minimize the 559 configuration. For BGP-controlled SDWAN, BGP UPDATE messages can 560 propagate each node's IPsec related attributes values for peers to 561 choose the common values supported, which is traditionally done by 562 IPsec IKEv2. 564 5. BGP Controlled SDWAN 566 5.1. BGP Walk Through for Homogeneous SDWAN 568 For the BGP-controlled homogeneous SDWAN, a C-PE can advertise its 569 attached client routes and the properties of the IPsec tunnel for 570 carrying the traffic towards the client routes in one BGP UPDATE 571 message. 573 In the Figure below, the BGP UPDATE message from C-PE2 to RR can 574 have the client routes encoded in the MP-NLRI Path Attribute and the 575 IPsec Tunnel associated information encoded in the Tunnel-Encap 576 [RFC9012] Path Attributes as described in the [SECURE-EVPN]. 578 +---+ 579 +---------|RR |----------+ 580 / Untrusted+---+ \ 581 / \ 582 C-PE1/ \ 583 +---------+ +------+ 584 --+---+---------------------------------> |-10.1.x.x/16 585 | / | |C-PE2 |- VLAN = 15 586 | / | +-----> | 587 --|/1.1.1.1 | | | |-12.1.1.x/24 588 +---------+ | +------+ 589 | 2.2.2.2 590 | 591 C-PE3 | 592 +---------+ | 593 --|---+---------------------------+ 594 | / | 595 | / | 596 --|/3.3.3.3 | 597 +---------+ 598 Figure 5: Homogeneous SDWAN 600 Alternatively, the C-PE2 can use two separate BGP UPDATE messages to 601 reduce the size of the BGP UPDATE messages, as described by Section 602 4 and 8 of [RFC9012]. UPDATE U1 has its Nexthop to the node loopback 603 address and is reclusively resolved to the IPsec tunnel detailed 604 attributes advertised by the UPDATE U2 for the Node Loopback 605 address: 607 Here are the details of the UPDATE messages: 609 - Suppose that a given packet P destined towards the client 610 addresses attached to C-PE2 (e.g., prefix 10.1.x.x/16) can be 611 carried by any IPsec tunnels terminated at C-PE2; 612 - The path along which P is to be forwarded is determined by BGP 613 UPDATE U1; 614 - UPDATE U1 does not have a Tunnel Encapsulation attribute; 615 - UPDATE U1 can include the Encapsulation Extended Community with 616 the option to have the Color Extended Community; 617 - The address of the next-hop of UPDATE U1 is router C-PE2; 618 - The best route to router C-PE2 is a BGP route advertised in 619 UPDATE U2; 620 - UPDATE U2 has a Tunnel Encapsulation attribute to describe the 621 IPsec detailed attributes. 623 UPDATE U1: 625 - MP-NLRI Path Attribute: 626 10.1.x.x/16 627 12.1.1.x/24 628 - Nexthop: 2.2.2.2 (C-PE2) 629 - Encapsulation Extended Community: Type = IPsec 631 UPDATE U2: 632 - MP-NLRI Path Attribute: 633 2.2.2.2 (C-PE2) 634 - Tunnel Encapsulation Path Attributes (as described in the 635 [SECURE-EVPN]) 637 If different client routes attached to C-PE2 need to be reached by 638 separate IPsec tunnels, the Color-Extended-Community [RFC9012] is 639 used to associate routes with the tunnels. See Section 8 of 640 [RFC9012]. 642 Suppose C-PE2 does not have the policy on the authorized peers for 643 the specific client routes. In that case, RR needs to check the 644 client routes policies to constrain the BGP UPDATE messages 645 propagation only to the remote authorized edge nodes. 647 5.2. BGP Walk Through for Hybrid WAN Underlay 649 In this scenario, some client routes can be forwarded by any tunnels 650 terminated at the edge node, and some client routes can be sent over 651 some specific tunnels (such as only MPLS VPN). 653 An edge node can use the Color Extended Community (Section 4 & 8 of 654 [RFC9012]) in its BGP UPDATE message to associate the client routes 655 with the specific tunnels. 657 For example, in Figure 5 above, suppose that Route 10.1.x.x/16 can 658 be carried by either MPLS or IPsec and Route 12.1.1.x/24 can only be 659 carried by MPLS; C-PE2 can use the following UPDATE messages: 661 UPDATE #1a for Route Route 10.1.x.x/16: 663 - MP-NLRI Path Attribute: 664 10.1.x.x/16 665 Nexthop: 2.2.2.2 (C-PE2) 666 - Encapsulation Extended Community: Type = SDWAN-Hybrid 667 - Color Extended Community: RED 669 UPDATE #1b for Route Route 12.1.1.x/24: 670 - MP-NLRI Path Attribute: 671 12.1.1.x/24 672 Nexthop: 2.2.2.2 (C-PE2) 673 - Encapsulation Extended Community: Type= SDWAN-Hybrid 674 - Color Extended Community: YELLOW 676 UPDATE #2a: for IPsec tunnels terminated at the node: 677 - MP-NLRI Path Attribute: 678 2.2.2.2 (C-PE2) 679 - Tunnel Encapsulation Path Attributes: TYPE=SDWAN-Hybrid 680 - Color Extended Community: RED 682 UPDATE #2b: for MPLS-in-GRE terminated at the node: 683 - MP-NLRI Path Attribute: 684 2.2.2.2 (C-PE2) 685 - Tunnel Encapsulation Path Attributes: TYPE=SDWAN-Hybrid 686 - Color Extended Community: YELLOW 688 SDWAN-Hybrid Tunnel Type is specified by [SDWAN-EDGE-Discovery]. 690 5.3. BGP Walk Through for Application Flow Based Segmentation 692 Suppose the application flow can be identified by the source or 693 destination IP addresses. In that case, constraining the BGP UPDATE 694 messages for the application only to the nodes that meet the 695 criteria of the application flow can achieve the Application Flow 696 based Segmentation described in Section 3.1.2. In the Figure below, 697 the following BGP Updates can be advertised to ensure that the 698 Payment Application only communicates with the Payment Gateway: 700 BGP UPDATE #1a from C-PE2 to RR for the P2P topology that is only 701 propagated to Payment GW node: 703 UPDATE #1a (only to the Payment GW node): 705 - MP-NLRI Path Attribute: 707 - 30.1.1.x/24 708 - Nexthop: 2.2.2.2 709 - Encapsulation extended community: Tunneltype=IPSEC 710 - Color Extended Community: BLUE 712 BGP UPDATE #1b from C-PE2 to RR for the routes to be reached by C- 713 PE1 and C-PE2: 715 - MP-NLRI Path Attribute: 716 - 10.1.x.x 717 - 12.4.x.x 718 - Nexthop:2.2.2.2 719 - Encapsulation extended community: Tunnel-type=IPSEC 720 - Color Extended Community: RED 722 BGP UPDATE #2 describes the IPsec detailed attributes for IPsec 723 tunnels terminated at C-PE2 2.2.2.2. 725 UPDATE #2a: for all IPsec SAs terminated at the node: 726 - MP-NLRI Path Attribute: 727 2.2.2.2 (C-PE2) 728 - Tunnel Encapsulation Path Attributes: TYPE=IPsec (for all IPsec 729 SAs) 730 - Color Extended Community: RED 732 UPDATE #2b: for the IPsec SA to the Payment Gateway: 733 - MP-NLRI Path Attribute: 734 2.2.2.2 (C-PE2) 735 - Tunnel Encapsulation Path Attributes: TYPE=IPsec (for the IPsec 736 SA to Payment GW). 737 - Color Extended Community: Blue 738 +-------+ 739 |Payment| 740 +-------->| GW |<-------+ 741 /Hub-spoke +-------+ \ 742 /for Payment App \ 743 C-PE1 / \ C-PE2 744 +------/--+ +----\-+ 745 --|-----/ | | -|- 30.1.1.x/24 746 + ---------------------------------------> |-10.1.x.x/16 747 | | | |- 748 | | +------------> |- 12.1.1.x/24 749 --|---------------------------+ | | 750 +---------+ +------> |- VLAN=25; 751 / +------+ 22.1.1.x/24 752 +---------+ / 753 --| -----------------------------+ 754 | C-PE3 | / 755 | | / 756 --| --------------------------+ 757 +---------+ 758 Figure 6: Application Based SDWAN Segmentation 760 5.4. Benefit of Using Recursive Next Hop Resolution 762 Using the Recursive Next Hop Resolution described in Section 8 of 763 [RFC9012], the clients' routes UPDATE messages become very compact, 764 and any changes of the underlay network tunnels attributes can be 765 advertised without client route update. This method is handy when 766 the underlay tunnels are IPsec based, which requires periodic 767 message exchange for the pairwise re-keying process. 769 5.5. Why BGP as Control Plane for SDWAN? 771 For an SDWAN network with a small number of nodes, the traditional 772 hub & spoke model utilizing NHRP or DSVPN/DMVPN protocol had worked 773 reasonably well. DSVPN/DMVPN has a hub node (or controller) managing 774 the edge nodes, including local & public addresses and tunnel 775 identifiers mapping. However, for a sizeable SDWAN network, say more 776 than 100 nodes with different underlays, the traditional approach 777 becomes very messy, complex, and error prone. 779 Here are some of the compelling reasons for using BGP: 781 - With a secure management channel already established between an 782 edge node and RR, RR can perform the peer authentication on behalf 783 of the edge node. Not only RR has policies on peer communication, 784 but RR also has the built-in capability to constrain the propagation 785 of the UPDATE messages to the authorized edge nodes [RFC4684]. 787 - When multiple IPsec tunnels are established between two pairwise 788 edge nodes, BGP Tunnel Attribute Update can associate multiple IPsec 789 tunnels with the client routes. In traditional IPsec VPN, separate 790 routing protocols must run in parallel in each IPsec Tunnel if the 791 client routes can be load shared among the IPsec tunnels. 793 - The IPsec tunnel's traffic selector or admission control can be 794 inherently realized by specifying importing/exporting the Route 795 Targets representing the SDWAN VPNs. 797 6. SDWAN Forwarding Model 799 This section describes how client traffic is forwarded in BGP 800 Controlled SDWAN for the use cases described in Section 3. 802 The procedures described in Section 6 of RFC8388 are applicable for 803 the SDWAN client traffic. Like the BGP-based VPN/EVPN client routes 804 UPDATE message, Route Target can distinguish routes from different 805 clients. 807 6.1. Forwarding Model for Homogeneous SDWAN 809 6.1.1. Network and Service Startup Procedures 811 A single IPsec security association (SA) protects data in one 812 direction. Under the homogeneous SDWAN Scenario, two SAs must be 813 present to secure traffic in both directions between two C-PE nodes, 814 two client ports, or two prefixes. Using Figure 2 of Section 3.2 as 815 an example, for client CN2 attached to C-PE1, C-PE3, and C-PE4 to 816 have full-mesh connection, six one-directional IPsec SAs must be 817 established: C-PE1 <-> C-PE3; C-PE1 <-> C-PE4; C-PE3 <-> C-PE4. 819 SDWAN services to clients can be IP-based or Ethernet-based. An 820 SDWAN node can learn client routes from the client-facing ports via 821 OSPF, RIP, BGP or Statically configuration for its IP-based 822 services. For Layer-2 SDWAN services, the relevant EVPN parameters, 823 such as the ESI (Ethernet Segment Identifier), EVI, CE-VID to EVI 824 mapping, can be configured in the same way as EVPN described in 825 RFC8388. 827 Instead of running IGP within each IPsec tunnel as done by the 828 traditional IPsec VPN, BGP UPDATE messages propagate the client 829 routes attached to SDWAN edge nodes. 831 In addition, BGP-RR (SDWAN Controller) facilitates the IPsec SA 832 establishment and rekey management described in [SECURE-EVPN]. The 833 Controller manages how client's routes are associated with 834 individual IPsec SA. Therefore, it is no longer necessary to 835 manually configure the IPsec tunnel's endpoint addresses on each 836 SDWAN edge node and set up policies for the allowed client prefixes. 838 6.1.2. Packet Walk-Through 840 For an IPsec SA terminated at a C-PE node, multiple client routes 841 can be multiplexed in the IPsec SA (or tunnel). Traffic to the 842 client prefixes is encapsulated in an inner tunnel, such as GRE or 843 VxLAN, carried by the IPsec SA ESP tunnel. Different client traffic 844 can be differentiated by a unique value in the inner encapsulation 845 key or ID field. 847 For unicast packets forwarding: 849 the C-PE node address (or loopback address) acts as the Next Hop 850 addresses for the prefixes attached to the C-PE nodes. 852 C-PE Node-based IPsec tunnel is inherently protected when the C-PE 853 has multiple WAN ports to different underlay paths. As shown in 854 Figure 2, when one of the underlay paths fails, the IPsec traffic 855 can be forwarded to or received from a different physical port. 857 When a C-PE receives a packet from its client port, the packet is 858 encapsulated inside the IPsec SA, whose destination address 859 matches the Next Hop address of the packet's destination and 860 forwarded to the target C-PE. 862 When a C-PE receives an IPsec encrypted packet from its WAN ports, 863 it decrypts the packet and forwards the inner packet to the client 864 port based on the inner packet's destination address. 866 For multicast packets forwarding: 868 IPsec was created to be a security protocol between two and only 869 two devices, so multicast service using IPsec is problematic. An 870 IPsec peer encrypts a packet so that only one other IPsec peer can 871 successfully perform the de-encryption. A straight way to forward 872 a multicast packet for the homogeneous SDWAN is to encapsulate the 873 multicast packet in separate unicast IPsec SA tunnels. More 874 optimized forwarding multicast packets for the homogeneous SDWAN 875 is out of the scope of this document. 877 6.2. Forwarding Model for Hybrid Underlay SDWAN 879 In this scenario as shown in Figure 3 of Section 3.3, traffic 880 forwarded over the trusted VPN paths can be native (i.e., 881 unencrypted). The traffic forwarded over untrusted networks need to 882 be protected by IPsec SA. 884 6.2.1. Network and Service Startup Procedures 886 Infrastructure setup: The proper MPLS infrastructure must be set up 887 among the edge nodes, i.e., the C-PE1/C-PE2/C-PE3/C-PE4 of Figure 3. 888 The IPsec SA between WAN ports or nodes must be set up as well. 889 IPsec SA related attributes on edge nodes can be distributed by BGP 890 UPDATE messages as described in Section 5. 892 There could be policies governing how flows can be forwarded, as 893 specified by MEF70.1. For example, "Private-only" indicates that 894 the flows can only traverse the MPLS VPN underlay paths. 896 6.2.2. Packet Walk-Through 898 For unicast packets forwarding: 900 Upon receiving a packet from a client port, if the packet belongs 901 to a flow that can only be forwarded over the MPLS VPN, the 902 forwarding processing is the same as the MPLS VPN. Otherwise, the 903 C-PE node can make the local decision in choosing the least cost 904 path, including the prior established MPLS paths and IPsec 905 Tunnels, to forward the packet. Packets forwarded over the trusted 906 MPLS VPN can be native without any additional encryption, while 907 the packets sent over the untrusted networks need to be encrypted 908 by IPsec SA. 910 For a C-PE with multiple WAN ports provided by different ISPs, 911 separate IPsec SAs can be established for the different WAN ports. 912 In this case, the C-PE have multiple IPsec tunnels in addition to 913 the MPLS path to choose from to forward the packets from the 914 client ports. 916 If the IPsec SA is chosen, the packet is encapsulated by the IPsec 917 inner packet header and encrypted by the IPsec SA before 918 forwarding to the WAN. 920 For packets received from a MPLS path, processing is the same as 921 MPLS VPN. 923 For IPsec SA encrypted packets received from the WAN ports, the 924 packets are decrypted, and the inner payload is decapsulated and 925 forward per the forwarding table of the C-PE. For all packets from 926 the Internet-facing WAN ports, the additional anti-DDoS mechanism 927 has to be enabled to prevent potential attacks from the Internet- 928 facing ports. Control Plane should not learn routes from the 929 Internet-facing WAN ports. 931 +---+ 932 +--------------|RR |----------+ 933 / +-+-+ \ 934 / \ 935 / \ 936 +----+ +---------+ packets encrypted over +--------+ +----+ 937 | CN3|--| A1-----+ Untrusted +------ B1 |--| CN1| 938 +----+ | C-PE-a A2-----+ +-------B2 C-PE-b| +----+ 939 |10.1.1.1 | |10.1.2.1| 940 +----+ | | +--+ +---+ | | +----+ 941 | CN2|--| A3 |PE+--------------+PE |---B3 |--| CN3| 942 +----+ +---------+ +--+ trusted +---+ +--------+ +----+ 943 | VPN | 944 +--------------+ 945 Figure 8: SDWAN with Hybrid Underlays 947 For multicast packets forwarding: 949 For multicast traffic, MPLS multicast [RFC6513, RFC6514, or 950 RFC7988] can be used to forward multicast traffic. 952 If IPsec tunnels are chosen for a multicast packet, the packet is 953 encapsulated and encrypted by multiple separate IPsec tunnels to 954 the desired destinations. 956 6.3. Forwarding Model for PE based SDWAN 958 6.3.1. Network and Service Startup Procedures 960 In this scenario, all PEs have secure interfaces facing the clients 961 and facing the MPLS backbone, with some PEs having extra connections 962 by untrusted public Internet. The public Internet paths are for 963 offloading low priority traffic when the MPLS paths get congested. 964 The PEs are already connected to their RRs, and the configurations 965 for the clients and policies are already established. 967 6.3.2. Packet Walk-Through 969 For PEs to offload some MPLS packets to the Internet path, each MPLS 970 packet is wrapped by an outer IP header as MPLS-in-IP or MPLS-in-GRE 972 [RFC4023]. The outer IP address can be an interface address or the 973 PE's loopback address. 975 When IPsec Tunnel mode is used to protect an MPLS-in-IP packet, the 976 entire MPLS-in-IP packet is placed after the IPsec tunnel header. 978 When IPsec transport mode is used to protect the MPLS packet, the 979 MPLS-in-IP packet's IP header becomes the outer IP header of the 980 IPsec packet, followed by an IPsec header, and then followed by the 981 MPLS label stack. The IPsec header must set the payload type to MPLS 982 by using the IP protocol number specified in section 3 of RFC4023. 984 If IPsec transport mode is applied to an MPLS-in-GRE packet, the GRE 985 header follows the IPsec header. 987 The IPsec SA's endpoints should not be the client-facing interface 988 addresses unless the traffic to/from those clients always goes 989 through the IPsec SA even when the MPLS backbone has enough capacity 990 to transport the traffic. 992 When the PEs' Internet-facing ports are behind the NAT [RFC3715], an 993 outer UDP field can be added outside the encrypted payload 994 [RFC3948]. Three UDP ports must be open on the PEs: UDP port 4500 995 (used for NAT traversal), UDP port 500 (used for IKE), and IP 996 protocol 50 (ESP). IPsec IKE (Internet Key Exchange) between the two 997 PEs would be over the NAT [RFC3947] as well. 999 Upon receiving a packet from a client port, the forwarding 1000 processing is the same as the MPLS VPN. If the MPLS backbone path to 1001 the destination is deemed congested, the IPsec tunnel towards the 1002 target PEs is used to encrypt the MPLS-in-IP packet. 1004 Upon receiving a packet from the Internet-facing WAN port, the 1005 packet is decrypted, and the inner MPLS payload is extracted to be 1006 sent to the MPLS VPN engine. 1008 Same as Scenario #2, the additional anti-DDoS mechanism must be 1009 enabled to prevent potential attacks from the Internet-facing port. 1010 Control Plane should not learn routes from the Internet-facing WAN 1011 ports. 1013 7. Manageability Considerations 1015 BGP-controlled SDWAN utilizes the BGP RR to facilitate the routes 1016 and underlay properties distribution among the authorized edge 1017 nodes. With RR having the preconfigured policies about the 1018 authorized peers, the peer-wise authentications of the IPsec IKE 1019 (Internet Key Exchange) are significantly simplified. 1021 8. Security Considerations 1023 Adding an Internet-facing WAN port to a C-PE can introduce the 1024 following security risks: 1026 1) Potential DDoS attacks from the Internet-facing ports. 1028 2) Potential risk of provider VPN network being injected with 1029 illegal traffic from the Internet-facing WAN ports. 1031 Therefore, the additional anti-DDoS mechanism must be enabled on all 1032 Internet-facing ports to prevent potential attacks from those ports. 1033 Control Plane should not learn any routes from the Internet-facing 1034 WAN ports. 1036 9. IANA Considerations 1038 No Action is needed. 1040 10. References 1042 10.1. Normative References 1044 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1045 Requirement Levels", BCP 14, RFC 2119, March 1997. 1047 [RFC4364] E. rosen, Y. Rekhter, "BGP/MPLS IP Virtual Private 1048 networks (VPNs)", Feb 2006. 1050 [RFC7296] C. Kaufman, et al, "Internet Key Exchange Protocol Version 1051 2 (IKEv2)", Oct 2014. 1053 [RFC7432] A. Sajassi, et al, "BGP MPLS-Based Ethernet VPN", Feb 1054 2015. 1056 [RFC8365] A. Sajassi, et al, "A network Virtualization Overlay 1057 Solution Using Ethernet VPN (EVPN)", March 2018. 1059 [RFC9012] K.Patel, et al "The BGP Tunnel Encapsulation Attribute", 1060 RFC9012, April 2021. 1062 10.2. Informative References 1064 [RFC8192] S. Hares, et al, "Interface to Network Security Functions 1065 (I2NSF) Problem Statement and Use Cases", July 2017 1067 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation Subsequent 1068 Address Family Identifier (SAFI) and the BGP Tunnel 1069 Encapsulation Attribute", April 2009. 1071 [RFC8388] J. Rabadan, et al, "Usage and Applicability of BGP MPLS- 1072 Based Ethernet VPN", May 2018. 1074 [Net2Cloud-Gap] L. Dunbar, A. Malis, C. Jacquenet, "Gap Analysis of 1075 Interconnecting Underlay with Cloud Overlay", draft-dm- 1076 net2cloud-gap-analysis-02, work in progress, Oct. 2018. 1078 [SDWAN-EDGE-Discovery] L. Dunbar, S. Hares, R. Raszuk, K. Majumdar, 1079 "BGP UPDATE for SDWAN Edge Discovery", draft-dunbar-idr- 1080 sdwan-edge-discovery-01, work-in-progress, Nov 2020. 1082 [VPN-over-Internet] E. Rosen, "Provide Secure Layer L3VPNs over 1083 Public Infrastructure", draft-rosen-bess-secure-l3vpn-00, 1084 work-in-progress, July 2018 1086 [DMVPN] Dynamic Multi-point VPN: 1087 https://www.cisco.com/c/en/us/products/security/dynamic- 1088 multipoint-vpn-dmvpn/index.html 1090 [DSVPN] Dynamic Smart VPN: 1091 http://forum.huawei.com/enterprise/en/thread-390771-1- 1092 1.html 1094 [SECURE-EVPN] A. Sajassi, et al, "Secure EVPN", draft-sajassi-bess- 1095 secure-evpn-01, Work-in-progress, March 2019. 1097 [SECURE-L3VPN] E. Rosen, R. Bonica, "Secure Layer L3VPN over Public 1098 Infrastructure", draft-rosen-bess-secure-l3vpn-00, Work- 1099 in-progress, June 2018. 1101 [ITU-T-X1036] ITU-T Recommendation X.1036, "Framework for creation, 1102 storage, distribution and enforcement of policies for 1103 network security", Nov 2007. 1105 [Net2Cloud-Problem] L. Dunbar and A. Malis, "Seamless Interconnect 1106 Underlay to Cloud Overlay Problem Statement", draft-rtgwg- 1107 net2cloud-problem-statement-12, March 2022 1109 [Net2Cloud-gap] L. Dunbar, A. Malis, and C. Jacquenet, "Gap Analysis 1110 of Interconnecting Underlay with Cloud Overlay", draft- 1111 rtgwg-net2cloud-gap-analysis-07, work-in-progress, July 1112 2020. 1114 11. Acknowledgments 1116 Acknowledgements to Adrian Farrel, Joel Halpern, John Scudder, 1117 Darren Dukes, Andy Malis and Donald Eastlake for their review and 1118 contributions. 1120 This document was prepared using 2-Word-v2.0.template.dot. 1122 Authors' Addresses 1124 Linda Dunbar 1125 Futurewei 1126 Email: ldunbar@futurewei.com 1128 James Guichard 1129 Futurewei 1130 Email: james.n.guichard@futurewei.com 1132 Ali Sajassi 1133 Cisco 1134 Email: sajassi@cisco.com 1136 John Drake 1137 Juniper 1138 Email: jdrake@juniper.net 1140 Basil Najem 1141 Bell Canada 1142 Email: basil.najem@bell.ca 1144 David Carrel 1145 IPsec Research 1146 Email: carrel@ipsec.org 1148 Ayan Banerjee 1149 Cisco 1150 Email: ayabaner@cisco.com