idnits 2.17.00 (12 Aug 2021) /tmp/idnits58840/draft-ietf-rtgwg-net2cloud-problem-statement-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 5 instances of too long lines in the document, the longest one being 121 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 3, 2019) is 1053 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'ITU-T-X1036' is defined on line 677, but no explicit reference was found in the text Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft A. Malis 3 Intended status: Informational Futurewei 4 Expires: Dec 2019 C. Jacquenet 5 Orange 6 M. Toy 7 Verizon 8 July 3, 2019 10 Dynamic Networks to Hybrid Cloud DCs Problem Statement 11 draft-ietf-rtgwg-net2cloud-problem-statement-03 13 Abstract 15 This document describes the problems that enterprises face today 16 when connecting their branch offices to dynamic workloads in third 17 party data centers (a.k.a. Cloud DCs). 19 It examines some of the approaches interconnecting cloud DCs with 20 enterprises' on-premises DCs & branch offices. This document also 21 describes some of the network problems that many enterprises face 22 when they have workloads & applications & data split among hybrid 23 data centers, especially for those enterprises with multiple sites 24 that are already interconnected by VPNs (e.g., MPLS L2VPN/L3VPN). 26 Current operational problems are examined to determine whether there 27 is a need to improve existing protocols or whether a new protocol is 28 necessary to solve them. 30 Status of this Memo 32 This Internet-Draft is submitted in full conformance with the 33 provisions of BCP 78 and BCP 79. 35 Internet-Drafts are working documents of the Internet Engineering 36 Task Force (IETF), its areas, and its working groups. Note that 37 other groups may also distribute working documents as Internet- 38 Drafts. 40 Internet-Drafts are draft documents valid for a maximum of six 41 months and may be updated, replaced, or obsoleted by other documents 42 at any time. It is inappropriate to use Internet-Drafts as 43 reference material or to cite them other than as "work in progress." 45 The list of current Internet-Drafts can be accessed at 46 http://www.ietf.org/ietf/1id-abstracts.txt 48 The list of Internet-Draft Shadow Directories can be accessed at 49 http://www.ietf.org/shadow.html 51 This Internet-Draft will expire on January 3, 2009. 53 Copyright Notice 55 Copyright (c) 2019 IETF Trust and the persons identified as the 56 document authors. All rights reserved. 58 This document is subject to BCP 78 and the IETF Trust's Legal 59 Provisions Relating to IETF Documents 60 (http://trustee.ietf.org/license-info) in effect on the date of 61 publication of this document. Please review these documents 62 carefully, as they describe your rights and restrictions with 63 respect to this document. Code Components extracted from this 64 document must include Simplified BSD License text as described in 65 Section 4.e of the Trust Legal Provisions and are provided without 66 warranty as described in the Simplified BSD License. 68 Table of Contents 70 1. Introduction...................................................3 71 1.1. On the evolution of Cloud DC connectivity.................3 72 1.2. The role of SD-WAN techniques in Cloud DC connectivity....4 73 2. Definition of terms............................................4 74 3. Current Practices in Interconnecting Enterprise Sites with Cloud 75 DCs...............................................................5 76 3.1. Multiple connection to workloads in a Cloud DC............5 77 3.2. Interconnect to Hybrid Cloud DCs..........................7 78 3.3. Connecting workloads among hybrid Cloud DCs...............8 79 4. Desired Properties for Networks that interconnect Hybrid Clouds9 80 5. Problems with MPLS-based VPNs extending to Hybrid Cloud DCs...10 81 6. Problem with using IPsec tunnels to Cloud DCs.................11 82 6.1. Complexity of multi-point any-to-any interconnection.....12 83 6.2. Poor performance over long distance......................12 84 6.3. Scaling Issues with IPsec Tunnels........................13 86 7. Problems of Using SD-WAN to connect to Cloud DCs..............13 87 7.1. SD-WAN among branch offices vs. interconnect to Cloud DCs14 88 8. End-to-End Security Concerns for Data Flows...................16 89 9. Requirements for Dynamic Cloud Data Center VPNs...............16 90 10. Security Considerations......................................17 91 11. IANA Considerations..........................................17 92 12. References...................................................17 93 12.1. Normative References....................................17 94 12.2. Informative References..................................17 95 13. Acknowledgments..............................................18 97 1. Introduction 99 1.1. On the evolution of Cloud DC connectivity 101 The ever-increasing use of cloud applications for communication 102 services change the way corporate business works and shares 103 information. Such cloud applications use resources hosted in third 104 party DCs that also host services for other customers. 106 With the advent of widely available third party cloud DCs in diverse 107 geographic locations and the advancement of tools for monitoring and 108 predicting application behaviors, it is technically feasible for 109 enterprises to instantiate applications and workloads in locations 110 that are geographically closest to their end-users. Such proximity 111 improves end-to-end latency and overall user experience. Conversely, 112 an enterprise can easily shutdown applications and workloads 113 whenever end-users are in motion (thereby modifying the networking 114 connection of subsequently relocated applications and workloads). In 115 addition, an enterprise may wish to take advantage of more and more 116 business applications offered by third party private cloud DCs. 118 Most of those enterprise branch offices & on-premises data centers 119 are already connected via VPNs, such as MPLS-based L2VPNs and 120 L3VPNs. Then connecting to the cloud-hosted resources may not be 121 straightforward if the provider of the VPN service does not have 122 direct connections to the corresponding cloud DCs. Under those 123 circumstances, the enterprise can upgrade the CPEs deployed in its 124 various premises to utilize SD-WAN techniques to reach cloud 125 resources (without any assistance from the VPN service provider), or 126 wait for their VPN service provider to make new agreements with data 127 center providers to connect to the cloud resources. Either way has 128 additional infrastructure and operational costs. 130 In addition, more enterprises are moving towards hybrid cloud DCs, 131 i.e. owned or operated by different Cloud operators, to maximize the 132 benefits of geographical proximity, elasticity and special features 133 offered by different cloud DCs. 135 1.2. The role of SD-WAN techniques in Cloud DC connectivity 137 This document discusses the issues associated with connecting 138 enterprise to their workloads/applications instantiated in multiple 139 third-party data centers (a.k.a. Cloud DCs). Very often, the actual 140 Cloud DCs that host the workloads/applications can be transient. . 142 SD-WAN, initially launched to maximize bandwidths between locations 143 by aggregating multiple paths managed by different service 144 providers, has expanded to include flexible, on-demand, application- 145 based connections established over any networks to access dynamic 146 workloads in Cloud DCs. 148 As a consequence, this document discusses the use of SD-WAN 149 techniques as a means to improve enterprise-to-cloud DC 150 connectivity. 152 2. Definition of terms 154 Cloud DC: Third party Data Centers that usually host applications 155 and workload owned by different organizations or 156 tenants. 158 Controller: Used interchangeably with SD-WAN controller to manage 159 SD-WAN overlay path creation/deletion and monitoring the 160 path conditions between two or more sites. 162 DSVPN: Dynamic Smart Virtual Private Network. DSVPN is a secure 163 network that exchanges data between sites without 164 needing to pass traffic through an organization's 165 headquarter virtual private network (VPN) server or 166 router. 168 Heterogeneous Cloud: applications and workloads split among Cloud 169 DCs owned or managed by different operators. 171 Hybrid Clouds: Hybrid Clouds refers to an enterprise using its own 172 on-premises DCs in addition to Cloud services provided 173 by one or more cloud operators. (e.g. AWS, Azure, 174 Google, Salesforces, SAP, etc). 176 SD-WAN: Software Defined Wide Area Network. In this document, 177 "SD-WAN" refers to the solutions of pooling WAN 178 bandwidth from multiple underlay networks to get better 179 WAN bandwidth management, visibility & control. When the 180 underlay networks are private networks, traffic can 181 traverse without additional encryption; when the 182 underlay networks are public, such as Internet, some 183 traffic needs to be encrypted when traversing through 184 (depending on user provided policies). 186 VPC: Virtual Private Cloud is a virtual network dedicated to 187 one client account. It is logically isolated from other 188 virtual networks in a Cloud DC. Each client can launch 189 his/her desired resources, such as compute, storage, or 190 network functions into his/her VPC. Most Cloud 191 operators' VPCs only support private addresses, some 192 support IPv4 only, others support IPv4/IPv6 dual stack. 194 3. Current Practices in Interconnecting Enterprise Sites with Cloud DCs 196 3.1. Multiple connection to workloads in a Cloud DC 198 Most Cloud operators offer some type of network gateway through 199 which an enterprise can reach their workloads hosted in the Cloud 200 DCs. For example, AWS (Amazon Web Services) offers the following 201 options to reach workloads in AWS Cloud DCs: 203 - AWS Internet gateway allows communication between instances in 204 AWS VPC and the internet. 205 - AWS Virtual gateway (vGW) where IPsec tunnels [RFC6071] are 206 established between an enterprise's own gateway and AWS vGW, so 207 that the communications between those gateways can be secured 208 from the underlay (which might be the public Internet). 209 - AWS Direct Connect, which allows enterprises to purchase direct 210 connect from network service providers to get a private leased 211 line interconnecting the enterprises gateway(s) and the AWS 212 Direct Connect routers. In addition, an AWS Transit Gateway 213 (https://aws.amazon.com/transit -gateway/) can be used to interconnect 214 multiple VPCs in different Availability Zones. AWS Transit 215 Gateway acts as a hub that controls how traffic is forwarded 216 among all the connected networks which act like spokes. 218 As an example, some branch offices of an enterprise can connect to 219 over the Internet to reach AWS's vGW via IPsec tunnels. Other branch 220 offices of the same enterprise can connect to AWS DirectConnect via 221 a private network (without any encryption). ). It is important for 222 enterprises to be able to observe the specific behaviors when 223 connected by different connections. 225 Figure below shows an example of some tenants' workloads are 226 accessible via a virtual router connected by AWS Internet Gateway; 227 some are accessible via AWS vGW, and others are accessible via AWS 228 Direct Connect. vR1 uses IPsec to establish secure tunnels over the 229 Internet to avoid paying extra fees for the IPsec features provided 230 by AWS vGW. Some tenants can deploy separate virtual routers to 231 connect to internet traffic and to traffic from the secure channels 232 from vGW and DirectConnect, e.g. vR1 & vR2. Others may have one 233 virtual router connecting to both types of traffic. Customer Gateway 234 can be customer owned router or ports physically connected to AWS 235 Direct Connect GW. 237 +------------------------+ 238 | ,---. ,---. | 239 | (TN-1 ) ( TN-2)| 240 | `-+-' +---+ `-+-' | 241 | +----|vR1|----+ | 242 | ++--+ | 243 | | +-+----+ 244 | | /Internet\ For External 245 | +-------+ Gateway +---------------------- 246 | \ / to reach via Internet 247 | +-+----+ 248 | | 249 | ,---. ,---. | 250 | (TN-1 ) ( TN-2)| 251 | `-+-' +---+ `-+-' | 252 | +----|vR2|----+ | 253 | ++--+ | 254 | | +-+----+ 255 | | / virtual\ For IPsec Tunnel 256 | +-------+ Gateway +---------------------- 257 | | \ / termination 258 | | +-+----+ 259 | | | 260 | | +-+----+ +------+ 261 | | / \ For Direct /customer\ 262 | +-------+ Gateway +----------+ gateway | 263 | \ / Connect \ / 264 | +-+----+ +------+ 265 | | 266 +------------------------+ 268 Figure 1: Examples of Multiple Cloud DC connections. 270 3.2. Interconnect to Hybrid Cloud DCs 272 It is likely that hybrid designs will become the rule for cloud 273 services, as more enterprises see the benefits of integrating public 274 and private cloud infrastructures. However, enabling the growth of 275 hybrid cloud deployments in the enterprise requires fast and safe 276 interconnection between public and private cloud services. 277 For an enterprise to connect to applications & workloads hosted in 278 multiple Cloud DCs, the enterprise can use IPsec tunnels established 279 over the Internet or a (virtualized) leased line service to connect 280 its on-premises gateways to each of the Cloud DC's gateways, virtual 281 routers instantiated in the Cloud DCs, or any other suitable design 282 (including a combination thereof). 284 Some enterprises prefer to instantiate their own virtual 285 CPEs/routers inside the Cloud DC to connect the workloads within the 286 Cloud DC. Then an overlay path is established between customer 287 gateways to the virtual CPEs/routers for reaching the workloads 288 inside the cloud DC. 290 3.3. Connecting workloads among hybrid Cloud DCs 292 There are multiple approaches to interconnect workloads among 293 different Cloud DCs: 295 a) Utilize Cloud DC provided inter/intra-cloud connectivity 296 services (e.g., AWS Transit Gateway) to connect workloads 297 instantiated in multiple VPCs. Such services are provided with 298 the cloud gateway to connect to external networks (e.g., AWS 299 DirectConnect Gateway). 300 b) Hairpin all traffic through the customer gateway, meaning all 301 workloads are directly connected to the customer gateway, so 302 that communications among workloads within one Cloud DC have to 303 traverse through the customer gateway. 304 c) Establish direct tunnels among different VPCs (Virtual Private 305 Clouds) via client's own virtual routers instantiated within 306 Cloud DCs. DMVPN (Dynamic Multipoint Virtual Private Network) 307 or DSVPN (Dynamic Smart VPN) techniques can be used to 308 establish direct Multi-point-to-Point or multi-point-to multi- 309 point tunnels among those client's own virtual routers. 311 Approach a) usually does not work if Cloud DCs are owned and managed 312 by different Cloud providers. 314 Approach b) creates additional transmission delay plus incurring 315 cost when exiting Cloud DCs. 317 For the Approach c), DMVPN or DSVPN use NHRP (Next Hop Resolution 318 Protocol) [RFC2735] so that spoke nodes can register their IP 319 addresses & WAN ports with the hub node. The IETF ION 320 (Internetworking over NBMA (non-broadcast multiple access) WG 321 standardized NHRP for connection-oriented NBMA network (such as ATM) 322 network address resolution more than two decades ago. 324 There are many differences between virtual routers in Public Cloud 325 DCs and the nodes in an NBMA network. NHRP cannot be used for 326 registering virtual routers in Cloud DCs unless an extension of such 327 protocols is developed for that purpose. Therefore, DMVPN and/or 328 DSVPN cannot be used directly for connecting workloads in hybrid 329 Cloud DCs. 331 Other protocols such as BGP can be used, as described in [BGP- 332 SDWAN]. 334 4. Desired Properties for Networks that interconnect Hybrid Clouds 335 The networks that interconnect hybrid cloud DCs must address the 336 following requirements: 337 - High availability to access all workloads in the desired cloud 338 DCs. 339 Many enterprises include cloud infrastructures in their 340 disaster recovery strategy, e.g., by enforcing periodic backup 341 policies within the cloud, or by running backup applications in 342 the Cloud, etc. Therefore, the connection to the cloud DCs may 343 not be permanent, but rather needs to be on-demand. 345 - Global reachability from different geographical zones, thereby 346 facilitating the proximity of applications as a function of the 347 end users' location, to improve latency. 348 - Elasticity: prompt connection to newly instantiated 349 applications at Cloud DCs when end-users' usages increase and 350 prompt release of connection after applications at locations 351 being removed when demands change. 352 Some enterprises have front-end web portals running in cloud 353 DCs and database servers in their on-premises DCs. Those Front- 354 end web portals need to be reachable from the public Internet. 355 The backend connection to the sensitive data in database 356 servers hosted in the on-premises DCs might need secure 357 connections. 359 - Scalable security management. IPsec is commonly used to 360 interconnect cloud gateways with CPEs deployed in the 361 enterprise premises. For enterprises with a large number or 362 branch offices, managing the IPsec's Security Associations 363 among many nodes can be very difficult. 365 5. Problems with MPLS-based VPNs extending to Hybrid Cloud DCs 367 Traditional MPLS-based VPNs have been widely deployed as an 368 effective way to support businesses and organizations that require 369 network performance and reliability. MPLS shifted the burden of 370 managing a VPN service from enterprises to service providers. The 371 CPEs attached to MPLS VPNs are also simpler and less expensive, 372 since they do not need to manage routes to remote sites; they simply 373 pass all outbound traffic to the MPLS VPN PEs to which the CPEs are 374 attached (albeit multi-homing scenarios require more processing 375 logic on CPEs). MPLS has addressed the problems of scale, 376 availability, and fast recovery from network faults, and 377 incorporated traffic-engineering capabilities. 379 However, traditional MPLS-based VPN solutions are sub-optimized for 380 connecting end-users to dynamic workloads/applications in cloud DCs 381 because: 383 - The Provider Edge (PE) nodes of the enterprise's VPNs might not 384 have direct connections to third party cloud DCs that are used 385 for hosting workloads with the goal of providing an easy access 386 to enterprises' end-users. 388 - It usually takes some time to deploy provider edge (PE) routers 389 at new locations. When enterprise's workloads are changed from 390 one cloud DC to another (i.e., removed from one DC and re- 391 instantiated to another location when demand changes), the 392 enterprise branch offices need to be connected to the new cloud 393 DC, but the network service provider might not have PEs located 394 at the new location. 396 One of the main drivers for moving workloads into the cloud is 397 the widely available cloud DCs at geographically diverse 398 locations, where apps can be instantiated so that they can be 399 as close to their end-users as possible. When the user base 400 changes, the applications may be migrated to a new cloud DC 401 location closest to the new user base. 403 - Most of the cloud DCs do not expose their internal networks, so 404 the MPLS-based VPNs can only reach Cloud DC's Gateways, not to 405 the workloads hosted inside. Even with AWS DirectConnect, the 406 connection only reaches the AWS DirectConnect Gateway. AWS 407 DirectConnect Gateway uses BGP to exchange all routes with 408 devices located behind the gateway, even including routes of 409 applications that might be physically located in different 410 geographical locations. There is no visibility on how the 411 applications/workloads are interconnected within a Cloud DC or 412 across multiple Cloud DCs. 414 - Extensive usage of Overlay by Cloud DCs: 416 Many cloud DCs use an overlay to connect their gateways to the 417 workloads located inside the DC. There is currently no standard 418 that specifies the interworking between the Cloud Overlay and 419 the enterprise' existing underlay networks. One of the 420 characteristics of overlay networks is that some of the WAN 421 ports of the edge nodes connect to third party networks. There 422 is therefore a need to propagate WAN port information to remote 423 authorized peers in third party network domains in addition to 424 route propagation. Such an exchange cannot happen before 425 communication between peers is properly secured. 427 Another roadblock is the lack of a standard way to express and 428 enforce consistent security policies for workloads that not only use 429 virtual addresses, but in which are also very likely hosted in 430 different locations within the Cloud DC [RFC8192]. The current VPN 431 path computation and bandwidth allocation schemes may not be 432 flexible enough to address the need for enterprises to rapidly 433 connect to dynamically instantiated (or removed) workloads and 434 applications regardless of their location/nature (i.e., third party 435 cloud DCs). 437 6. Problem with using IPsec tunnels to Cloud DCs 438 As described in the previous section, many Cloud operators expose 439 their gateways for external entities (which can be enterprises 440 themselves) to directly establish IPsec tunnels. Enterprises can 441 also instantiate virtual routers within Cloud DCs to connect to 442 their on-premises devices via IPsec tunnels. If there is only one 443 enterprise location that needs to reach the Cloud DC, an IPsec 444 tunnel is a very convenient solution. 446 However, many medium-to-large enterprises usually have multiple 447 sites and multiple data centers. For workloads and apps hosted in 448 cloud DCs, multiple sites need to communicate securely with those 449 cloud workloads and apps. This section documents some of the issues 450 associated with using IPsec tunnels to connect enterprise premises 451 with cloud gateways. 453 6.1. Complexity of multi-point any-to-any interconnection 455 The dynamic workload instantiated in cloud DC needs to communicate 456 with multiple branch offices and on-premises data centers. Most 457 enterprises need multi-point interconnection among multiple 458 locations, which can be provided by means of MPLS L2/L3 VPNs. 460 Using IPsec overlay paths to connect all branches & on-premises data 461 centers to cloud DCs requires CPEs to manage routing among Cloud DCs 462 gateways and the CPEs located at other branch locations, which can 463 dramatically increase the complexity of the design, possibly at the 464 cost of jeopardizing the CPE performance. 466 The complexity of requiring CPEs to maintain routing among other 467 CPEs is one of the reasons why enterprises migrated from Frame Relay 468 based services to MPLS-based VPN services. 470 MPLS-based VPNs have their PEs directly connected to the CPEs. 471 Therefore, CPEs only need to forward all traffic to the directly 472 attached PEs, which are therefore responsible for enforcing the 473 routing policy within the corresponding VPNs. Even for multi-homed 474 CPEs, the CPEs only need to forward traffic among the directly 475 connected PEs. However, when using IPsec tunnels between CPEs and 476 Cloud DCs, the CPEs need to compute, select, establish and maintain 477 routes for traffic to be forwarded to Cloud DCs, to remote CPEs via 478 VPN, or directly. 480 6.2. Poor performance over long distance 482 When enterprise CPEs or gateways are far away from cloud DC gateways 483 or across country/continent boundaries, performance of IPsec tunnels 484 over the public Internet can be problematic and unpredictable. Even 485 though there are many monitoring tools available to measure delay 486 and various performance characteristics of the network, the 487 measurement for paths over the Internet is passive and past 488 measurements may not represent future performance. 490 Many cloud providers can replicate workloads in different available 491 zones. An App instantiated in a cloud DC closest to clients may have 492 to cooperate with another App (or its mirror image) in another 493 region or database server(s) in the on-premises DC. This kind of 494 coordination requires predicable networking behavior/performance 495 among those locations. 497 6.3. Scaling Issues with IPsec Tunnels 499 IPsec can achieve secure overlay connections between two locations 500 over any underlay network, e.g., between CPEs and Cloud DC Gateways. 502 If there is only one enterprise location connected to the cloud 503 gateway, a small number of IPsec tunnels can be configured on-demand 504 between the on-premises DC and the Cloud DC, which is an easy and 505 flexible solution. 507 However, for multiple enterprise locations to reach workloads hosted 508 in cloud DCs, the cloud DC gateway needs to maintain multiple IPsec 509 tunnels to all those locations (e.g., as a hub & spoke topology). 510 For a company with hundreds or thousands of locations, there could 511 be hundreds (or even thousands) of IPsec tunnels terminating at the 512 cloud DC gateway, which is not only very expensive (because Cloud 513 Operators usually charge their customers based on connections), but 514 can be very processing intensive for the gateway. Many cloud 515 operators only allow a limited number of (IPsec) tunnels & bandwidth 516 to each customer. Alternatively, you could use a solution like 517 group encryption where a single IPsec SA is necessary at the GW but 518 the drawback here is key distribution and maintenance of a key 519 server, etc. 521 7. Problems of Using SD-WAN to connect to Cloud DCs 522 SD-WAN can establish parallel paths over multiple underlay networks 523 between two locations on-demand, for example, to support the 524 connections established between two CPEs interconnected by a 525 traditional MPLS VPN ([RFC4364] or [RFC4664]) or by IPsec [RFC6071] 526 tunnels. 528 SD-WAN lets enterprises augment their current VPN network with cost- 529 effective, readily available Broadband Internet connectivity, 530 enabling some traffic offloading to paths over the Internet 531 according to differentiated, possibly application-based traffic 532 forwarding policies, or when the MPLS VPN connection between the two 533 locations is congested, or otherwise undesirable or unavailable. 535 7.1. SD-WAN among branch offices vs. interconnect to Cloud DCs 537 SD-WAN interconnection of branch offices is not as simple as it 538 appears. For an enterprise with multiple sites, using SD-WAN overlay 539 paths among sites requires each CPE to manage all the addresses that 540 local hosts have the potential to reach, i.e., map internal VPN 541 addresses to appropriate SD-WAN paths. This is similar to the 542 complexity of Frame Relay based VPNs, where each CPE needed to 543 maintain mesh routing for all destinations if they were to avoid an 544 extra hop through a hub router. Even though SD-WAN CPEs can get 545 assistance from a central controller (instead of running a routing 546 protocol) to resolve the mapping between destinations and SD-WAN 547 paths, SD-WAN CPEs are still responsible for routing table 548 maintenance as remote destinations change their attachments, e.g., 549 the dynamic workload in other DCs are de-commissioned or added. 551 Even though originally envisioned for interconnecting branch 552 offices, SD-WAN offers a very attractive way for enterprises to 553 connect to Cloud DCs. 555 The SD-WAN for interconnecting branch offices and the SD-WAN for 556 interconnecting to Cloud DCs have some differences: 558 - SD-WAN for interconnecting branch offices usually have two end- 559 points (e.g., CPEs) controlled by one entity (e.g., a 560 controller or management system operated by the enterprise). 561 - SD-WAN for Cloud DC interconnects may consider CPEs owned or 562 managed by the enterprise, while remote end-points are being 563 managed or controlled by Cloud DCs (For the ease of 564 description, let's call such CPEs asymmetrically-managed CPEs). 566 - Cloud DCs may have different entry points (or devices) with one 567 entry point that terminates a private direct connection (based 568 upon a leased line for example) and other entry points being 569 devices terminating the IPsec tunnels, as shown in Figure 2. 571 Therefore, the SD-WAN design becomes asymmetric. 572 +------------------------+ 573 | ,---. ,---. | 574 | (TN-1 ) ( TN-2)| TN: Tenant applications/workloads 575 | `-+-' +---+ `-+-' | 576 | +----|vR1|----+ | 577 | ++--+ | 578 | | +-+----+ 579 | | /Internet\ One path via 580 | +-------+ Gateway +---------------------+ 581 | \ / Internet \ 582 | +-+----+ \ 583 +------------------------+ \ 584 \ 585 +------------------------+ native traffic \ 586 | ,---. ,---. | without encryption| 587 | (TN-3 ) ( TN-4)| | 588 | `-+-' +--+ `-+-' | | +------+ 589 | +----|vR|-----+ | +----+ CPE | 590 | ++-+ | | +------+ 591 | | +-+----+ | 592 | | / virtual\ One path via IPsec Tunnel | 593 | +-------+ Gateway +-------------------------- + 594 | \ / Encrypted traffic over| 595 | +-+----+ public network | 596 +------------------------+ | 597 | 598 +------------------------+ | 599 | ,---. ,---. | Native traffic | 600 | (TN-5 ) ( TN-6)| without encryption | 601 | `-+-' +--+ `-+-' | over secure network| 602 | +----|vR|-----+ | | 603 | ++-+ | | 604 | | +-+----+ +------+ | 605 | | / \ Via Direct /customer\ | 606 | +-------+ Gateway +----------+ gateway |-----+ 607 | \ / Connect \ / 608 | +-+----+ +------+ 609 +------------------------+Customer GW has physical connection to AWS GW 611 Figure 2: Different Underlays to Reach Cloud DC 613 8. End-to-End Security Concerns for Data Flows 615 When IPsec tunnels established from enterprise on-premises CPEs 616 are terminated at the Cloud DC gateway where the workloads or 617 applications are hosted, some enterprises have concerns regarding 618 traffic to/from their workload being exposed to others behind the 619 data center gateway (e.g., exposed to other organizations that 620 have workloads in the same data center). 621 To ensure that traffic to/from workloads is not exposed to 622 unwanted entities, IPsec tunnels may go all the way to the 623 workload (servers, or VMs) within the DC. 625 9. Requirements for Dynamic Cloud Data Center VPNs 627 In order to address the aforementioned issues, any solution for 628 enterprise VPNs that includes connectivity to dynamic workloads or 629 applications in cloud data centers should satisfy a set of 630 requirements: 632 - The solution should allow enterprises to take advantage of the 633 current state-of-the-art in VPN technology, in both traditional 634 MPLS-based VPNs and IPsec-based VPNs (or any combination 635 thereof) that run over the public Internet. 636 - The solution should not require an enterprise to upgrade all 637 their existing CPEs. 638 - The solution should support scalable IPsec key management among 639 all nodes involved in DC interconnect schemes. 640 - The solution needs to support easy and fast, on-the-fly, VPN 641 connections to dynamic workloads and applications in third 642 party data centers, and easily allow these workloads to migrate 643 both within a data center and between data centers. 644 - Allow VPNs to provide bandwidth and other performance 645 guarantees. 646 - Be a cost-effective solution for enterprises to incorporate 647 dynamic cloud-based applications and workloads into their 648 existing VPN environment. 650 10. Security Considerations 652 The draft discusses security requirements as a part of the problem 653 space, particularly in sections 4, 5, and 8. 655 Solution drafts resulting from this work will address security 656 concerns inherent to the solution(s), including both protocol 657 aspects and the importance (for example) of securing workloads in 658 cloud DCs and the use of secure interconnection mechanisms. 660 11. IANA Considerations 662 This document requires no IANA actions. RFC Editor: Please remove 663 this section before publication. 665 12. References 667 12.1. Normative References 669 12.2. Informative References 671 [RFC2735] B. Fox, et al "NHRP Support for Virtual Private 672 networks". Dec. 1999. 674 [RFC8192] S. Hares, et al "Interface to Network Security Functions 675 (I2NSF) Problem Statement and Use Cases", July 2017 677 [ITU-T-X1036] ITU-T Recommendation X.1036, "Framework for creation, 678 storage, distribution and enforcement of policies for 679 network security", Nov 2007. 681 [RFC6071] S. Frankel and S. Krishnan, "IP Security (IPsec) and 682 Internet Key Exchange (IKE) Document Roadmap", Feb 2011. 684 [RFC4364] E. Rosen and Y. Rekhter, "BGP/MPLS IP Virtual Private 685 Networks (VPNs)", Feb 2006 687 [RFC4664] L. Andersson and E. Rosen, "Framework for Layer 2 Virtual 688 Private Networks (L2VPNs)", Sept 2006. 690 [BGP-SDWAN] L. Dunbar, et al. "BGP Extension for SDWAN Overlay 691 Networks", draft-dunbar-idr-bgp-sdwan-overlay-ext-03, 692 work-in-progress, Nov 2018. 694 13. Acknowledgments 696 Many thanks to Chris Bowers, Ignas Bagdonas, Michael Huang, Liu Yuan 697 Jiao, Katherine Zhao, and Jim Guichard for the discussion and 698 contributions. 700 Authors' Addresses 702 Linda Dunbar 703 Futurewei 704 Email: Linda.Dunbar@futurewei.com 706 Andrew G. Malis 707 Futurewei 708 Email: agmalis@gmail.com 710 Christian Jacquenet 711 Orange 712 Rennes, 35000 713 France 714 Email: Christian.jacquenet@orange.com 716 Mehmet Toy 717 Verizon 718 One Verizon Way 719 Basking Ridge, NJ 07920 720 Email: mehmet.toy@verizon.com