idnits 2.17.00 (12 Aug 2021) /tmp/idnits23427/draft-dunbar-6man-5g-edge-compute-sticky-service-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (March 7, 2022) is 68 days in the past. Is this intentional? Checking references for intended status: Full Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'A-ER' is mentioned on line 281, but not defined == Missing Reference: 'RFC8799' is mentioned on line 404, but not defined == Unused Reference: 'RFC4364' is defined on line 786, but no explicit reference was found in the text == Unused Reference: '3GPP-EdgeComputing' is defined on line 801, but no explicit reference was found in the text == Unused Reference: 'RFC5521' is defined on line 826, but no explicit reference was found in the text == Unused Reference: 'BGP-SDWAN-Port' is defined on line 830, but no explicit reference was found in the text == Unused Reference: 'SDWAN-EDGE-Discovery' is defined on line 836, but no explicit reference was found in the text == Unused Reference: 'Tunnel-Encap' is defined on line 841, but no explicit reference was found in the text ** Downref: Normative reference to an Proposed Standard RFC: RFC 4364 == Outdated reference: A later version (-02) exists of draft-dunbar-ippm-5g-edge-compute-ip-layer-metrics-00 -- Unexpected draft version: The latest known version of draft-dunbar-lsr-5g-edge-compute-ospf-ext is -04, but you're referring to -05. == Outdated reference: A later version (-06) exists of draft-dunbar-idr-5g-edge-compute-app-meta-data-02 == Outdated reference: A later version (-04) exists of draft-dunbar-idr-sdwan-edge-discovery-00 == Outdated reference: draft-ietf-idr-tunnel-encaps has been published as RFC 9012 Summary: 1 error (**), 0 flaws (~~), 14 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group L. Dunbar 2 Internet Draft J. Kaippallimalil 3 Intended status: Standard Futurewei 4 Expires: September 7, 2022 5 March 7, 2022 7 IPv6 Solution for 5G Edge Computing Sticky Service 8 draft-dunbar-6man-5g-edge-compute-sticky-service-06 10 Abstract 12 This draft describes the IPv6-based solutions that can stick an 13 application flow originated from a mobile device to the same 14 ANYCAST server location when the mobile device moves from one 15 5G cell site to another. 17 Status of this Memo 19 This Internet-Draft is submitted in full conformance with the 20 provisions of BCP 78 and BCP 79. 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. This document may not be 24 modified, and derivative works of it may not be created, except 25 to publish it as an RFC and to translate it into languages 26 other than English. 28 Internet-Drafts are working documents of the Internet 29 Engineering Task Force (IETF), its areas, and its working 30 groups. Note that other groups may also distribute working 31 documents as Internet-Drafts. 33 Internet-Drafts are draft documents valid for a maximum of six 34 months and may be updated, replaced, or obsoleted by other 35 documents at any time. It is inappropriate to use Internet- 36 Drafts as reference material or to cite them other than as 37 "work in progress." 39 The list of current Internet-Drafts can be accessed at 40 http://www.ietf.org/ietf/1id-abstracts.txt 42 Internet-Draft IPv6 for 5G Edge Sticky Service 44 The list of Internet-Draft Shadow Directories can be accessed 45 at http://www.ietf.org/shadow.html 47 This Internet-Draft will expire on April 7, 2021. 49 Copyright Notice 51 Copyright (c) 2021 IETF Trust and the persons identified as the 52 document authors. All rights reserved. 54 This document is subject to BCP 78 and the IETF Trust's Legal 55 Provisions Relating to IETF Documents 56 (http://trustee.ietf.org/license-info) in effect on the date of 57 publication of this document. Please review these documents 58 carefully, as they describe your rights and restrictions with 59 respect to this document. Code Components extracted from this 60 document must include Simplified BSD License text as described 61 in Section 4.e of the Trust Legal Provisions and are provided 62 without warranty as described in the Simplified BSD License. 64 Table of Contents 66 1. Introduction.................................................. 3 67 1.1. 5G Edge Computing Background.......................... 3 68 1.2. 5G Edge Computing Network Properties.................. 4 69 1.3. Problem #1: Discovery of Edge Application Server...... 5 70 1.4. Problem #2: sticking to original App Server........... 6 71 2. Conventions used in this document............................. 7 72 3. Stick a Flow to an ANYCAST Server............................. 9 73 4. Sticky flow for QUIC based Applications....................... 9 74 5. Other Solutions within a Limited Domain...................... 10 75 5.1. Use Case of 5G Edge Computing in a limited domain.... 10 76 5.2. End Node Based Sticky Service Solution............... 10 77 5.2.1. Edge Controller Based Solution.................. 11 78 5.3. Sticky Egress Address Discovery...................... 12 79 5.4. Sticky-Dst-SubTLV in Destination Extension Header.... 12 80 5.5. Processing at the Ingress router..................... 13 81 6. Tunnel based Sticky Service Solution......................... 13 82 6.1. Desired functions by the Network Controller.......... 14 83 6.2. Ingress and Egress Routers Processing Behavior....... 14 84 6.3. A Solution without the Communication with 5G system.. 16 85 6.4. A Solution that depends on the communication with 5G 86 system.................................................... 16 87 7. Expanding APN6 for Sticky Service information................ 17 88 7.1. Sticky Service ID encoded in the Application-aware ID 17 90 Internet-Draft IPv6 for 5G Edge Sticky Service 92 7.2. Sticky Service Sub-TLV encoded in APN6 Service-para 93 option.................................................... 18 94 8. Manageability Considerations................................. 18 95 9. Security Considerations...................................... 18 96 10. IANA Considerations......................................... 18 97 11. References.................................................. 18 98 11.1. Normative References................................ 18 99 11.2. Informative References.............................. 19 100 12. Acknowledgments............................................. 20 102 1. Introduction 104 1.1. 5G Edge Computing Background 106 As described in [5G-EC-Metrics], one application in 5G Edge 107 Computing environment can have multiple application servers 108 hosted in different Edge Computing data centers close in 109 proximity. Those Edge Computing (mini) data centers are usually 110 very close to, or co-located with, 5G base stations, to 111 minimize latency and optimize the performances. 113 When a mobile device sends packets using the destination 114 address from a DNS reply or its own cache, the packets are 115 carried by a GTP tunnel from the 5G eNB to the 5G UPF-PSA (User 116 Plan Function - PDU Session Anchor). The UPF-PSA decapsulates 117 the 5G GTP outer header and forwards the packets from the 118 mobile devices to the Ingress router of the Edge Computing (EC) 119 Local Data Network (LDN). The LDN for 5G EC, the IP Networks, 120 is responsible for forwarding the packets to the intended 121 destinations. 123 When the mobile device moves out of coverage of its current gNB 124 (next-generation Node B) (gNB1), handover procedures are 125 initiated, and the 5G SMF (Session Management Function) selects 126 a new UPF-PSA. The standard handover procedures are described 127 in 3GPP TS 23.501 and TS 23.502. When the handover process is 128 complete, the mobile device might be anchored to a new UPF-PSA. 129 5G Session Management function (SMF) may maintain a path from 130 the old UPF to the new UPF for a short period of time for SSC 131 [Session and Service Continuity] mode 3 to make the handover 132 process more seamless. 134 Internet-Draft IPv6 for 5G Edge Sticky Service 136 1.2. 5G Edge Computing Network Properties 138 In this document, 5G Edge Computing Network refers to multiple 139 Local IP Data Networks (LDN) in one region that interconnect 140 the Edge Computing mini-data centers. Those IP LDN networks are 141 the N6 interfaces from 3GPP 5G perspective. 143 The ingress routers to the 5G Edge Computing Network are 144 directly connected to 5G UPFs. The egress routers to the 5G 145 Edge Computing Network are the routers that have a direct link 146 to the Edge Computing servers. The servers and the egress 147 routers are co-located. Some of those mini Edge Computing Data 148 centers may have Virtual switches or Top of Rack switches 149 between the egress routers and the servers. But transmission 150 delay between the egress routers and the Edge Computing servers 151 is very small, which is considered negligible in this document. 153 When multiple Edge Computing Servers attached to one App Layer 154 Load Balancer, only the App Layer Load Balancer address is 155 visible to the 5G Edge Computing Network. How the App Layer 156 Load balancer manages the individual servers is out of the 157 scope of the document. 159 The Edge Computer Services are registered services that need to 160 utilize the network topology and balance among multiple mini 161 Edge Computing Data Centers with the same ANYCAST address. 162 Majority services are not registered 5G Edge Computing 163 Services. 165 Internet-Draft IPv6 for 5G Edge Sticky Service 167 +--+ 168 |MD|---\+---------+ +------------------+ 169 +--+ | 5G | +---------+ | S1: aa08::4450 | 170 +--+ | Site +--++---+ +----+ | 171 |MD|----| A |PSA| Ra| | R1 | S2: aa08::4460 | 172 +--+ | +---+---+ +----+ | 173 +---+ | | | | | S3: aa08::4470 | 174 |MD1|---/+---------+ | | +------------------+ 175 +---+ |IP Network | L-DN1 176 |(3GPP N6) | 177 | | | +------------------+ 178 | MB1 | | | S1: aa08::4450 | 179 | moves to | +----+ | 180 | Site B | | R3 | S2: aa08::4460 | 181 v | +----+ | 182 | | | S3: aa08::4470 | 183 | | +------------------+ 184 | | L-DN3 185 +--+ | | 186 |MD|---\+---------+ | | +------------------+ 187 +--+ | 5G | | | | S1: aa08::4450 | 188 +--+ | Site +--++-+--+ +----+ | 189 |MD|----| B |PSA| Rb | | R2 | S2: aa08::4460 | 190 +--+ | +--++----+ +----+ | 191 +--+ | | +-----------+ | S3: aa08::4470 | 192 |MD|---/+---------+ +------------------+ 193 +--+ L-DN2 194 Figure 1: App Servers in different edge DCs 196 1.3. Problem #1: Discovery of Edge Application Server 198 Key Issue #1 identified by 3GPP Edge Computing Study [TR 199 23.748] is that one application service might be served by 200 multiple Edge Application Servers typically deployed in 201 different sites. These multiple Edge Application Server 202 instances that host same content or service may use a single IP 203 address (anycast address) or different IP addresses. 205 Key Issue #2 identified by 3GPP Edge Computing Study [TR 206 23.748] is Edge server relocation. 208 Internet-Draft IPv6 for 5G Edge Sticky Service 210 Application Server discovery and relocation can be achieved by 211 running IGP/BGP routing protocols among the routers in LDN. 213 Increasingly, ANYCAST is used extensively by various 214 application providers because it is possible to dynamically 215 load balance across multiple locations of the same address 216 based on network conditions. When multiple servers in different 217 locations have the same IP address (ANYCAST), the routers see 218 multiple paths to the IP address. The IGP/BGP routing protocols 219 can inform all the nodes where the servers are and when servers 220 move to new locations. 222 Application Server location selection using Anycast address 223 leverages the proximity information present in the network 224 routing layer and eliminates the single point of failure and 225 bottleneck at the DNS resolvers and application layer load 226 balancers. Another benefit of using ANYCAST address is removing 227 the dependency on mobile devices that use their cached IP 228 addresses instead of querying DNS when they move to a new 229 location. 231 However, having multiple locations for the same ANYCAST address 232 in the 5G Edge Computing environment can be problematic because 233 all those edge computing Data Centers can be close in 234 proximity. There might not be any difference in the routing 235 cost to reach the Application Servers in different Edge DCs. 236 The same routing cost to multiple locations can cause packets 237 from one flow to be forwarded to different locations, which can 238 cause service glitches. 240 1.4. Problem #2: sticking to original App Server 242 When a mobile device moves to a new location but continues the 243 same application flow, the router connected to the new UPF 244 might choose the App Server closer to the new location. As 245 shown in the figure below, when the MD1 in 5G-site-A moves to 246 the 5G-Site-B, the router directly connected to 5G PSA2 might 247 forward the packets destined towards the S1: aa08::4450 to the 248 server located in L-DN2 because L-DN2 has the lowest cost based 249 on routing. This is not the desired behavior for some services, 250 which are called Sticky Services in this document. 252 Even for some advanced applications with built-in mechanisms to 253 re-sync the communications at the application layer after 254 switching to a new location, service glitches are often 255 experienced. 257 Internet-Draft IPv6 for 5G Edge Sticky Service 259 It worth noting that not all services need to be sticky. We 260 assume only a subset of services are, and the Network is 261 informed of the services that need to be sticky, usually by 262 requests from application developers or controllers. 264 This document describes an IPv6-based network layer solution to 265 stick the packets belonging to the same flow of a mobile device 266 to its original App Server location after the mobile device is 267 anchored to a new nearby UPF-PSA. 269 Note: for ease of description, the Edge Computing Server, 270 Application Server, or App Server are used interchangeably 271 throughout this document. 273 2. Conventions used in this document 275 APN6 Application aware network using IPv6. The term 276 "Application" has very broad meanings. In this 277 document the term "Application" refers to any 278 applications that use ANYCAST servers in the 5G 279 Edge Computing Environment. 281 A-ER: Egress Router to an Application Server, [A-ER] is 282 used to describe the last router that the 283 Application Server is attached. For 5G EC 284 environment, the A-ER can be the gateway router to 285 a (mini) Edge Computing Data Center. 287 Application Server: An application server is a physical or 288 virtual server that host the software system for 289 the application. 291 Application Server Location: Represent a cluster of servers at 292 one location serving the same Application. One 293 application may have a Layer 7 Load balancer, whose 294 address(es) are reachable from external IP network, 295 in front of a set of application servers. From IP 296 network perspective, this whole group of servers 297 are considered as the Application server at the 298 location. 300 Internet-Draft IPv6 for 5G Edge Sticky Service 302 Edge Application Server: used interchangeably with Application 303 Server throughout this document. 305 EC: Edge Computing 307 Edge Hosting Environment: An environment providing support 308 required for Edge Application Server's execution. 310 NOTE: The above terminologies are the same as those 311 used in 3GPP TR 23.758 313 Edge DC: Edge Data Center, which provides the Edge Computing 314 Hosting Environment. It might be co-located with or 315 very close to a 5G Base Station. 317 gNB next generation Node B 319 L-DN: Local Data Network 321 MD: Mobile Device, which is the same as the UE (User 322 Equipment) used in 3GPP. The term "mobile device" 323 is used instead of UE to emphasize on sticking 324 services originated from the devices that are 325 mobile to same server. 327 PSA: PDU Session Anchor (UPF) 329 SSC: Session and Service Continuity 331 UE: User Equipment. UE is same as a mobile device in 332 this document. 334 UPF: User Plane Function 336 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 337 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", 338 "MAY", and "OPTIONAL" in this document are to be interpreted as 339 described in BCP 14 [RFC2119] [RFC8174] when, and only when, 340 they appear in all capitals, as shown here. 342 Internet-Draft IPv6 for 5G Edge Sticky Service 344 3. Stick a Flow to an ANYCAST Server 346 When servers attached to different egress routers are assigned 347 with the same IP address, the routers in the LDN see multiple 348 paths to the IP address. The Egress nodes' unicast addresses 349 are the Next Hops (i.e., R1, R2, and R3) to reach the Edge 350 Computing server ANYCAST address. 352 The routers choose the lowest cost path. [5G-EC-OSPF-EXT] and 353 [5G-EC-BGP-EXT] describe the OSPF and BGP extension to 354 propagate additional costs about the site where the servers are 355 located so that the site costs can be incorporated into the 356 path computation. 358 Flow sticking to one server is not the same as flow nailing 359 down to the same server. When the network cost is significantly 360 increased, such as the mobile device moving to a very far away 361 location or the extreme case of link failure to the original 362 server, another server with the same IP address is selected. 364 The Flow Affinity feature, which most commercial routers 365 support today, can ensure packets belonging to one flow be 366 forwarded along the same path to the same egress router, which 367 then delivers the packets to the attached server. 369 Editor's note: for IPv6 traffic, Flow Affinity can be supported 370 by the Local Data Network (LDN) routers forwarding the packets 371 with the same Flow Label in the packets' IPv6 Header along the 372 same path towards the same egress router. For IPv4 traffic, 5 373 tuples in the IPv4 header can be used to achieve the Flow 374 Affinity. 376 When a UE moves to a different cell site, the packets from the 377 UE might enter the 5G LDN from a different UPF. Suppose the 378 handover to the new cell site is in the middle of a flow from 379 the UE. In that case, the new ingress router directly connected 380 to the new UPF needs to have the original egress router 381 information to stick the flow from the UE to the original 382 egress router. The original egress router is called Sticky 383 Egress throughout this document. 385 4. Sticky flow for QUIC based Applications 387 For applications using QUIC transport protocol, ANYCAST 388 stickiness are supported natively. During the initial 389 handshake, QUIC servers can provide a "preferred address" (IP 390 or IPv6 and port number), and the client can immediately 391 migrate the connection to use that address. This was 393 Internet-Draft IPv6 for 5G Edge Sticky Service 395 specifically designed to support servers listening on anycast 396 addresses, so the connection can be pinned to a unicast address 397 specific to the server. 399 5. Other Solutions within a Limited Domain 401 This section describes some sticky flow solutions within a 402 limited domain [RFC8799] for applications not based on QUIK. 404 Within a limited domain [RFC8799], mobile devices, edge 405 servers, and network functions are under one administrative 406 domain. Therefore, it is feasible for mobile devices to perform 407 specific actions. 409 5.1. Use Case of 5G Edge Computing in a limited domain. 411 Some 5G Connected devices, such as drones for fighting natural 412 disasters or robots in Industry 4.0 environments, need ultra- 413 low latency responses from their analytic servers. To reach 414 ultra-low latency, those analytic functions can be hosted on 415 servers very close to radio towers. 417 All the functions (including networking and analytics) and 418 devices are administrated by one operator. Network devices 419 within the 5G LDN limited domain might be provided by different 420 vendors, therefore needing interoperable solutions. 422 5.2. End Node Based Sticky Service Solution 424 The End-Node-based Sticky Service solution needs IPv6 mobile 425 devices to insert the Destination Option header extracted from 426 the packet received from the network side to the IPv6 Header of 427 the next packet if the next packet belongs to the same flow. 428 This action dramatically simplifies the processing at the LDN's 429 Ingress routers. 431 Here are some assumptions for the End-Node based Sticky Service 432 solution: 434 - The mobile devices are under the same administrative 435 control as the Edge computing servers. 436 - If an Edge Computing service needs to be sticky in the 5G 437 Edge Computing environment, the corresponding service ID 438 is registered with the 5G Edge Computing controller. The 439 Sticky Service ID can be the IP address (unicast or 440 ANYCAST) of the server. 442 Internet-Draft IPv6 for 5G Edge Sticky Service 444 Here is the overview of the End-Node based Sticky Service 445 solution: 446 - Each ANYCAST Edge Computing server either learns or is 447 informed of the unicast Sticky Egress address (Section 3). 448 The goal is to deliver packets belonging to one flow to 449 the same Sticky Egress address for the ANYCAST address. 450 - When an Edge Computing server sends data packets back to a 451 client (or the mobile device), it inserts the Sticky-Dst- 452 SubTLV (described in Section 4.4) into the packets' 453 Destination Option Header. 454 - The client (or the mobile device) needs to copy the 455 Destination Option Header from the received packet to the 456 next packet's Destination Header if the next packet 457 belongs to the same flow as the previous packet. 458 - If the following conditions are true, the ingress router 459 encapsulates the packet from the client in a tunnel whose 460 outer destination address is set to the Sticky Egress 461 Address extracted from the packet's Sticky-Dst-SubTLV: 462 o The destination of the packet from the client-side 463 matches with one of the Sticky Service ACLs 464 configured on the ingress router of the LDN, 465 o the packet header has the Destination Option present 466 with Sticky-Dst-SubTLV. 467 - Else (i.e., one of the conditions above is not true), the 468 ingress node uses its algorithm, such as the least cost as 469 described in [5G-EC-Metrics], to select the optimal Sticky 470 Egress address for forwarding the packet. 472 5.2.1. Edge Controller Based Solution. 474 To be added. 475 [Editor's note: can consider adding something along the 476 line of the following, which is suggested by the email: 477 say 5G/MEC control plane can tell the UE what address to 478 use, it does NOT mean a UE will query whenever it is 479 anchored to a new UPF. The initial query when it needs a 480 service will return the unicast address of a server based 482 Internet-Draft IPv6 for 5G Edge Sticky Service 484 on all kinds of information/constraints, including the 485 server load information talked about in draft-dunbar-idr- 486 5g-edge-compute-app-meta-data. After that, the server 487 won't change until new server is indeed needed (this is 488 what "sticky service" is about, right). When a server 489 change is indeed needed, the 5G/MEC control plane will 490 tell the UE the new unicast address to use and tell the 491 servers to move the corresponding application data when 492 necessary. 493 ] 494 5.3. Sticky Egress Address Discovery 496 To an App server with ANYCAST address, the Sticky Egress 497 address is the same as its default Gateway address. 499 To prevent malicious entities sending DDOS attacks to routers 500 within 5G EC LDN, e.g., the Sticky Egress address that is 501 encoded in the Destination option header in the packets sent 502 back to the clients, a proxy Sticky Egress address can be 503 encoded in the Destination option header. The proxy Sticky 504 Egress address is only recognizable by the 5G EC LDN ingress 505 nodes, i.e., the Ra and Rb in Figure 1, but not routable in 506 other networks. The LDN ingress routers can translate the proxy 507 Sticky Egress to a routable address for the Sticky Egress node 508 after the source addresses of the packets are authenticated. 510 5.4. Sticky-Dst-SubTLV in Destination Extension Header 512 A new Sticky-Dst-SubTLV is specified as below, which can be 513 inserted into the IPv6 Destination Options header. The IPv6 514 Destination Option Header is specified by [RFC8200] as having a 515 Next Header value of 60: 517 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 518 | Next Header | Hdr Ext Len | | 519 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 520 | | 521 | Sticky-Dst-SubTLV | 522 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 524 Sticky-Dst-SubTLV is specified as: 526 Internet-Draft IPv6 for 5G Edge Sticky Service 528 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 529 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 530 | Sticky-Type | Len | AFI | Reserved | 531 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 532 |Sticky Egress address (IPv4 or IPv6) for reaching the ANYCAST | 533 ~ ~ 534 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 536 Sticky-Type = 1: indicate the Sticky Egress unicast address 537 at encoded in the Sticky-Dst-SubTLV. 539 5.5. Processing at the Ingress router 541 - An Ingress router is configured with an ACL for filtering 542 out the applications that need sticky service. 544 Note, not all applications need sticky service. Using ACL 545 can significantly reduce the processing on the routers. 547 - When an Ingress router receives a packet from the 5G side 548 that matches the ACL, the Ingress router extracts the 549 Sticky-Dst-SubTLV from the packet IPv6 header if the field 550 exists in the packet header. 551 - Encapsulate the packet with the tunnel type that are 552 supported by the original Sticky Egress node, using the 553 extracted Sticky Egress address in the destination field 554 of the outer Header, and forward the packet. 556 Note: if the proxy Sticky Egress address is encoded in the 557 Sticky-Dst-SubTLV, the ingress router needs to translate 558 the proxy Sticky Egress address to a routable address. 560 If none of the above conditions are met, the ingress router 561 uses its algorithm to select the optimal Sticky Egress node 562 to forward the packet. 564 6. Tunnel based Sticky Service Solution 566 For environments that mobile devices cannot change their 567 processing behavior as described in Section 4, a Tunnel based 569 Internet-Draft IPv6 for 5G Edge Sticky Service 571 Sticky Service solution can be used. This solution does not 572 depend on mobile device's behavior. However, this solution does 573 require ingress routers to filter out the registered sticky 574 services and might need some level of assistance from the LDN 575 network controller. 577 6.1. Desired functions by the Network Controller 579 6.2. Ingress and Egress Routers Processing Behavior 581 The solution assumes that both ingress routers and egress 582 routers support at least one type of tunnel and are configured 583 with ACLs to filter out packets whose destination or source 584 addresses match with the Sticky Service Identifier. The 585 solution also assumes there are only limited number of Sticky 586 Services to be supported. 587 An ingress router needs to build a Sticky-Service-Table, with 588 the following minimum attributes. The Sticky-Service-Table is 589 initialized to be empty. 590 - Sticky Service ID 591 - Flow Label 592 - Sticky Egress address 593 - Timer 595 Editor's Note: 596 When a mobile device moves from one 5G Site to another, the 597 same mobile device will have a new IP address. "Flow Label + 598 Sticky Service ID" stays the same when a mobile device is 599 anchored to a new PSA. Therefore, this solution uses "Flow 600 Label + Sticky Service ID" to identify a sticky flow. Since 601 the chance of different mobile devices sending packets to the 602 same ANYCAST address using the same Flow Label is very low, 603 it is with high probability that "Flow Label + Sticky Service 604 ID" can uniquely identify a flow. When multiple mobile 605 devices using the same Flow Label sending packets to the same 606 ANYCAST address, the solution described in this section will 607 stick the flows to the same ANYCAST server attached to the 608 Sticky Egress router. This behavior doesn't cause any harm. 610 Internet-Draft IPv6 for 5G Edge Sticky Service 612 Each entry in the Sticky-Service-Table has a Timer because a 613 sticky service is no longer sticky if there are no packets of 614 the same flow destined towards the service ID for a period of 615 time. The Timer should be larger than a typical TCP session 616 Timeout value. An entry is automatically removed from the 617 Sticky-Service-Table when its timer expires. 618 Note: since there are only small number of Sticky services, the 619 Sticky-Service-Table is not very large. 620 When an ingress router receives a packet from a mobile device 621 matching with one of the Sticky Service ACLs and there is no 622 entry in the Sticky-Service-Table matching the Flow Label and 623 the Sticky Service ID, the ingress router considers the packet 624 to be the first packet of the flow. There is no need to 625 sticking the packet to any location. The ingress router uses 626 its own algorithm to select the optimal egress node as the 627 Sticky Egress address for the ANYCAST address, encapsulates the 628 packet with a tunnel that is supported by the egress node. The 629 tunnel's destination address is set to the egress node address. 631 When an egress router receives a packet from an attached host 632 with the packet's source address matching with one of the 633 Sticky Service IDs, the egress router encapsulates the packet 634 with a tunnel that is supported by the ingress router and the 635 tunnel's destination address is set to the ingress router 636 address. An Egress router learns the ingress router address for 637 a mobile device IP address via BGP UPDATE messages. 639 When an ingress router receives a packet in a tunnel from any 640 egress router and the packet's source address matches with a 641 Sticky Service ID, the egress router address is set as the 642 Sticky Egress address for the Sticky Service ID. The ingress 643 router adds the entry of "Sticky-Service-ID + Flow Label + the 644 associated Sticky Egress address + Timer" to the Sticky- 645 Service-Table if the entry doesn't exist yet in the table. If 646 the entry exists, the ingress router refreshes the Timer of the 647 entry in the table. 649 When the ingress router receives the subsequent packets of a 650 flow from the 5G side matching with an Sticky Service ID and 651 the Sticky-Service ID exists in the Sticky-Service-Table, the 652 ingress router uses the Sticky Egress address found in the 653 Sticky-Service-Table to encapsulate the packet and refresh the 654 Timer of the entry. If the Sticky-Service ID doesn't exist in 655 the table, the ingress router considers the packet as the first 656 packet of a flow. 658 Internet-Draft IPv6 for 5G Edge Sticky Service 660 The subsequent sections describe how ingress nodes prorogate 661 their Sticky-Service-Table to their neighboring ingress nodes. 662 The propagation is for neighboring ingress nodes to be informed 663 of the Sticky Egress address to a sticky service if a mobile 664 device moves to a new neighboring 5G site resulting in 665 anchoring to a new ingress node. 667 6.3. A Solution without the Communication with 5G system. 669 When a mobile device moves to a very far away 5G site, say a 670 different geographic region, the benefit of sticking to the 671 original ANYCAST server is out weighted by network delay. Then, 672 there is no point sending packets to the Sticky Egress node if 673 the ingress router very far away. Therefore, it is necessary 674 for each ingress router to have a group of neighboring ingress 675 routers that are not too far away from the potential Sticky 676 Egress nodes selected by the ingress router. This group of 677 ingress routers is called the Neighboring Ingress Group. Each 678 ingress router can either automatically discover its 679 Neighboring Ingress Group by routing protocols or is configured 680 by its controller. It is out of the scope of this document on 681 how ingress nodes discover its Neighboring Ingress Group. 683 Each ingress node needs to periodically advertise its Sticky- 684 Service-Table to the routers within its Neighboring Ingress 685 Group. 687 Upon receiving the Sticky-Service-Table from routers in its 688 Neighboring Ingress Group, each ingress router merges the 689 entries from the received Sticky-Service-Table to its own. 691 The ingress and the egress nodes perform the same actions as 692 described in Section 5.1. 694 6.4. A Solution that depends on the communication with 5G system 696 In this scenario, there is communication with 5G System and 697 network get notified by a mobile device is anchored to a new 698 PSA. 700 When a mobile device is re-anchoring from PSA1 to PSA2, 5GC EC 701 management system sends a notification to the router that is 702 directly connected to PSA1. The notification includes the 703 address of the new PSA that the mobile device is to be 704 anchored, i.e. the PSA2, and the mobile device's new IP 705 address. 707 Internet-Draft IPv6 for 5G Edge Sticky Service 709 In this scenario, the Sticky Service can be uniquely identified 710 by "Sticky Service ID" + "mobile device address". the Sticky- 711 Service-Table should include the following attributes: 713 - Sticky Service ID 714 - mobile device address 715 - Sticky Egress address 716 - Timer 718 Upon receiving the notification from the 5G EC management 719 system, the ingress router (i.e. the one directly connected to 720 the old PSA) sends the specific entry of the Sticky-Service 721 Table, i.e. "Sticky Service ID" + mobile device address + 722 Sticky Egress + Timer to the router directly connected to the 723 new PSA. 725 Upon receiving the entry, the ingress router merges the entry 726 into its own Sticky-Service-Table. 728 The ingress and egress router processing are the same as 729 described in Section 5.1 except a flow is now uniquely 730 identified by the "Sticky Service ID" + "mobile device address" 731 instead of "Sticky Service ID" + "Flow Label". 733 7. Expanding APN6 for Sticky Service information 735 The Application-aware ID and Service-Para Option described 736 [APN6] can be expanded to include the sticky service 737 information. 739 7.1. Sticky Service ID encoded in the Application-aware ID 741 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 742 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 743 | Sticky Level |StickyServiceID| Reserved | Flow ID | 744 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 746 Sticky Level: represent how important for an application to 747 stick to its ANYCAST servers. Some applications may prefer one 748 flow sticking to the original ANYCAST server, but not required. 749 Some applications may require the stickiness. 751 StickyServiceID: the ANYCAST address of the application 752 servers. 754 Internet-Draft IPv6 for 5G Edge Sticky Service 756 The Reserved field can be used for future to identifier the 5G 757 access domain for the flow. 759 Flow ID: the identifier for the flow that needs to stick to a 760 specific ANYCAST server. 762 7.2. Sticky Service Sub-TLV encoded in APN6 Service-para option 764 The Sticky-Dst-SubTLV described in the Section 4.2 of this 765 document can be included in the Service-Para Sub-TLVs field. 767 8. Manageability Considerations 769 To be added. 771 9. Security Considerations 773 To be added. 775 10. IANA Considerations 777 To be added. 779 11. References 781 11.1. Normative References 783 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 784 Requirement Levels", BCP 14, RFC 2119, March 1997. 786 [RFC4364] E. rosen, Y. Rekhter, "BGP/MPLS IP Virtual Private 787 networks (VPNs)", Feb 2006. 789 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in 790 RFC 2119 Key Words", BCP 14, RFC 8174, DOI 791 10.17487/RFC8174, May 2017, . 794 Internet-Draft IPv6 for 5G Edge Sticky Service 796 [RFC8200] s. Deering R. Hinden, "Internet Protocol, Version 6 797 (IPv6) Specification", July 2017 799 11.2. Informative References 801 [3GPP-EdgeComputing] 3GPP TR 23.748, "3rd Generation 802 Partnership Project; Technical Specification Group 803 Services and System Aspects; Study on enhancement of 804 support for Edge Computing in 5G Core network (5GC)", 805 Release 17 work in progress, Aug 2020. 807 [5G-EC-Metrics] L. Dunbar, H. Song, J. Kaippallimalil, "IP 808 Layer Metrics for 5G Edge Computing Service", draft- 809 dunbar-ippm-5g-edge-compute-ip-layer-metrics-00, 810 work-in-progress, Oct 2020. 812 [5G-EC-OSPF-EXT] L. Dunbar, H.Chen, A. Wang, "OSPF extension 813 for 5G Edge Computing Service", draft-dunbar-lsr-5g- 814 edge-compute-ospf-ext-05, work-in-progress, March 815 2021. 817 [5G-EC-BGP-EXT] L. Dunbar, K. Majumdar, H. Wang, "BGP NLRI App 818 Meta Data for 5G Edge Computing Service", draft- 819 dunbar-idr-5g-edge-compute-app-meta-data-02, work-in- 820 progress, March 2021. 822 [APN6] Z. Li, et al, "Application-aware IPv6 Networking 823 (APN6) Encapsulation", draft-li-6man-app-aware-ipv6- 824 network-03, work-in-progress, Feb 2021. 826 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation 827 Subsequent Address Family Identifier (SAFI) and the 828 BGP Tunnel Encapsulation Attribute", April 2009. 830 [BGP-SDWAN-Port] L. Dunbar, H. Wang, W. Hao, "BGP Extension for 831 SDWAN Overlay Networks", draft-dunbar-idr-bgp-sdwan- 832 overlay-ext-03, work-in-progress, Nov 2018. 834 Internet-Draft IPv6 for 5G Edge Sticky Service 836 [SDWAN-EDGE-Discovery] L. Dunbar, S. Hares, R. Raszuk, K. 837 Majumdar, "BGP UPDATE for SDWAN Edge Discovery", 838 draft-dunbar-idr-sdwan-edge-discovery-00, work-in- 839 progress, July 2020. 841 [Tunnel-Encap] E. Rosen, et al "The BGP Tunnel Encapsulation 842 Attribute", draft-ietf-idr-tunnel-encaps-10, Aug 843 2018. 845 12. Acknowledgments 847 Acknowledgements to Gyan Mishra, Jeffrey Zhang, Joel Halpern, 848 Ron Bonica, Donald Eastlake, and Eduard Vasilenko for their 849 review and contributions. 851 This document was prepared using 2-Word-v2.0.template.dot. 853 Authors' Addresses 855 Linda Dunbar 856 Futurewei 857 Email: ldunbar@futurewei.com 859 John Kaippallimalil 860 Futurewei 861 Email: john.kaippallimalil@futurewei.com