idnits 2.17.00 (12 Aug 2021) /tmp/idnits23996/draft-ietf-quic-multipath-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'SHOULD not' in this paragraph: Both endpoints, namely the client and the server, can close a path, by sending PATH_ABANDON frame (see Section 12.1) which abandons the path with a corresponding Path Identifier. Once a path is marked as "abandoned", it means that the resources related to the path, such as the used connection IDs, can be released. However, information related to data delivered over that path SHOULD not be released immediately as acknowledgments can still be received or other frames that also may trigger retransmission of data on another path. -- The document date (7 March 2022) is 68 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '0' on line 855 == Missing Reference: 'X' is mentioned on line 946, but not defined == Missing Reference: 'Y' is mentioned on line 910, but not defined -- Looks like a reference, but probably isn't: '1' on line 915 == Missing Reference: 'U' is mentioned on line 913, but not defined -- Looks like a reference, but probably isn't: '2' on line 913 == Missing Reference: 'V' is mentioned on line 915, but not defined == Unused Reference: 'QUIC-Invariants' is defined on line 1233, but no explicit reference was found in the text Summary: 0 errors (**), 0 flaws (~~), 6 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 QUIC Working Group Y. Liu, Ed. 3 Internet-Draft Y. Ma 4 Intended status: Standards Track Alibaba Inc. 5 Expires: 8 September 2022 Q. De Coninck, Ed. 6 O. Bonaventure 7 UCLouvain 8 C. Huitema 9 Private Octopus Inc. 10 M. Kuehlewind, Ed. 11 Ericsson 12 7 March 2022 14 Multipath Extension for QUIC 15 draft-ietf-quic-multipath-01 17 Abstract 19 This document specifies a multipath extension for the QUIC protocol 20 to enable the simultaneous usage of multiple paths for a single 21 connection. 23 Discussion Venues 25 This note is to be removed before publishing as an RFC. 27 Discussion of this document takes place on the QUIC Working Group 28 mailing list (quic@ietf.org), which is archived at 29 https://mailarchive.ietf.org/arch/browse/quic/. 31 Source for this draft and an issue tracker can be found at 32 https://github.com/mirjak/draft-lmbdhk-quic-multipath. 34 Status of This Memo 36 This Internet-Draft is submitted in full conformance with the 37 provisions of BCP 78 and BCP 79. 39 Internet-Drafts are working documents of the Internet Engineering 40 Task Force (IETF). Note that other groups may also distribute 41 working documents as Internet-Drafts. The list of current Internet- 42 Drafts is at https://datatracker.ietf.org/drafts/current/. 44 Internet-Drafts are draft documents valid for a maximum of six months 45 and may be updated, replaced, or obsoleted by other documents at any 46 time. It is inappropriate to use Internet-Drafts as reference 47 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on 8 September 2022. 50 Copyright Notice 52 Copyright (c) 2022 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 57 license-info) in effect on the date of publication of this document. 58 Please review these documents carefully, as they describe your rights 59 and restrictions with respect to this document. Code Components 60 extracted from this document must include Revised BSD License text as 61 described in Section 4.e of the Trust Legal Provisions and are 62 provided without warranty as described in the Revised BSD License. 64 Table of Contents 66 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 67 1.1. Conventions and Definitions . . . . . . . . . . . . . . . 4 68 2. High-level overview . . . . . . . . . . . . . . . . . . . . . 5 69 3. Handshake Negotiation and Transport Parameter . . . . . . . . 6 70 4. Path Setup and Removal . . . . . . . . . . . . . . . . . . . 7 71 4.1. Path Initiation . . . . . . . . . . . . . . . . . . . . . 8 72 4.2. Path Close . . . . . . . . . . . . . . . . . . . . . . . 8 73 4.2.1. Use PATH_ABANDON Frame to Close a Path . . . . . . . 8 74 4.2.2. Effect of RETIRE_CONNECTION_ID Frame . . . . . . . . 9 75 4.2.3. Idle Timeout . . . . . . . . . . . . . . . . . . . . 10 76 4.3. Path States . . . . . . . . . . . . . . . . . . . . . . . 10 77 5. Congestion Control . . . . . . . . . . . . . . . . . . . . . 12 78 6. Computing Path RTT . . . . . . . . . . . . . . . . . . . . . 12 79 7. Packet Scheduling . . . . . . . . . . . . . . . . . . . . . . 13 80 8. Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . 14 81 9. Packet Number Space and Use of Connection ID . . . . . . . . 14 82 9.1. Using One Packet Number Space . . . . . . . . . . . . . . 15 83 9.1.1. Sending Acknowledgements and Handling Ranges . . . . 15 84 9.1.2. ACK Delay Considerations . . . . . . . . . . . . . . 16 85 9.2. Using Multiple Packet Number Spaces . . . . . . . . . . . 16 86 9.2.1. Packet Protection for QUIC Multipath . . . . . . . . 17 87 9.2.2. Key Update for QUIC Multipath . . . . . . . . . . . . 18 88 10. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 18 89 10.1. Path Establishment . . . . . . . . . . . . . . . . . . . 18 90 10.2. Path Closure . . . . . . . . . . . . . . . . . . . . . . 19 91 11. Implementation Considerations . . . . . . . . . . . . . . . . 21 92 11.1. Handling different PMTU sizes . . . . . . . . . . . . . 21 93 12. New Frames . . . . . . . . . . . . . . . . . . . . . . . . . 22 94 12.1. PATH_ABANDON Frame . . . . . . . . . . . . . . . . . . . 22 95 12.2. ACK_MP Frame . . . . . . . . . . . . . . . . . . . . . . 24 97 13. Error Codes . . . . . . . . . . . . . . . . . . . . . . . . . 24 98 14. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25 99 15. Security Considerations . . . . . . . . . . . . . . . . . . . 26 100 16. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 26 101 17. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 26 102 18. References . . . . . . . . . . . . . . . . . . . . . . . . . 26 103 18.1. Normative References . . . . . . . . . . . . . . . . . . 26 104 18.2. Informative References . . . . . . . . . . . . . . . . . 27 105 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 28 107 1. Introduction 109 This document specifies an extension to QUIC version 1 110 [QUIC-TRANSPORT] to enable the simultaneous usage of multiple paths 111 for a single connection. 113 This proposal is based on several basic design points: 115 * Re-use as much as possible mechanisms of QUIC version 1. In 116 particular this proposal uses path validation as specified for 117 QUIC version 1 and aims to re-use as much as possible of QUIC's 118 connection migration. 120 * Use the same packet header formats as QUIC version 1 to avoid the 121 risk of packets being dropped by middleboxes (which may only 122 support QUIC version 1) 124 * Congestion Control must be per-path (following [QUIC-TRANSPORT]) 125 which usually also requires per-path RTT measurements 127 * PMTU discovery should be performed per-path 129 * A path is determined by the 4-tuple of source and destination IP 130 address as well as source and destination port. Therefore there 131 can be at most one active paths/connection ID per 4-tuple. 133 The path management specified in Section 9 of [QUIC-TRANSPORT] 134 fulfills multiple goals: it directs a peer to switch sending through 135 a new preferred path, and it allows the peer to release resources 136 associated with the old path. Multipath requires several changes to 137 that mechanism: 139 * Allow simultaneous transmission of non probing frames on multiple 140 paths. 142 * Continue using an existing path even if non-probing frames have 143 been received on another path. 145 * Manage the removal of paths that have been abandoned. 147 As such this extension specifies a departure from the specification 148 of path management in Section 9 of [QUIC-TRANSPORT] and therefore 149 requires negotiation between the two endpoints using a new transport 150 parameter, as specified in Section 3. 152 This proposal supports the negotiation of either the use of one 153 packet number space for all paths or the use of separate packet 154 number spaces per path. While both approaches are supported by the 155 specification in this version of the document, the intention for the 156 final publication of a multipath extension for QUIC is to choose one 157 option in order to avoid incompatibility. More evaluation and 158 implementation experience is needed to select one approach before 159 final publication. Some discussion about pros and cons can be found 160 here: https://github.com/mirjak/draft-lmbdhk-quic- 161 multipath/blob/master/presentations/PacketNumberSpace_s.pdf 163 As currently defined in this version of the draft the use of multiple 164 packet number spaces requires the use of connection IDs is both 165 directions. Today's deployments often only use destination 166 connection ID when sending packets from the client to the server as 167 this addresses the most important use cases for migration, like NAT 168 rebinding or mobility events. Further discussion and work is 169 required to evaluate if the use of multiple packet number spaces 170 could be supported as well when the connection ID is only present in 171 one direction. 173 This proposal does not cover address discovery and management. 174 Addresses and the actual decision process to setup or tear down paths 175 are assumed to be handled by the application that is using the QUIC 176 multipath extension. Further, this proposal only specifies a simple 177 basic packet scheduling algorithm in order to provide some basic 178 implementation guidance. However, more advanced algorithms as well 179 as potential extensions to enhance signaling of the current path 180 state are expected as future work. 182 1.1. Conventions and Definitions 184 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 185 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 186 "OPTIONAL" in this document are to be interpreted as described in BCP 187 14 [RFC2119] [RFC8174] when, and only when, they appear in all 188 capitals, as shown here. 190 We assume that the reader is familiar with the terminology used in 191 [QUIC-TRANSPORT]. In addition, we define the following terms: 193 * Path: refers to the 4-tuple {source IP address, source port 194 number, destination IP address, destination port number}. A path 195 refers to "network path" used in [QUIC-TRANSPORT]. 197 * Path Identifier (Path ID): An identifier that is used to identify 198 a path in a QUIC connection at an endpoint. Path Identifier is 199 used in multipath control frames (etc. PATH_ABANDON frame) to 200 identify a path. By default, it is defined as the sequence number 201 of the destination Connection ID used for sending packets on that 202 particular path, but alternative definitions can be used if the 203 length of that connection ID is zero. 205 * Packet Number Space Identifier (PN Space ID): An identifier that 206 is used to distinguish packet number spaces for different paths. 207 It is used in 1-RTT packets and ACK_MP frames. Each node 208 maintains a list of "Received Packets" for each of the CID that it 209 provided to the peer, which is used for acknowledging packets 210 received with that CID. 212 The difference between Path Identifier and Packet Number Space 213 Identifier, is that the Path Identifier is used in multipath control 214 frames to identify a path, and the Packet Number Space Identifier is 215 used in 1-RTT packets and ACK_MP frames to distinguish packet number 216 spaces for different paths. Both identifiers have the same value, 217 which is the sequence number of the connection ID, if a non-zero 218 connection ID is used. If the connection ID is zero length, the 219 Packet Number Space Identifier is 0, while the Path Identifier is 220 selected on path establishment. 222 2. High-level overview 224 The multipath extensions to QUIC proposed in this document enable the 225 simultaneous utilization of different paths to exchange non-probing 226 QUIC frames for a single connection. This contrasts with the base 227 QUIC protocol [QUIC-TRANSPORT] that includes a connection migration 228 mechanism that selects only one path to exchange such frames. 230 A multipath QUIC connection starts with a QUIC handshake as a regular 231 QUIC connection. See further Section Section 3. The peers use the 232 enable_multipath transport parameter during the handshake to 233 negotiate the utilization of the multipath capabilities. The 234 active_connection_id_limit transport parameter limits the maximum 235 number of active paths that can be used during a connection. A 236 multipath QUIC connection is thus an established QUIC connection 237 where the enable_multipath transport parameter has been successfully 238 negotiated. 240 To add a new path to an existing multipath QUIC connection, a client 241 starts a path validation on the chosen path, as further described in 242 Section Section 4. In this version of the document, a QUIC server 243 does not initiate the creation of a path, but it can validate a new 244 path created by a client. A new path can only be used once it has 245 been validated. Each endpoint associates a Path identifier to each 246 path. This identifier is notably used when a peer sends a 247 PATH_ABANDON frame to indicate that it has closed the path whose 248 identifier is contained in the PATH_ABANDON frame. 250 In addition to these core features, an application using Multipath 251 QUIC will typically need additional algorithms to handle the number 252 of active paths and how they are used to send packets. As these 253 differ depending on the application's requirements, their 254 specification is out of scope of this document. 256 3. Handshake Negotiation and Transport Parameter 258 This extension defines a new transport parameter, used to negotiate 259 the use of the multipath extension during the connection handshake, 260 as specified in [QUIC-TRANSPORT]. The new transport parameter is 261 defined as follow: 263 * name: enable_multipath (TBD - experiments use 0xbabf) 265 * value: 0 (default) for disabled. 267 Endpoints use 2-bits in the value field for negotiating one or more 268 PN spaces, available option values are listed in Table 1 : 270 +========+=================================================+ 271 | Option | Definition | 272 +========+=================================================+ 273 | 0x0 | don't support multipath | 274 +--------+-------------------------------------------------+ 275 | 0x1 | only support one PN space for multipath | 276 +--------+-------------------------------------------------+ 277 | 0x2 | only support multiple PN spaces for multipath | 278 +--------+-------------------------------------------------+ 279 | 0x3 | support both one PN space and multiple PN space | 280 +--------+-------------------------------------------------+ 282 Table 1: Available value for enable_multipath 284 If for any one of the endpoints the parameter is absent or set to 0, 285 or if the two endpoints select incompatible values, one proposing 0x1 286 and the other proposing 0x2, the endpoints MUST fallback to 287 [QUIC-TRANSPORT] with single path and MUST NOT use any frame or 288 mechanism defined in this document. 290 If an endpoint proposes the value 0x3, the value proposed by the 291 other is accepted. If both endpoints propose the value 0x3, the 292 value 0x2 is negotiated. 294 If endpoint receives unexpected value for the transport parameter 295 "enable_multipath", it MUST treat this as a connection error of type 296 MP_CONNECTION_ERROR and close the connection. 298 This extension does not change the definition of any transport 299 parameter defined in Section 18.2. of [QUIC-TRANSPORT]. 301 Inline with the definition in [QUIC-TRANSPORT] 302 disable_active_migration also disables multipath support, except 303 "after a client has acted on a preferred_address transport parameter" 304 Section 18.2. of [QUIC-TRANSPORT]. 306 The transport parameter "active_connection_id_limit" [QUIC-TRANSPORT] 307 limits the number of usable Connection IDs, and also limits the 308 number of concurrent paths. For the QUIC multipath extension this 309 limit even applies when no connection ID is exposed in the QUIC 310 header. 312 4. Path Setup and Removal 314 After completing the handshake, endpoints have agreed to enable 315 multipath feature and can start using multiple paths. This document 316 does not specify how an endpoint that is reachable via several 317 addresses announces these addresses to the other endpoint. In 318 particular, if the server uses the preferred_address transport 319 parameter, clients SHOULD NOT assume that the initial server address 320 and the addresses contained in this parameter can be simultaneously 321 used for multipath. Furthermore, this document does not discuss when 322 a client decides to initiate a new path. We delegate such discussion 323 in separate documents. 325 This proposal adds one multipath control frame for path management: 327 * PATH_ABANDON frame for the receiver side to abandon the path 328 Section 12.1 330 All the new frames are sent in 1-RTT packets [QUIC-TRANSPORT]. 332 4.1. Path Initiation 334 When the multipath option is negotiated, clients that want to use an 335 additional path MUST first initiate the Address Validation procedure 336 with PATH_CHALLENGE and PATH_RESPONSE frames described in Section 8 337 of [QUIC-TRANSPORT]. After receiving packets from the client on the 338 new paths, the servers MAY in turn attempt to validate these paths 339 using the same mechanisms. 341 If validation succeed, the client can send non-probing, 1-RTT packets 342 on the new paths. In contrast with the specification in Section 9 of 343 [QUIC-TRANSPORT], the server MUST NOT assume that receiving non- 344 probing packets on a new path indicates an attempt to migrate to that 345 path. Instead, servers SHOULD consider new paths over which non- 346 probing packets have been received as available for transmission. 348 4.2. Path Close 350 Each endpoint manages the set of paths that are available for 351 transmission. At any time in the connection, each endpoint can 352 decide to abandon one of these paths, following for example changes 353 in local connectivity or changes in local preferences. After an 354 endpoint abandons a path, the peer will not receive any more non- 355 probing packets on that path. 357 An endpoint that wants to close a path SHOULD NOT rely on implicit 358 signals like idle time or packet losses, but instead SHOULD use 359 explicit request to terminate path by sending the PATH_ABANDON frame 360 (see Section 12.1). 362 4.2.1. Use PATH_ABANDON Frame to Close a Path 364 Both endpoints, namely the client and the server, can close a path, 365 by sending PATH_ABANDON frame (see Section 12.1) which abandons the 366 path with a corresponding Path Identifier. Once a path is marked as 367 "abandoned", it means that the resources related to the path, such as 368 the used connection IDs, can be released. However, information 369 related to data delivered over that path SHOULD not be released 370 immediately as acknowledgments can still be received or other frames 371 that also may trigger retransmission of data on another path. 373 The endpoint sending the PATH_ABANDON frame SHOULD consider a path as 374 abandoned when the packet that contained the PATH_ABANDON frame is 375 acknowledged. When releasing resources of a path, the endpoint 376 SHOULD send a RETIRE_CONNECTION_ID frame for the connection IDs used 377 on the path, if any. 379 The receiver of a PATH_ABANDON frame SHOULD NOT release its resources 380 immediately but SHOULD wait for the receive of the 381 RETIRE_CONNECTION_ID frame for the used connection IDs or 3 RTOs. 383 Usually it is expected that the PATH_ABANDON frame is used by the 384 client to indicate to the server that path conditions have changed 385 such that the path is or will be not usable anymore, e.g. in case of 386 an mobility event. The PATH_ABANDON frame therefore indicates to the 387 receiving peer that the sender does not intend to send any packets on 388 that path anymore but also recommends to the receiver that no packets 389 should be sent in either direction. The receiver of an PATH_ABANDON 390 frame MAY also send an PATH_ABANDON frame to signal its own 391 willingness to not send any packet on this path anymore. 393 If connection IDs are used, PATH_ABANDON frames can be sent on any 394 path, not only the path that is intended to be closed. Thus a path 395 can be abandoned even if connectivity on that path is already broken. 396 If no connection IDs are used and the PATH_ABANDON frame has to sent 397 on the path that is intended to be closed, it is possible that the 398 packet containing the PATH_ABANDON frame or the packet containing the 399 ACK for the PATH_ABANDON frame cannot be received anymore and the 400 endpoint might need to rely on an idle time out to close the path, as 401 described in Section Section 4.2.3. 403 Retransmittable frames, that have previously been send on the 404 abandoned path and are considered lost, SHOULD be retransmitted on a 405 different path. 407 If a PATH_ABANDON frame is received for the only active path of a 408 QUIC connection, the receiving peer SHOULD send a CONNECTION_CLOSE 409 frame and enters the closing state. If the client received a 410 PATH_ABANDON frame for the last open path, it MAY instead try to open 411 a new path, if available, and only initiate connection closure if 412 path validation fails or a CONNECTION_CLOSE frame is received from 413 the server. Similarly the server MAY wait for a short, limited time 414 such as one RTO if a path probing packet is received on a new path 415 before sending the CONNECTION_CLOSE frame. 417 4.2.2. Effect of RETIRE_CONNECTION_ID Frame 419 Receiving a RETIRE_CONNECTION_ID frame causes the endpoint to discard 420 the resources associated with that connection ID. If the connection 421 ID was used by the peer to identify a path from the peer to this 422 endpoint, the resources include the list of received packets used to 423 send acknowledgements. The peer MAY decide to keep sending data 424 using the same IP addresses and UDP ports previously associated with 425 the connection ID, but MUST use a different connection ID when doing 426 so. 428 4.2.3. Idle Timeout 430 [QUIC-TRANSPORT] allows for closing of connections if they stay idle 431 for too long. The connection idle timeout in multipath QUIC is 432 defined as "no packet received on any path for the duration of the 433 idle timeout". When only one path is available, servers MUST follow 434 the specifications in [QUIC-TRANSPORT]. 436 When more than one path is available, hosts shall monitor the arrival 437 of non-probing packets and the acknowledgements for the packets sent 438 over each path. Hosts SHOULD stop sending traffic on a path if for 439 at least max_idle_timeout milliseconds (a) no non-probing packet was 440 received or (b) no non-probing packet sent over this path was 441 acknowledged, but MAY ignore that rule if it would disqualify all 442 available paths. To avoid idle timeout of a path, endpoints can send 443 ack-eliciting packets such as packets containing PING frames 444 (Section 19.2 of [QUIC-TRANSPORT]) on that path to keep it alive. 445 Sending periodic PING frames also helps prevent middlebox timeout, as 446 discussed in Section 10.1.2 of [QUIC-TRANSPORT]. 448 Server MAY release the resource associated with paths for which no 449 non-probing packet was received for a sufficiently long path-idle 450 delay, but SHOULD only release resource for the last available path 451 if no traffic is received for the duration of the idle timeout, as 452 specified in Section 10.1 of [QUIC-TRANSPORT]. This means if all 453 paths remain idle for the idle timeout, the connection is implicitly 454 closed. 456 Server implementations need to select the sub-path idle timeout as a 457 trade- off between keeping resources, such as connection IDs, in use 458 for an excessive time or having to promptly reestablish a path after 459 a spurious estimate of path abandonment by the client. 461 4.3. Path States 463 Figure 1 shows the states that an endpoint's path can have. 465 o 466 | PATH_CHALLENGE sent/received on new path 467 v 468 +------------+ Path validation abandoned 469 | Validating |----------------------------------+ 470 +------------+ | 471 | | 472 | PATH_RESPONSE received | 473 | | 474 v Associated CID have been retired | 475 +------------+ Path's idle timeout | 476 | Active |----------------------------------+ 477 +------------+ | 478 | | 479 | PATH_ABANDONED sent/received | 480 v | 481 +------------+ | 482 | Closing | | 483 +------------+ | 484 | | 485 | Associated CID have been retired | 486 | Path's idle timeout | 487 v | 488 +------------+ | 489 | Closed |<---------------------------------+ 490 +------------+ 492 Figure 1: States of a path 494 In non-final states, hosts have to track the following information. 496 * Associated 4-tuple: The tuple (source IP, source port, destination 497 IP, destination port) used by the endhost to send packets over the 498 path. 500 * Associated Destination Connection ID: The Connection ID used to 501 send packets over the path. 503 If multiple packet number spaces are used over the connection, hosts 504 MUST also track the following information. 506 * Path Packet Number Space: The endpoint maintains a separate packet 507 number for sending and receiving packets over this path. Packet 508 number considerations described in [QUIC-TRANSPORT] apply within 509 the given path. 511 In the "Active" state, hosts MUST also track the following 512 information. 514 * Associated Source Connection ID: The Connection ID used to receive 515 packets over the path. 517 A path in the "Validating" state performs path validation as 518 described in Section 8.2 of [QUIC-TRANSPORT]. An endhost should not 519 send non-probing frames on a path in "Validating" state, as it has no 520 guarantee that packets will actually reach the peer. 522 The endhost can use all the paths in the "Active" state, provided 523 that the congestion control and flow control currently allow sending 524 of new data on a path. 526 In the "Closing" state, the endhost SHOULD NOT send packets on this 527 path anymore, as there is no guarantee that the peer can still map 528 the packets to the connection. The endhost SHOULD wait for the 529 acknowledgment of the PATH_ABANDONED frame before moving the path to 530 the "Closed" state to ensure a graceful termination of the path. 532 When a path reaches the "Closed" state, the endhost releases all the 533 path's associated resources. Consequently, the endhost is not able 534 to send nor receive packets on this path anymore. 536 5. Congestion Control 538 Senders MUST manage per-path congestion status, and MUST NOT send 539 more data on a given path than congestion control on that path 540 allows. This is already a requirement of [QUIC-TRANSPORT]. 542 When a Multipath QUIC connection uses two or more paths, there is no 543 guarantee that these paths are fully disjoint. When two (or more 544 paths) share the same bottleneck, using a standard congestion control 545 scheme could result in an unfair distribution of the bandwidth with 546 the multipath connection getting more bandwidth than competing single 547 paths connections. Multipath TCP uses the LIA congestion control 548 scheme specified in [RFC6356] to solve this problem. This scheme can 549 immediately be adapted to Multipath QUIC. Other coupled congestion 550 control schemes have been proposed for Multipath TCP such as [OLIA]. 552 6. Computing Path RTT 554 Acknowledgement delays are the sum of two one-way delays, the delay 555 on the packet sending path and the delay on the return path chosen 556 for the acknowledgements. When different paths have different 557 characteristics, this can cause acknowledgement delays to vary 558 widely. Consider for example multipath transmission using both a 559 terrestrial path, with a latency of 50ms in each direction, and a 560 geostationary satellite path, with a latency of 300ms in both 561 directions. The acknowledgement delay will depend on the combination 562 of paths used for the packet transmission and the ACK transmission, 563 as shown in Table 2. 565 +======================+=============+===========+ 566 | ACK Path \ Data path | Terrestrial | Satellite | 567 +======================+=============+===========+ 568 | Terrestrial | 100ms | 350ms | 569 +----------------------+-------------+-----------+ 570 | Satellite | 350ms | 600ms | 571 +----------------------+-------------+-----------+ 573 Table 2: Example of ACK delays using multiple 574 paths 576 Using the default algorithm specified in [QUIC-RECOVERY] would result 577 in suboptimal performance, computing average RTT and standard 578 deviation from series of different delay measurements of different 579 combined paths. At the same time, early tests showed that it is 580 desirable to send ACKs through the shortest path, because a shorter 581 ACK delay results in a tighter control loop and better performances. 582 The tests also showed that it is desirable to send copies of the ACKs 583 on multiple paths, for robustness if a path experiences sudden 584 losses. 586 An early implementation mitigated the delay variation issue by using 587 time stamps, as specified in [QUIC-Timestamp]. When the timestamps 588 are present, the implementation can estimate the transmission delay 589 on each one-way path, and can then use these one way delays for more 590 efficient implementations of recovery and congestion control 591 algorithms. 593 If timestamps are not available, implementations could estimate one 594 way delays using statistical techniques. For example, in the example 595 shown in Table 1, implementations can use use "same path" 596 measurements to estimate the one way delay of the terrestrial path to 597 about 50ms in each direction, and that of the satellite path to about 598 300ms. Further measurements can then be used to maintain estimates 599 of one way delay variations, using logical similar to Kalman filters. 600 But statistical processing is error-prone, and using time stamps 601 provides more robust measurements. 603 7. Packet Scheduling 605 The transmission of QUIC packets on a regular QUIC connection is 606 regulated by the arrival of data from the application and the 607 congestion control scheme. QUIC packets can only be sent when the 608 congestion window of at least one path is open. 610 Multipath QUIC implementations also need to include a packet 611 scheduler that decides, among the paths whose congestion window is 612 open, the path over which the next QUIC packet will be sent. Many 613 factors can influence the definition of these algorithms and their 614 precise definition is outside the scope of this document. Various 615 packet schedulers have been proposed and implemented, notably for 616 Multipath TCP. A companion draft [I-D.bonaventure-iccrg-schedulers] 617 provides several general-purpose packet schedulers depending on the 618 application goals. 620 8. Recovery 622 Simultaneous use of multiple paths enables different retransmission 623 strategies to cope with losses such as: a) retransmitting lost frames 624 over the same path, b) retransmitting lost frames on a different or 625 dedicated path, and c) duplicate lost frames on several paths (not 626 recommended for general purpose use due to the network overhead). 627 While this document does not preclude a specific strategy, more 628 detailed specification is out of scope. 630 9. Packet Number Space and Use of Connection ID 632 If the connection ID is present (non-zero length) in the packet 633 header, the connection ID is used to identify the path. If no 634 connection ID is present, the 4 tuple identifies the path. The 635 initial path that is used during the handshake (and multipath 636 negotiation) has the path ID 0 and therefore all 0-RTT packets are 637 also tracked and processed with the path ID 0. For 1-RTT packets the 638 path ID is the sequence number of the Destination Connection ID 639 present in the packet header, as defined in Section 5.1.1 of 640 [QUIC-TRANSPORT], or also 0 if the Connection ID is zero-length. 642 If non-zero-length Connection IDs are used, an endpoint MUST use 643 different Connection IDs on different paths. Still, the receiver may 644 observe the same Connection ID used on different 4-tuples due to, 645 e.g., NAT rebinding. In such case, the receiver reacts as specified 646 in Section 9.3 of [QUIC-TRANSPORT]. 648 Acknowledgements of Initial and Handshake packets MUST be carried 649 using ACK frames, as specified in [QUIC-TRANSPORT]. The ACK frames, 650 as defined in [QUIC-TRANSPORT], do not carry path identifiers. If 651 for any reason ACK frames are received in 1-RTT packets while the 652 state of multipath negotiation is ambiguous, they MUST be interpreted 653 as acknowledging packets sent on path 0. 655 Endpoints negotiate the use of one packet number space for all paths 656 or separate packet number spaces per path during the connection 657 handshake Section 3. While separate packet number spaces allow for 658 more efficient ACK encoding, especially when paths have highly 659 different latencies, this approach requires the use of a connection 660 ID. Therefore use of a single number space can be beneficial when 661 endpoints use zero-length connection ID for less overhead. 663 9.1. Using One Packet Number Space 665 If the multipath option is negotiated to use one packet number space 666 for all paths, the packet sequence numbers are allocated from the 667 common number space, so that, for example, packet number N could be 668 sent on one path and packet number N+1 on another. 670 ACK frames report the numbers of packets that have been received so 671 far, regardless of the path on which they have been received. That 672 means the senders needs to maintain an association between sent 673 packet numbers and the path over which these packets were sent. This 674 is necessary to implement per path congestion control. 676 When a packet is acknowledged, the state of the congestion control 677 MUST be updated for the path where the acknowledged packet was 678 originally sent. The RTT is calculated based on the delay between 679 the transmission of that packet and its first acknowledgement (see 680 Section 6) and is used to update the RTT statistics for the sending 681 path. 683 Also loss detection MUST be adapted to allow for different RTTs on 684 different paths. For example, timer computations should take into 685 account the RTT of the path on which a packet was sent. Detections 686 based on packet numbers shall compare a given packet number to the 687 highest packet number received for that path. 689 9.1.1. Sending Acknowledgements and Handling Ranges 691 If senders decide to send packets on paths with different 692 transmission delays, some packets will very likely be received out of 693 order. This will cause the ACK frames to carry multiple ranges of 694 received packets. The large number of range increases the size of 695 ACK frames, causing transmission and processing overhead. 697 The size and overhead of the ACK frames can be controlled by the 698 combination of one or several of the following: 700 * Not transmitting again ACK ranges that were present in an ACK 701 frame acknowledged by the peer. 703 * Delay acknowledgements to allow for arrival of "hole filling" 704 packets. 706 * Limit the total number of ranges sent in an ACK frame. 708 * Limiting the number of transmissions of a specific ACK range, on 709 the assumption that a sufficient number of transmissions almost 710 certainly ensures reception by the peer. 712 * Send multiple messages for a given path in a single socket 713 operation, so that a series of packets sent from a single path 714 uses a series of consecutive sequence numbers without creating 715 holes. 717 9.1.2. ACK Delay Considerations 719 The ACK Delay field of the ACK frame is relative to the largest 720 acknowledged packet number (see Section 13.2.5 of [QUIC-TRANSPORT]). 721 When using paths with different transmission delays, the reported 722 host delay will most of the time relate to the path with the shortest 723 latency. To collect ACK delays on all the paths, hosts should rely 724 on time stamps as described in [QUIC-Timestamp]. 726 9.2. Using Multiple Packet Number Spaces 728 If the multipath option is enabled with a value of 2, each path has 729 its own packet number space for transmitting 1-RTT packets and a new 730 ACK frame format is used as specified in Section 12.2. Compared to 731 the QUIC version 1 ACK frame, the ACK_MP frames additionally contains 732 a Packet Number Space Identifier (PN Space ID). The PN Space ID used 733 to distinguish packet number spaces for different paths and is simply 734 derived from the sequence number of Destination Connection ID. 735 Therefore, the packet number space for 1-RTT packets can be 736 identified based on the Destination Connection ID in each packets. 738 As soon as the negotiation of multipath support with value 2 is 739 completed, endpoints SHOULD use ACK_MP frames instead of ACK frames 740 for acknowledgements of 1-RTT packets on path 0, as well as for 0-RTT 741 packets that are acknowledged after the handshake concluded. 743 Following [QUIC-TRANSPORT], each endpoint uses NEW_CONNECTION_ID 744 frames to issue usable connections IDs to reach it. Before an 745 endpoint adds a new path by initiating path validation, it MUST check 746 whether at least one unused Connection ID is available for each side. 748 If the transport parameter "active_connection_id_limit" is negotiated 749 as N, the server provided N Connection IDs, and the client is already 750 actively using N paths, the limit is reached. If the client wants to 751 start a new path, it has to retire one of the established paths. 753 ACK_MP frame Section 12.2 can be returned via either a different 754 path, or the same path identified by the Path Identifier, based on 755 different strategies of sending ACK_MP frames. 757 Using multiple packet number spaces requires changes in the way AEAD 758 is applied for packet protection, as explained in Section 9.2.1, and 759 tighter constraints for key updates, as explained in Section 9.2.2. 761 9.2.1. Packet Protection for QUIC Multipath 763 Packet protection for QUIC version 1 is specified in Section 5 of 764 [QUIC-TLS]. The general principles of packet protection are not 765 changed for QUIC Multipath. No changes are needed for setting packet 766 protection keys, initial secrets, header protection, use of 0-RTT 767 keys, receiving out-of-order protected packets, receiving protected 768 packets, or retry packet integrity. However, the use of multiple 769 number spaces for 1-RTT packets requires changes in AEAD usage. 771 Section 5.3 of [QUIC-TLS] specifies AEAD usage, and in particular the 772 use of a nonce, N, formed by combining the packet protection IV with 773 the packet number. If multiple packet number spaces are used, the 774 packet number alone would not guarantee the uniqueness of the nonce. 776 In order to guarantee the uniqueness of the nonce, the nonce N is 777 calculated by combining the packet protection IV with the packet 778 number and with the path identifier. 780 The path ID for 1-RTT packets is the sequence number of of 781 [QUIC-TRANSPORT], or zero if the Connection ID is zero-length. 782 Section 19 of [QUIC-TRANSPORT] encodes the Connection ID Sequence 783 Number as a variable-length integer, allowing values up to 2^62-1; in 784 this specification a range of less than 2^32-1 values MUST be used 785 before updating the packet protection key. 787 To calculate the nonce, a 96 bit path-and-packet-number is composed 788 of the 32 bit Connection ID Sequence Number in byte order, two zero 789 bits, and the 62 bits of the reconstructed QUIC packet number in 790 network byte order. If the IV is larger than 96 bits, the path-and- 791 packet-number is left-padded with zeros to the size of the IV. The 792 exclusive OR of the padded packet number and the IV forms the AEAD 793 nonce. 795 For example, assuming the IV value is 6b26114b9cba2b63a9e8dd4f, the 796 connection ID sequence number is 3, and the packet number is aead, 797 the nonce will be set to 6b2611489cba2b63a9e873e2. 799 9.2.2. Key Update for QUIC Multipath 801 The Key Phase bit update process for QUIC version 1 is specified in 802 Section 6 of [QUIC-TLS]. The general principles of key update are 803 not changed in this specification. Following QUIC version 1, the Key 804 Phase bit is used to indicate which packet protection keys are used 805 to protect the packet. The Key Phase bit is toggled to signal each 806 subsequent key update. Because of network delays, packets protected 807 with the older key might arrive later than the packets protected with 808 the new key. Therefore, the endpoint needs to retain old packet keys 809 to allow these delayed packets to be processed and it must 810 distinguish between the new key and the old key. In QUIC version 1, 811 this is done using packet numbers so that the rule is made simple: 812 Use the older key if packet number is lower than any packet number 813 frame the current key phase. 815 When using multiple packet number spaces on different paths, some 816 care is needed when initiating the Key Update process, as different 817 paths use different packet number spaces but share a single key. 818 When a key update is initiated on one path, packets sent to another 819 path needs to know when the transition is complete. Otherwise, it is 820 possible that the other paths send packets with the old keys, but 821 skip sending any packets in the current key phase and directly jump 822 to sending packet in the next key phase. When that happens, as the 823 endpoint can only retain two sets of packet protection keys with the 824 1-bit Key Phase bit, the other paths cannot distinguish which key 825 should be used to decode received packets, which results in a key 826 rotation synchronization problem. 828 To address such a synchronization issue, if key update is initialized 829 on one path, the sender SHOULD send at least one packet with the new 830 key on all active paths. Further, an endpoint MUST NOT initiate a 831 subsequent key update until a packet with the current key has been 832 acknowledged on each path. 834 Following Section 5.4 of [QUIC-TLS], the Key Phase bit is protected, 835 so sending multiple packets with Key Phase bit flipping at the same 836 time should not cause linkability issue. 838 10. Examples 840 10.1. Path Establishment 842 Figure 2 illustrates an example of new path establishment using 843 multiple packet number spaces. 845 Client Server 847 (Exchanges start on default path) 848 1-RTT[]: NEW_CONNECTION_ID[C1, Seq=1] --> 849 <-- 1-RTT[]: NEW_CONNECTION_ID[S1, Seq=1] 850 <-- 1-RTT[]: NEW_CONNECTION_ID[S2, Seq=2] 851 ... 852 (starts new path) 853 1-RTT[0]: DCID=S2, PATH_CHALLENGE[X] --> 854 Checks AEAD using nonce(CID sequence 2, PN 0) 855 <-- 1-RTT[0]: DCID=C1, PATH_RESPONSE[X], PATH_CHALLENGE[Y], 856 ACK_MP[Seq=2,PN=0] 857 Checks AEAD using nonce(CID sequence 1, PN 0) 858 1-RTT[1]: DCID=S2, PATH_RESPONSE[Y], 859 ACK_MP[Seq=1, PN=0], ... --> 861 Figure 2: Example of new path establishment 863 In Figure Figure 2, the endpoints first exchange new available 864 Connection IDs with the NEW_CONNECTION_ID frame. In this example the 865 client provides one Connection ID (C1 with sequence number 1), and 866 server provides two Connection IDs (S1 with sequence number 1, and S2 867 with sequence number 2). 869 Before the client opens a new path by sending an packet on that path 870 with a PATH_CHALLENGE frame, it has to check whether there is an 871 unused Connection IDs available for each side. In this example the 872 client chooses the Connection ID S2 as the Destination Connection ID 873 in the new path. 875 If the client has used all the allocated CID, it is supposed to 876 retire those that are not used anymore, and the server is supposed to 877 provide replacements, as specified in [QUIC-TRANSPORT]. Usually it 878 is desired to provide one more connection ID as currently in used, to 879 allow for new paths or migration. 881 10.2. Path Closure 883 In this example the client detects the network environment change 884 (client's 4G/Wi-Fi is turned off, Wi-Fi signal is fading to a 885 threshold, or the quality of RTT or loss rate is becoming worse) and 886 wants to close the initial path. 888 Figure 3 illustrates an example of path closing when both the client 889 and the server use non-zero-length CIDs. For the first path, the 890 server's 1-RTT packets use DCID C1, which has a sequence number of 1; 891 the client's 1-RTT packets use DCID S2, which has a sequence number 892 of 2. For the second path, the server's 1-RTT packets use DCID C2, 893 which has a sequence number of 2; the client's 1-RTT packets use DCID 894 S3, which has a sequence number of 3. Note that the paths use 895 different packet number spaces. In this case, the client is going to 896 close the first path. It identifies the path by the sequence number 897 of the received packet's DCID over that path (path identifier type 898 0x00), hence using the path_id 1. Optionally, the server confirms 899 the path closure by sending an PATH_ABANDON frame using the sequence 900 number of the received packet's DCID over that path (path identifier 901 type 0x00) as path identifier, which corresponds to the path_id 2. 902 Both the client and the server can close the path after receiving the 903 RETIRE_CONNECTION_ID frame for that path. 905 Client Server 907 (client tells server to abandon a path) 908 1-RTT[X]: DCID=S2 PATH_ABANDON[path_id_type=0, path_id=1]-> 909 (server tells client to abandon a path) 910 <-1-RTT[Y]: DCID=C1 PATH_ABANDON[path_id_type=0, path_id=2], 911 ACK_MP[Seq=2, PN=X] 912 (client retires the corresponding CID) 913 1-RTT[U]: DCID=S3 RETIRE_CONNECTION_ID[2], ACK_MP[Seq=1, PN=Y] -> 914 (server retires the corresponding CID) 915 <- 1-RTT[V]: DCID=C2 RETIRE_CONNECTION_ID[1], ACK_MP[Seq=3, PN=U] 917 Figure 3: Example of closing a path when both the client and the 918 server choose to receive non-zero-length CIDs. 920 Figure 4 illustrates an example of path closing when the client 921 chooses to receive zero-length CIDs while the server chooses to 922 receive non-zero-length CIDs. Because there is a zero-length CID in 923 one direction, single packet number spaces are used. For the first 924 path, the client's 1-RTT packets use DCID S2, which has a sequence 925 number of 2. For the second path, the client's 1-RTT packets use 926 DCID S3, which has a sequence number of 3. Again, in this case, the 927 client is going to close the first path. Because the client now 928 receives zero-length CID packets, it needs to use path identifier 929 type 0x01, which identifies a path by the DCID sequence number of the 930 packets it sends over that path, and hence, it uses a path_id 2 in 931 its PATH_ABANDON frame. The server SHOULD stop sending new data on 932 the path indicated by the PATH_ABANDON frame after receiving it. 933 However, The client may want to repeat the PATH_ABANDON frame if it 934 sees the server continuing to send data. When the client's 935 PATH_ABANDON frame is acknowledged, it sends out a 936 RETIRE_CONNECTION_ID frame for the CID used on the first path. The 937 server can readily close the first path when it receives the 938 RETIRE_CONNECTION_ID frame from the client. However, since the 939 client will not receive a RETIRE_CONNECTION_ID frame, after sending 940 out the RETIRE_CONNECTION_ID frame, the client waits for 3 RTO before 941 closing the path. 943 Client Server 945 (client tells server to abandon a path) 946 1-RTT[X]: DCID=S2 PATH_ABANDON[path_id_type=1, path_id=2]-> 947 (server stops sending on that path after 948 receiving PATH_ABANDON) 949 (client retires the corresponding CID 950 after PATH_ABANDON is acknowledged) 951 1-RTT[X+1]: DCID=S3 RETIRE_CONNECTION_ID[2]-> 953 Figure 4: Example of closing a path when the client chooses to 954 receive zero- length CIDs while the server chooses to receive 955 non-zero-length CIDs 957 11. Implementation Considerations 959 11.1. Handling different PMTU sizes 961 An implementation should take care to handle different PMTU sizes 962 across multiple paths. One simple option if the PMTUs are relatively 963 similar is to apply the minimum PMTU of all paths to each path. The 964 benefit of such an approach is to simplify retransmission processing 965 as the content of lost packets initially sent on one path can be sent 966 on another path without further frame scheduling adaptations. 968 12. New Frames 970 All the new frames MUST only be sent in 1-RTT packet, and MUST NOT 971 use other encryption levels. 973 If an endpoint receives multipath-specific frames from packets of 974 other encryption levels, it MUST return MP_PROTOCOL_VIOLATION as a 975 connection error and close the connection. 977 12.1. PATH_ABANDON Frame 979 The PATH_ABANDON frame informs the peer to abandon a path. More 980 complex path management can be made possible with additional 981 extensions (e.g., PATH_STATUS frame in [I-D.liu-multipath-quic] ). 983 PATH_ABANDON frames are formatted as shown in Figure 5. 985 PATH_ABANDON Frame { 986 Type (i) = TBD-03 (experiments use 0xbaba05), 987 Path Identifier (..), 988 Error Code (i), 989 Reason Phrase Length (i), 990 Reason Phrase (..), 991 } 993 Figure 5: PATH_ABANDON Frame Format 995 PATH_ABANDON frames contain the following fields: 997 Path Identifier: An identifier of the path, which is formatted as 998 shown in Figure 6. 1000 * Identifier Type: Identifier Type field is set to indicate the type 1001 of path identifier. 1003 - Type 0: Refer to the connection identifier issued by the sender 1004 of the control frame. Note that this is the connection 1005 identifier used by the peer when sending packets on the to-be- 1006 closed path. This method SHOULD be used if this connection 1007 identifier is non-zero length. This method MUST NOT be used if 1008 this connection identifier is zero-length. 1010 - Type 1: Refer to the connection identifier issued by the 1011 receiver of the control frame. Note that this is the 1012 connection identifier used by the sender when sending packets 1013 on the to-be-closed path. This method MUST NOT be used if this 1014 connection identifier is zero-length. 1016 - Type 2: Refer to the path over which the control frame is sent 1017 or received. 1019 * Path Identifier Content: A variable-length integer specifying the 1020 path identifier. If Identifier Type is 2, the Path Identifier 1021 Content MUST be empty. 1023 Path Identifier { 1024 Identifier Type (i) = 0x00..0x02, 1025 [Path Identifier Content (i)], 1026 } 1028 Figure 6: Path Identifier Format 1030 Note: If the receiver of the PATH_ABANDON frame is using non-zero 1031 length Connection ID on that path, endpoint SHOULD use type 0x00 for 1032 path identifier in the control frame. If the receiver of the 1033 PATH_ABANDON frame is using zero-length Connection ID, but the peer 1034 is using non-zero length Connection ID on that path, endpoints SHOULD 1035 use type 0x01 for path identifier. If both endpoints are using 1036 0-length Connection IDs on that path, endpoints SHOULD only use type 1037 0x02 for path identifier. 1039 Error Code: A variable-length integer that indicates the reason for 1040 abandoning this path. 1042 Reason Phrase Length: A variable-length integer specifying the 1043 length of the reason phrase in bytes. Because an PATH_ABANDON 1044 frame cannot be split between packets, any limits on packet size 1045 will also limit the space available for a reason phrase. 1047 Reason Phrase: Additional diagnostic information for the closure. 1048 This can be zero length if the sender chooses not to give details 1049 beyond the Error Code value. This SHOULD be a UTF-8 encoded 1050 string [RFC3629], though the frame does not carry information, 1051 such as language tags, that would aid comprehension by any entity 1052 other than the one that created the text. 1054 PATH_ABANDON frames SHOULD be acknowledged. If a packet containing a 1055 PATH_ABANDON frame is considered lost, the peer SHOULD repeat it. 1057 If the Identifier Type is 0x00 or 0x01, PATH_ABANDON frames MAY be 1058 sent on any path, not only the path identified by the Path Identifier 1059 Content field. If the Identifier Type if 0x02, the PATH_ABANDON 1060 frame MUST only be sent on the path that is intended to be abandoned. 1062 12.2. ACK_MP Frame 1064 The ACK_MP frame (types TBD-00 and TBD-01; experiments use 1065 0xbaba00..0xbaba01) is an extension of the ACK frame defined by 1066 [QUIC-TRANSPORT]. It is used to acknowledge packets that were sent 1067 on different paths when using multiple packet number spaces. If the 1068 frame type is TBD-01, ACK_MP frames also contain the sum of QUIC 1069 packets with associated ECN marks received on the connection up to 1070 this point. 1072 ACK_MP frame is formatted as shown in Figure 7. 1074 ACK_MP Frame { 1075 Type (i) = TBD-00..TBD-01 (experiments use 0xbaba00..0xbaba01), 1076 Packet Number Space Identifier (i), 1077 Largest Acknowledged (i), 1078 ACK Delay (i), 1079 ACK Range Count (i), 1080 First ACK Range (i), 1081 ACK Range (..) ..., 1082 [ECN Counts (..)], 1083 } 1085 Figure 7: ACK_MP Frame Format 1087 Compared to the ACK frame specified in [QUIC-TRANSPORT], the 1088 following field is added. 1090 Packet Number Space Identifier: An identifier of the path packet 1091 number space, which is the sequence number of Destination Connection 1092 ID of the 1-RTT packets which are acknowledged by the ACK_MP frame. 1093 If the endpoint receives 1-RTT packets with zero-length Connection 1094 ID, it SHOULD use Packet Number Space Identifier 0 in ACK_MP frames. 1095 If an endpoint receives a ACK_MP frame with a non-existing packet 1096 number space ID, it MUST treat this as a connection error of type 1097 MP_PROTOCOL_VIOLATION and close the connection. 1099 When using a single packet number space, endhosts MUST NOT send 1100 ACK_MP frames. If an endhost receives an ACK_MP frame while a single 1101 packet number space was negotiated, it MUST treat this as a 1102 connection error of type MP_PROTOCOL_VIOLATION and close the 1103 connection. 1105 13. Error Codes 1107 Multipath QUIC transport error codes are 62-bit unsigned integers 1108 following [QUIC-TRANSPORT]. 1110 This section lists the defined multipath QUIC transport error codes 1111 that can be used in a CONNECTION_CLOSE frame with a type of 0x1c. 1112 These errors apply to the entire connection. 1114 MP_PROTOCOL_VIOLATION (experiments use 0xba01): An endpoint detected 1115 an error with protocol compliance that was not covered by more 1116 specific error codes. 1118 14. IANA Considerations 1120 This document defines a new transport parameter for the negotiation 1121 of enable multiple paths for QUIC, and two new frame types. The 1122 draft defines provisional values for experiments, but we expect IANA 1123 to allocate short values if the draft is approved. 1125 The following entry in Table 3 should be added to the "QUIC Transport 1126 Parameters" registry under the "QUIC Protocol" heading. 1128 +==============================+==================+===============+ 1129 | Value | Parameter Name. | Specification | 1130 +==============================+==================+===============+ 1131 | TBD (experiments use 0xbabf) | enable_multipath | Section 3 | 1132 +------------------------------+------------------+---------------+ 1134 Table 3: Addition to QUIC Transport Parameters Entries 1136 The following frame types defined in Table 4 should be added to the 1137 "QUIC Frame Types" registry under the "QUIC Protocol" heading. 1139 +==============================+==============+===============+ 1140 | Value | Frame Name | Specification | 1141 +==============================+==============+===============+ 1142 | TBD-00 - TBD-01 (experiments | ACK_MP | Section 12.2 | 1143 | use 0xbaba00-0xbaba01) | | | 1144 +------------------------------+--------------+---------------+ 1145 | TBD-02 (experiments use | PATH_ABANDON | Section 12.1 | 1146 | 0xbaba05) | | | 1147 +------------------------------+--------------+---------------+ 1149 Table 4: Addition to QUIC Frame Types Entries 1151 The following transport error code defined in Table 5 should be added 1152 to the "QUIC Transport Error Codes" registry under the "QUIC 1153 Protocol" heading. 1155 +==============+=======================+============+===============+ 1156 | Value | Code |Description | Specification | 1157 +==============+=======================+============+===============+ 1158 | TBD | MP_PROTOCOL_VIOLATION |Multipath | Section 13 | 1159 | (experiments | |protocol | | 1160 | use 0xba01) | |violation | | 1161 +--------------+-----------------------+------------+---------------+ 1163 Table 5: Error Code for Multipath QUIC 1165 15. Security Considerations 1167 TBD 1169 16. Contributors 1171 This document is a collaboration of authors that combines work from 1172 three proposals. Further contributors that were also involved one of 1173 the original proposals are: 1175 * Qing An 1177 * Zhenyu Li 1179 17. Acknowledgments 1181 TBD 1183 18. References 1185 18.1. Normative References 1187 [QUIC-TLS] Thomson, M., Ed. and S. Turner, Ed., "Using TLS to Secure 1188 QUIC", RFC 9001, DOI 10.17487/RFC9001, May 2021, 1189 . 1191 [QUIC-TRANSPORT] 1192 Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based 1193 Multiplexed and Secure Transport", RFC 9000, 1194 DOI 10.17487/RFC9000, May 2021, 1195 . 1197 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1198 Requirement Levels", BCP 14, RFC 2119, 1199 DOI 10.17487/RFC2119, March 1997, 1200 . 1202 [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 1203 10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, November 1204 2003, . 1206 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1207 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1208 May 2017, . 1210 18.2. Informative References 1212 [I-D.bonaventure-iccrg-schedulers] 1213 Bonaventure, O., Piraux, M., Coninck, Q. D., Baerts, M., 1214 Paasch, C., and M. Amend, "Multipath schedulers", Work in 1215 Progress, Internet-Draft, draft-bonaventure-iccrg- 1216 schedulers-02, 25 October 2021, 1217 . 1220 [I-D.liu-multipath-quic] 1221 Liu, Y., Ma, Y., Huitema, C., An, Q., and Z. Li, 1222 "Multipath Extension for QUIC", Work in Progress, 1223 Internet-Draft, draft-liu-multipath-quic-04, 5 September 1224 2021, . 1227 [OLIA] Khalili, R., Gast, N., Popovic, M., Upadhyay, U., and J. 1228 Le Boudec, "MPTCP is not pareto-optimal: performance 1229 issues and a possible solution", Proceedings of the 8th 1230 international conference on Emerging networking 1231 experiments and technologies, ACM , 2012. 1233 [QUIC-Invariants] 1234 Thomson, M., "Version-Independent Properties of QUIC", 1235 RFC 8999, DOI 10.17487/RFC8999, May 2021, 1236 . 1238 [QUIC-RECOVERY] 1239 Iyengar, J., Ed. and I. Swett, Ed., "QUIC Loss Detection 1240 and Congestion Control", RFC 9002, DOI 10.17487/RFC9002, 1241 May 2021, . 1243 [QUIC-Timestamp] 1244 Huitema, C., "Quic Timestamps For Measuring One-Way 1245 Delays", Work in Progress, Internet-Draft, draft-huitema- 1246 quic-ts-07, 6 March 2022, 1247 . 1250 [RFC6356] Raiciu, C., Handley, M., and D. Wischik, "Coupled 1251 Congestion Control for Multipath Transport Protocols", 1252 RFC 6356, DOI 10.17487/RFC6356, October 2011, 1253 . 1255 Authors' Addresses 1257 Yanmei Liu (editor) 1258 Alibaba Inc. 1259 Email: miaoji.lym@alibaba-inc.com 1261 Yunfei Ma 1262 Alibaba Inc. 1263 Email: yunfei.ma@alibaba-inc.com 1265 Quentin De Coninck (editor) 1266 UCLouvain 1267 Email: quentin.deconinck@uclouvain.be 1269 Olivier Bonaventure 1270 UCLouvain 1271 Email: olivier.bonaventure@uclouvain.be 1273 Christian Huitema 1274 Private Octopus Inc. 1275 Email: huitema@huitema.net 1277 Mirja Kuehlewind (editor) 1278 Ericsson 1279 Email: mirja.kuehlewind@ericsson.com