idnits 2.17.00 (12 Aug 2021) /tmp/idnits55405/draft-irtf-nwcrg-coding-and-congestion-12.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 547 has weird spacing: '...packets inf...' -- The document date (22 February 2022) is 81 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: draft-ietf-quic-datagram has been published as RFC 9221 Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NWCRG N. Kuhn 3 Internet-Draft CNES 4 Intended status: Informational E. Lochin 5 Expires: 26 August 2022 ENAC 6 F. Michel 7 UCLouvain 8 M. Welzl 9 University of Oslo 10 22 February 2022 12 Coding and congestion control in transport 13 draft-irtf-nwcrg-coding-and-congestion-12 15 Abstract 17 Forward Erasure Correction (FEC) is a reliability mechanism that is 18 distinct and separate from the retransmission logic in reliable 19 transfer protocols such as TCP. FEC coding can help deal with losses 20 at the end of transfers or with networks having non-congestion 21 losses. However, FEC coding mechanisms should not hide congestion 22 signals. This memo offers a discussion of how FEC coding and 23 congestion control can coexist. Another objective is to encourage 24 the research community to also consider congestion control aspects 25 when proposing and comparing FEC coding solutions in communication 26 systems. 28 This document is the product of the Coding for Efficient Network 29 Communications Research Group (NWCRG). The scope of the document is 30 end-to-end communications: FEC coding for tunnels is out-of-the scope 31 of the document. 33 Status of This Memo 35 This Internet-Draft is submitted in full conformance with the 36 provisions of BCP 78 and BCP 79. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF). Note that other groups may also distribute 40 working documents as Internet-Drafts. The list of current Internet- 41 Drafts is at https://datatracker.ietf.org/drafts/current/. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on 26 August 2022. 50 Copyright Notice 52 Copyright (c) 2022 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 57 license-info) in effect on the date of publication of this document. 58 Please review these documents carefully, as they describe your rights 59 and restrictions with respect to this document. Code Components 60 extracted from this document must include Revised BSD License text as 61 described in Section 4.e of the Trust Legal Provisions and are 62 provided without warranty as described in the Revised BSD License. 64 Table of Contents 66 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 67 2. Context . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 68 2.1. Fairness, Quantifying and Limiting Harm, and Policy 69 Concerns . . . . . . . . . . . . . . . . . . . . . . . . 4 70 2.2. Separate channels, separate entities . . . . . . . . . . 5 71 2.3. Relation between transport layer and application 72 requirements . . . . . . . . . . . . . . . . . . . . . . 7 73 2.4. Scope of the document concerning transport multipath and 74 multi-streams applications . . . . . . . . . . . . . . . 8 75 2.5. Types of coding . . . . . . . . . . . . . . . . . . . . . 9 76 3. FEC above the transport . . . . . . . . . . . . . . . . . . . 10 77 3.1. Fairness and impact on non-coded flows . . . . . . . . . 11 78 3.2. Congestion control and recovered symbols . . . . . . . . 11 79 3.3. Interactions between congestion control and coding 80 rates . . . . . . . . . . . . . . . . . . . . . . . . . . 11 81 3.4. On useless repair symbols . . . . . . . . . . . . . . . . 12 82 3.5. On partial ordering at FEC level . . . . . . . . . . . . 12 83 3.6. On partial reliability at FEC level . . . . . . . . . . . 12 84 3.7. On multipath transport and FEC mechanism . . . . . . . . 12 85 4. FEC within the transport . . . . . . . . . . . . . . . . . . 12 86 4.1. Fairness and impact on non-coded flows . . . . . . . . . 14 87 4.2. Interactions between congestion control and coding 88 rates . . . . . . . . . . . . . . . . . . . . . . . . . . 14 89 4.3. On useless repair symbols . . . . . . . . . . . . . . . . 14 90 4.4. On partial ordering at FEC and/or transport level . . . . 15 91 4.5. On partial reliability at FEC level . . . . . . . . . . . 15 92 4.6. On transport multipath and subpath FEC coding rate . . . 15 93 5. FEC below the transport . . . . . . . . . . . . . . . . . . . 15 94 5.1. Fairness and impact on non-coded flows . . . . . . . . . 17 95 5.2. Congestion control and recovered symbols . . . . . . . . 17 96 5.3. Interactions between congestion control and coding 97 rates . . . . . . . . . . . . . . . . . . . . . . . . . . 17 99 5.4. On useless repair symbols . . . . . . . . . . . . . . . . 17 100 5.5. On partial ordering at FEC level with in-order delivery 101 transport . . . . . . . . . . . . . . . . . . . . . . . . 17 102 5.6. On partial reliability at FEC level . . . . . . . . . . . 18 103 5.7. FEC not aware of transport multipath . . . . . . . . . . 18 104 6. Research recommendations and questions . . . . . . . . . . . 18 105 6.1. Activities related to congestion control and coding . . . 18 106 6.2. Open research questions . . . . . . . . . . . . . . . . . 18 107 6.2.1. Parameter derivation . . . . . . . . . . . . . . . . 19 108 6.2.2. New signaling methods and fairness . . . . . . . . . 19 109 6.3. Recommendations and advice for evaluating coding 110 mechanisms . . . . . . . . . . . . . . . . . . . . . . . 20 111 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 20 112 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 113 9. Security Considerations . . . . . . . . . . . . . . . . . . . 20 114 10. Informative References . . . . . . . . . . . . . . . . . . . 21 115 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23 117 1. Introduction 119 There are cases where deploying FEC coding improves the performance 120 of a transmission. As an example, it may take time for a sender to 121 detect transfer tail losses (losses that occur at the end of a 122 transfer, where, e.g., TCP obtains no more ACKs that would enable it 123 to quickly repair the loss via retransmission). Allowing the 124 receiver to recover such losses instead of having to rely on a 125 retransmission could improve the experience of applications using 126 short flows. Another example is a network where non-congestion 127 losses are persistent and prevent a sender from exploiting the link 128 capacity. 130 Coding and the loss detection of congestion controls are two distinct 131 and separate reliability mechanisms that is distinct and separate 132 from the loss detection of congestion controls. Since FEC coding 133 repairs losses, blindly applying FEC may easily lead to an 134 implementation that also hides a congestion signal from the sender. 135 It is important to ensure that such information hiding does not 136 occur, because loss may be the only congestion signal available to 137 the sender (e.g. TCP [RFC5681]). 139 FEC coding and congestion control can be seen as two separate 140 channels. In practice, implementations may mix the signals that are 141 exchanged on these channels. This memo offers a discussion of how 142 FEC coding and congestion control coexist. Another objective is to 143 encourage the research community also to consider congestion control 144 aspects when proposing and comparing FEC coding solutions in 145 communication systems. This document does not aim at proposing 146 guidelines for characterizing FEC coding solutions. 148 We consider three architectures for end-to-end unicast data transfer: 150 * with FEC coding in the application (above the transport) 151 (Section 3), 153 * within the transport (Section 4), or 155 * directly below the transport (Section 5). 157 A typical scenario for the considerations in this document is a 158 client browsing the web or watching a live video. 160 This document represents the collaborative work and consensus of the 161 Coding for Efficient Network Communications Research Group (NWCRG); 162 it is not an IETF product and is not a standard. The document 163 follows the terminology proposed in the taxonomy document [RFC8406]. 165 2. Context 167 2.1. Fairness, Quantifying and Limiting Harm, and Policy Concerns 169 Traffic from or to different end users may share various types of 170 bottlenecks. When such a shared bottleneck does not implement some 171 form of flow protection, the share of the available capacity between 172 single flows can help assess when one flow starves the other. 174 As one example, for residential accesses, the data rate can be 175 guaranteed for the customer premises equipment, but not necessarily 176 for the end user. The quality of service that guarantees fairness 177 between the different clients can be seen as a policy concern 178 [I-D.briscoe-tsvarea-fair]. 180 While past efforts have focused on achieving fairness, quantifying 181 and limiting harm caused by new algorithms (or algorithms with 182 coding) is more practical [BEYONDJAIN]. This document considers 183 fairness as the impact of the addition of coded flows on non-coded 184 flows when they share the same bottleneck. It is assumed that the 185 non-coded flows respond to congestion signals from the network. This 186 document does not contribute to the definition of fairness at a wider 187 scale. 189 2.2. Separate channels, separate entities 191 Figure 1 and Figure 2 present the notations that will be used in this 192 document and introduces the Forward Erasure Correction (FEC) and 193 Congestion Control (CC) channels. The Forward Erasure Correction 194 channel carries repair symbols (from the sender to the receiver) and 195 information from the receiver to the sender (e.g. signaling which 196 symbols have been recovered, loss rate prior and/or after decoding, 197 etc.). The Congestion Control channel carries network packets from a 198 sender to a receiver, and packets signaling information about the 199 network (number of packets received vs. lost, Explicit Congestion 200 Notification (ECN) [RFC3168] marks, etc.) from the receiver to the 201 sender. The network packets that are sent by the Congestion Control 202 channel may be composed of source packets and/or repair symbols. 204 SENDER RECEIVER 206 +------+ +------+ 207 | | ----- network packets ---->| | 208 | CC | | CC | 209 | | <--- network information ---| | 210 +------+ +------+ 212 Figure 1: Congestion Control (CC) channel 214 SENDER RECEIVER 216 +------+ +------+ 217 | | source and/or | | 218 | | ----- repair symbols ---->| | 219 | FEC | | FEC | 220 | | signaling | | 221 | | <--- recovered symbols ----| | 222 +------+ +------+ 224 Figure 2: Forward Erasure Correction (FEC) channel 226 Inside a host, the CC and FEC entities can be regarded as 227 conceptually separate: 229 | ^ | ^ 230 | source | coding |packets | sending 231 | packets | rate |requirements | rate (or 232 v | v | window) 233 +---------------+source +-----------------+ 234 | FEC |and/or | CC | 235 | |repair | |network 236 | |symbols | |packets 237 +---------------+==> +-----------------+==> 238 ^ ^ 239 | signaling | network 240 | recovered symbols | information 242 Figure 3: Separate entities (sender-side) 244 | | 245 | source and/or | network 246 | repair symbols | packets 247 v v 248 +---------------+ +-----------------+ 249 | FEC |signaling | CC | 250 | |recovered | |network 251 | |symbols | |information 252 +---------------+==> +-----------------+==> 254 Figure 4: Separate entities (receiver-side) 256 Figure 3 and Figure 4 provide more details than Figure 1 and 257 Figure 2. Some elements are introduced: 259 * 'network information' (input control plane for the transport 260 including CC): refers not only to the network information that is 261 explicitly signaled from the receiver, but all the information a 262 congestion control obtains from a network. 264 * 'requirements' (input control plane for the transport including 265 CC): refers to application requirements such as upper/lower rate 266 bounds, periods of quiescence, or a priority. 268 * 'sending rate (or window)' (output control plane for the transport 269 including CC): refers to the rate at which a congestion control 270 decides to transmit packets based on 'network information'. 272 * 'signaling recovered symbols' (input control plane for the FEC): 273 refers to the information a FEC sender can obtain from a FEC 274 receiver about the performance of the FEC solution as seen by the 275 receiver. 277 * 'coding rate' (output control plane for the FEC): refers to the 278 coding rate that is used by the FEC solution (i.e. proportion of 279 transmitted symbols that carry useful data). 281 * 'network packets' (output data plane for the CC): refers to the 282 data that is transmitted by a CC sender to a CC receiver. The 283 network packets may contain source and/or repair symbols. 285 * 'source and/or repair symbols' (data plane for the FEC): refers to 286 the data that is transmitted by a FEC sender to a FEC receiver. 287 The sender can decide to send source symbols only (meaning that 288 the coding rate is 0), repair symbols only (if the solution 289 decides not to send the original source symbols) or a mix of both. 291 The inputs to FEC (incoming data packets without repair symbols, and 292 signaling from the receiver about losses and/or recovered symbols) 293 are distinct from the inputs to CC. The latter calculates a sending 294 rate or window from network information, and it takes the packet to 295 send as input, sometimes along with application requirements such as 296 upper/lower rate bounds, periods of quiescence, or a priority. It is 297 not clear that the ACK signals feeding into a congestion control 298 algorithm are useful to FEC in their raw form, and vice versa - 299 information about recovered blocks may be quite irrelevant to a CC 300 algorithm. 302 2.3. Relation between transport layer and application requirements 304 The choice of the adequate transport layer may be related to 305 application requirements and the services offered by a transport 306 protocol [RFC8095]: 308 * The transport layer may implement a retransmission mechanism to 309 guarantee the reliability of a data transfer (e.g. TCP). 310 Depending on how the FEC and CC functions are scheduled (FEC above 311 CC (Section 3), FEC in CC (Section 4), FEC below CC (Section 5)), 312 the impact of reliable transport on the FEC reliability mechanisms 313 is different. 315 The transport layer may provide an unreliable transport service (e.g. 316 UDP or DCCP [RFC4340]) or a partially reliable transport service 317 (e.g. SCTP with the partial reliability extension [RFC3758] or QUIC 318 with the unreliable datagram extension [I-D.ietf-quic-datagram]). 319 Depending on the amount of redundancy and network conditions, there 320 could be cases where it becomes impossible to carry traffic. This is 321 further discussed in Section 3 where "FEC above CC" case is assessed 322 and in Section 4 and in Section 5 where "FEC in CC" and "FEC below 323 CC" are assessed. 325 2.4. Scope of the document concerning transport multipath and multi- 326 streams applications 328 The application layer can be composed of several streams above FEC 329 and transport layers instances. The transport layer can exploit a 330 multipath mechanism. The different streams could exploit different 331 paths between the sender and the receiver. Moreover, a single-stream 332 application could also exploit a multipath transport mechanism. This 333 section describes what is in the scope of this document in regards 334 with multi-streams applications and multipath transport protocols. 336 The different combinations between multi-stream applications and 337 multipath transport are the following: (1) one application layer 338 stream as input packets above a combination of FEC and multipath 339 (Mpath) transport layers (Figure 5), and (2) multiple application 340 layer streams as input packets above a combination of FEC and 341 multipath (Mpath) or single path (Spath) transport layers (Figure 6). 342 This document further details cases I (in Section 3.7), II (in 343 Section 4.6) and III (in Section 5.7) illustrated in Figure 5. Cases 344 IV, V and VI of Figure 6 are related to how multiple streams are 345 managed by a single transport or FEC layer: this does not directly 346 concerns the interaction between FEC and the transport and is out of 347 the scope of this document. 349 CASE I CASE II CASE III 350 +---------------+ +---------------+ +---------------+ 351 | Stream 1 | | Stream 2 | | Stream 3 | 352 +---------------+ +---------------+ +---------------+ 354 +---------------+ +---------------+ +---------------+ 355 | FEC | | FEC | |Mpath Transport| 356 +---------------+ | in | +---------------+ 357 |Mpath Transport| 358 +---------------+ | | +-----+ +-----+ 359 |Mpath Transport| | | |Flow1|...|FlowM| 360 +---------------+ +---------------+ +-----+ +-----+ 362 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 363 |Flow1|...|FlowM| |Flow1|...|FlowM| | FEC |...| FEC | 364 +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ 366 Figure 5: Transport multipath and single stream applications - in 367 the scope of the document 369 CASE IV CASE V CASE VI 370 +-------+ +-------+ +-------+ +-------+ +-------+ +-------+ 371 |Stream1|...|StreamM| |Stream1|...|StreamM| |Stream1|...|StreamM| 372 +-------+ +-------+ +-------+ +-------+ +-------+ +-------+ 374 +-------------------+ +-------------------+ +-------------------+ 375 | | | FEC | | Mpath Transport | 376 | FEC | +-------------------+ +-------------------+ 377 | above/in/below | 378 | Spath Transport | +-------------------+ +-------------------+ 379 | | | Mpath Transport | | FEC | 380 +-------------------+ +-------------------+ +-------------------+ 382 +-------------------+ +-----+ +-----+ +-----+ +-----+ 383 | Flow | |Flow1| ... |FlowM| |Flow1| ... |FlowM| 384 +-------------------+ +-----+ +-----+ +-----+ +-----+ 386 Figure 6: Transport single path, transport multipath and multi-stream 387 applications - out of the scope of the document 389 2.5. Types of coding 391 [RFC8406] summarizes recommended terminology for Network Coding 392 concepts and constructs. In particular, the document identifies the 393 following coding types (among many others): 395 * Block Coding: Coding technique where the input Flow must first be 396 segmented into a sequence of blocks; FEC encoding and decoding are 397 performed independently on a per-block basis. 399 * Sliding Window Coding: general class of coding techniques that 400 rely on a sliding encoding window. 402 The decoding scheme may not be able to decode all the symbols. The 403 chance of decoding the erased packets depends on the size of the 404 encoding window, the coding rate and the distribution of erasure in 405 the transmission channel. The FEC channel may let the client 406 transmit information related to the need of supplementary symbols to 407 adapt the level of reliability. Partial and full reliability could 408 be envisioned. 410 * Full reliability: The receiver may hold symbols until the decoding 411 of source symbols is possible. In particular, if the codec does 412 not enable a subset of the system to be inverted, the receiver 413 would have to wait for a certain minimum amount of repair packets 414 before it can recover all the source symbols. 416 * Partial reliability: The receiver cannot deliver source symbols 417 that could not have been decoded to the upper layer. For a fixed 418 size of encoding window (for Sliding Window Coding) or of blocks 419 (for Block Coding) containing the source symbols, increasing the 420 amount of repair symbols would increase the chances of recovering 421 the erased symbols. However, this would impact on memory 422 requirements, on the cost of encoding and decoding processes and 423 on the network overhead. 425 3. FEC above the transport 427 | source ^ source 428 | packets | packets 429 v | 430 +-------------+ +-------------+ 431 |FEC | signaling|FEC | 432 | | recovered| | 433 | | symbols| | 434 | | <==| | 435 +-------------+ +-------------+ 436 | source ^ ^ source 437 | and/or | sending | and/or 438 | repair | rate | repair 439 | symbols | (or window) | symbols 440 v | | 441 +-------------+ +-------------+ 442 |Transport | network|Transport | 443 |(incl. CC) | information| | 444 | |network <==| | 445 | |packets | | 446 +-------------+==> +-------------+ 448 SENDER RECEIVER 450 Figure 7: FEC above the transport 452 Figure 7 presents an architecture where FEC operates on top of the 453 transport. 455 The advantage of this approach is that the FEC overhead does not 456 contribute to congestion in the network when congestion control is 457 implemented at the transport layer, because the repair symbols are 458 sent following the congestion window or rate determined by the CC 459 mechanism. This can result in improved quality of experience for 460 latency sensitive applications such as Voice over IP (VoIP) or any 461 not-fully reliable services. 463 This approach requires that the transport protocol does not implement 464 a fully reliable in-order data transfer service (e.g., like TCP). 465 QUIC with unreliable datagram extension [I-D.ietf-quic-datagram] is 466 an example of a protocol for which this is relevant. In cases where 467 the partially reliable transport is blocked and a fall-back to a 468 reliable transport is proposed, there is a risk for bad interactions 469 between reliability at the transport level and coding schemes. For 470 reliable transfers, coding usage does not guarantee better 471 performance; instead, it would mainly reduce goodput. 473 3.1. Fairness and impact on non-coded flows 475 The addition of coding within the flow does not influence the 476 interaction between coded and non-coded flows. This interaction 477 would mainly depend on the congestion controls associated with each 478 flow. 480 3.2. Congestion control and recovered symbols 482 The congestion control mechanism receives network packets and may not 483 be able to differentiate repair symbols from actual source ones. 484 This differentiation requires a transport protocol providing more 485 than the services described in [RFC8095], in particular specifically 486 indicating what information has been repaired. The relevance of 487 adding coding at the application layer is related to the needs of the 488 application. For real-time applications using an unreliable or 489 partially reliable transport, this approach may reduce the number of 490 losses perceived by the application. 492 3.3. Interactions between congestion control and coding rates 494 The coding rate applied at the application layer mainly depends on 495 the available rate or congestion window given by the congestion 496 control underneath. The coding rate could be adapted to avoid adding 497 overhead when the minimum required data rate of the application is 498 not provided by the congestion control underneath. When the 499 congestion control allows sending faster than the application needs, 500 adding coding can reduce packet losses and improve the quality of 501 experience (provided that an unreliable or partially reliable 502 transport is used). 504 3.4. On useless repair symbols 506 The only case where adding useless repair symbols does not obviously 507 result in reduced goodput is when the application rate is limited 508 (e.g., VoIP traffic). In this case, useless repair symbols would 509 only impact the amount of data generated in the network. Extra data 510 in the network can, however, increase the likelihood of increasing 511 delay and/or packet loss, which could provoke a congestion control 512 reaction that would degrade goodput. 514 3.5. On partial ordering at FEC level 516 Irrespective of the transport protocol, a FEC mechanism does not 517 require to implement a reordering mechanism if the application does 518 not need it. However, if the application needs in-order delivery of 519 packets, a reordering mechanism at the receiver is required. 521 3.6. On partial reliability at FEC level 523 The application may require partial reliability. In this case, the 524 coding rate of a FEC mechanism could be adapted based on inputs from 525 the application and the trade-off between latency and packet loss. 526 Partial reliability impacts the type of FEC and type of codec that 527 can be used, such as discussed in Section 2.5. 529 3.7. On multipath transport and FEC mechanism 531 Whether the transport protocol exploits multiple paths or not does 532 not have an impact on the FEC mechanism. 534 4. FEC within the transport 536 | source ^ source 537 | packets | packets 538 v | 539 +------------+ +------------+ 540 | Transport | | Transport | 541 | | | | 542 | +---+ +--+ | signaling| +---+ +--+ | 543 | |FEC| |CC| | recovered| |FEC| |CC| | 544 | +---+ +--+ | symbols| +---+ +--+ | 545 | | <==| | 546 | |network network| | 547 | |packets information| | 548 +------------+ ==> <==+------------+ 550 SENDER RECEIVER 551 Figure 8: FEC in the transport 553 Figure 8 presents an architecture where FEC operates within the 554 transport. The repair symbols are sent within what the congestion 555 window or calculated rate allows, such as in [CTCP]. 557 The advantage of this approach is that it allows a joint optimization 558 of CC and FEC. Moreover, the transmission of repair symbols does not 559 add congestion in potentially congested networks but helps repair 560 lost packets (such as tail losses). This joint optimization is the 561 key to prevent flows to consume the whole available capacity. The 562 amount of repair traffic injected should not lead to congestion. As 563 denoted in [I-D.singh-rmcat-adaptive-fec], an increase of the repair 564 ratio should be done conjointly with a decrease of the source sending 565 rate. 567 The drawback of this approach is that it may require specific 568 signaling and transport services that may not be described in 569 [RFC8095]. Therefore, development and maintenance may require 570 specific efforts at both transport and coding level and the design of 571 the solution may end up being complex to suit different deployment 572 needs. 574 For reliable transfers, including redundancy reduces goodput for long 575 transfers but the amount of repair symbols can be adapted, e.g. 576 depending on the congestion window size. There is a trade-off 577 between 1) the capacity that could have been exploited by application 578 data instead of transmitting source packets, and 2) the benefits 579 derived from transmitting repair symbols (e.g. unlocking the receive 580 buffer if it is limiting). The coding ratio needs to be carefully 581 designed. For small files, sending repair symbols when there is no 582 more data to transmit could help to reduce the transfer time. 583 Sending repair symbols can avoid the silence period between the 584 transmission of the last packet in the send buffer and 1) firing a 585 retransmission of lost packets, or 2) the transmission of new 586 packets. 588 Examples of the solution could be to add a given percentage of the 589 congestion window or rate as supplementary symbols, or to send a 590 fixed amount of repair symbols at a fixed rate. The redundancy flow 591 can be decorrelated from the congestion control that manages source 592 packets: a separate congestion control entity could be introduced to 593 manage the amount of recovered symbols to transmit on the FEC 594 channel. The separate congestion control instances could be made to 595 work together while adhering to priorities, as in coupled congestion 596 control for RTP media [RFC8699] in case all traffic can be assumed to 597 take the same path, or otherwise with a multipath congestion window 598 coupling mechanism as in Multipath TCP [RFC6356]. Another 599 possibility would be to exploit a lower than best-effort congestion 600 control [RFC6297] for repair symbols. 602 4.1. Fairness and impact on non-coded flows 604 Specific interaction between congestion controls and coding schemes 605 can be proposed (see Section 4.2 and Section 4.3). If no specific 606 interaction is introduced, the coding scheme may hide congestion 607 losses from the congestion controller and the description of 608 Section 5 may apply. 610 4.2. Interactions between congestion control and coding rates 612 The receiver can differentiate between source packets and repair 613 symbols. The receiver may indicate both the number of source packets 614 received and repair symbols that were actually useful in the recovery 615 process of packets. The congestion control at the sender can then 616 exploit this information to tune congestion control behavior. 618 There is an important flexibility in the trade-off, inherent to the 619 use of coding, between (1) reducing goodput when useless repair 620 symbols are transmitted and (2) helping to recover from losses 621 earlier than with retransmissions. The receiver may indicate to the 622 sender the number of packets that have been received or recovered. 623 The sender may use this information to tune the coding ratio. For 624 example, coupling an increased transmission rate with an increasing 625 or decreasing coding rate could be envisioned. A server may use a 626 decreasing coding rate as a probe of the channel capacity and adapt 627 the congestion control transmission rate. 629 4.3. On useless repair symbols 631 The sender may exploit the information given by the receiver to 632 reduce the number of useless repair symbols, and improve goodput. 634 4.4. On partial ordering at FEC and/or transport level 636 The application may require in-order delivery of packets. In this 637 case, both FEC and transport layer mechanisms should guarantee that 638 packets are delivered in order. If partial ordering is requested by 639 the application, both the FEC and transport could relax the 640 constraints related to in-order delivery: partial ordering impacts 641 both the congestion control and the type of FEC and type of codec 642 that can be used, mostly at the receiver that may need to implement 643 partial reordering. 645 4.5. On partial reliability at FEC level 647 The application may require partial reliability. The reliability 648 offered by FEC may be sufficient, with no retransmission required. 649 This depends on application needs and the trade-off between latency 650 and loss. Partial reliability impacts the type of FEC and type of 651 codec that can be used, such as discussed in Section 2.5. 653 4.6. On transport multipath and subpath FEC coding rate 655 The sender may adapt the coding rate of each of the single subpaths, 656 whether the congestion control is coupled or not. There is an 657 important flexibility on how the coding rate is tuned depending on 658 the characteristics of each subpath. 660 5. FEC below the transport 662 | source ^ source 663 | packets | packets 664 v | 665 +--------------+ +--------------+ 666 |Transport | network|Transport | 667 |(including CC)| information| | 668 | | <==| | 669 +--------------+ +--------------+ 670 | network packets ^ network packets 671 v | 672 +--------------+ +--------------+ 673 | FEC |source | FEC | 674 | |and/or signaling| | 675 | |repair recovered| | 676 | |symbols symbols| | 677 | |==> <==| | 678 +--------------+ +--------------+ 680 SENDER RECEIVER 681 Figure 9: FEC below the transport 683 Figure 9 presents an architecture where FEC is applied end-to-end 684 below the transport layer, but above the link layer. Note that it is 685 common to apply FEC at the link layer on one or more of the links 686 that make up the end-to-end path. The application of FEC at the link 687 layer contributes to the total capacity that a link exposes to upper 688 layers, but may not be visible to either the end-to-end sender or 689 receiver, if the end-to-end sender and receiver are separated by more 690 than one link, and therefore is out of scope for this document. This 691 includes the use of FEC on top of a link layer in scenarios where the 692 link is known by configuration. In the scenario considered here, the 693 repair symbols are not visible to the end-to-end congestion 694 controller and may be sent on top of what is allowed by the 695 congestion control. 697 Including redundancy adds traffic without reducing goodput but incurs 698 potential fairness issues. The effective bit-rate is higher than the 699 CC's computed fair share due to the transmission of repair symbols, 700 and losses are hidden from the transport. This may cause a problem 701 for loss-based congestion detection, but it is not a problem for 702 delay-based congestion detection. 704 The advantage of this approach is that it can result in performance 705 gains when there are persistent transmission losses along the path. 707 The drawback of this approach is that it can induce congestion in 708 already congested networks. The coding ratio needs to be carefully 709 designed. 711 Examples of the solution could be to add a given percentage of the 712 congestion window or rate as supplementary symbols, or to send a 713 fixed amount of repair symbols at a fixed rate. The redundancy flow 714 can be decorrelated from the congestion control that manages source 715 packets: a separate congestion control entity could be introduced to 716 manage the amount of recovered symbols to transmit on the FEC 717 channel. The separate congestion control instances could be made to 718 work together while adhering to priorities, as in coupled congestion 719 control for RTP media [RFC8699] in case all traffic can be assumed to 720 take the same path, or otherwise with a multipath congestion window 721 coupling mechanism as in Multipath TCP [RFC6356]. Another 722 possibility would be to exploit a lower than best-effort congestion 723 control [RFC6297] for repair symbols. 725 5.1. Fairness and impact on non-coded flows 727 The coding scheme may hide congestion losses from the congestion 728 controller. There are cases where this can drastically reduce the 729 goodput of non-coded flows. Depending on the congestion control, it 730 may be possible to signal to the congestion control mechanism that 731 there was congestion (loss) even when a packet has been recovered, 732 e.g. using ECN, to reduce the impact on the non-coded flows (see 733 Section 5.2 and [TENTET]). 735 5.2. Congestion control and recovered symbols 737 The congestion control may not be aware of the existence of a coding 738 scheme underneath it. The congestion control may behave as if no 739 coding scheme had been introduced. The only way for a coding channel 740 to indicate that symbols have been lost but recovered is to exploit 741 existing signaling that is understood by the congestion control 742 mechanism. An example would be to indicate to a TCP sender that a 743 packet has been received, yet congestion has occurred, by using ECN 744 signaling [TENTET]. 746 5.3. Interactions between congestion control and coding rates 748 The coding rate can be tuned depending on the number of recovered 749 symbols and the rate at which the sender transmits data. If the 750 coding scheme is not aware of the congestion control implementation, 751 it is hard for the coding scheme to apply the relevant coding rate. 753 5.4. On useless repair symbols 755 Useless repair symbols only impact the load on the network without 756 actual gain for the coded flow. Using feedback signaling, FEC 757 mechanisms can measure the ratio between the number of symbols that 758 were actually used and the number of symbols that useless, and adjust 759 the coding rate. 761 5.5. On partial ordering at FEC level with in-order delivery transport 763 The transport above the FEC channel may support out-of-order delivery 764 of packets: reordering mechanisms at the receiver may not be 765 necessary. In cases where the transport requires in-order delivery, 766 the FEC channel may need to implement a reordering mechanism. 767 Otherwise, spurious retransmissions may occur at the transport level. 769 5.6. On partial reliability at FEC level 771 The transport or application layer above the FEC channel may require 772 partial reliability only. FEC may provide an unnecessary service 773 unless it is aware of the reliability requirements. Partial 774 reliability impacts the type of FEC and type of codec that can be 775 used, such as discussed in Section 2.5. 777 5.7. FEC not aware of transport multipath 779 The transport may exploit multiple paths without the FEC channel 780 being aware of it. If FEC is aware that multiple paths are in use, 781 FEC can be applied to all subflows as an aggregate, or to each of the 782 subflows individually. If FEC is not aware that multiple paths are 783 in use, FEC can only be applied to each subflow individually. When 784 FEC is applied to all the flows as an aggregate, the varying 785 characteristics of the individual paths may lead to a risk for the 786 coding rate to be inadequate for the characteristics of the 787 individual paths. 789 6. Research recommendations and questions 791 This section provides a short state-of-the art overview of activities 792 related to congestion control and coding. The objective is to 793 identify open research questions and contribute to advice when 794 evaluating coding mechanisms. 796 6.1. Activities related to congestion control and coding 798 We map activities related to congestion control and coding with the 799 organization presented in this document: 801 * For the FEC above transport case: [RFC8680]. 803 * For the FEC within transport case: 804 [I-D.swett-nwcrg-coding-for-quic], [QUIC-FEC], [RFC5109]. 806 * For the FEC below transport case: [NCTCP], 807 [I-D.detchart-nwcrg-tetrys]. 809 6.2. Open research questions 811 There is a general trade-off, inherent to the use of coding, between 812 (1) reducing goodput when useless repair symbols are transmitted and 813 (2) helping to recover from transmission and congestion losses. 815 6.2.1. Parameter derivation 817 There is a trade-off related to the amount of redundancy to add, as a 818 function of the transport layer protocol and application 819 requirements. 821 [RFC8095] describes the mechanisms provided by existing IETF 822 protocols such as TCP, SCTP or RTP. [RFC8406] describes the variety 823 of coding techniques. The number of combinations makes the 824 determination of an optimum parameters derivation very complex. This 825 depends on application requirements and deployment context. 827 Appendix C of [RFC8681] describes how to tune the parameters for 828 target use-case. However, this discussion does not integrate 829 congestion-controlled end points. 831 Research question 1 : "Is there a way to dynamically adjust the codec 832 characteristics depending on the transmission channel, the transport 833 protocol and application requirements ?" 835 Research question 2 : "Should we apply specific per-stream FEC 836 mechanisms when multiple streams with different reliability needs are 837 carried out ?" 839 6.2.2. New signaling methods and fairness 841 Recovering lost symbols may hide congestion losses from the 842 congestion control. Disambiguate acked packets from rebuilt packets 843 would help the sender adapt its sending rate accordingly. There are 844 opportunities for introducing interaction between congestion control 845 and coding schemes to improve the quality of experience while 846 guaranteeing fairness with other flows. 848 Some existing solutions already propose to disambiguate acked packets 849 from rebuilt packets [QUIC-FEC]. New signaling methods and FEC- 850 recovery-aware congestion controls could be proposed. This would 851 allow the design of adaptive coding rates. 853 Research question 3 : "Should we quantify the harm that a coded flow 854 would induce on a non-coded flow ? How can this be reduced while 855 still benefiting from advantages brought by FEC ?" 857 Research question 4 : "If transport and FEC senders are collocated 858 and close to the client, and FEC is applied only on the last mile, 859 e.g. to ignore losses on a noisy wireless link, would this raise 860 fairness issues ?" 861 Research question 5 : "Should we propose a generic API to allow 862 dynamic interactions between a transport protocol and a coding scheme 863 ? This should consider existing APIs between application and 864 transport layers." 866 6.3. Recommendations and advice for evaluating coding mechanisms 868 Research Recommendation 1: "From a congestion control point-of-view, 869 a recovered packet must be considered as a lost packet. This does 870 not apply to the usage of FEC on a path that is known to be lossy." 872 Research Recommendation 2: "New research contributions should be 873 mapped following the organization of this document (above, below, in 874 the congestion control) and should consider congestion control 875 aspects when proposing and comparing FEC coding solutions in 876 communication systems." 878 Research Recommendation 3: "When a research work aims at improving 879 throughput by hiding the packet loss signal from congestion control 880 (e.g., because the path between the sender and receiver is known to 881 consist of a noisy wireless link), the authors should 1) discuss the 882 advantages of using the proposed FEC solution compared to replacing 883 the congestion control by one that ignores a portion of the 884 encountered losses, 2) critically discuss the impact of hiding packet 885 loss from the congestion control mechanism." 887 7. Acknowledgements 889 Many thanks to Spencer Dawkins, Dave Oran, Carsten Bormann, Vincent 890 Roca and Marie-Jose Montpetit for their useful comments that helped 891 improve the document. 893 8. IANA Considerations 895 This memo includes no request to IANA. 897 9. Security Considerations 899 FEC and CC schemes can contribute to DoS attacks. Moreover, the 900 transmission of signaling messages from the client to the server 901 should be protected and reliable otherwise an attacker may compromise 902 FEC rate adaptation. Indeed, an attacker could either modify the 903 values indicated by the client or drop signaling messages. 905 In case of FEC below the transport, the aggregate rate of source and 906 repair packets may exceed the rate at which a congestion control 907 mechanism allows an application to send. This could result in an 908 application obtaining more than its fair share of the network 909 capacity. 911 10. Informative References 913 [BEYONDJAIN] 914 Ware (et al.), R., "Beyond Jain's Fairness Index: Setting 915 the Bar For The Deployment of Congestion Control 916 Algorithms", HotNets '19 10.1145/3365609.3365855, 2019. 918 [CTCP] Kim (et al.), M., "Network Coded TCP (CTCP)", 919 arXiv 1212.2291v3, 2013. 921 [I-D.briscoe-tsvarea-fair] 922 Briscoe, B., "Flow Rate Fairness: Dismantling a Religion", 923 Work in Progress, Internet-Draft, draft-briscoe-tsvarea- 924 fair-02, 11 July 2007, . 927 [I-D.detchart-nwcrg-tetrys] 928 Detchart, J., Lochin, E., Lacan, J., and V. Roca, "Tetrys, 929 an On-the-Fly Network Coding protocol", Work in Progress, 930 Internet-Draft, draft-detchart-nwcrg-tetrys-08, 17 October 931 2021, . 934 [I-D.ietf-quic-datagram] 935 Pauly, T., Kinnear, E., and D. Schinazi, "An Unreliable 936 Datagram Extension to QUIC", Work in Progress, Internet- 937 Draft, draft-ietf-quic-datagram-10, 4 February 2022, 938 . 941 [I-D.singh-rmcat-adaptive-fec] 942 Singh, V., Nagy, M., Ott, J., and L. Eggert, "Congestion 943 Control Using FEC for Conversational Media", Work in 944 Progress, Internet-Draft, draft-singh-rmcat-adaptive-fec- 945 03, 20 March 2016, . 948 [I-D.swett-nwcrg-coding-for-quic] 949 Swett, I., Montpetit, M., Roca, V., and F. Michel, "Coding 950 for QUIC", Work in Progress, Internet-Draft, draft-swett- 951 nwcrg-coding-for-quic-04, 9 March 2020, 952 . 955 [NCTCP] Sundararajan (et al.), J., "Network Coding Meets TCP: 956 Theory and Implementation", IEEE 957 INFOCOM 10.1109/JPROC.2010.2093850, 2009. 959 [QUIC-FEC] Michel (et al.), F., "QUIC-FEC: Bringing the benefits of 960 Forward Erasure Correction to QUIC", IFIP 961 Networking 10.23919/IFIPNetworking.2019.8816838, 2019. 963 [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition 964 of Explicit Congestion Notification (ECN) to IP", 965 RFC 3168, DOI 10.17487/RFC3168, September 2001, 966 . 968 [RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P. 969 Conrad, "Stream Control Transmission Protocol (SCTP) 970 Partial Reliability Extension", RFC 3758, 971 DOI 10.17487/RFC3758, May 2004, 972 . 974 [RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram 975 Congestion Control Protocol (DCCP)", RFC 4340, 976 DOI 10.17487/RFC4340, March 2006, 977 . 979 [RFC5109] Li, A., Ed., "RTP Payload Format for Generic Forward Error 980 Correction", RFC 5109, DOI 10.17487/RFC5109, December 981 2007, . 983 [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion 984 Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, 985 . 987 [RFC6297] Welzl, M. and D. Ros, "A Survey of Lower-than-Best-Effort 988 Transport Protocols", RFC 6297, DOI 10.17487/RFC6297, June 989 2011, . 991 [RFC6356] Raiciu, C., Handley, M., and D. Wischik, "Coupled 992 Congestion Control for Multipath Transport Protocols", 993 RFC 6356, DOI 10.17487/RFC6356, October 2011, 994 . 996 [RFC8095] Fairhurst, G., Ed., Trammell, B., Ed., and M. Kuehlewind, 997 Ed., "Services Provided by IETF Transport Protocols and 998 Congestion Control Mechanisms", RFC 8095, 999 DOI 10.17487/RFC8095, March 2017, 1000 . 1002 [RFC8406] Adamson, B., Adjih, C., Bilbao, J., Firoiu, V., Fitzek, 1003 F., Ghanem, S., Lochin, E., Masucci, A., Montpetit, M-J., 1004 Pedersen, M., Peralta, G., Roca, V., Ed., Saxena, P., and 1005 S. Sivakumar, "Taxonomy of Coding Techniques for Efficient 1006 Network Communications", RFC 8406, DOI 10.17487/RFC8406, 1007 June 2018, . 1009 [RFC8680] Roca, V. and A. Begen, "Forward Error Correction (FEC) 1010 Framework Extension to Sliding Window Codes", RFC 8680, 1011 DOI 10.17487/RFC8680, January 2020, 1012 . 1014 [RFC8681] Roca, V. and B. Teibi, "Sliding Window Random Linear Code 1015 (RLC) Forward Erasure Correction (FEC) Schemes for 1016 FECFRAME", RFC 8681, DOI 10.17487/RFC8681, January 2020, 1017 . 1019 [RFC8699] Islam, S., Welzl, M., and S. Gjessing, "Coupled Congestion 1020 Control for RTP Media", RFC 8699, DOI 10.17487/RFC8699, 1021 January 2020, . 1023 [TENTET] Lochin, E., "On the joint use of TCP and Network Coding", 1024 NWCRG session IETF 100, 2017. 1026 Authors' Addresses 1028 Nicolas Kuhn 1029 CNES 1030 Email: nicolas.kuhn.ietf@gmail.com 1032 Emmanuel Lochin 1033 ENAC 1034 Email: emmanuel.lochin@enac.fr 1036 Francois Michel 1037 UCLouvain 1038 Email: francois.michel@uclouvain.be 1039 Michael Welzl 1040 University of Oslo 1041 Email: michawe@ifi.uio.no