idnits 2.17.00 (12 Aug 2021) /tmp/idnits18906/draft-han-tsvwg-ip-transport-qos-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet has text resembling RFC 2119 boilerplate text. -- The document date (October 19, 2018) is 1310 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC2581' is defined on line 1130, but no explicit reference was found in the text == Unused Reference: 'I-D.falk-xcp-spec' is defined on line 1156, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-aqm-codel' is defined on line 1177, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-aqm-fq-codel' is defined on line 1182, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-aqm-pie' is defined on line 1189, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-tcpm-dctcp' is defined on line 1205, but no explicit reference was found in the text == Unused Reference: 'I-D.sridharan-tcpm-ctcp' is defined on line 1216, but no explicit reference was found in the text == Unused Reference: 'TCP-vegas' is defined on line 1262, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2581 (Obsoleted by RFC 5681) == Outdated reference: draft-ietf-aqm-codel has been published as RFC 8289 == Outdated reference: draft-ietf-aqm-fq-codel has been published as RFC 8290 == Outdated reference: draft-ietf-aqm-pie has been published as RFC 8033 == Outdated reference: draft-ietf-detnet-use-cases has been published as RFC 8578 == Outdated reference: draft-ietf-spring-segment-routing has been published as RFC 8402 == Outdated reference: draft-ietf-tcpm-dctcp has been published as RFC 8257 Summary: 1 error (**), 0 flaws (~~), 16 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TSVWG Working Group L. Han 3 Internet-Draft Y. Qu 4 Intended status: Informational L. Dong 5 Expires: April 22, 2019 R. Li 6 Huawei Technologies 7 T. Nadeau 8 Lucid Vision 9 K. Smith 10 Vodafone 11 J. Tantsura 12 Apstra 13 October 19, 2018 15 Resource Reservation Protocol for IP Transport QoS 16 draft-han-tsvwg-ip-transport-qos-01 18 Abstract 20 IP is designed for use in Best Effort Networks, which are networks 21 that provide no guarantee that data is delivered, or that delivery 22 meets any specified quality of service parameters. However there are 23 new applications requiring IP to provide deterministic services in 24 terms of bandwidth and latency, such as network based AR/VR 25 (Augmented Reality and Virtual Reality), industrial internet. This 26 document proposes a solution in IPv6 that can be used by transport 27 layer protocols to guarantee certain level of service quality. This 28 new service is fined-grained and could apply to individual or 29 aggregated TCP/UDP flow(s). 31 Status of This Memo 33 This Internet-Draft is submitted in full conformance with the 34 provisions of BCP 78 and BCP 79. 36 Internet-Drafts are working documents of the Internet Engineering 37 Task Force (IETF). Note that other groups may also distribute 38 working documents as Internet-Drafts. The list of current Internet- 39 Drafts is at https://datatracker.ietf.org/drafts/current/. 41 Internet-Drafts are draft documents valid for a maximum of six months 42 and may be updated, replaced, or obsoleted by other documents at any 43 time. It is inappropriate to use Internet-Drafts as reference 44 material or to cite them other than as "work in progress." 46 This Internet-Draft will expire on April 22, 2019. 48 Copyright Notice 50 Copyright (c) 2018 IETF Trust and the persons identified as the 51 document authors. All rights reserved. 53 This document is subject to BCP 78 and the IETF Trust's Legal 54 Provisions Relating to IETF Documents 55 (https://trustee.ietf.org/license-info) in effect on the date of 56 publication of this document. Please review these documents 57 carefully, as they describe your rights and restrictions with respect 58 to this document. Code Components extracted from this document must 59 include Simplified BSD License text as described in Section 4.e of 60 the Trust Legal Provisions and are provided without warranty as 61 described in the Simplified BSD License. 63 Table of Contents 65 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 66 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 67 3. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 6 68 3.1. Design Targets . . . . . . . . . . . . . . . . . . . . . 6 69 3.2. Scope and Assumptions . . . . . . . . . . . . . . . . . . 7 70 3.3. Sub-layer in IP for Transport Control . . . . . . . . . . 7 71 3.4. IP In-band signaling . . . . . . . . . . . . . . . . . . 8 72 3.5. IPv6 Approach . . . . . . . . . . . . . . . . . . . . . . 9 73 4. Key Messages and Parameters . . . . . . . . . . . . . . . . . 10 74 4.1. Setup and Setup State Report messages . . . . . . . . . . 10 75 4.2. Forwarding State and Forwarding State Report messages 11 76 4.3. Hop Number . . . . . . . . . . . . . . . . . . . . . . . 12 77 4.4. Flow Identifying Method and Service ID . . . . . . . . . 13 78 4.5. QoS State and life of Time . . . . . . . . . . . . . . . 14 79 4.6. Authentication . . . . . . . . . . . . . . . . . . . . . 14 80 5. Packet Forwarding . . . . . . . . . . . . . . . . . . . . . . 14 81 5.1. Basic Hardware Capability . . . . . . . . . . . . . . . . 15 82 5.2. Flow Identification in Packet Forwarding . . . . . . . . 15 83 5.3. QoS Forwarding State Detection and Failure Handling . . . 16 84 6. Details of Working with Transport Layer . . . . . . . . . . . 17 85 6.1. Working with TCP . . . . . . . . . . . . . . . . . . . . 17 86 6.2. Working with UDP and other Protocols . . . . . . . . . . 19 87 7. Additional Considerations . . . . . . . . . . . . . . . . . . 20 88 7.1. User and Application driven . . . . . . . . . . . . . . . 20 89 7.2. Traffic Management in Host . . . . . . . . . . . . . . . 21 90 7.3. Heterogeneous Network . . . . . . . . . . . . . . . . . . 21 91 7.4. Proxy Control . . . . . . . . . . . . . . . . . . . . . . 22 92 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 93 9. Security Considerations . . . . . . . . . . . . . . . . . . . 23 94 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 24 95 10.1. Normative References . . . . . . . . . . . . . . . . . . 24 96 10.2. Informative References . . . . . . . . . . . . . . . . . 25 97 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 27 98 Appendix B. Message Objects . . . . . . . . . . . . . . . . . . 28 99 B.1. Setup State Object . . . . . . . . . . . . . . . . . . . 28 100 B.2. Bandwidth Object . . . . . . . . . . . . . . . . . . . . 30 101 B.3. Burst Msg . . . . . . . . . . . . . . . . . . . . . . . . 30 102 B.4. Latency Object . . . . . . . . . . . . . . . . . . . . . 31 103 B.5. Authentication Object . . . . . . . . . . . . . . . . . . 31 104 B.6. OAM Object . . . . . . . . . . . . . . . . . . . . . . . 32 105 B.7. Forwarding State Object . . . . . . . . . . . . . . . . . 33 106 B.8. Setup State Report Object . . . . . . . . . . . . . . . . 33 107 B.9. Forward State Report Object . . . . . . . . . . . . . . . 34 108 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 35 110 1. Introduction 112 Recently, more and more new applications for The Internet are 113 emerging. These applications have a number of key requirements that 114 are common to all such as that their required bandwidth is very high 115 and/or latency is very low compared with traditional applications 116 like most of web and video applications. 118 For example, network based Augmented Reality (AR) or Virtual Reality 119 (VR) applications may need hundreds of Mbps bandwidth (throughput) 120 and a low single digit millisecond latency. Moreover, the difference 121 between mean bit rate and peak bit rate may be significant due to the 122 choice of compression algorithm 123 [I-D.han-iccrg-arvr-transport-problem]. This may result in large 124 bursts, and make traffic management more difficult. 126 Some future applications may expect networks to provide a bounded 127 latency service. One such example is tactile network [Tactile]. 129 With the technology development in 5G [HU5G][QU2016] and beyond, the 130 wireless access network is also increasing the demand for the Ultra- 131 Reliable and Low-Latency Communications (URLLC). This also leads to 132 the question of whether IP can provide such service in an Evolved 133 Packet Core (EPC)[EPC] network. IP is becoming more and more 134 important in the EPC when the Multi-access Edge Computing (MEC)[MEC] 135 for 5G requires the cloud and data service to move closer to eNodeB 136 [eNodeB]. 138 [I-D.ietf-detnet-use-cases] identifies some use cases from different 139 industries which have a common need for "deterministic flows". Such 140 flows require guaranteed bandwidth and bounded latency. 142 Traditionally, an IP network provides an unreliable or best-effort 143 datagram service over a collection of underlying networks (i.e.: 145 ethernet, ATM, etc...). Integrated services(IntServ) [RFC3175] 146 specifies a fine-grained QoS system, which requires all routers along 147 the traffic path to support it and maintain the states for resource 148 reserved IP flow(s), so it is difficult to scale up to keep track of 149 all the reservations. Differentiated services (DiffServ) [RFC2475] 150 specifies a simple and scalable mechanism to classify traffic and 151 provide more coarse QoS, however because it can only specify per-hop 152 behaviors (PHBs), and how individual routers deal with the DS 153 [RFC2474] field is configuration specific. It is difficult to 154 provide consistent resource reservation for specified class of 155 traffic, thus hard to support the end-to-end bandwidth or latency 156 guarantee. 158 The transport layer (TCP/UDP) on top of IP is based on the best- 159 effort-only service, which has influenced the transport layer 160 evolution for quite long time, and results in some widely accepted 161 assumptions and solutions, such as: 163 1. The IP layer can only provide basic P2P (point to point) or P2MP 164 (point to multi-point) end-to-end connectivity in the Internet, 165 but the connectivity is not reliable and does not guarantee any 166 quality of service to end-user or application, such as bandwidth, 167 packet loss, latency etc. Due to this assumption, the transport 168 layer or application must have its own control mechanism in 169 congestion and flow to obtain the reliable and satisfactory 170 service to cooperate with the under layer network quality. 172 2. The transport layer assumes that the IP layer can only process 173 all IP flows equally in the hardware since the best effort 174 service is actually an un-differentiated service. The process 175 includes scheduling, queuing and forwarding. Thus, the transport 176 layer must behave nicely and friendly to make sure all flows will 177 only obtain its own faired share of resource, and no one could 178 consume more and no one could be starved. 180 This document proposes a new IP transport service that guarantees 181 bandwidth and latency for new applications. The scope and criteria 182 for the new technology will also be discussed. This new IP transport 183 service is designed to be supplementary to regular IP transport 184 services, only meant to be used for special applications that are 185 bandwidth and/or latency sensitive. 187 2. Terminology 189 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 190 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 191 "OPTIONAL" in this document are to be interpreted as described in BCP 192 14 RFC 2119 [RFC2119] RFC 8174 [RFC8174] when, and only when, they 193 appear in all capitals, as shown here. 195 Abbreviations used in this documents: 197 E2E 198 End-to-end. 200 EH 201 IPv6 Extension Header or Extension Option. 203 QoS 204 Quality of Service. 206 OAM 207 Operation and Management. 209 In-band Signaling 210 In telecommunications, in-band signaling is sending control 211 information within the same band or channel used for voice or 212 video. 214 Out-of-band Signaling 215 out-of-band signaling is that the control information sent over 216 a different channel, or even over a separate network. 218 IP flow 219 For non-IPSec, an IP flow is identified by the source, 220 destination IP address, the protocol number, the source and 221 destination port number. 223 IP path 224 An IP path is the route that IP flow will traverse. It could 225 be the shortest path determined by routing protocols (IGP or 226 BPG), or the explicit path such as segment routing 227 [I-D.ietf-spring-segment-routing]. 229 QoS channel 230 A forwarding channel that is QoS guaranteed. It provides 231 additional QoS service to IP forwarding. A QoS channel can be 232 used for one or multiple IP flows depending on the granularity 233 of in-band signaling. 235 CIR 236 Committed Information Rate. 238 PIR 239 Peak Information Rate. 241 HbH-EH 242 IPv6 Hop-by-Hop Extension Header. 244 Dst-EH 245 IPv6 Destination Extension Header. 247 HbH-EH-aware node 248 Network nodes that are configured to process the IPv6 Hop-by- 249 Hop Extension Header. 251 3. Overview 253 Semiconductor chip technology has advanced significantly in the last 254 decade, and as such the widely used network processing and forwarding 255 process can now not only forward packets at line speed, but also 256 easily support other feature processing such as QoS for DiffServ/ 257 MPLS, Access Control List (ACL), fire wall, and Deep Packet 258 Inspection (DPI). 260 This advancement enables network processors to do the general process 261 to handle simple control messages for traffic management, such as 262 signaling for hardware programming, congestion state report, OAM, 263 etc. So now it's possible to treat some TCP/IP flows differently 264 from others and give them specified resource are feasible now by 265 using network processor. 267 This document proposes a deterministic IP transport service, which 268 can provide guaranteed bandwidth and latency. The solution is based 269 on the QoS implemented in network processor through in-band 270 signaling. 272 3.1. Design Targets 274 The proposed transport service is expected to satisfy the following 275 criteria: 277 o End user or application may directly use the new service. 279 o The new service can coexist with the current transport service and 280 is backward compatible. 282 o Service providers can manage the new service. 284 o Performance and scalability targets of this new service are 285 practical for vendors to achieve. 287 o The new service is transport agnostic. TCP, UDP and other 288 transport protocols on top of IP can use it. 290 3.2. Scope and Assumptions 292 The initial aim is to propose a solution for IPv6. To limit the 293 scope of the document and simplify the design and solution, the 294 following constraints are given: 296 1. The new service with QoS is aimed to be supplementary to regular 297 IP service. It is targeted for the applications that are 298 bandwidth and/or latency sensitive. It is not intended to 299 replace the TCP/IP variants that have been proved to be efficient 300 and successful for current applications. 302 2. The new service is limited within one administrative domain, even 303 it does not exclude the possibilities of extending the mechanism 304 for inter-domain scenarios. Currently only inter-domain security 305 is considered, and the inter-domain SLA, accounting and other 306 issues are not discussed. 308 3. Due to high bandwidth requirement of new service for individual 309 flow, the total number of the flows with the new service cannot 310 be high for a port, or a system. From another point of view, the 311 new service is targeted for applications that really need it, the 312 number of supported applications/users should be controlled and 313 cannot be unlimited. Hence the scalability requirement for the 314 new service is limited. 316 4. The new service must be able to coexist with the regular 317 transport service in the same hardware, and be backward 318 compatible. Also, a transport flow can switch between regular 319 transport and new service without service interruption. 321 3.3. Sub-layer in IP for Transport Control 323 In order to provide some new features for the layer above IP, it is 324 very useful to introduce an additional sub-layer, Transport Control, 325 between layer 3 (IP) and layer 4 (TCP/UDP). The new layer belongs to 326 IP, and is present only when the system needs to provide extra 327 control for the upper layer, in addition to the normal IP forwarding. 328 Fig 1. illustrates a new stack with the sub-layer. 330 +=========================+ 331 | APP | 332 +=========================+ 333 | TCP/UDP | 334 +=========================+ 335 | Transport Ctl | 336 +-+-+-+-+-+-+-+-+-+-+-+-+-+ 337 | IP | 338 +=========================+ 339 | Network Access | 340 +=========================+ 342 Figure 1: The new stack with a sub-layer in Layer 3 344 The new sub-layer is always bound with IP layer and can provide 345 support of the features for upper layer, such as: 347 In-band Signaling 348 The IP header with the new sub-layer can carry the signaling 349 information for the devices on the IP path. The information may 350 include all QoS related parameters used for hardware programming. 352 Congestion control 353 The congestion state in each device on the path can be detected 354 and notified to the source of flows by the sub-layer; The dynamic 355 congestion control instruction can also be carried by the sub- 356 layer and examined by network devices on the IP path. 358 IP Path OAM 359 The OAM instruction can be carried in the sub-layer, and the OAM 360 state can be notified to the source of flows by the sub-layer. 361 The OAM includes the path and device property detection, QoS 362 forwarding diagnosis and report. 364 IPv4 could use the IP option for the purpose of the sub-layer. But 365 due to the limit size of the IP option, the functionalities, 366 scalability of the layer is restricted. 368 IPv6 can realize the sub-layer easily using IPv6 extension header 369 [RFC8200]. The document will focus on the solution for IPv6 using 370 different IPv6 extension headers. 372 3.4. IP In-band signaling 374 In-band signaling messages are carried along with the payload. It is 375 guaranteed that the signaling follows the same path as the data flow, 376 and this can bring up some advantages that other methods can hardly 377 provide: 379 Diagnosis 380 The in-band signaling message takes the same path, same hops, same 381 processing at each hop as the data packet, this will make the 382 diagnosis for both signaling and data path easier. 384 Simplicity 385 The in-band signaling message is forwarded with the normal data 386 packet, it does not need to run a separate protocol. This will 387 dramatically reduce the complexity of the control. 389 Performance and scalability 390 Due to the simplicity of in-band signaling for control, it is 391 easier to provide a better performance and scalability for a new 392 future. 394 Note, the requirement of IP in-band signaling was proposed before by 395 John Harper [I-D.harper-inband-signalling-requirements]. And the in- 396 band QoS signaling for IPv6 was simply discussed in 397 [I-D.roberts-inband-qos-ipv6]. Unfortunately, both works did not 398 continue. 400 This document proposes a detailed solution for QoS service using in- 401 band signaling, and it also tries to address issues raised by 402 previous proposals, such as security, scalability and performance. 404 3.5. IPv6 Approach 406 IPv6 extension header is used for signaling. There are two types of 407 extension header used for the purpose of transport QoS control, one 408 is the hop-by-hop EH (HbH-EH) and another is the destination EH (Dst- 409 EH). 411 The HbH-EH may be examined and processed by the nodes that are 412 explicitly configured to do so [RFC8200], and these nodes are called 413 HbH-EH-aware nodes. Note, not all nodes along a patch need to HbH- 414 EH-aware. HbH-EH is used to carry the QoS requirement for dedicated 415 flow(s) and then the information is intercepted by HbH-EH-aware nodes 416 on the path to program hardware accordingly. 418 The destination EH will only be examined and processed by the 419 destination device that is associated with the destination IPv6 420 address in the IPv6 header. This EH is used to send the QoS related 421 report information directly to the source of the signaling at other 422 end. 424 The following figure illustrates the path setup process: 426 Setup Message (bandwidth, latency, setup state) 428 +------------------------HbH-EH------ -----------------+ 429 | | | | 430 | | | | 431 +------+ +------------+ +-----+ +------------+ +------+ 432 | | | | | | | | | | 433 | host |---| R1 |---| R2 |---| R3 |---|server| 434 | | |HbH-EH-aware| | | |HbH-EH-aware| | | 435 | | | | | | | | | | 436 +------+ +------------+ +-----+ +------------+ +------+ 437 | | 438 | | 439 +----------------------- Dst-EH ------------------------+ 441 Setup State Report Message (setup state report) 443 Figure 2: Path Setup 445 Using the figure.2 for illustration, to set up a path with resource 446 reservation, a setup message including QoS requirements, such as max/ 447 min bandwidth, burst size, the latency, and the setup state is sent 448 from the host to the server. After each HbH-EH-aware node along the 449 path receives the message, it reads the QoS information and programs 450 the hardware for resource reservation, queuing management etc. The 451 setup state object is updated at each HbH-EH-aware node to include 452 the QoS programming and provisioning result and the necessary 453 hardware reference information for IP forwarding with QoS. After the 454 setup message reaches the server, the server will send a setup state 455 report message encoded as Dst-EH to the host. The setup state report 456 message carries the path setup results from the setup state object. 458 4. Key Messages and Parameters 460 4.1. Setup and Setup State Report messages 462 Setup message is intended to program the hardware for QoS channel on 463 the IP path from the source to the destination expressed in IPv6 464 header. It is embedded as the HbH-EH in an IPv6 packet and will be 465 processed at each HbH-EH-aware node. For the simplicity, performance 466 and scalability purpose, not all routers along the path need do the 467 processing or be HbH-EH-aware. For different QoS requirements and 468 scenarios, different criteria can be used to configure HbH-EH-aware 469 nodes. 471 A throttle router is the device that an interested TCP/UDP session 472 cannot get the enough bandwidth to support its application, and it 473 will also contribute more to latency than non-throttle routers. The 474 regular throttle routers include the BRAS (broadband remote access 475 server) in broadband access network, the PGW (PDN Gateway) in LTE 476 network etc. In more general case, any routers which aggregated 477 traffic may become as a throttle router. Throttle routers should be 478 configured to process HbH-EH when: 480 o Reserved bandwidth is required: The throttle router is the 481 critical point to be configured to process the hop-by-hop EH for 482 the bandwidth reservation. Moreover, the direction of congestion 483 must be considered. 485 o Bounded latency is required: In theory, each router and switch 486 could contribute some delay to the end-to-end latency, but the 487 throttle router will contribute more than non-throttle routers, 488 and slow device will contribute more than fast device. We can use 489 OAM to detect the latency contribution in a network, and configure 490 those worst-cast devices to process the HbH-EH. 492 Setup State Report message is the message sent from the destination 493 host to the source host (from the point of view of the Setup 494 message). The message is embedded into the Dst-EH in any data 495 packet. The Setup State Report in the message is just a copy from 496 the Setup message received at the destination host for a typical TCP 497 session. The message is used at the source host to forward the 498 packet later and to do the congestion control. 500 ::= [ ] 502 [ ] [ ] 504 [ ] [ ] 506 ::= 508 [ ] 510 4.2. Forwarding State and Forwarding State Report messages 512 After the QoS is programmed by the in-band signaling, the specified 513 IP flows can be processed and forwarded for the QoS requirement. 514 There are two ways for host to use the QoS channel for associated TCP 515 session: 517 1. Host directly send the IP packet without any changes to the 518 packet, this is for the following cases: 520 * The hardware was programmed to use the tuples in IP header as 521 identification for QoS process (SIS = 0), and 523 * The packet does not function to collect the QoS forwarding 524 state on the path. 526 2. Host add the Forward State message into a data packet's IP header 527 as HbH-EH and send the packet, this is for the cases: 529 * The hardware was programmed to use the Service ID as 530 identification for QoS process (SIS != 0). 532 * The hardware was programmed to use the tuples in IP header as 533 identification for QoS process (SIS = 0), and the data packet 534 functions to collect the QoS forwarding state on the path. 535 This is the situation that host wants to detect the QoS 536 forwarding state for the purpose of failure handling (See 537 section 4.3). 539 Forwarding State message format is shown in the Section 6.7. It is 540 used to notify the service ID and also update QoS forwarding state 541 for the hops that are HbH-EH-aware nodes. 543 After Forwarding State message is reaching the destination host, the 544 host is supposed to retrieve it and form a Forwarding State Report 545 message, and carry it in any data packet as the Dst-EH, then send it 546 to the host in the reverse direction. 548 ::= [ 549 ] 550 [ ] [ ] 552 ::= 554 [ ] 556 4.3. Hop Number 558 This is the parameter for total number of HbH-EH-aware nodes on the 559 path. It is the field "Hop_num" in Setup message, and is used to 560 locate the bit position for "Setup State" and the "Service ID" in 561 "Service ID List". The value of "Hop_num" must be decremented at 562 each HbH-EH-aware node. At the receiving host of the in-band 563 signaling, the Hop_num must be zero. 565 The source host must know the exact hop number, and setup the initial 566 value in the Setup message. The exact hop number can be detected 567 using OAM message. 569 4.4. Flow Identifying Method and Service ID 571 A QoS channel might be enforced for a group of flows or a delicate 572 flow, and flow identifying method means the way of identifying a flow 573 or a group of flows that can use a HW programmed QoS channel. 574 Different levels of flow granularities to support QoS are defined as 575 below: 577 Flow level 578 The flow identification could be 5 tuples for non IPSec IPv6 579 packet: the source, destination IP address, protocol number, 580 source and destination port number, and also could be 3 tuples for 581 IPSec IPv6 packet: the source, destination IP address and the flow 582 label. 584 Address level In-band Signaling 585 A flow of packets share the same source, destination IP address, 586 but with different protocol number. This is the scenario that the 587 signaling is for the aggregated flows which have the same source, 588 destination address. i.e, All TCP/UDP flows between the same 589 client and same server (only one address for client and one for 590 server) 592 Transport level In-band Signaling 593 Packets share the same source, destination IP address, protocol 594 number, but with different source or destination port number (non- 595 IPSec) or different flow label (IPSec). This could be for the 596 aggregated TCP or UDP flows that started and terminated at the 597 same IP addresses. 599 DiffServ level In-band Signaling 600 Packets share the same DSCP value. This means aggregated 601 differentiated service flows that have the same DSCP value. The 602 DSCP value is determined by the 6 most-significant bits in 8-bits 603 DiffServ field for IPv4 or 8-bits Traffic Class field for IPv6. 605 There are two ways for flow identifying. One is by tuple or DSCP 606 value in IP header, another is by a local significant number, called 607 service ID, generated and maintained in a router. When "Service ID 608 Size" (SIS) is zero, it means the "Flow identification method" (FI) 609 is used for both control plane and data plane. When "SIS" is not 610 zero, it means "FI" is only used in signaling of setting up the QoS 611 channel, and the data plane will only use the "Service ID". The use 612 of local generated number to identify flow is to speed up the flow 613 lookup and QoS process for data plane. 615 The "Service ID List" is a list of "Service ID" for all hops that are 616 HbH-EH-aware nodes on the IP path. When a router receives a HbH-EH, 617 it may generate a service ID for the flow(s) that is defined by the 618 Flow Identifying Method in "FI". Then the router must attach the 619 service ID value to the end of the Service ID List. After the packet 620 reaches the destination host, the Service ID List will be that the 621 1st router's service ID as the list header, and the last router's 622 service ID as the list tail. 624 4.5. QoS State and life of Time 626 After a router is programmed for a QoS, a QoS state is created. The 627 QoS state life is determined by the "Time" in the Setup message. 628 Whenever there is a packet processed by a QoS state, the associated 629 timer for the QoS state is reset. If the timer of a QoS state is 630 expired, the QoS state will be erased and the associated resource 631 will be released. 633 In order to keep the QoS state active, a application at source host 634 can send some zero size of data to refresh the QoS state. 636 When the Time is set to zero, it means the life of the QoS State will 637 be kept until the de-programming message is received. 639 4.6. Authentication 641 The in-band signaling is designed to have a basic security mechanism 642 to protect the integrity of a signaling message. The Authentication 643 message is to attach to a signaling message, the source host 644 calculates the harsh value of a key and all invariable part of a 645 signaling message (Setup message: ver, FI, R, SIS, P, Time; Bandwidth 646 message, Latency message, Burst message). The key is only known to 647 the hosts and all HbH-EH-aware nodes. The securely distribution of 648 the key is out the scope of the document. 650 5. Packet Forwarding 652 To achieve the required QoS, after the path setup with guaranteed 653 bandwidth there are some requirements to be met during data 654 forwarding. These include the hardware capability, the scheme for 655 the data forwarding, QoS processing, state report, etc. 657 5.1. Basic Hardware Capability 659 Section 4 explains how QoS guaranteed path can be set up and the 660 corresponding messages used, however different implementations may 661 vary in details. To achieve the satisfactory targets for performance 662 and scalability, the protocol must be cooperated with capable 663 hardware to provide the desired fine-grained QoS for different 664 transport. 666 In our experiment to implement the feature for TCP, we used a network 667 processor with traffic management feature. The traffic management 668 can provide the fine-grained QoS for any configured flow(s). 670 The following capabilities are RECOMMENDED: 672 1. The in-banding signaling is processed in network processor 673 without punting to controller CPU for help 675 2. The QoS forwarding state is kept and maintained in network 676 processor without the involvement from controller CPU. 678 3. The QoS state has a life of a pre-configured time and will be 679 automatically deleted if there is no data packet processed by 680 that QoS state. The timer can be changed on the fly. 682 4. The data forwarding does not need to be done at the controller 683 CPU, or so called slow path. It is at the same hardware as the 684 normal IP forwarding. For any IP packet, the QoS forwarding is 685 executed first. Normal forwarding will be executed if there is 686 no QoS state associated with the identification of the flow. 688 5. The QoS forwarding and normal forwarding can be switched on the 689 fly. 691 The details of data plane and hardware related implementations, such 692 as traffic classification, shaping, queuing and scheduling, are out 693 of scope of this document. The report of [NGP] has given some 694 experiments and results by using commercial hardwares. 696 5.2. Flow Identification in Packet Forwarding 698 Flow identification in Packet Forwarding is same as the QoS channel 699 establishment by Setup message. It is to forward a packet with a 700 specified QoS process if the packet is identified to be belonging to 701 specified flow(s). 703 There are two method used in data forwarding to identify flows: 705 1. Hardware was programmed to use tuples in IP header implicitly. 706 This is indicated by that the "SIS" is zero or the Service ID is 707 not used. When a packet is received, its tuples are looked up 708 according to the value of "FI". If there is a QoS table has 709 match for the packet, the packet will be processed by the QoS 710 state found in the QoS table. This method does not need any EH 711 added into the data packet unless the data packet function to 712 collect the QoS forwarding state on the path. 714 2. Hardware was programmed to use service ID to identify flows. 715 This is indicated by that the "SIS" is not zero. When a packet 716 is received, the service ID associated with the hop is retrieved 717 and looked up for the QoS table. If it has match for the packet, 718 the packet will be processed by the QoS state entry found in the 719 QoS table. 721 5.3. QoS Forwarding State Detection and Failure Handling 723 QoS forwarding may fail due to different reasons: 725 1. Hardware failure in HbH-EH-aware node. 727 2. IP path change due to link failure, node failure or routing 728 changes; And the IP path change has impact to the HbH-EH-aware 729 node. 731 3. Network topology change; and the change leads to the changes of 732 HbH-EH-aware nodes. 734 Application may need to be aware of the service status of QoS 735 guarantee when the application is using a TCP session with QoS. In 736 order to provide such feature, the TCP stack in the source host can 737 detect the QoS forwarding state by sending TCP data packet with 738 Forwarding State message coded as HbH-EH. After the TCP data packet 739 reaches the destination host, the host will copy the forwarding state 740 into a Forwarding State Report message, and send it with another TCP 741 packet (for example, TCP-ACK) in reverse direction to the source 742 host. Thereafter, the source host can obtain the QoS forwarding 743 state on all HbH-EH-aware nodes. 745 A host can do the QoS forwarding state detection by three ways: on 746 demand, periodically or constantly. 748 After a host detects that there is QoS forwarding state failure, it 749 can repair such failure by sending another Setup message embedded 750 into a HbH-EH of any TCP packet. This repairing can handle all 751 failure case mentioned above. 753 If a failure cannot be repaired, host will be notified, and 754 appropriate action can be taken, see section 7.1 756 6. Details of Working with Transport Layer 758 The proposed new IP service is transport agnostic, which means any 759 transport layer protocol can use it. 761 6.1. Working with TCP 763 Considering TCP as the most widely used transport layer protocol, 764 this document uses TCP as an example of transport protocol to show 765 how it works with the proposed IP service. 767 The following is the list of messages for signaling and associated 768 data forwarding. 770 o Setup: This is for the setup of QoS channel through the IP path. 772 o Bandwidth: This is the required bandwidth for the QoS channel. It 773 has minimum (CIR) and maximum bandwidth (PIR). 775 o Latency: This is the required latency for the QoS channel, it is 776 the bounded latency for each hop on the path. This is not the end 777 to end latency. 779 o Burst: This is the required burst for the QoS channel, it is the 780 maximum burst size. 782 o Authentication: This is the security message for a in-band 783 signaling. 785 o OAM: This is the Operation and Management message for the QoS 786 channel. 788 o Setup State Report: This is the state report of a setup message. 790 o Forwarding State: This is the forwarding state message used for 791 data packet. 793 o Forwarding State Report: This is the forwarding state report of a 794 QoS channel. 796 There are three scenarios of QoS signaling for TCP session setup with 797 QoS 799 1. Upstream: This is for the direction of client to server. A 800 application decides to open a TCP session with upstream QoS (for 801 uploading), it will call TCP API to open a socket and connect to 802 a server. The client host will form a TCP SYN packet with the 803 HbH-EH in the IPv6 header. The EH includes Setup message and 804 Bandwidth message, and optionally Latency, Burst, Authentication 805 and OAM messages. The packet is forwarded at each hop. Each 806 HbH-EH-aware nodes will process the signaling message to finish 807 the following tasks before forwarding the packet to next hop: 809 * Retrieve the QoS parameters to program the Hardware, it 810 includes: FL, Time, Bandwidth, Latency, Burst 812 * Update the field in the EH, it includes: Hop_number, 813 Total_latency, and possibly Service ID List 815 When the server receives the TCP SYN, the Host kernel will also 816 check the HbH-EH while punting the TCP packet to the TCP stack 817 for processing. If the HbH-EH is present and the Report bit is 818 set, the Host kernel must form a new Setup State Report message, 819 all fields in the message must be copied from the Setup message 820 in the HbH-EH. When the TCP stack is sending the TCP-SYNACK to 821 the client, the kernel must add the Setup State Report message as 822 a Dst-EH in the IPv6 header. After this, the IPv6 packet is 823 complete and can be sent to wire; When the client receives the 824 TCP-SYNACK, the Host kernel will check the Dst-EH while punting 825 the TCP packet to the TCP stack for processing. If the Dst-EH is 826 present and the Setup State Report message is valid, the kernel 827 must read the Setup State Report message. Depending on the setup 828 state, the client will operate according to description in 829 section 7.1 831 2. Downstream: This is for the direction of server to client. A 832 application decides to open a TCP session with downstream QoS 833 (for downloading), it will call TCP API to open a socket and 834 connect to a server. The client host will form a TCP SYN packet 835 with the Dst-EH in the IPv6 header. The EH includes Bandwidth 836 message, and optionally Latency, Burst messages. The packet is 837 forwarded at each hop. Each hop will not process the Dst-EH. 838 When the server receives the TCP SYN, the Host kernel will check 839 the Dst-EH while punting the TCP packet to the TCP stack for 840 processing. If the Dst-EH is present, the Host kernel will 841 retrieve the QoS requirement information from Bandwidth, Latency 842 and Burst message, and check the QoS policy for the user. If the 843 user is allowed to get the service with the expected QoS, the 844 server will form a Setup message similar to the case of client to 845 server, and add it as the HbH-EH in the IPv6 header, and send the 846 TCP-SYNACK to client. Each HbH-EH-aware nodes on the path from 847 server to client will process the message similar to the case of 848 client to server. After the client receives the TCP-SYNACK, The 849 client will send the Setup State Report message to server as the 850 Dst-EH in the TCP-ACK. Finally the server receives the TC-ACK 851 and Setup State Report message, it can send the data to the 852 established session according to the pre-negotiated QoS 853 requirements. 855 3. Bi-direction: This is the case that the client wants to setup a 856 session with bi-direction QoS guarantee. The detailed operations 857 are actually a combination of Upstream and Downstream described 858 above. 860 After a QoS channel is setup, the in-band signaling message can still 861 be exchanged between two hosts, there are two scenarios for this. 863 1. Modify QoS on the fly: When the pre-set QoS parameters need to be 864 adjusted, the application at source host can re-send a new in- 865 band signaling message, the message can be embedded into any TCP 866 packet as a IPv6 HbH-EH. The QoS modification should not impact 867 the established TCP session and programmed QoS service. Thus, 868 there is no service impacted during the QoS modification. 869 Depending on the hardware performance, the signaling message can 870 be sent with TCP packet with different data size. If the 871 performance is high, the signaling message can be sent with any 872 TCP packet; otherwise, the signaling message should be sent with 873 small size TCP packet or zero-size TCP packet (such as TCP ACK). 874 Modification of QoS on the fly is a very critical feature for the 875 so called "Application adaptive QoS transport service". With 876 this service, an application (or the proxy from a service 877 provider) could setup an optimized CIR for different stage of 878 application for the economical and efficient purpose. For 879 example, in the transport of compressed video, the I-frame has 880 big size and cannot be lost, but P-frame and B-frame both have 881 smaller size and can tolerate some loss. There are much more 882 P-frame and B-frame than I-frame in videos with smooth changes 883 and variations in images [I-D.han-iccrg-arvr-transport-problem]. 884 Based on this characteristics, application can request a 885 relatively small CIR for the time of P-frame and P-frame, and 886 request a big CIR for the time of I-frame. 888 2. Repairing of the QoS channel: This is the case the QoS channel 889 was broken and need to be repaired, see section 5.3. 891 6.2. Working with UDP and other Protocols 893 There are other transport layer protocols, such as UDP, QUICK and 894 SCTP, and for these protocols similar strategy as TCP can be applied. 895 The to establish a closed-loop for the transport control. 897 For protocols with natively bi-directional control mechanism such as 898 SCTP, only some QoS control functionalities for the protocol need to 899 be added. The mechanism for TCP can be borrowed for such job. There 900 will be the QoS setup for one directional data stream, and QoS setup 901 state report for another directional data stream. The protocol may 902 also have functionalities in the stack to handle the adjustment of 903 the behaviour for different QoS setup and setup states. 905 For protocols that natively lack the feed-back control mechanism to 906 form a closed-loop such as UDP, this mechanism needs to be added into 907 the streams. There are two options to realize this: 909 1. Modify the protocol itself to have some state machine to 910 establish the closed-loop for the protocol. This can be done in 911 the kernel of the OS by modifying the protocol stack. 913 2. Modify the user data stream to introduce the closed-loop scheme, 914 this becomes as application work. It is up to application to add 915 or modify codes for the state machine of the closed-loop control. 917 7. Additional Considerations 919 This document only covers the details of setting up a path with QoS 920 using IPv6, and TCP is used as an example of transport layer protocol 921 to achieve flow level service. Only basic scenarios are covered, and 922 there are lots of open issues to be researched. The following is a 923 non-comprehensive list, and they can be addressed in separate drafts. 925 7.1. User and Application driven 927 The QoS transport service is initiated and controlled by end user's 928 application. Following tasks are done in host: 930 1. The detailed QoS parameters in signaling message are set by end 931 user application. New socket option must be added, the option is 932 a place holder for QoS parameters (Setup, Bandwidth, etc.), Setup 933 State Report and Forwarding State Report messages. 935 2. The Setup State Report and Forwarding State Report message 936 received at host are processed by transport service in kernel. 937 The Setup State Report message processed at host can result in 938 the notification to the application whether the setup is 939 successful. If the setup is successful, the application can 940 start to use the socket having the QoS support; If the setup is 941 failed, the application may have three choices: 943 * Lower the QoS requirement and re-setup a new QoS channel with 944 new in-band signaling message. 946 * Use the TCP session as traditional transport without any QoS 947 support. 949 * Lookup the service provider for help to locate the problem in 950 network. 952 7.2. Traffic Management in Host 954 In order to better accommodate this new IP in-band service, the OS on 955 a host may be changed in traffic management related areas. There are 956 two parts for traffic management to be changed: one is to manage 957 traffic going out a host's shared links, and the other is congestion 958 control for TCP flows. 960 1. For current traffic management in a host, all TCP/UDP sessions 961 will share the bandwidth for all egress links. For the purpose 962 to work with the differentiated service provided by under layer 963 network in bandwidth and latency, the kernel may allocate 964 expected resource to applications that are using the QoS 965 transport service. For example, kernel can queue different 966 packets from different applications or users to different queue 967 and schedule them in different priority. Only after this change, 968 some application can use more bandwidth and get less queuing 969 delay for a link than others. 971 2. The congestion control in a host manages the behavior of TCP 972 flow(s). This includes important features like slow start, AIMD, 973 fast retransmit, selective ACK, etc. To accommodate the benefit 974 of the QoS guaranteed transport service, the congestion control 975 can be much simpler [I-D.han-tsvwg-cc]. The new congestion 976 control is related to the implementation of QoS guarantee. 977 Following is a simple congestion control algorithm assuming that 978 the CIR is guaranteed and PIR is shared between flows: 980 * There is no slow start, the TCP can start sending traffic at 981 the rate of CIR. 983 * The AIMD is kept, but the range of the sawtooth pattern should 984 be maintained between CIR and PIR. 986 * Other congestion control features can be kept. 988 7.3. Heterogeneous Network 990 When an IP network is connected with a non-IP network, such as MPLS 991 or Ethernet network, the in-band signaling should also work in that 992 network to achieve an end-to-end connection. The behavior, protocol 993 and rules in the interworking with non-IP network is out of the scope 994 of this draft, and further research needs to be done to solve the 995 problem. 997 7.4. Proxy Control 999 It is expected that in a real service provider network, the in-band 1000 signaling will be checked, filtered and managed at proxy routers. It 1001 serves the following purposes: 1003 1. A proxy can check if the in-band signaling from an end user meets 1004 the SLA compliance. This adds extra security and DOS attack 1005 prevention. 1007 2. A proxy can collect the statistics for user's TCP flows and check 1008 the in-band signaling for accounting and charging. 1010 3. A proxy can insert and process appropriate in-band signaling for 1011 TCP flows if the host does not support this new feature. This 1012 can provide backward compatibility, also enable the host to use 1013 the new feature. 1015 8. IANA Considerations 1017 This document defines a new option type for the Hop-by-Hop Options 1018 header and the Destination Options header. According to [RFC8200], 1019 the detailed value are: 1021 +-----------+----------------+---------------------+---------------+ 1022 | | Binary Value | | | 1023 | Hex Value +----+---+-------+ Description | Reference | 1024 | | act|chg| rest | | | 1025 +-----------+----+---+-------+---------------------+---------------+ 1026 | 0x0 | 00 | 0 | 10000 | In-band Signaling | Section 4 | 1027 | | | | | | in this doc | 1028 +-----------+----+---+-------+---------------------+---------------+ 1030 Figure 3: The New Option Type 1032 1. The highest-order 2 bits: 00, indicating if the processing IPv6 1033 node does not recognize the Option type, skip over this option and 1034 continue processing the header. 1036 2. The third-highest-order bit: 0, indicating the Option Data does 1037 not change en route. 1039 3. The low-order 5 bits: 10000, assigned by IANA. 1041 This document also defines a 4-bit subtype field, for which IANA will 1042 create and will maintain a new sub-registry entitled "In-band 1043 signaling Subtypes" under the "Internet Protocol Version 6 (IPv6) 1044 Parameters" [IPv6_Parameters] registry. Initial values for the 1045 subtype registry are given below 1047 +-------+------------+-----------------------------+---------------+ 1048 | Type | Mnemonic | Description | Reference | 1049 +-------+------------+-----------------------------+---------------+ 1050 | 0 | SETUP | Setup object | Appendix B | 1051 +-------+------------+-----------------------------+---------------+ 1052 | 1 | BANDWIDTH | Bandwidth object | Appendix B | 1053 +-------+------------+-----------------------------+---------------+ 1054 | 2 | BURST | Burst object | Appendix B | 1055 +-------+------------+-----------------------------+---------------+ 1056 | 3 | LATENCY | Latency object | Appendix B | 1057 +-------+------------+-----------------------------+---------------+ 1058 | 4 | AUTH | Authentication object | Appendix B | 1059 +-------+------------+-----------------------------+---------------+ 1060 | 5 | OAM | OAM object | Appendix B | 1061 +-------+------------+-----------------------------+---------------+ 1062 | 6 | FWD STATE | Forward state | Appendix B | 1063 +-------+------------+-----------------------------+---------------+ 1064 | 7 |SETUP REPORT| Setup state report | Appendix B | 1065 +-------+------------+-----------------------------+---------------+ 1066 | 8 | FWD REPORT | Forwarding state report | Appendix B | 1067 +-------+------------+-----------------------------+---------------+ 1069 Figure 4: The In-band Signaling Sub Type 1071 9. Security Considerations 1073 It is important to guarantee that the resource reservation is used by 1074 authenticated users, and false signaling should not be accepted or 1075 processed. The following aspects may be considered: 1077 Authentication of user 1079 If an user is interested in using this new service, the user 1080 should sign up to a service provider. Service provider should 1081 do the proper authentication check for a new user, and 1082 establish account for the user. 1084 After the sign up, a user should provide a security key to the 1085 service provider through a secured channel (https, registered 1086 mail, etc.), or the key could be generated and given to user by 1087 the service provider. Service provider should distribute the 1088 security key of the user to different network device. More 1089 specifically, the security key should be distributed securely 1090 to all HbH-EH-aware nodes for an open network, or the proxy for 1091 a closed network. 1093 Proxy 1095 Proxy or gateway is the 1st network device connecting to 1096 customer's devices (Host, phone, etc.) that can generate the 1097 signaling for resource reservation. The functionality of the 1098 Proxy is to check if the signaling is allowed to go through 1099 SP's network. This can be done by checking the signaling 1100 integrity and other info associated with the user, such as the 1101 source/destination IP address, the account balance, the user's 1102 privilege, etc. 1104 Authentication of signaling message 1106 The signaling for resource reservation should be checked at 1107 each HbH-EH-aware nodes or a proxy node. 1109 Service ID is originally used for performance improvement of 1110 forwarding with QoS, and it can also provide additional security 1111 protection of forwarding resource in data plane. Service ID in each 1112 HbH-EH-aware node is to represent an IP flow with programmed QoS 1113 service, and it is a local significant number generated by a router 1114 to identify a flow that was offered QoS service. So, the router can 1115 periodically change the number for the same flow to protect any 1116 middle box sniffing for DOS attacking. It can be done by host 1117 periodical send out in-band signaling with the same QoS parameters 1118 and obtain the new Service ID and Service ID List for the use of next 1119 data forwarding. 1121 10. References 1123 10.1. Normative References 1125 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1126 Requirement Levels", BCP 14, RFC 2119, 1127 DOI 10.17487/RFC2119, March 1997, 1128 . 1130 [RFC2581] Allman, M., Paxson, V., and W. Stevens, "TCP Congestion 1131 Control", RFC 2581, DOI 10.17487/RFC2581, April 1999, 1132 . 1134 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1135 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1136 May 2017, . 1138 [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1139 (IPv6) Specification", STD 86, RFC 8200, 1140 DOI 10.17487/RFC8200, July 2017, 1141 . 1143 10.2. Informative References 1145 [eNodeB] wikipedia, "eNodeB", 2018, 1146 . 1148 [EPC] 3GPP, "The Evolved Packet Core", 2018, 1149 . 1152 [HU5G] Huawei, "5G Vision: 100 Billion connections, 1 ms Latency, 1153 and 10 Gbps Throughput", 2015, 1154 . 1156 [I-D.falk-xcp-spec] 1157 Falk, A., "Specification for the Explicit Control Protocol 1158 (XCP)", draft-falk-xcp-spec-03 (work in progress), July 1159 2007. 1161 [I-D.han-iccrg-arvr-transport-problem] 1162 Han, L. and K. Smith, "Problem Statement: Transport 1163 Support for Augmented and Virtual Reality Applications", 1164 draft-han-iccrg-arvr-transport-problem-01 (work in 1165 progress), March 2017. 1167 [I-D.han-tsvwg-cc] 1168 Han, L., Qu, Y., and T. Nadeau, "A New Congestion Control 1169 in Bandwidth Guaranteed Network", draft-han-tsvwg-cc-00 1170 (work in progress), March 2018. 1172 [I-D.harper-inband-signalling-requirements] 1173 Harper, J., "Requirements for In-Band QoS Signalling", 1174 draft-harper-inband-signalling-requirements-00 (work in 1175 progress), January 2007. 1177 [I-D.ietf-aqm-codel] 1178 Nichols, K., Jacobson, V., McGregor, A., and J. Iyengar, 1179 "Controlled Delay Active Queue Management", draft-ietf- 1180 aqm-codel-06 (work in progress), December 2016. 1182 [I-D.ietf-aqm-fq-codel] 1183 Hoeiland-Joergensen, T., McKenney, P., 1184 dave.taht@gmail.com, d., Gettys, J., and E. Dumazet, "The 1185 FlowQueue-CoDel Packet Scheduler and Active Queue 1186 Management Algorithm", draft-ietf-aqm-fq-codel-06 (work in 1187 progress), March 2016. 1189 [I-D.ietf-aqm-pie] 1190 Pan, R., Natarajan, P., Baker, F., and G. White, "PIE: A 1191 Lightweight Control Scheme To Address the Bufferbloat 1192 Problem", draft-ietf-aqm-pie-10 (work in progress), 1193 September 2016. 1195 [I-D.ietf-detnet-use-cases] 1196 Grossman, E., "Deterministic Networking Use Cases", draft- 1197 ietf-detnet-use-cases-19 (work in progress), October 2018. 1199 [I-D.ietf-spring-segment-routing] 1200 Filsfils, C., Previdi, S., Ginsberg, L., Decraene, B., 1201 Litkowski, S., and R. Shakir, "Segment Routing 1202 Architecture", draft-ietf-spring-segment-routing-15 (work 1203 in progress), January 2018. 1205 [I-D.ietf-tcpm-dctcp] 1206 Bensley, S., Eggert, L., Thaler, D., Balasubramanian, P., 1207 and G. Judd, "Datacenter TCP (DCTCP): TCP Congestion 1208 Control for Datacenters", draft-ietf-tcpm-dctcp-03 (work 1209 in progress), November 2016. 1211 [I-D.roberts-inband-qos-ipv6] 1212 Roberts, L. and J. Harford, "In-Band QoS Signaling for 1213 IPv6", draft-roberts-inband-qos-ipv6-00 (work in 1214 progress), July 2005. 1216 [I-D.sridharan-tcpm-ctcp] 1217 Sridharan, M., Tan, K., Bansal, D., and D. Thaler, 1218 "Compound TCP: A New TCP Congestion Control for High-Speed 1219 and Long Distance Networks", draft-sridharan-tcpm-ctcp-02 1220 (work in progress), November 2008. 1222 [IPv6_Parameters] 1223 IANA, "Internet Protocol Version 6 (IPv6) Parameters", 1224 2015, . 1227 [MEC] ETSI, "Multi-access Edge Computing", 2018, 1228 . 1231 [NGP] ETSI, "Next Generation Protocols (NGP); Recommendation for 1232 New Transport Technologies", 2018, 1233 . 1236 [QU2016] Qualcomm, "Leading the world to 5G", 2016, 1237 . 1240 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1241 "Definition of the Differentiated Services Field (DS 1242 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1243 DOI 10.17487/RFC2474, December 1998, 1244 . 1246 [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., 1247 and W. Weiss, "An Architecture for Differentiated 1248 Services", RFC 2475, DOI 10.17487/RFC2475, December 1998, 1249 . 1251 [RFC3175] Baker, F., Iturralde, C., Le Faucheur, F., and B. Davie, 1252 "Aggregation of RSVP for IPv4 and IPv6 Reservations", 1253 RFC 3175, DOI 10.17487/RFC3175, September 2001, 1254 . 1256 [Tactile] JDavid Szabo, et al. Proceedings of European Wireless 1257 2015; 21th European Wireless Conference, "Towards the 1258 Tactile Internet: Decreasing Communication Latency with 1259 Network Coding and Software Defined Networking", 2015, 1260 . 1262 [TCP-vegas] 1263 Peterson, L., "TCP Vegas: New Techniques for Congestion 1264 Detection and Avoidance - CiteSeer page on the 1994 1265 SIGCOMM paper", 1994. 1267 [TCP_Targets] 1268 Andreas Benthin, Stefan Mischke, University of Paderborn, 1269 "Bandwidth Allocation of TCP", 2004. 1271 Appendix A. Acknowledgements 1273 The authors are very grateful to Fred Baker for his valuable 1274 contributions to this document. 1276 We appreciate the following people who made lots of contributions to 1277 this draft: Guoping Li, Boyan Tu, and Xuefei Tan, and thank Huawei 1278 Nanjing research team led by Feng Li to provide the Product on 1279 Concept (POC) development and test, the team members include Fengxin 1280 Sun, Xingwang Zhou, and Weiguang Wang. We also like to thank other 1281 people involved in the discussion of solution: Tao Ma from Future 1282 Network Strategy dept. 1284 Appendix B. Message Objects 1286 This section defines detailed objects used in different messages. 1288 B.1. Setup State Object 1289 0 1 2 3 1290 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1291 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1292 |0 0 0 0|ver| FI|R|SIS|P| Time | Hop_num |u| Total_latency | 1293 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1294 | State for each hop index | 1295 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1296 | Service ID list for hops | 1297 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-. . . + 1299 Type = 0, Setup state; 1301 Version: The version of the protocol for the QoS 1303 FI: Flow identification method, 1305 0: 5 tuples; 1: src,dst,port; 2: src,dst; 3: DSCP 1307 R: If the destination host report the received Setup state to 1309 the src address by Destination EH. 0: dont report; 1: report 1311 SIS: Service ID size; 0: 0bits, 1: 16bits, 2: 20bits, 3: 32bits 1313 P: 0: program HW for the QoS from src to dst; 1315 1: De-program HW for the QoS from src to dst 1317 Time: The life time of QoS forwarding state in second. 1319 Hop_num: The total hop number on the path set by host. It must be 1321 decremented at each hop after the processing. 1323 u: the unit of latency, 0: ms; 1: us 1325 Total_latency : Latency accumulated from each hop, each hop will 1327 add the latency in the device to this value. 1329 Figure 5: The Setup State Object 1331 Setup state for each hop index: each bit is the setup state on each 1332 hop on the path, 0: failed; 1: success. The 1st hop is at the most 1333 significant bit. 1335 Service ID list for hops: it is for all hops on the path, each 1336 service ID bit size is defined in SIS. The 1st service ID is at the 1337 top of the stack. Each hop add its service ID at the correct 1338 position indexed by the current hop number for the router. 1340 The Setup object is embedded into the hop-by-hop EH to setup the QoS 1341 in the device on the IP forwarding path. To keep the whole setup 1342 message size unchanged at each hop, the total hop number must be 1343 known at the source host. The total hop number can be detected by 1344 OAM. The service ID list is empty before the 1st hop receives the 1345 in-band signaling. Each hop then fill up the associated service ID 1346 into the correct place determined by the index of the hop. 1348 B.2. Bandwidth Object 1350 0 1 2 3 1351 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1352 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1353 |0 0 0 1| reserved | Minimum bandwidth | 1354 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1355 | Maximum bandwidth | 1356 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1358 Type = 1, 1360 Minimum bandwidth : The minimum bandwidth required, or CIR, unit 1361 Mbps 1363 Maximum bandwidth : The maximum bandwidth required, or PIR, unit 1364 Mbps 1366 Figure 6: The Bandwidth Object 1368 B.3. Burst Msg 1369 0 1 2 3 1370 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1371 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1372 |0 0 1 0| Burst size | 1373 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1375 Type = 2, 1377 Burst size : The burst size, unit M bytes 1379 Figure 7: The burst message 1381 B.4. Latency Object 1383 0 1 2 3 1384 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1385 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1386 |0 0 1 1|u| Latency | 1387 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1389 Type = 3, 1391 u: the unit of the latency 1393 0: ms; 1: us 1395 Latency: Expected maximum latency for each hop 1397 Figure 8: The Latency Object 1399 B.5. Authentication Object 1400 0 1 2 3 1401 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1402 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1403 |0 1 0 0| MAC_ALG | res | MAC data (variable length) | 1404 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+. . .+ 1406 Type = 4, 1408 MAC_ALG: Message Authentication Algorithm 1410 0: MD5; 1:SHA-0; 2: SHA-1; 3: SHA-256; 4: SHA-512 1412 MAC data: Message Authentication Data; 1414 Res: Reserved bits 1416 Size of signaling data (opt_len): Size of MAC data + 2 1418 MD5: 18; SHA-0: 22; SHA-1: 22; SHA-256: 34; SHA-512: 66 1420 Figure 9: The Authentication Object 1422 B.6. OAM Object 1424 0 1 2 3 1425 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1426 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1427 |0 1 0 1| OAM_t | OAM_len | OAM data (variable length) | 1428 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+. . .+ 1430 Type = 5, 1432 OAM_t : OAM type 1434 OAM_len : 8-bit unsigned integer. Length of the OAM data, in octets; 1436 OAM data: OAM data, details of OAM data are TBD. 1438 Figure 10: The OAM Object 1440 B.7. Forwarding State Object 1442 0 1 2 3 1443 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1444 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1445 |0 1 1 0|ver| FI|R|SIS|P| Time | Hop_num |u| Total_latency | 1446 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1447 | Forwarding state for each hop index | 1448 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1449 | Service ID list for hops | 1450 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-. . . + 1452 Type = 6, Forwarding state; 1454 All parameter definitions and process in the 1st row are same in 1456 the setup message. 1458 Forward state for each hop index : each bit is the fwd state on 1459 each 1461 hop on the path, 0: failed; 1: success; The 1st hop is at the 1463 most significant bit. 1465 Service ID list for hops: it is for all hops on the path, each index 1466 bit 1468 size is defined in SIS. The list is from the setup report message. 1470 Figure 11: The Forwarding State Object 1472 B.8. Setup State Report Object 1473 0 1 2 3 1474 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1475 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1476 |0 1 1 1|ver|H|u| Total_latency | Reserved | 1477 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1478 | State for each hop index | 1479 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1480 | Service ID list for hops | 1481 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-. . . + 1483 Type = 7, Setup state report; 1485 H: Hop number bit. When a host receives a setup message and form 1487 a setup report message, it must check if the Hop_num in setup 1489 message is zero. If it is zero, the H bit is set to one, and if 1491 it is not zero, the H bit is clear. This will notify the source 1493 of setup message that if the original Hop_num was correct. 1495 Following are directly copied from the setup message: 1497 u, Total_latency; 1499 State for each hop index 1501 Service ID list for hops. 1503 Figure 12: The Setup State Report Object 1505 B.9. Forward State Report Object 1506 0 1 2 3 1507 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1508 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1509 |1 0 0 0|ver|H|u| Total_latency | Reserved | 1510 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1511 | Forwarding state for each hop index | 1512 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1514 Type = 8, Forwarding state report; 1516 H: Hop number bit. When a host receives a Forward State message 1518 and form a Forward State Report message, it must check if the 1520 Hop_num in Forward State message is zero. If it is zero, the H bit 1522 is set to one, and if it is not zero, the H bit is clear. 1524 This will notify the source of Forward State message that if the 1526 original Hop_num was set correct. 1528 Following are directly copied from the Forward State message: 1530 u, Total_latency; 1532 Forwarding State for each hop index 1534 Figure 13: The Fwd State Report Object 1536 Authors' Addresses 1538 Lin Han 1539 Huawei Technologies 1540 2330 Central Expressway 1541 Santa Clara, CA 95050 1542 USA 1544 Phone: +10 408 330 4613 1545 Email: lin.han@huawei.com 1546 Yingzhen Qu 1547 Huawei Technologies 1548 2330 Central Expressway 1549 Santa Clara, CA 95050 1550 USA 1552 Email: yingzhen.qu@huawei.com 1554 Lijun Dong 1555 Huawei Technologies 1556 2330 Central Expressway 1557 Santa Clara, CA 95050 1558 USA 1560 Email: lijun.dong@huawei.com 1562 Richard Li 1563 Huawei Technologies 1564 2330 Central Expressway 1565 Santa Clara, CA 95050 1566 USA 1568 Email: renwei.li@huawei.com 1570 Thomas Nadeau 1571 Lucid Vision 1572 Hampton NH 03842 1573 USA 1575 Email: tnadeau@lucidvision.com 1577 Kevin Smith 1578 Vodafone 1579 UK 1581 Email: Kevin.Smith@vodafone.com 1583 Jeff Tantsura 1584 Apstra 1586 Email: jefftant.ietf@gmail.com