idnits 2.17.00 (12 Aug 2021) /tmp/idnits17901/draft-han-tsvwg-ip-transport-qos-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 2 instances of too long lines in the document, the longest one being 2 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (July 2, 2018) is 1419 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC2581' is defined on line 1089, but no explicit reference was found in the text == Unused Reference: 'I-D.falk-xcp-spec' is defined on line 1104, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-aqm-codel' is defined on line 1125, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-aqm-fq-codel' is defined on line 1130, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-aqm-pie' is defined on line 1137, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-tcpm-dctcp' is defined on line 1149, but no explicit reference was found in the text == Unused Reference: 'I-D.sridharan-tcpm-ctcp' is defined on line 1160, but no explicit reference was found in the text == Unused Reference: 'TCP-vegas' is defined on line 1181, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2581 (Obsoleted by RFC 5681) == Outdated reference: draft-ietf-aqm-codel has been published as RFC 8289 == Outdated reference: draft-ietf-aqm-fq-codel has been published as RFC 8290 == Outdated reference: draft-ietf-aqm-pie has been published as RFC 8033 == Outdated reference: draft-ietf-spring-segment-routing has been published as RFC 8402 == Outdated reference: draft-ietf-tcpm-dctcp has been published as RFC 8257 Summary: 2 errors (**), 0 flaws (~~), 15 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 TSVWG Working Group L. Han 3 Internet-Draft Y. Qu 4 Intended status: Informational L. Dong 5 Expires: January 3, 2019 Huawei Technologies 6 T. Nadeau 7 Lucid Vision 8 K. Smith 9 Vodafone 10 July 2, 2018 12 Resource Reservation Protocol for IP Transport QoS 13 draft-han-tsvwg-ip-transport-qos-00 15 Abstract 17 IP has been known as best-effort data transmission. However there 18 are new applications requiring IP to provide deterministic services 19 in terms of bandwidth and latency, such as network based AR/VR 20 (Augmented Reality and Virtual Reality), industrial internet. This 21 document proposes a solution in IPv6 that can be used by transport 22 layer protocols to guarantee certain level of service quality. This 23 new service is fined-grained and could apply to individual or 24 aggregated TCP/UDP flow(s). 26 Status of This Memo 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF). Note that other groups may also distribute 33 working documents as Internet-Drafts. The list of current Internet- 34 Drafts is at https://datatracker.ietf.org/drafts/current/. 36 Internet-Drafts are draft documents valid for a maximum of six months 37 and may be updated, replaced, or obsoleted by other documents at any 38 time. It is inappropriate to use Internet-Drafts as reference 39 material or to cite them other than as "work in progress." 41 This Internet-Draft will expire on January 3, 2019. 43 Copyright Notice 45 Copyright (c) 2018 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents 50 (https://trustee.ietf.org/license-info) in effect on the date of 51 publication of this document. Please review these documents 52 carefully, as they describe your rights and restrictions with respect 53 to this document. Code Components extracted from this document must 54 include Simplified BSD License text as described in Section 4.e of 55 the Trust Legal Provisions and are provided without warranty as 56 described in the Simplified BSD License. 58 Table of Contents 60 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 61 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 62 3. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 5 63 3.1. Design Targets . . . . . . . . . . . . . . . . . . . . . 6 64 3.2. Scope and Assumptions . . . . . . . . . . . . . . . . . . 6 65 3.3. Sub-layer in IP for Transport Control . . . . . . . . . . 7 66 3.4. IP In-band signaling . . . . . . . . . . . . . . . . . . 8 67 3.5. IPv6 Approach . . . . . . . . . . . . . . . . . . . . . . 8 68 4. Key Messages and Parameters . . . . . . . . . . . . . . . . . 10 69 4.1. Setup and Setup State Report messages . . . . . . . . . . 10 70 4.2. Forwarding State and Forwarding State Report messages . . 11 71 4.3. Hop Number . . . . . . . . . . . . . . . . . . . . . . . 11 72 4.4. Flow Identifying Method and Service ID . . . . . . . . . 12 73 4.5. QoS State and life of Time . . . . . . . . . . . . . . . 13 74 4.6. Authentication . . . . . . . . . . . . . . . . . . . . . 13 75 5. Packet Forwarding . . . . . . . . . . . . . . . . . . . . . . 13 76 5.1. Basic Hardware Capability . . . . . . . . . . . . . . . . 14 77 5.2. Flow Identification in Packet Forwarding . . . . . . . . 14 78 5.3. QoS Forwarding State Detection and Failure Handling . . . 15 79 6. Details of Working with Transport Layer . . . . . . . . . . . 16 80 6.1. Working with TCP . . . . . . . . . . . . . . . . . . . . 16 81 6.2. Working with UDP and other Protocols . . . . . . . . . . 18 82 7. Additional Considerations . . . . . . . . . . . . . . . . . . 19 83 7.1. User and Application driven . . . . . . . . . . . . . . . 19 84 7.2. Traffic Management in Host . . . . . . . . . . . . . . . 20 85 7.3. Heterogeneous Network . . . . . . . . . . . . . . . . . . 20 86 7.4. Proxy Control . . . . . . . . . . . . . . . . . . . . . . 21 87 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 88 9. Security Considerations . . . . . . . . . . . . . . . . . . . 22 89 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 90 10.1. Normative References . . . . . . . . . . . . . . . . . . 23 91 10.2. Informative References . . . . . . . . . . . . . . . . . 24 92 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 26 93 Appendix B. Message Objects . . . . . . . . . . . . . . . . . . 26 94 B.1. Setup State Object . . . . . . . . . . . . . . . . . . . 26 95 B.2. Bandwidth Object . . . . . . . . . . . . . . . . . . . . 27 96 B.3. Burst Msg . . . . . . . . . . . . . . . . . . . . . . . . 27 97 B.4. Latency Object . . . . . . . . . . . . . . . . . . . . . 27 98 B.5. Authentication Object . . . . . . . . . . . . . . . . . . 28 99 B.6. OAM Object . . . . . . . . . . . . . . . . . . . . . . . 28 100 B.7. Forwarding State Object . . . . . . . . . . . . . . . . . 29 101 B.8. Setup State Report Object . . . . . . . . . . . . . . . . 29 102 B.9. Forward State Report Object . . . . . . . . . . . . . . . 30 103 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 30 105 1. Introduction 107 Recently, more and more new applications for The Internet are 108 emerging. These applications have a number of key requirements that 109 are common to all such as that their required bandwidth is very high 110 and/or latency is very low compared with traditional applications 111 like most of web and video applications. 113 For example, network based AR or VR applications may need a couple of 114 hundred Mbps bandwidth (throughput) and a low single digit 115 millisecond latency. Moreover, the difference between mean bit rate 116 and peak bit rate is significant due to the usage of compression 117 algorithm [I-D.han-iccrg-arvr-transport-problem], and this may cause 118 large burst and make traffic management more difficult. 120 Some future applications may expect network to provide a bounded 121 latency service, and one such example is tactile network [Tactile]. 123 With the technology development in 5G [HU5G][QU2016] and beyond, the 124 wireless access network is also rising the demand for the Ultra- 125 Reliable and Low-Latency Communications (URLLC), this also leads to 126 the question if IP can provide such service in Evolved Packet Core 127 (EPC) network. IP is becoming more and more important in EPC when 128 the Multi-access Edge Computing (MEC) for 5G requires the cloud and 129 data service moving closer to eNodeB. 131 Traditional IP network provides an unreliable or best-effort data 132 gram service over some underlying network (i.e.: ethernet, ATM, 133 etc...). The transport layer (TCP/UDP) on top of IP is based on this 134 fundamental architecture. The best-effort-only service has 135 influenced the transport evolution for quite long time, and results 136 in some widely accepted assumptions and solutions, such as: 138 1. The IP layer can only provide basic P2P (point to point) or P2MP 139 (point to multi-point) end-to-end connectivity in the Internet, 140 but the connectivity is not reliable and does not guarantee any 141 quality of service to end-user or application, such as bandwidth, 142 packet loss, latency etc. Due to this assumption, the transport 143 layer or application must have its own control mechanism in 144 congestion and flow to obtain the reliable and satisfactory 145 service to cooperate with the under layer network quality. 147 2. The transport layer assumes that the IP layer can only process 148 all IP flows equally in the hardware since the best effort 149 service is actually an un-differentiated service. The process 150 includes scheduling, queuing and forwarding. Thus, the transport 151 layer must behave nicely and friendly to make sure all flows will 152 only obtain its own faired share of resource, and no one could 153 consume more and no one could be starved. 155 This document proposes a new IP transport service that guarantees 156 bandwidth and latency for new applications. The scope and criteria 157 for the new technology will also be discussed. This new IP transport 158 service is designed to be supplementary to regular IP transport 159 services, only meant to be used for special applications that are 160 bandwidth and/or latency sensitive. 162 2. Terminology 164 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 165 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 166 document are to be interpreted as described in RFC 2119 [RFC2119]. 168 Abbreviations used in this documents: 170 E2E 171 End-to-end. 173 EH 174 IPv6 Extension Header or Extension Option. 176 QoS 177 Quality of Service. 179 OAM 180 Operation and Management. 182 In-band Signaling 183 In telecommunications, in-band signaling is sending control 184 information within the same band or channel used for voice or 185 video. 187 Out-of-band Signaling 188 out-of-band signaling is that the control information sent over 189 a different channel, or even over a separate network. 191 IP flow 192 For non-IPSec, an IP flow is identified by the source, 193 destination IP address, the protocol number, the source and 194 destination port number. 196 IP path 197 An IP path is the route that IP flow will traverse. It could 198 be the shortest path determined by routing protocols (IGP or 199 BPG), or the explicit path such as segment routing 200 [I-D.ietf-spring-segment-routing]. 202 QoS channel 203 A forwarding channel that is QoS guaranteed. It provides 204 additional QoS service to IP forwarding. A QoS channel can be 205 used for one or multiple IP flows depending on the granularity 206 of in-band signaling. 208 CIR 209 Committed Information Rate. 211 PIR 212 Peak Information Rate. 214 HbH-EH 215 IPv6 Hop-by-Hop Extension Header. 217 Dst-EH 218 IPv6 Destination Extension Header. 220 HbH-EH-aware node 221 Network nodes that are configured to process the IPv6 Hop-by- 222 Hop Extension Header. 224 3. Overview 226 Semiconductor chip technology has advanced significantly in the last 227 decade, and as such the widely used network processing and forwarding 228 process can now not only forward packets at line speed, but also 229 easily support other feature processing such as Qos for DiffServ/ 230 MPLS, Access Control List (ACL), fire wall, and Deep Packet 231 Inspection (DPI). 233 This advancement enables network processors to do the general process 234 to handle simple control messages for traffic management, such as 235 signaling for hardware programming, congestion state report, OAM, 236 etc. So now it's possible to treat some TCP/IP flows differently 237 from others and give them specified resource are feasible now by 238 using network processor. 240 This document proposes a deterministic IP transport service, which 241 can provide guaranteed bandwidth and latency. The solution is based 242 on the QoS implemented in network processor through in-band 243 signaling. 245 3.1. Design Targets 247 The proposed transport service is expected to satisfy the following 248 criteria: 250 o End user or application may directly use the new service. 252 o The new service can coexist with the current transport service and 253 is backward compatible. 255 o Service providers can manage the new service. 257 o Performance and scalability targets of this new service are 258 practical for vendors to achieve. 260 o The new service is transport agnostic. TCP, UDP and other 261 transport protocols on top of IP can use it. 263 3.2. Scope and Assumptions 265 The initial aim is to propose a solution for IPv6. To limit the 266 scope of the document and simplify the design and solution, the 267 following constraints are given: 269 1. The new service with QoS is aimed to be supplementary to regular 270 IP service. It is targeted for the applications that are 271 bandwidth and/or latency sensitive. It is not intended to 272 replace the TCP/IP variants that have been proved to be efficient 273 and successful for current applications. 275 2. The new service is limited within one administrative domain, even 276 it does not exclude the possibilities of extending the mechanism 277 for inter-domain scenarios. Currently only inter-domain security 278 is considered, and the inter-domain SLA, accounting and other 279 issues are not discussed. 281 3. Due to high bandwidth requirement of new service for individual 282 flow, the total number of the flows with the new service cannot 283 be high for a port, or a system. From another point of view, the 284 new service is targeted for applications that really need it, the 285 number of supported applications/users should be controlled and 286 cannot be unlimited. Hence the scalability requirement for the 287 new service is limited. 289 4. The new service must be able to coexist with the regular 290 transport service in the same hardware, and be backward 291 compatible. Also, a transport flow can switch between regular 292 transport and new service without service interruption. 294 3.3. Sub-layer in IP for Transport Control 296 In order to provide some new features for the layer above IP, it is 297 very useful to introduce an additional sub-layer, Transport Control, 298 between layer 3 (IP) and layer 4 (TCP/UDP). The new layer belongs to 299 IP, and is present only when the system needs to provide extra 300 control for the upper layer, in addition to the normal IP forwarding. 301 Fig 1. illustrates a new stack with the sub-layer. 303 +=========================+ 304 | APP | 305 +=========================+ 306 | TCP/UDP | 307 +=========================+ 308 | Transport Ctl | 309 +-+-+-+-+-+-+-+-+-+-+-+-+-+ 310 | IP | 311 +=========================+ 312 | Network Access | 313 +=========================+ 315 Figure 1: The new stack with a sub-layer in Layer 3 317 The new sub-layer is always bound with IP layer and can provide 318 support of the features for upper layer, such as: 320 In-band Signaling 321 The IP header with the new sub-layer can carry the signaling 322 information for the devices on the IP path. The information may 323 include all QoS related parameters used for hardware programming. 325 Congestion control 326 The congestion state in each device on the path can be detected 327 and notified to the source of flows by the sub-layer; The dynamic 328 congestion control instruction can also be carried by the sub- 329 layer and examined by network devices on the IP path. 331 IP Path OAM 332 The OAM instruction can be carried in the sub-layer, and the OAM 333 state can be notified to the source of flows by the sub-layer. 334 The OAM includes the path and device property detection, QoS 335 forwarding diagnosis and report. 337 IPv4 could use the IP option for the purpose of the sub-layer. But 338 due to the limit size of the IP option, the functionalities, 339 scalability of the layer is restricted. 341 IPv6 can realize the sub-layer easily using IPv6 extension header 342 [RFC8200]. The document will focus on the solution for IPv6 using 343 different IPv6 extension headers. 345 3.4. IP In-band signaling 347 In-band signaling messages are carried along with the payload. It is 348 guaranteed that the signaling follows the same path as the data flow, 349 and this can bring up some advantages that other methods can hardly 350 provide: 352 Diagnosis 353 The in-band signaling message takes the same path, same hops, same 354 processing at each hop as the data packet, this will make the 355 diagnosis for both signaling and data path easier. 357 Simplicity 358 The in-band signaling message is forwarded with the normal data 359 packet, it does not need to run a separate protocol. This will 360 dramatically reduce the complexity of the control. 362 Performance and scalability 363 Due to the simplicity of in-band signaling for control, it is 364 easier to provide a better performance and scalability for a new 365 future. 367 Note, the requirement of IP in-band signaling was proposed before by 368 John Harper [I-D.harper-inband-signalling-requirements]. And the in- 369 band QoS signaling for IPv6 was simply discussed in 370 [I-D.roberts-inband-qos-ipv6]. Unfortunately, both works did not 371 continue. 373 This document proposes a detailed solution for QoS service using in- 374 band signaling, and it also tries to address issues raised by 375 previous proposals, such as security, scalability and performance. 377 3.5. IPv6 Approach 379 IPv6 extension header is used for signaling. There are two types of 380 extension header used for the purpose of transport QoS control, one 381 is the hop-by-hop EH (HbH-EH) and another is the destination EH (Dst- 382 EH). 384 The HbH-EH may be examined and processed by the nodes that are 385 explicitly configured to do so [RFC8200], and these nodes are called 386 HbH-EH-aware nodes. Note, not all nodes along a patch need to HbH- 387 EH-aware. HbH-EH is used to carry the QoS requirement for dedicated 388 flow(s) and then the information is intercepted by HbH-EH-aware nodes 389 on the path to program hardware accordingly. 391 The destination EH will only be examined and processed by the 392 destination device that is associated with the destination IPv6 393 address in the IPv6 header. This EH is used to send the QoS related 394 report information directly to the source of the signaling at other 395 end. 397 The following figure illustrates the path setup process: 399 Setup Message (bandwidth, latency, setup state) 400 +------------------------HbH-EH------ -----------------+ 401 | | | | 402 | | | | 403 +------+ +------------+ +-----+ +------------+ +------+ 404 | | | | | | | | | | 405 | host |---| R1 |---| R2 |---| R3 |---|server| 406 | | |HbH-EH-aware| | | |HbH-EH-aware| | | 407 | | | | | | | | | | 408 +------+ +------------+ +-----+ +------------+ +------+ 409 | | 410 | | 411 +----------------------- Dst-EH ------------------------+ 412 Setup State Report Message (setup state report) 414 Figure 2: Path Setup 416 Using the figure.2 for illustration, to set up a path with resource 417 reservation, a setup message including QoS requirements, such as max/ 418 min bandwidth, burst size, the latency, and the setup state is sent 419 from the host to the server. After each HbH-EH-aware node along the 420 path receives the message, it reads the QoS information and programs 421 the hardware for resource reservation, queuing management etc. The 422 setup state object is updated at each HbH-EH-aware node to include 423 the QoS programming and provisioning result and the necessary 424 hardware reference information for IP forwarding with QoS. After the 425 setup message reaches the server, the server will send a setup state 426 report message encoded as Dst-EH to the host. The setup state report 427 message carries the path setup results from the setup state object. 429 4. Key Messages and Parameters 431 4.1. Setup and Setup State Report messages 433 Setup message is intended to program the hardware for QoS channel on 434 the IP path from the source to the destination expressed in IPv6 435 header. It is embedded as the HbH-EH in an IPv6 packet and will be 436 processed at each HbH-EH-aware node. For the simplicity, performance 437 and scalability purpose, not all routers along the path need do the 438 processing or be HbH-EH-aware. For different QoS requirements and 439 scenarios, different criteria can be used to configure HbH-EH-aware 440 nodes. 442 A throttle router is the device that an interested TCP/UDP session 443 cannot get the enough bandwidth to support its application, and it 444 will also contribute more to latency than non-throttle routers. The 445 regular throttle routers include the BRAS (broadband remote access 446 server) in broadband access network, the PGW (PDN Gateway) in LTE 447 network etc. In more general case, any routers which aggregated 448 traffic may become as a throttle router. Throttle routers should be 449 configured to process HbH-EH when: 451 o Reserved bandwidth is required: The throttle router is the 452 critical point to be configured to process the hop-by-hop EH for 453 the bandwidth reservation. Moreover, the direction of congestion 454 must be considered. 456 o Bounded latency is required: In theory, each router and switch 457 could contribute some delay to the end-to-end latency, but the 458 throttle router will contribute more than non-throttle routers, 459 and slow device will contribute more than fast device. We can use 460 OAM to detect the latency contribution in a network, and configure 461 those worst-cast devices to process the HbH-EH. 463 Setup State Report message is the message sent from the destination 464 host to the source host (from the point of view of the Setup 465 message). The message is embedded into the Dst-EH in any data 466 packet. The Setup State Report in the message is just a copy from 467 the Setup message received at the destination host for a typical TCP 468 session. The message is used at the source host to forward the 469 packet later and to do the congestion control. 471 ::= [ ] 472 [ ] [ ] 473 [ ] [ ] 474 ::= 475 [ ] 477 4.2. Forwarding State and Forwarding State Report messages 479 After the QoS is programmed by the in-band signaling, the specified 480 IP flows can be processed and forwarded for the QoS requirement. 481 There are two ways for host to use the QoS channel for associated TCP 482 session: 484 1. Host directly send the IP packet without any changes to the 485 packet, this is for the following cases: 487 * The hardware was programmed to use the tuples in IP header as 488 identification for QoS process (SIS = 0), and 490 * The packet does not function to collect the QoS forwarding 491 state on the path. 493 2. Host add the Forward State message into a data packet's IP header 494 as HbH-EH and send the packet, this is for the cases: 496 * The hardware was programmed to use the Service ID as 497 identification for QoS process (SIS != 0). 499 * The hardware was programmed to use the tuples in IP header as 500 identification for QoS process (SIS = 0), and the data packet 501 functions to collect the QoS forwarding state on the path. 502 This is the situation that host wants to detect the QoS 503 forwarding state for the purpose of failure handling (See 504 section 4.3). 506 Forwarding State message format is shown in the Section 6.7. It is 507 used to notify the service ID and also update QoS forwarding state 508 for the hops that are HbH-EH-aware nodes. 510 After Forwarding State message is reaching the destination host, the 511 host is supposed to retrieve it and form a Forwarding State Report 512 message, and carry it in any data packet as the Dst-EH, then send it 513 to the host in the reverse direction. 515 ::= [ ] 516 [ ] [ ] 517 ::= 518 [ ] 520 4.3. Hop Number 522 This is the parameter for total number of HbH-EH-aware nodes on the 523 path. It is the field "Hop_num" in Setup message, and is used to 524 locate the bit position for "Setup State" and the "Service ID" in 525 "Service ID List". The value of "Hop_num" must be decremented at 526 each HbH-EH-aware node. At the receiving host of the in-band 527 signaling, the Hop_num must be zero. 529 The source host must know the exact hop number, and setup the initial 530 value in the Setup message. The exact hop number can be detected 531 using OAM message. 533 4.4. Flow Identifying Method and Service ID 535 A QoS channel might be enforced for a group of flows or a delicate 536 flow, and flow identifying method means the way of identfying a flow 537 or a group of flows that can use a HW programmed QoS channel. 538 Different levels of flow granularities to support QoS are defined as 539 below: 541 Flow level 542 The flow identification could be 5 tuples for non IPSec IPv6 543 packet: the source, destination IP address, protocol number, 544 source and destination port number, and also could be 3 tuples for 545 IPSec IPv6 packet: the source, destination IP address and the flow 546 label. 548 Address level In-band Signaling 549 A flow of packets share the same source, destination IP address, 550 but with different protocol number. This is the scenario that the 551 signaling is for the aggregated flows which have the same source, 552 destination address. i.e, All TCP/UDP flows between the same 553 client and same server (only one address for client and one for 554 server) 556 Transport level In-band Signaling 557 Packets share the same source, destination IP address, protocol 558 number, but with different source or destination port number (non- 559 IPSec) or different flow label (IPSec). This could be for the 560 aggregated TCP or UDP flows that started and terminated at the 561 same IP addresses. 563 DiffServ level In-band Signaling 564 Packets share the same DSCP value. This means aggregated 565 differentiated service flows that have the same DSCP value. The 566 DSCP value is determined by the 6 most-significant bits in 8-bits 567 DiffServ field for IPv4 or 8-bits Traffic Class field for IPv6. 569 There are two ways for flow identifying. One is by tuple or DSCP 570 value in IP header, another is by a local significant number, called 571 service ID, generated and maintained in a router. When "Service ID 572 Size" (SIS) is zero, it means the "Flow identification method" (FI) 573 is used for both control plane and data plane. When "SIS" is not 574 zero, it means "FI" is only used in signaling of setting up the QoS 575 channel, and the data plane will only use the "Service ID". The use 576 of local generated number to identify flow is to speed up the flow 577 lookup and QoS process for data plane. 579 The "Service ID List" is a list of "Service ID" for all hops that are 580 HbH-EH-aware nodes on the IP path. When a router receives a HbH-EH, 581 it may generate a service ID for the flow(s) that is defined by the 582 Flow Identifying Method in "FI". Then the router must attach the 583 service ID value to the end of the Service ID List. After the packet 584 reaches the destination host, the Service ID List will be that the 585 1st router's service ID as the list header, and the last router's 586 service ID as the list tail. 588 4.5. QoS State and life of Time 590 After a router is programmed for a QoS, a QoS state is created. The 591 QoS state life is determined by the "Time" in the Setup message. 592 Whenever there is a packet processed by a QoS state, the associated 593 timer for the QoS state is reset. If the timer of a QoS state is 594 expired, the QoS state will be erased and the associated resource 595 will be released. 597 In order to keep the QoS state active, a application at source host 598 can send some zero size of data to refresh the QoS state. 600 When the Time is set to zero, it means the life of the QoS State will 601 be kept until the de-programming message is received. 603 4.6. Authentication 605 The in-band signaling is designed to have a basic security mechanism 606 to protect the integrity of a signaling message. The Authentication 607 message is to attach to a signaling message, the source host 608 calculates the harsh value of a key and all invariable part of a 609 signaling message (Setup message: ver, FI, R, SIS, P, Time; Bandwidth 610 message, Latency message, Burst message). The key is only known to 611 the hosts and all HbH-EH-aware nodes. The securely distribution of 612 the key is out the scope of the document. 614 5. Packet Forwarding 616 To achieve the required QoS, after the path setup with guaranteed 617 bandwidth there are some requirements to be met during data 618 forwarding. These include the hardware capability, the scheme for 619 the data forwarding, QoS processing, state report, etc. 621 5.1. Basic Hardware Capability 623 Section 4 explains how QoS guaranteed path can be set up and the 624 corresponding messages used, however different implementations may 625 vary in details. To achieve the satisfactory targets for performance 626 and scalability, the protocol must be cooperated with capable 627 hardware to provide the desired fine-grained QoS for different 628 transport. 630 In our experiment to implement the feature for TCP, we used a network 631 processor with traffic management feature. The traffic management 632 can provide the fine-grained QoS for any configured flow(s). 634 The following capabilities are RECOMMENDED: 636 1. The in-banding signaling is processed in network processor 637 without punting to controller CPU for help 639 2. The QoS forwarding state is kept and maintained in network 640 processor without the involvement from controller CPU. 642 3. The QoS state has a life of a pre-configured time and will be 643 automatically deleted if there is no data packet processed by 644 that QoS state. The timer can be changed on the fly. 646 4. The data forwarding does not need to be done at the controller 647 CPU, or so called slow path. It is at the same hardware as the 648 normal IP forwarding. For any IP packet, the QoS forwarding is 649 executed first. Normal forwarding will be executed if there is 650 no QoS state associated with the identification of the flow. 652 5. The QoS forwarding and normal forwarding can be switched on the 653 fly. 655 5.2. Flow Identification in Packet Forwarding 657 Flow identification in Packet Forwarding is same as the QoS channel 658 establishment by Setup message. It is to forward a packet with a 659 specified QoS process if the packet is identified to be belonging to 660 specified flow(s). 662 There are two method used in data forwarding to identify flows: 664 1. Hardware was programmed to use tuples in IP header implicitly. 665 This is indicated by that the "SIS" is zero or the Service ID is 666 not used. When a packet is received, its tuples are looked up 667 according to the value of "FI". If there is a QoS table has 668 match for the packet, the packet will be processed by the QoS 669 state found in the QoS table. This method does not need any EH 670 added into the data packet unless the data packet function to 671 collect the QoS forwarding state on the path. 673 2. Hardware was programmed to use service ID to identify flows. 674 This is indicated by that the "SIS" is not zero. When a packet 675 is received, the service ID associated with the hop is retrieved 676 and looked up for the QoS table. If it has match for the packet, 677 the packet will be processed by the QoS state entry found in the 678 QoS table. 680 5.3. QoS Forwarding State Detection and Failure Handling 682 QoS forwarding may fail due to different reasons: 684 1. Hardware failure in HbH-EH-aware node. 686 2. IP path change due to link failure, node failure or routing 687 changes; And the IP path change has impact to the HbH-EH-aware 688 node. 690 3. Network topology change; and the change leads to the changes of 691 HbH-EH-aware nodes. 693 Application may need to be aware of the service status of QoS 694 guarantee when the application is using a TCP session with QoS. In 695 order to provide such feature, the TCP stack in the source host can 696 detect the QoS forwarding state by sending TCP data packet with 697 Forwarding State message coded as HbH-EH. After the TCP data packet 698 reaches the destination host, the host will copy the forwarding state 699 into a Forwarding State Report message, and send it with another TCP 700 packet (for example, TCP-ACK) in reverse direction to the source 701 host. Thereafter, the source host can obtain the QoS forwarding 702 state on all HbH-EH-aware nodes. 704 A host can do the QoS forwarding state detection by three ways: on 705 demand, periodically or constantly. 707 After a host detects that there is QoS forwarding state failure, it 708 can repair such failure by sending another Setup message embedded 709 into a HbH-EH of any TCP packet. This repairing can handle all 710 failure case mentioned above. 712 If a failure cannot be repaired, host will be notified, and 713 appropriate action can be taken, see section 7.1 715 6. Details of Working with Transport Layer 717 The proposed new IP service is transport agnostic, which means any 718 transport layer protocol can use it. 720 6.1. Working with TCP 722 Considering TCP as the most widely used transport layer protocol, 723 this document uses TCP as an example of transport protocol to show 724 how it works with the proposed IP service. 726 The following is the list of messages for signaling and associated 727 data forwarding. 729 o Setup: This is for the setup of QoS channel through the IP path. 731 o Bandwidth: This is the required bandwidth for the QoS channel. It 732 has minimum (CIR) and maximum bandwidth (PIR). 734 o Latency: This is the required latency for the QoS channel, it is 735 the bounded latency for each hop on the path. This is not the end 736 to end latency. 738 o Burst: This is the required burst for the QoS channel, it is the 739 maximum burst size. 741 o Authentication: This is the security message for a in-band 742 signaling. 744 o OAM: This is the Operation and Management message for the QoS 745 channel. 747 o Setup State Report: This is the state report of a setup message. 749 o Forwarding State: This is the forwarding state message used for 750 data packet. 752 o Forwarding State Report: This is the forwarding state report of a 753 QoS channel. 755 There are three scenarios of QoS signaling for TCP session setup with 756 QoS 758 1. Upstream: This is for the direction of client to server. A 759 application decides to open a TCP session with upstream QoS (for 760 uploading), it will call TCP API to open a socket and connect to 761 a server. The client host will form a TCP SYN packet with the 762 HbH-EH in the IPv6 header. The EH includes Setup message and 763 Bandwidth message, and optionally Latency, Burst, Authentication 764 and OAM messages. The packet is forwarded at each hop. Each 765 HbH-EH-aware nodes will process the signaling message to finish 766 the following tasks before forwarding the packet to next hop: 768 * Retrieve the QoS parameters to program the Hardware, it 769 includes: FL, Time, Bandwidth, Latency, Burst 771 * Update the field in the EH, it includes: Hop_number, 772 Total_latency, and possibly Service ID List 774 When the server receives the TCP SYN, the Host kernel will also 775 check the HbH-EH while punting the TCP packet to the TCP stack 776 for processing. If the HbH-EH is present and the Report bit is 777 set, the Host kernel must form a new Setup State Report message, 778 all fields in the message must be copied from the Setup message 779 in the HbH-EH. When the TCP stack is sending the TCP-SYNACK to 780 the client, the kernel must add the Setup State Report message as 781 a Dst-EH in the IPv6 header. After this, the IPv6 packet is 782 complete and can be sent to wire; When the client receives the 783 TCP-SYNACK, the Host kernel will check the Dst-EH while punting 784 the TCP packet to the TCP stack for processing. If the Dst-EH is 785 present and the Setup State Report message is valid, the kernel 786 must read the Setup State Report message. Depending on the setup 787 state, the client will operate according to description in 788 section 7.1 790 2. Downstream: This is for the direction of server to client. A 791 application decides to open a TCP session with downstream QoS 792 (for downloading), it will call TCP API to open a socket and 793 connect to a server. The client host will form a TCP SYN packet 794 with the Dst-EH in the IPv6 header. The EH includes Bandwidth 795 message, and optionally Latency, Burst messages. The packet is 796 forwarded at each hop. Each hop will not process the Dst-EH. 797 When the server receives the TCP SYN, the Host kernel will check 798 the Dst-EH while punting the TCP packet to the TCP stack for 799 processing. If the Dst-EH is present, the Host kernel will 800 retrieve the QoS requirement information from Bandwidth, Latency 801 and Burst message, and check the QoS policy for the user. If the 802 user is allowed to get the service with the expected QoS, the 803 server will form a Setup message similar to the case of client to 804 server, and add it as the HbH-EH in the IPv6 header, and send the 805 TCP-SYNACK to client. Each HbH-EH-aware nodes on the path from 806 server to client will process the message similar to the case of 807 client to server. After the client receives the TCP-SYNACK, The 808 client will send the Setup State Report message to server as the 809 Dst-EH in the TCP-ACK. Finally the server receives the TC-ACK 810 and Setup State Report message, it can send the data to the 811 established session according to the pre-negotiated QoS 812 requirements. 814 3. Bi-direction: This is the case that the client wants to setup a 815 session with bi-direction QoS guarantee. The detailed operations 816 are actually a combination of Upstream and Downstream described 817 above. 819 After a QoS channel is setup, the in-band signaling message can still 820 be exchanged between two hosts, there are two scenarios for this. 822 1. Modify QoS on the fly: When the pre-set QoS parameters need to be 823 adjusted, the application at source host can re-send a new in- 824 band signaling message, the message can be embedded into any TCP 825 packet as a IPv6 HbH-EH. The QoS modification should not impact 826 the established TCP session and programmed QoS service. Thus, 827 there is no service impacted during the QoS modification. 828 Depending on the hardware performance, the signaling message can 829 be sent with TCP packet with different data size. If the 830 performance is high, the signaling message can be sent with any 831 TCP packet; otherwise, the signaling message should be sent with 832 small size TCP packet or zero-size TCP packet (such as TCP ACK). 833 Modification of QoS on the fly is a very critical feature for the 834 so called "Application adaptive QoS transport service". With 835 this service, an application (or the proxy from a service 836 provider) could setup an optimized CIR for different stage of 837 application for the economical and efficient purpose. For 838 example, in the transport of compressed video, the I-frame has 839 big size and cannot be lost, but P-frame and B-frame both have 840 smaller size and can tolerate some loss. There are much more 841 P-frame and B-frame than I-frame in videos with smooth changes 842 and variations in images [I-D.han-iccrg-arvr-transport-problem]. 843 Based on this characteristics, application can request a 844 relatively small CIR for the time of P-frame and P-frame, and 845 request a big CIR for the time of I-frame. 847 2. Repairing of the QoS channel: This is the case the QoS channel 848 was broken and need to be repaired, see section 5.3. 850 6.2. Working with UDP and other Protocols 852 There are other transport layer protocols, such as UDP, QUICK and 853 SCTP, and for these protocols similar strategy as TCP can be applied. 854 The to establish a closed-loop for the transport control. 856 For protocols with natively bi-directional control mechanism such as 857 SCTP, only some QoS control functionalities for the protocol need to 858 be added. The mechanism for TCP can be borrowed for such job. There 859 will be the QoS setup for one directional data stream, and QoS setup 860 state report for another directional data stream. The protocol may 861 also have functionalities in the stack to handle the adjustment of 862 the behaviour for different QoS setup and setup states. 864 For protocols that natively lack the feed-back control mechanism to 865 form a closed-loop such as UDP, this mechanism needs to be added into 866 the streams. There are two options to realize this: 868 1. Modify the protocol itself to have some state machine to 869 establish the closed-loop for the protocol. This can be done in 870 the kernel of the OS by modifying the protocol stack. 872 2. Modify the user data stream to introduce the closed-loop scheme, 873 this becomes as application work. It is up to application to add 874 or modify codes for the state machine of the closed-loop control. 876 7. Additional Considerations 878 This document only covers the details of setting up a path with QoS 879 using IPv6, and TCP is used as an example of transport layer protocol 880 to achieve flow level service. Only basic scenarios are covered, and 881 there are lots of open issues to be researched. The following is a 882 non-comprehensive list, and they can be addressed in separate drafts. 884 7.1. User and Application driven 886 The QoS transport service is initiated and controlled by end user's 887 application. Following tasks are done in host: 889 1. The detailed QoS parameters in signaling message are set by end 890 user application. New socket option must be added, the option is 891 a place holder for QoS parameters (Setup, Bandwidth, etc.), Setup 892 State Report and Forwarding State Report messages. 894 2. The Setup State Report and Forwarding State Report message 895 received at host are processed by transport service in kernel. 896 The Setup State Report message processed at host can result in 897 the notification to the application whether the setup is 898 successful. If the setup is successful, the application can 899 start to use the socket having the QoS support; If the setup is 900 failed, the application may have three choices: 902 * Lower the QoS requirement and re-setup a new QoS channel with 903 new in-band signaling message. 905 * Use the TCP session as traditional transport without any QoS 906 support. 908 * Lookup the service provider for help to locate the problem in 909 network. 911 7.2. Traffic Management in Host 913 In order to better accommodate this new IP in-band service, the OS on 914 a host may be changed in traffic management related areas. There are 915 two parts for traffic management to be changed: one is to manage 916 traffic going out a host's shared links, and the other is congestion 917 control for TCP flows. 919 1. For current traffic management in a host, all TCP/UDP sessions 920 will share the bandwidth for all egress links. For the purpose 921 to work with the differentiated service provided by under layer 922 network in bandwidth and latency, the kernel may allocate 923 expected resource to applications that are using the QoS 924 transport service. For example, kernel can queue different 925 packets from different applications or users to different queue 926 and schedule them in different priority. Only after this change, 927 some application can use more bandwidth and get less queuing 928 delay for a link than others. 930 2. The congestion control in a host manages the behavior of TCP 931 flow(s). This includes important features like slow start, AIMD, 932 fast retransmit, selective ACK, etc. To accommodate the benefit 933 of the QoS guaranteed transport service, the congestion control 934 can be much simpler [I-D.han-tsvwg-cc]. The new congestion 935 control is related to the implementation of QoS guarantee. 936 Following is a simple congestion control algorithm assuming that 937 the CIR is guaranteed and PIR is shared between flows: 939 * There is no slow start, the TCP can start sending traffic at 940 the rate of CIR. 942 * The AIMD is kept, but the range of the sawtooth pattern should 943 be maintained between CIR and PIR. 945 * Other congestion control features can be kept. 947 7.3. Heterogeneous Network 949 When an IP network is connected with a non-IP network, such as MPLS 950 or Ethernet network, the in-band signaling should also work in that 951 network to achieve an end-to-end connection. The behavior, protocol 952 and rules in the interworking with non-IP network is out of the scope 953 of this draft, and further research needs to be done to solve the 954 problem. 956 7.4. Proxy Control 958 It is expected that in a real service provider network, the in-band 959 signaling will be checked, filtered and managed at proxy routers. It 960 serves the following purposes: 962 1. A proxy can check if the in-band signaling from an end user meets 963 the SLA compliance. This adds extra security and DOS attack 964 prevention. 966 2. A proxy can collect the statistics for user's TCP flows and check 967 the in-band signaling for accounting and charging. 969 3. A proxy can insert and process appropriate in-band signaling for 970 TCP flows if the host does not support this new feature. This 971 can provide backward compatibility, also enable the host to use 972 the new feature. 974 8. IANA Considerations 976 This document defines a new option type for the Hop-by-Hop Options 977 header and the Destination Options header. According to [RFC8200], 978 the detailed value are: 980 +-----------+----------------+---------------------+---------------+ 981 | | Binary Value | | | 982 | Hex Value +----+---+-------+ Description | Reference | 983 | | act|chg| rest | | | 984 +-----------+----+---+-------+---------------------+---------------+ 985 | 0x0 | 00 | 0 | 10000 | In-band Signaling | Section 4 | 986 | | | | | | in this doc | 987 +-----------+----+---+-------+---------------------+---------------+ 989 Figure 3: The New Option Type 991 1. The highest-order 2 bits: 00, indicating if the processing IPv6 992 node does not recognize the Option type, skip over this option and 993 continue processing the header. 995 2. The third-highest-order bit: 0, indicating the Option Data does 996 not change en route. 998 3. The low-order 5 bits: 10000, assigned by IANA. 1000 This document also defines a 4-bit subtype field, for which IANA will 1001 create and will maintain a new sub-registry entitled "In-band 1002 signaling Subtypes" under the "Internet Protocol Version 6 (IPv6) 1003 Parameters" [IPv6_Parameters] registry. Initial values for the 1004 subtype registry are given below 1006 +-------+------------+-----------------------------+---------------+ 1007 | Type | Mnemonic | Description | Reference | 1008 +-------+------------+-----------------------------+---------------+ 1009 | 0 | SETUP | Setup object | Appendix B | 1010 +-------+------------+-----------------------------+---------------+ 1011 | 1 | BANDWIDTH | Bandwidth object | Appendix B | 1012 +-------+------------+-----------------------------+---------------+ 1013 | 2 | BURST | Burst object | Appendix B | 1014 +-------+------------+-----------------------------+---------------+ 1015 | 3 | LATENCY | Latency object | Appendix B | 1016 +-------+------------+-----------------------------+---------------+ 1017 | 4 | AUTH | Authentication object | Appendix B | 1018 +-------+------------+-----------------------------+---------------+ 1019 | 5 | OAM | OAM object | Appendix B | 1020 +-------+------------+-----------------------------+---------------+ 1021 | 6 | FWD STATE | Forward state | Appendix B | 1022 +-------+------------+-----------------------------+---------------+ 1023 | 7 |SETUP REPORT| Setup state report | Appendix B | 1024 +-------+------------+-----------------------------+---------------+ 1025 | 8 | FWD REPORT | Forwarding state report | Appendix B | 1026 +-------+------------+-----------------------------+---------------+ 1028 Figure 4: The In-band Signaling Sub Type 1030 9. Security Considerations 1032 It is important to guarantee that the resource reservation is used by 1033 authenticated users, and false signaling should not be accepted or 1034 processed. The following aspects may be considered: 1036 Authentication of user 1038 If an user is interested in using this new service, the user 1039 should sign up to a service provider. Service provider should 1040 do the proper authentication check for a new user, and 1041 establish account for the user. 1043 After the sign up, a user should provide a security key to the 1044 service provider through a secured channel (https, registered 1045 mail, etc.), or the key could be generated and given to user by 1046 the service provider. Service provider should distribute the 1047 security key of the user to different network device. More 1048 specifically, the security key should be distributed securely 1049 to all HbH-EH-aware nodes for an open network, or the proxy for 1050 a closed network. 1052 Proxy 1054 Proxy or gateway is the 1st network device connecting to 1055 customer's devices (Host, phone, etc.) that can generate the 1056 signaling for resource reservation. The functionality of the 1057 Proxy is to check if the signaling is allowed to go through 1058 SP's network. This can be done by checking the signaling 1059 integrity and other info associated with the user, such as the 1060 source/destination IP address, the account balance, the user's 1061 privilege, etc. 1063 Authentication of signaling message 1065 The signaling for resource reservation should be checked at 1066 each HbH-EH-aware nodes or a proxy node. 1068 Service ID is originally used for performance improvement of 1069 forwarding with QoS, and it can also provide additional security 1070 protection of forwarding resource in data plane. Service ID in each 1071 HbH-EH-aware node is to represent an IP flow with programmed QoS 1072 service, and it is a local significant number generated by a router 1073 to identify a flow that was offered QoS service. So, the router can 1074 periodically change the number for the same flow to protect any 1075 middle box sniffing for DOS attacking. It can be done by host 1076 periodical send out in-band signaling with the same QoS parameters 1077 and obtain the new Service ID and Service ID List for the use of next 1078 data forwarding. 1080 10. References 1082 10.1. Normative References 1084 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1085 Requirement Levels", BCP 14, RFC 2119, 1086 DOI 10.17487/RFC2119, March 1997, 1087 . 1089 [RFC2581] Allman, M., Paxson, V., and W. Stevens, "TCP Congestion 1090 Control", RFC 2581, DOI 10.17487/RFC2581, April 1999, 1091 . 1093 [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1094 (IPv6) Specification", STD 86, RFC 8200, 1095 DOI 10.17487/RFC8200, July 2017, 1096 . 1098 10.2. Informative References 1100 [HU5G] Huawei, "5G Vision: 100 Billion connections, 1 ms Latency, 1101 and 10 Gbps Throughput", 2015, 1102 . 1104 [I-D.falk-xcp-spec] 1105 Falk, A., "Specification for the Explicit Control Protocol 1106 (XCP)", draft-falk-xcp-spec-03 (work in progress), July 1107 2007. 1109 [I-D.han-iccrg-arvr-transport-problem] 1110 Han, L. and K. Smith, "Problem Statement: Transport 1111 Support for Augmented and Virtual Reality Applications", 1112 draft-han-iccrg-arvr-transport-problem-01 (work in 1113 progress), March 2017. 1115 [I-D.han-tsvwg-cc] 1116 Han, L., Qu, Y., and T. Nadeau, "A New Congestion Control 1117 in Bandwidth Guaranteed Network", draft-han-tsvwg-cc-00 1118 (work in progress), March 2018. 1120 [I-D.harper-inband-signalling-requirements] 1121 Harper, J., "Requirements for In-Band QoS Signalling", 1122 draft-harper-inband-signalling-requirements-00 (work in 1123 progress), January 2007. 1125 [I-D.ietf-aqm-codel] 1126 Nichols, K., Jacobson, V., McGregor, A., and J. Iyengar, 1127 "Controlled Delay Active Queue Management", draft-ietf- 1128 aqm-codel-06 (work in progress), December 2016. 1130 [I-D.ietf-aqm-fq-codel] 1131 Hoeiland-Joergensen, T., McKenney, P., 1132 dave.taht@gmail.com, d., Gettys, J., and E. Dumazet, "The 1133 FlowQueue-CoDel Packet Scheduler and Active Queue 1134 Management Algorithm", draft-ietf-aqm-fq-codel-06 (work in 1135 progress), March 2016. 1137 [I-D.ietf-aqm-pie] 1138 Pan, R., Natarajan, P., Baker, F., and G. White, "PIE: A 1139 Lightweight Control Scheme To Address the Bufferbloat 1140 Problem", draft-ietf-aqm-pie-10 (work in progress), 1141 September 2016. 1143 [I-D.ietf-spring-segment-routing] 1144 Filsfils, C., Previdi, S., Ginsberg, L., Decraene, B., 1145 Litkowski, S., and R. Shakir, "Segment Routing 1146 Architecture", draft-ietf-spring-segment-routing-15 (work 1147 in progress), January 2018. 1149 [I-D.ietf-tcpm-dctcp] 1150 Bensley, S., Eggert, L., Thaler, D., Balasubramanian, P., 1151 and G. Judd, "Datacenter TCP (DCTCP): TCP Congestion 1152 Control for Datacenters", draft-ietf-tcpm-dctcp-03 (work 1153 in progress), November 2016. 1155 [I-D.roberts-inband-qos-ipv6] 1156 Roberts, L. and J. Harford, "In-Band QoS Signaling for 1157 IPv6", draft-roberts-inband-qos-ipv6-00 (work in 1158 progress), July 2005. 1160 [I-D.sridharan-tcpm-ctcp] 1161 Sridharan, M., Tan, K., Bansal, D., and D. Thaler, 1162 "Compound TCP: A New TCP Congestion Control for High-Speed 1163 and Long Distance Networks", draft-sridharan-tcpm-ctcp-02 1164 (work in progress), November 2008. 1166 [IPv6_Parameters] 1167 IANA, "Internet Protocol Version 6 (IPv6) Parameters", 1168 2015, . 1171 [QU2016] Qualcomm, "Leading the world to 5G", 2016, 1172 . 1175 [Tactile] JDavid Szabo, et al. Proceedings of European Wireless 1176 2015; 21th European Wireless Conference, "Towards the 1177 Tactile Internet: Decreasing Communication Latency with 1178 Network Coding and Software Defined Networking", 2015, 1179 . 1181 [TCP-vegas] 1182 Peterson, L., "TCP Vegas: New Techniques for Congestion 1183 Detection and Avoidance - CiteSeer page on the 1994 1184 SIGCOMM paper", 1994. 1186 [TCP_Targets] 1187 Andreas Benthin, Stefan Mischke, University of Paderborn, 1188 "Bandwidth Allocation of TCP", 2004. 1190 Appendix A. Acknowledgements 1192 We appreciate the following people who made lots of contributions to 1193 this draft: Guoping Li, Boyan Tu, and Xuefei Tan, and thank Huawei 1194 Nanjing research team led by Feng Li to provide the Product on 1195 Concept (POC) development and test, the team members include Fengxin 1196 Sun, Xingwang Zhou, and Weiguang Wang. We also like to thank other 1197 people involved in the discussion of solution: Tao Ma from Future 1198 Network Strategy dept. 1200 Appendix B. Message Objects 1202 This section defines detailed objects used in different messages. 1204 B.1. Setup State Object 1206 0 1 2 3 1207 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1208 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1209 |0 0 0 0|ver| FI|R|SIS|P| Time | Hop_num |u| Total_latency | 1210 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1211 | State for each hop index | 1212 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1213 | Service ID list for hops | 1214 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-. . . + 1216 Type = 0, Setup state; 1217 Version: The version of the protocol for the QoS 1218 FI: Flow identification method, 1219 0: 5 tuples; 1: src,dst,port; 2: src,dst; 3: DSCP 1220 R: If the destination host report the received Setup state to 1221 the src address by Destination EH. 0: dont report; 1: report 1222 SIS: Service ID size; 0: 0bits, 1: 16bits, 2: 20bits, 3: 32bits 1223 P: 0: program HW for the QoS from src to dst; 1224 1: De-program HW for the QoS from src to dst 1225 Time: The life time of QoS forwarding state in second. 1226 Hop_num: The total hop number on the path set by host. It must be 1227 decremented at each hop after the processing. 1228 u: the unit of latency, 0: ms; 1: us 1229 Total_latency : Latency accumulated from each hop, each hop will 1230 add the latency in the device to this value. 1232 Figure 5: The Setup State Object 1234 Setup state for each hop index: each bit is the setup state on each 1235 hop on the path, 0: failed; 1: success. The 1st hop is at the most 1236 significant bit. 1238 Service ID list for hops: it is for all hops on the path, each 1239 service ID bit size is defined in SIS. The 1st service ID is at the 1240 top of the stack. Each hop add its service ID at the correct 1241 position indexed by the current hop number for the router. 1243 The Setup object is embedded into the hop-by-hop EH to setup the QoS 1244 in the device on the IP forwarding path. To keep the whole setup 1245 message size unchanged at each hop, the total hop number must be 1246 known at the source host. The total hop number can be detected by 1247 OAM. The service ID list is empty before the 1st hop receives the 1248 in-band signaling. Each hop then fill up the associated service ID 1249 into the correct place determined by the index of the hop. 1251 B.2. Bandwidth Object 1253 0 1 2 3 1254 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1255 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1256 |0 0 0 1| reserved | Minimum bandwidth | 1257 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1258 | Maximum bandwidth | 1259 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1261 Type = 1, 1262 Minimum bandwidth : The minimum bandwidth required, or CIR, unit Mbps 1263 Maximum bandwidth : The maximum bandwidth required, or PIR, unit Mbps 1265 Figure 6: The Bandwidth Object 1267 B.3. Burst Msg 1269 0 1 2 3 1270 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1271 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1272 |0 0 1 0| Burst size | 1273 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1275 Type = 2, 1276 Burst size : The burst size, unit M bytes 1278 Figure 7: The burst message 1280 B.4. Latency Object 1281 0 1 2 3 1282 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1283 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1284 |0 0 1 1|u| Latency | 1285 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1287 Type = 3, 1288 u: the unit of the latency 1289 0: ms; 1: us 1290 Latency: Expected maximum latency for each hop 1292 Figure 8: The Latency Object 1294 B.5. Authentication Object 1296 0 1 2 3 1297 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1298 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1299 |0 1 0 0| MAC_ALG | res | MAC data (variable length) | 1300 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+. . .+ 1302 Type = 4, 1303 MAC_ALG: Message Authentication Algorithm 1304 0: MD5; 1:SHA-0; 2: SHA-1; 3: SHA-256; 4: SHA-512 1305 MAC data: Message Authentication Data; 1306 Res: Reserved bits 1307 Size of signaling data (opt_len): Size of MAC data + 2 1308 MD5: 18; SHA-0: 22; SHA-1: 22; SHA-256: 34; SHA-512: 66 1310 Figure 9: The Authentication Object 1312 B.6. OAM Object 1314 0 1 2 3 1315 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1316 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1317 |0 1 0 1| OAM_t | OAM_len | OAM data (variable length) | 1318 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+. . .+ 1320 Type = 5, 1321 OAM_t : OAM type 1322 OAM_len : 8-bit unsigned integer. Length of the OAM data, in octets; 1323 OAM data: OAM data, details of OAM data are TBD. 1325 Figure 10: The OAM Object 1327 B.7. Forwarding State Object 1329 0 1 2 3 1330 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1331 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1332 |0 1 1 0|ver| FI|R|SIS|P| Time | Hop_num |u| Total_latency | 1333 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1334 | Forwarding state for each hop index | 1335 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1336 | Service ID list for hops | 1337 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-. . . + 1339 Type = 6, Forwarding state; 1340 All parameter definitions and process in the 1st row are same in 1341 the setup message. 1342 Forward state for each hop index : each bit is the fwd state on each 1343 hop on the path, 0: failed; 1: success; The 1st hop is at the 1344 most significant bit. 1345 Service ID list for hops: it is for all hops on the path, each index bit 1346 size is defined in SIS. The list is from the setup report message. 1348 Figure 11: The Forwarding State Object 1350 B.8. Setup State Report Object 1352 0 1 2 3 1353 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1354 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1355 |0 1 1 1|ver|H|u| Total_latency | Reserved | 1356 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1357 | State for each hop index | 1358 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1359 | Service ID list for hops | 1360 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-. . . + 1362 Type = 7, Setup state report; 1363 H: Hop number bit. When a host receives a setup message and form 1364 a setup report message, it must check if the Hop_num in setup 1365 message is zero. If it is zero, the H bit is set to one, and if 1366 it is not zero, the H bit is clear. This will notify the source 1367 of setup message that if the original Hop_num was correct. 1368 Following are directly copied from the setup message: 1369 u, Total_latency; 1370 State for each hop index 1371 Service ID list for hops. 1373 Figure 12: The Setup State Report Object 1375 B.9. Forward State Report Object 1377 0 1 2 3 1378 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1379 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1380 |1 0 0 0|ver|H|u| Total_latency | Reserved | 1381 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1382 | Forwarding state for each hop index | 1383 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1385 Type = 8, Forwarding state report; 1386 H: Hop number bit. When a host receives a Forward State message 1387 and form a Forward State Report message, it must check if the 1388 Hop_num in Forward State message is zero. If it is zero, the H bit 1389 is set to one, and if it is not zero, the H bit is clear. 1390 This will notify the source of Forward State message that if the 1391 original Hop_num was set correct. 1392 Following are directly copied from the Forward State message: 1393 u, Total_latency; 1394 Forwarding State for each hop index 1396 Figure 13: The Fwd State Report Object 1398 Authors' Addresses 1400 Lin Han 1401 Huawei Technologies 1402 2330 Central Expressway 1403 Santa Clara, CA 95050 1404 USA 1406 Phone: +10 408 330 4613 1407 Email: lin.han@huawei.com 1409 Yingzhen Qu 1410 Huawei Technologies 1411 2330 Central Expressway 1412 Santa Clara, CA 95050 1413 USA 1415 Email: yingzhen.qu@huawei.com 1416 Lijun Dong 1417 Huawei Technologies 1418 2330 Central Expressway 1419 Santa Clara, CA 95050 1420 USA 1422 Email: lijun.dong@huawei.com 1424 Thomas Nadeau 1425 Lucid Vision 1426 Hampton NH 03842 1427 USA 1429 Email: tnadeau@lucidvision.com 1431 Kevin Smith 1432 Vodafone 1433 UK 1435 Email: Kevin.Smith@vodafone.com