idnits 2.17.00 (12 Aug 2021) /tmp/idnits62723/draft-ietf-detnet-data-plane-framework-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 193 has weird spacing: '...e stack v ...' -- The document date (April 23, 2020) is 758 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: draft-ietf-detnet-flow-information-model has been published as RFC 9016 == Outdated reference: draft-ietf-detnet-ip has been published as RFC 8939 == Outdated reference: draft-ietf-detnet-mpls has been published as RFC 8964 == Outdated reference: draft-ietf-detnet-mpls-over-tsn has been published as RFC 9037 == Outdated reference: draft-ietf-detnet-mpls-over-udp-ip has been published as RFC 9025 == Outdated reference: draft-ietf-detnet-security has been published as RFC 9055 == Outdated reference: draft-ietf-pce-pcep-extension-for-pce-controller has been published as RFC 9050 == Outdated reference: draft-ietf-spring-srv6-network-programming has been published as RFC 8986 -- Obsolete informational reference (is this intentional?): RFC 5575 (Obsoleted by RFC 8955) Summary: 0 errors (**), 0 flaws (~~), 10 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet B. Varga, Ed. 3 Internet-Draft J. Farkas 4 Intended status: Informational Ericsson 5 Expires: October 25, 2020 L. Berger 6 LabN Consulting, L.L.C. 7 A. Malis 8 Malis Consulting 9 S. Bryant 10 Futurewei Technologies 11 April 23, 2020 13 DetNet Data Plane Framework 14 draft-ietf-detnet-data-plane-framework-05 16 Abstract 18 This document provides an overall framework for the DetNet data 19 plane. It covers concepts and considerations that are generally 20 common to any Deterministic Networking data plane specification. 22 Status of This Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at https://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on October 25, 2020. 39 Copyright Notice 41 Copyright (c) 2020 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (https://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 57 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 58 2.1. Terms Used in This Document . . . . . . . . . . . . . . . 4 59 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 4 60 3. DetNet Data Plane Overview . . . . . . . . . . . . . . . . . 4 61 3.1. Data Plane Characteristics . . . . . . . . . . . . . . . 6 62 3.1.1. Data Plane Technology . . . . . . . . . . . . . . . . 6 63 3.1.2. Encapsulation . . . . . . . . . . . . . . . . . . . . 6 64 3.2. DetNet-specific Metadata . . . . . . . . . . . . . . . . 7 65 3.3. DetNet IP Data Plane . . . . . . . . . . . . . . . . . . 8 66 3.4. DetNet MPLS Data Plane . . . . . . . . . . . . . . . . . 8 67 3.5. Further DetNet Data Plane Considerations . . . . . . . . 9 68 3.5.1. Per Flow Related Functions . . . . . . . . . . . . . 9 69 3.5.2. Service Protection . . . . . . . . . . . . . . . . . 11 70 3.5.3. Aggregation Considerations . . . . . . . . . . . . . 13 71 3.5.4. End-System-Specific Considerations . . . . . . . . . 14 72 3.5.5. Sub-Network Considerations . . . . . . . . . . . . . 15 73 4. Controller Plane (Management and Control) 74 Considerations . . . . . . . . . . . . . . . . . . . . . . . 15 75 4.1. DetNet Controller Plane Requirements . . . . . . . . . . 15 76 4.2. Generic Controller Plane Considerations . . . . . . . . . 17 77 4.2.1. Flow Aggregation Control . . . . . . . . . . . . . . 17 78 4.2.2. Explicit Routes . . . . . . . . . . . . . . . . . . . 18 79 4.2.3. Contention Loss and Jitter Reduction . . . . . . . . 19 80 4.2.4. Bidirectional Traffic . . . . . . . . . . . . . . . . 19 81 4.3. Packet Replication, Elimination, and Ordering (PREOF) . . 20 82 5. Security Considerations . . . . . . . . . . . . . . . . . . . 20 83 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 84 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 21 85 8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 22 86 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 22 87 9.1. Normative References . . . . . . . . . . . . . . . . . . 22 88 9.2. Informative References . . . . . . . . . . . . . . . . . 22 89 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25 91 1. Introduction 93 DetNet (Deterministic Networking) provides a capability to carry 94 specified unicast or multicast data flows for real-time applications 95 with extremely low packet loss rates and assured maximum end-to-end 96 delivery latency. A description of the general background and 97 concepts of DetNet can be found in [RFC8655]. 99 This document describes the concepts needed by any DetNet data plane 100 specification (i.e., the DetNet-specific use of packet header fields) 101 and provides considerations for any corresponding implementation. It 102 covers the building blocks that provide the DetNet service, the 103 DetNet service sub-layer and the DetNet forwarding sub-layer 104 functions as described in the DetNet Architecture. 106 The DetNet Architecture models the DetNet related data plane 107 functions decomposed into two sub-layers: a service sub-layer and a 108 forwarding sub-layer. The service sub-layer is used to provide 109 DetNet service protection and reordering. The forwarding sub-layer 110 leverages Traffic Engineering mechanisms and provides congestion 111 protection (low loss, assured latency, and limited out-of-order 112 delivery). A particular forwarding sub-layer may have capabilities 113 that are not available on other forwarding-sub layers. DetNet makes 114 use of the existing forwarding sub-layers with their respective 115 capabilities and does not require 1:1 equivalence between different 116 forwarding sub-layer capabilities. 118 As part of the service sub-layer functions, this document describes 119 typical DetNet node data plane operation. It describes the 120 functionality and operation of the Packet Replication (PRF), Packet 121 Elimination (PEF), and the Packet Ordering (POF) functions within the 122 service sub-layer. Furthermore, it describes the forwarding sub- 123 layer. 125 As defined in [RFC8655], DetNet flows may be carried over network 126 technologies that can provide the DetNet required service 127 characteristics. For example, DetNet MPLS flows can be carried over 128 IEEE 802.1 Time Sensitive Network (TSN) [IEEE802.1TSNTG] sub- 129 networks. However, IEEE 802.1 TSN support is not required in DetNet. 130 TSN frame preemption is an example of a forwarding layer capability 131 that is typically not replicated in other forwarding technologies. 132 Most of DetNet benefits can be gained by running over a data link 133 layer that has not been specifically enhanced to support all TSN 134 capabilities but for certain networks and traffic mixes delay and 135 jitter performance may vary due to the forwarding sub-layer intrinsic 136 properties. 138 Different application flows, such as Ethernet or IP, can be mapped on 139 top of DetNet. DetNet can optionally reuse header information 140 provided by, or shared with, applications. An example of shared 141 header fields can be found in [I-D.ietf-detnet-ip]. 143 This document also covers basic concepts related to the controller 144 plane and Operations, Administration, and Maintenance (OAM). Data 145 plane OAM specifics are out of scope for this document. 147 2. Terminology 149 2.1. Terms Used in This Document 151 This document uses the terminology established in the DetNet 152 architecture [RFC8655], and the reader is assumed to be familiar with 153 that document and its terminology. 155 2.2. Abbreviations 157 The following abbreviations are used in this document: 159 BGP Border Gateway Protocol. 160 d-CW DetNet Control Word. 161 DetNet Deterministic Networking. 162 DN DetNet. 163 GMPLS Generalized Multiprotocol Label Switching. 164 GRE Generic Routing Encapsulation. 165 IPSec IP Security. 166 L2 Layer 2. 167 LSP Label Switched Path. 168 MPLS Multiprotocol Label Switching. 169 OAM Operations, Administration, and Maintenance. 170 PCEP Path Computation Element Communication Protocol. 171 PEF Packet Elimination Function. 172 PRF Packet Replication Function. 173 PREOF Packet Replication, Elimination and Ordering Functions. 174 POF Packet Ordering Function. 175 PSN Packet Switched Network. 176 QoS Quality of Service. 177 S-Label DetNet "service" label. 178 TDM Time-Division Multiplexing. 179 TSN Time-Sensitive Network. 180 YANG Yet Another Next Generation. 182 3. DetNet Data Plane Overview 184 This document describes how application flows, or app-flows, are 185 carried over DetNet networks. The DetNet Architecture [RFC8655] 186 models the DetNet-related data plane functions as decomposed into two 187 sub-layers: a service sub-layer and a forwarding sub-layer. 189 Figure 1, reproduced from [RFC8655], shows a logical DetNet service 190 with the two sub-layers. 192 | packets going | ^ packets coming ^ 193 v down the stack v | up the stack | 194 +-----------------------+ +-----------------------+ 195 | Source | | Destination | 196 +-----------------------+ +-----------------------+ 197 | Service sub-layer: | | Service sub-layer: | 198 | Packet sequencing | | Duplicate elimination | 199 | Flow replication | | Flow merging | 200 | Packet encoding | | Packet decoding | 201 +-----------------------+ +-----------------------+ 202 | Forwarding sub-layer: | | Forwarding sub-layer: | 203 | Resource allocation | | Resource allocation | 204 | Explicit routes | | Explicit routes | 205 +-----------------------+ +-----------------------+ 206 | Lower layers | | Lower layers | 207 +-----------------------+ +-----------------------+ 208 v ^ 209 \_________________________/ 211 Figure 1: DetNet data plane protocol stack 213 The DetNet forwarding sub-layer may be directly provided by the 214 DetNet service sub-layer, for example by IP tunnels or MPLS. 215 Alternatively, an overlay approach may be used in which the packet is 216 natively carried between key nodes within the DetNet network (say 217 between PREOF nodes) and a sub-layer is used to provide the 218 information needed to reach the next hop in the overlay. 220 The forwarding sub-layer provides the QoS related functions needed by 221 the DetNet flow. It may do this directly through the use of queuing 222 techniques and traffic engineering methods, or it may do this through 223 the assistance of its underlying connectivity. For example it may 224 call upon Ethernet TSN capabilities defined in IEEE 802.1 TSN 225 [IEEE802.1TSNTG]. The forwarding sub-layer uses buffer resources for 226 packet queuing, as well as reservation and allocation of bandwidth 227 capacity resources. 229 The service sub-layer provides additional support beyond the 230 connectivity function of the forwarding sub-layer. An example of 231 this is Packet Replication, Elimination, and Ordering functions see 232 Section 4.3. The ordering function (POF) uses sequence numbers added 233 to packets enabling a range of packet order protection from simple 234 ordering and dropping out-of-order packets to more complex reordering 235 of a fixed number of out-of-order, minimally delayed packets. 236 Reordering requires buffer resources and has impact on the delay and 237 jitter of packets in the DetNet flow. 239 The method of instantiating each of the layers is specific to the 240 particular DetNet data plane method, and more than one approach may 241 be applicable to a given bearer network type. 243 3.1. Data Plane Characteristics 245 There are two major characteristics to the data plane: the technology 246 and the encapsulation, as discussed below. 248 3.1.1. Data Plane Technology 250 The DetNet data plane is provided by the DetNet service and 251 forwarding sub layers. The DetNet service sub-layer generally 252 provides its functions for the DetNet application flows by using or 253 applying existing standardized headers and/or encapsulations. The 254 Detnet forwarding sub-layer may provide capabilities leveraging that 255 same header or encapsulation technology (e.g., DN IP or DN MPLS) or 256 it may be achieved by other technologies (e.g., Figure 2). DetNet is 257 currently defined for operation over packet-switched (IP) networks or 258 label-switched (MPLS) networks. 260 3.1.2. Encapsulation 262 DetNet encodes specific flow attributes (flow identity and sequence 263 number) in packets. For example, in DetNet IP, zero encapsulation is 264 used and no sequence number is available, and in DetNet MPLS, DetNet- 265 specific information may be added explicitly to the packets in the 266 format of S-label and d-CW [I-D.ietf-detnet-mpls] . 268 The encapsulation of a DetNet flow allows it to be sent over a data 269 plane technology other than its native type. DetNet uses header 270 information to perform traffic classification, i.e., identify DetNet 271 flows, and provide DetNet service and forwarding functions. As 272 mentioned above, DetNet may add headers, as is the case for DN MPLS, 273 or may use headers that are already present, as is the case in DN IP. 274 Figure 2 illustrates some relationships between the components. 276 +-----+ 277 | TSN | 278 +-------+ +-+-----+-+ 279 | DN IP | | DN MPLS | 280 +--+--+----+----+ +-+---+-----+-+ 281 | TSN | DN MPLS | | TSN | DN IP | 282 +-----+---------+ +-----+-------+ 284 Figure 2: DetNet Service Examples 286 The use of encapsulation is also required if additional information 287 (metadata) is needed by the DetNet data plane and there is either no 288 ability to include it in the client data packet, or the specification 289 of the client data plane does not permit the modification of the 290 packet to include additional data. An example of such metadata is 291 the inclusion of a sequence number required by the PREOF function. 293 Encapsulation may also be used to carry or aggregate flows for 294 equipment with limited DetNet capability. 296 3.2. DetNet-specific Metadata 298 The DetNet data plane can provide or carry the following metadata: 300 1. Flow-ID 302 2. Sequence Number 304 The DetNet data plane framework supports a Flow-ID (for 305 identification of the flow or aggregate flow) and/or a Sequence 306 Number (for PREOF) for each DetNet flow. The DetNet Service sub- 307 layer requires both; the DetNet forwarding sub-layer requires only 308 Flow-ID. Metadata can also be used for OAM indications and 309 instrumentation of DetNet data plane operation. 311 Metadata inclusion can be implicit or explicit. Explicit inclusions 312 involve a dedicated header field that is used to include metadata in 313 a DetNet packet. In the implicit method, part of an already existing 314 header field is used to encode the metadata. 316 Explicit inclusion of metadata is possible through the use of IP 317 options or IP extension headers. New IP options are almost 318 impossible to get standardized or to deploy in an operational network 319 and will not be discussed further in this text. IPv6 extensions 320 headers are finding popularity in current IPv6 development work, 321 particularly in connection with Segment Routing of IPv6 (SRv6) and IP 322 OAM. The design of a new IPv6 extension header or the modification 323 of an existing one is a technique available in the tool box of the 324 DetNet IP data plane designer. 326 Explicit inclusion of metadata in an IP packet is also possible 327 through the inclusion of an MPLS label stack and the MPLS DetNet 328 Control Word using one of the methods for carrying MPLS over IP 329 [I-D.ietf-detnet-mpls-over-udp-ip]. This is described in more detail 330 in Section 3.5.5. 332 Implicit metadata in IP can be included through the use of the 333 network programming paradigm 335 [I-D.ietf-spring-srv6-network-programming] in which the suffix of an 336 IPv6 address is used to encode additional information for use by the 337 network of the receiving host. 339 Some MPLS examples of implicit metadata include the sequence number 340 for use by the PREOF function, or even all the essential information 341 being included into the DetNet over MPLS label stack (the DetNet 342 Control Word and the DetNet Service label). 344 3.3. DetNet IP Data Plane 346 An IP data plane may operate natively or through the use of an 347 encapsulation. Many types of IP encapsulation can satisfy DetNet 348 requirements and it is anticipated that more than one encapsulation 349 may be deployed, for example GRE, IPSec. 351 One method of operating an IP DetNet data plane without encapsulation 352 is to use "6-tuple" based flow identification, where "6-tuple" refers 353 to information carried in IP and higher layer protocol headers. 354 General background on the use of IP headers, and "6-tuples", to 355 identify flows and support Quality of Service (QoS) can be found in 356 [RFC3670]. [RFC7657] provides useful background on differentiated 357 services (DiffServ) and "tuple" based flow identification. DetNet 358 flow aggregation may be enabled via the use of wildcards, masks, 359 prefixes and ranges. The operation of this method is described in 360 detail in [I-D.ietf-detnet-ip]. 362 The DetNet forwarding plane may use explicit route capabilities and 363 traffic engineering capabilities to provide a forwarding sub-layer 364 that is responsible for providing resource allocation and explicit 365 routes. It is possible to include such information in a native IP 366 packet explicitly, or implicitly. 368 3.4. DetNet MPLS Data Plane 370 MPLS provides a forwarding sub-layer for traffic over implicit and 371 explicit paths to the point in the network where the next DetNet 372 service sub-layer action needs to take place. It does this through 373 the use of a stack of one or more labels with various forwarding 374 semantics. 376 MPLS also provides the ability to identify a service instance that is 377 used to process the packet through the use of a label that maps the 378 packet to a service instance. 380 In cases where metadata is needed to process an MPLS encapsulated 381 packet at the service sub-layer, the d-CW [I-D.ietf-detnet-mpls], 382 [RFC4385] can be used. Although such d-CWs are frequently 32 bits 383 long, there is no architectural constraint on the size of this 384 structure, only the requirement that it is fully understood by all 385 parties operating on it in the DetNet service sub-layer. The 386 operation of this method is described in detail in 387 [I-D.ietf-detnet-mpls]. 389 3.5. Further DetNet Data Plane Considerations 391 This section provides informative considerations related to providing 392 DetNet service to flows which are identified based on their header 393 information. 395 3.5.1. Per Flow Related Functions 397 At a high level, the following functions are provided on a per flow 398 basis. 400 3.5.1.1. Reservation and Allocation of resources 402 Resources might be reserved in order to make them available for 403 allocation to specific DetNet flows. This can eliminate packet 404 contention and packet loss for DetNet traffic. This also can reduce 405 jitter for DetNet traffic. Resources allocated to a DetNet flow 406 protect it from other traffic flows. On the other hand, DetNet flows 407 are assumed to behave with respect to the reserved traffic profile. 408 It must be possible to detect misbehaving DetNet flows and to ensure 409 that they do not compromise QoS of other flows. Queuing, policing, 410 and shaping policies can be used to ensure that the allocation of 411 resources reserved for DetNet is met. 413 3.5.1.2. Explicit routes 415 A flow can be routed over a specific, pre-computed path. This allows 416 control of the network delay by steering the packet with the ability 417 to influence the physical path. Explicit routes complement 418 reservation by ensuring that a consistent path can be associated with 419 its resources for the duration of that path. Coupled with the 420 traffic mechanism, this limits misordering and bounds latency. 421 Explicit route computation can encompass a wide set of constraints 422 and can optimize the path for a certain characteristic, e.g., highest 423 bandwidth or lowest jitter. In these cases the "best" path for any 424 set of characteristics may not be a shortest path. The selection of 425 path can take into account multiple network metrics. Some of these 426 metrics are measured and distributed by the routing system as traffic 427 engineering metrics. 429 3.5.1.3. Service protection 431 Service protection involves use of multiple packet streams using 432 multiple paths, for example 1+1 or 1:1 linear protection. For 433 DetNet, this primarily relates to packet replication and elimination 434 capabilities. MPLS offers a number of protection schemes. MPLS 435 hitless protection can be used to switch traffic to an already 436 established path in order to restore delivery rapidly after a 437 failure. Path changes, even in the case of failure recovery, can 438 lead to the out of order delivery of data requiring packet ordering 439 functions either within the DetNet service or at a high layer in the 440 application traffic. Establishment of new paths after a failure is 441 out of scope for DetNet services. 443 3.5.1.4. Network Coding 445 Network Coding [nwcrg], not to be confused with network programming, 446 comprises several techniques where multiple data flows are encoded. 447 These resulting flows can then be sent on different paths. The 448 encoding operation can combine flows and error recovery information. 449 When the encoded flows are decoded and recombined the original flows 450 can be recovered. Note that Network coding uses an alternative to 451 packet by packet PREOF. Therefore, for certain network topologies 452 and traffic loads, Network Coding can be used to improve a network's 453 throughput, efficiency, latency, and scalability, as well as 454 resilience to partition, attacks, and eavesdropping, as compared to 455 traditional methods. DetNet could use Network coding as an 456 alternative to other protection means. Network coding is often 457 applied in wireless networks and is being explored for other network 458 types. 460 3.5.1.5. Load sharing 462 Use of packet-by-packet distribution of the same DetNet flow over 463 multiple paths is not recommended except for the cases listed above 464 where PREOF is utilized to improve protection of traffic and maintain 465 order. Packet by packet load sharing, e.g., via ECMP or UCMP, 466 impacts ordering and possibly jitter. 468 3.5.1.6. Troubleshooting 470 Detnet leverages many different forwarding sub-layers, each of which 471 supports various tools to troubleshoot connectivity, for example 472 identification of misbehaving flows. The DetNet Service layer can 473 leverage existing mechanisms to troubleshoot or monitor flows, such 474 as those in use by IP and MPLS networks. At the Application layer a 475 client of a DetNet service can use existing techniques to detect and 476 monitor delay and loss. 478 3.5.1.7. Flow recognition for analytics 480 Network analytics can be inherited from the technologies of the 481 Service and Forwarding sub-layers. At the DetNet service edge, 482 packet and bit counters (e.g. sent, received, dropped, and out-of- 483 sequence) can be maintained. 485 3.5.1.8. Correlation of events with flows 487 The provider of a DetNet service may provide other capabilities to 488 monitor flows, such as more detailed loss statistics and time 489 stamping of events. The details of these capabilities are out of 490 scope for this document. 492 3.5.2. Service Protection 494 Service protection allows DetNet services to increase reliability and 495 maintain a DetNet Service Assurance in the case of network congestion 496 or network failure. Detnet relies on the underlying technology 497 capabilities for various protection schemes. Protection schemes 498 enable partial or complete coverage of the network paths and active 499 protection with combinations of PRF, PEF, and POF. 501 3.5.2.1. Linear Service Protection 503 An example DetNet MPLS network fragment and packet flow is 504 illustrated in Figure 3. 506 1 1.1 1.1 1.2.1 1.2.1 1.2.2 507 CE1----EN1--------R1-------R2-------R3--------EN2-----CE2 508 \ 1.2.1 / / 509 \1.2 /-----+ / 510 +------R4------------------------+ 511 1.2.2 513 Figure 3: Example Packet Flow in DetNet Protected Network 515 In Figure 3 the numbers are used to identify the instance of a 516 packet. Packet 1 is the original packet, and packets 1.1, and 1.2 517 are two first generation copies of packet 1. Packet 1.2.1 is a 518 second generation copy of packet 1.2, etc. Note that these numbers 519 never appear in the packet, and are not to be confused with sequence 520 numbers, labels or any other identifier that appears in the packet. 521 They simply indicate the generation number of the original packet so 522 that its passage through the network fragment can be identified to 523 the reader. 525 Customer Equipment CE1 sends a packet into the DetNet enabled 526 network. This is packet (1). Edge Node EN1 encapsulates the packet 527 as a DetNet packet and sends it to Relay node R1 (packet 1.1). EN1 528 makes a copy of the packet (1.2), encapsulates it and sends this copy 529 to Relay node R4. 531 Note that along the path from EN1 to R1 there may be zero or more 532 nodes which, for clarity, are not shown. The same is true for any 533 other path between two DetNet entities shown in Figure 3. 535 Relay node R4 has been configured to send one copy of the packet to 536 Relay Node R2 (packet 1.2.1) and one copy to Edge Node EN2 (packet 537 1.2.2). 539 R2 receives packet copy 1.2.1 before packet copy 1.1 arrives, and, 540 having been configured to perform packet elimination on this DetNet 541 flow, forwards packet 1.2.1 to Relay Node R3. Packet copy 1.1 is of 542 no further use and so is discarded by R2. 544 Edge Node EN2 receives packet copy 1.2.2 from R4 before it receives 545 packet copy 1.2.1 from R2 via relay Node R3. EN2 therefore strips 546 any DetNet encapsulation from packet copy 1.2.2 and forwards the 547 packet to CE2. When EN2 receives the later packet copy 1.2.1 this is 548 discarded. 550 The above is of course illustrative of many network scenarios that 551 can be configured. 553 This example also illustrates 1:1 protection scheme meaning there is 554 traffic over each segment of the end-to-end path. Local DetNet relay 555 nodes determine which packets are eliminated and which packets are 556 forwarded. A 1+1 scheme where only one path is used for traffic at a 557 time could use the same topology. In this case there is no PRF 558 function and traffic is switched upon detection of failure. An OAM 559 scheme that monitors the paths to detect the loss of path or traffic 560 is required to initiate the switch. A POF may still be used in this 561 case to prevent misordering of packets. In both cases the protection 562 paths are established and maintained for the duration of the DetNet 563 service. 565 3.5.2.2. Path Differential Delay 567 In the preceding example, proper working of duplicate elimination and 568 reordering of packets are dependent on the number of out-of-order 569 packets that can be buffered and the delay difference of arriving 570 packets. DetNet uses flow-specific requirements (e.g., maximum 571 number of out-of-order packets, maximum latency of the flow) for 572 configuration of POF-related buffers. If the differential delay 573 between paths is excessively large or there is excessive mis-ordering 574 of the packets, then packets may be dropped instead of being 575 reordered. Likewise, PEF uses the sequence number to identify 576 duplicate packets, and large differential delays combined with high 577 numbers of packets may exceed the ability of the PEF to work 578 properly. 580 3.5.2.3. Ring Service Protection 582 Ring protection may also be supported if the underlying technology 583 supports it. Many of the same concepts apply, however rings are 584 normally 1+1 protection for data efficiency reasons. [RFC8227] is an 585 example of MPLS-TP data plane that supports Ring protection. 587 3.5.3. Aggregation Considerations 589 The DetNet data plane also allows for the aggregation of DetNet 590 flows, which can improve scalability by reducing the per-hop state. 591 How this is accomplished is data plane or control plane dependent. 592 When DetNet flows are aggregated, transit nodes provide service to 593 the aggregate and not on a per-DetNet flow basis. When aggregating 594 DetNet flows, the flows should be compatible, i.e., the same or very 595 similar QoS and CoS characteristics. In this case, nodes performing 596 aggregation will ensure that per-flow service requirements are 597 achieved. 599 If bandwidth reservations are used, the reservation should be the sum 600 of all the individual reservations; in other words, the reservations 601 should not add up to an over-subscription of bandwidth reservation. 602 If maximum delay bounds are used, the system should ensure that the 603 aggregate does not exceed the delay bounds of the individual flows. 605 When an encapsulation is used, the choice of reserving a maximum 606 resource level and then tracking the services in the aggregated 607 service or adjusting the aggregated resources as the services are 608 added is implementation and technology specific. 610 DetNet flows at edges must be able to handle rejection to an 611 aggregation group due to lack of resources as well as conditions 612 where requirements are not satisfied. 614 3.5.3.1. IP Aggregation 616 IP aggregation has both data plane and controller plane aspects. For 617 the data plane, flows may be aggregated for treatment based on shared 618 characteristics such as 6-tuple. Alternatively, an IP encapsulation 619 may be used to tunnel an aggregate number of DetNet Flows between 620 relay nodes. 622 3.5.3.2. MPLS Aggregation 624 MPLS aggregation also has data plane and controller plane aspects. 625 MPLS flows are often tunneled in a forwarding sub-layer, under the 626 reservation associated with that MPLS tunnel. 628 3.5.4. End-System-Specific Considerations 630 Data-flows requiring DetNet service are generated and terminated on 631 end-systems. Encapsulation depends on the application and its 632 preferences. For example, in a DetNet MPLS domain the sub-layer 633 functions use the d-CWs, S-Labels and F-Labels to provide DetNet 634 services. However, an application may exchange further flow related 635 parameters (e.g., time-stamp), which are not provided by DetNet 636 functions. 638 As a general rule, DetNet domains are capable of forwarding any 639 DetNet flows and the DetNet domain does not mandate the end-system or 640 edge node encapsulation format. Unless there is a proxy of some form 641 present, end-systems peer with similar end-systems using the same 642 application encapsulation format. For example, as shown in Figure 4, 643 IP applications peer with IP applications and Ethernet applications 644 peer with Ethernet applications. 646 +-----+ 647 | X | +-----+ 648 +-----+ | X | 649 | Eth | ________ +-----+ 650 +-----+ _____ / \ | Eth | 651 \ / \__/ \___ +-----+ 652 \ / \ / 653 0======== tunnel-1 ========0_ 654 | \ 655 \ | 656 0========= tunnel-2 =========0 657 / \ __/ \ 658 +-----+ \__ DetNet MPLS domain / \ 659 | X | \ __ / +-----+ 660 +-----+ \_______/ \_____/ | X | 661 | IP | +-----+ 662 +-----+ | IP | 663 +-----+ 665 Figure 4: End-Systems and The DetNet MPLS Domain 667 3.5.5. Sub-Network Considerations 669 Any of the DetNet service types may be transported by another DetNet 670 service. MPLS nodes may be interconnected by different sub-network 671 technologies, which may include point-to-point links. Each of these 672 sub-network technologies needs to provide appropriate service to 673 DetNet flows. In some cases, e.g., on dedicated point-to-point links 674 or TDM technologies, all that is required is for a DetNet node to 675 appropriately queue its output traffic. In other cases, DetNet nodes 676 will need to map DetNet flows to the flow semantics (i.e., 677 identifiers) and mechanisms used by an underlying sub-network 678 technology. Figure 5 shows several examples of header formats that 679 can be used to carry DetNet MPLS flows over different sub-network 680 technologies. L2 represents a generic layer-2 encapsulation that 681 might be used on a point-to-point link. TSN represents the 682 encapsulation used on an IEEE 802.1 TSN network, as described in 683 [I-D.ietf-detnet-mpls-over-tsn]. UDP/IP represents the encapsulation 684 used on a DetNet IP PSN, as described in 685 [I-D.ietf-detnet-mpls-over-udp-ip]. 687 +------+ +------+ +------+ 688 App-Flow | X | | X | | X | 689 +-----+======+--+======+--+======+-----+ 690 DetNet-MPLS | d-CW | | d-CW | | d-CW | 691 +------+ +------+ +------+ 692 |Labels| |Labels| |Labels| 693 +-----+======+--+======+--+======+-----+ 694 Sub-Network | L2 | | TSN | | UDP | 695 +------+ +------+ +------+ 696 | IP | 697 +------+ 698 | L2 | 699 +------+ 701 Figure 5: Example DetNet MPLS Sub-Network Formats 703 4. Controller Plane (Management and Control) Considerations 705 4.1. DetNet Controller Plane Requirements 707 The Controller Plane corresponds to the aggregation of the Control 708 and Management Planes discussed in [RFC7426] and [RFC8655]. While 709 more details of any DetNet controller plane are out of the scope of 710 this document, there are particular considerations and requirements 711 for such that result from the unique characteristics of the DetNet 712 architecture and data plane as defined herein. 714 The primary requirements of the DetNet controller plane are that it 715 must be able to: 717 o Instantiate DetNet flows in a DetNet domain (which may include 718 some or all of explicit path determination, link bandwidth 719 reservations, restricting flows to IEEE 802.1 TSN links, node 720 buffer and other resource reservations, specification of required 721 queuing disciplines along the path, ability to manage 722 bidirectional flows, etc.) as needed for a flow. 724 o In the case of MPLS, manage DetNet S-Label and F-Label allocation 725 and distribution, where the DetNet MPLS encapsulation is in use 726 see [I-D.ietf-detnet-mpls]. 728 o Support DetNet flow aggregation. 730 o Advertise static and dynamic node and link resources such as 731 capabilities and adjacencies to other network nodes (for dynamic 732 signaling approaches) or to network controllers (for centralized 733 approaches). 735 o Scale to handle the number of DetNet flows expected in a domain 736 (which may require per-flow signaling or provisioning). 738 o Provision flow identification information at each of the nodes 739 along the path. Flow identification may differ depending on the 740 location in the network and the DetNet functionality (e.g. transit 741 node vs. relay node). 743 These requirements, as stated earlier, could be satisfied using 744 distributed control protocol signaling (such as RSVP-TE), centralized 745 network management provisioning mechanisms (such as BGP, PCEP, YANG 746 [I-D.ietf-detnet-flow-information-model], etc.) or hybrid 747 combinations of the two, and could also make use of MPLS-based 748 segment routing. 750 In the abstract, the results of either distributed signaling or 751 centralized provisioning are equivalent from a DetNet data plane 752 perspective - flows are instantiated, explicit routes are determined, 753 resources are reserved, and packets are forwarded through the domain 754 using the DetNet data plane. 756 However, from a practical and implementation standpoint, they are not 757 equivalent at all. Some approaches are more scalable than others in 758 terms of signaling load on the network. Some can take advantage of 759 global tracking of resources in the DetNet domain for better overall 760 network resource optimization. Some are more resilient than others 761 if link, node, or management equipment failures occur. While a 762 detailed analysis of the control plane alternatives is out of the 763 scope of this document, the requirements from this document can be 764 used as the basis of a later analysis of the alternatives. 766 4.2. Generic Controller Plane Considerations 768 This section covers control plane considerations that are independent 769 of the data plane technology used for DetNet service delivery. 771 While the management plane and control planes are traditionally 772 considered separately, from the data plane perspective there is no 773 practical difference based on the origin of flow provisioning 774 information, and the DetNet architecture [RFC8655] refers to these 775 collectively as the 'Controller Plane'. This document therefore does 776 not distinguish between information provided by distributed control 777 plane protocols, e.g., RSVP-TE [RFC3209] and [RFC3473], or by 778 centralized network management mechanisms, e.g., RestConf [RFC8040], 779 YANG [RFC7950], and the Path Computation Element Communication 780 Protocol (PCEP) [I-D.ietf-pce-pcep-extension-for-pce-controller] or 781 any combination thereof. Specific considerations and requirements 782 for the DetNet Controller Plane are discussed in Section 4.1. 784 Each respective data plane document also covers the control plane 785 considerations for that technology. For example, 786 [I-D.ietf-detnet-ip] covers IP control plane normative considerations 787 and [I-D.ietf-detnet-mpls] covers MPLS control plane normative 788 considerations. 790 4.2.1. Flow Aggregation Control 792 Flow aggregation means that multiple app-flows are served by a single 793 new DetNet flow. There are many techniques to achieve aggregation. 794 For example, in the case of IP, IP flows that share 6-tuple 795 attributes or flow identifiers at the DetNet sub-layer can be 796 grouped. Another example includes aggregation accomplished through 797 the use of hierarchical LSPs in MPLS and tunnels. 799 Control of aggregation involves a set of procedures listed here. 800 Aggregation may use some or all of these capabilities and the order 801 may vary: 803 o Traffic engineering resource collection and distribution: 805 Available resources are tracked through control plane or 806 management plane databases and distributed amongst controllers 807 or nodes that can manage resources. 809 o Path computation and resource allocation: 811 When DetNet services are provisioned or requested, one or more 812 paths meeting the requirements are selected and the resources 813 verified and recorded. 815 o Resource assignment and data plane co-ordination: 817 The assignment of resources along the path depends on the 818 technology and includes assignment of specific links, 819 coordination of queueing, and other traffic management 820 capabilities such as policing and shaping. 822 o Assigned Resource recording and updating: 824 Depending on the specific technology, the assigned resources 825 are updated and distributed in the databases, preventing over- 826 subscription. 828 4.2.2. Explicit Routes 830 Explicit routes are used to ensure that packets are routed through 831 the resources that have been reserved for them, and hence provide the 832 DetNet application with the required service. A requirement for the 833 DetNet Controller Plane will be the ability to assign a particular 834 identified DetNet IP flow to a path through the DetNet domain that 835 has been assigned the required nodal resources. This provides the 836 appropriate traffic treatment for the flow and also includes 837 particular links as a part of the path that are able to support the 838 DetNet flow. For example, by using IEEE 802.1 TSN links (as 839 discussed in [I-D.ietf-detnet-mpls-over-tsn] ) DetNet parameters can 840 be maintained. Further considerations and requirements for the 841 DetNet Controller Plane are discussed in Section 4.1. 843 Whether configuring, calculating and instantiating these routes is a 844 single-stage or multi-stage process, or in a centralized or 845 distributed manner, is out of scope of this document. 847 There are several approaches that could be used to provide explicit 848 routes and resource allocation in the DetNet forwarding sub-layer. 849 For example: 851 o The path could be explicitly set up by a controller which 852 calculates the path and explicitly configures each node along that 853 path with the appropriate forwarding and resource allocation 854 information. 856 o The path could use a distributed control plane such as RSVP 857 [RFC2205] or RSVP-TE [RFC3473] extended to support DetNet IP 858 flows. 860 o The path could be implemented using IPv6-based segment routing 861 when extended to support resource allocation. 863 See Section 4.1 for further discussion of these alternatives. In 864 addition, [RFC2386] contains useful background information on QoS- 865 based routing, and [RFC5575] discusses a specific mechanism used by 866 BGP for traffic flow specification and policy-based routing. 868 4.2.3. Contention Loss and Jitter Reduction 870 As discussed in Section 1, this document does not specify the 871 mechanisms needed to eliminate packet contention, packet loss or 872 reduce jitter for DetNet flows at the DetNet forwarding sub-layer. 873 The ability to manage node and link resources to be able to provide 874 these functions is a necessary part of the DetNet controller plane. 875 It is also necessary to be able to control the required queuing 876 mechanisms used to provide these functions along a flow's path 877 through the network. See [I-D.ietf-detnet-ip] and Section 4.1 for 878 further discussion of these requirements. Some forms of protection 879 may minimize packet loss or change jitter characteristics in the 880 cases where packets are reordered when out-of-order packets are 881 received at the service sub-layer. 883 4.2.4. Bidirectional Traffic 885 In many cases DetNet flows can be considered unidirectional and 886 independent. However, there are cases where the DetNet service 887 requires bidirectional traffic from a DetNet application service 888 perspective. IP and MPLS typically treat each direction separately 889 and do not force interdependence of each direction. MPLS has 890 considered bidirectional traffic requirements and the MPLS 891 definitions from [RFC5654] are useful to illustrate terms such as 892 associated bidirectional flows and co-routed bidirectional flows. 893 MPLS defines a point-to-point associated bidirectional LSP as 894 consisting of two unidirectional point-to-point LSPs, one from A to B 895 and the other from B to A, which are regarded as providing a single 896 logical bidirectional forwarding path. This is analogous to standard 897 IP routing. MPLS defines a point-to-point co-routed bidirectional 898 LSP as an associated bidirectional LSP which satisfies the additional 899 constraint that its two unidirectional component LSPs follow the same 900 path (in terms of both nodes and links) in both directions. An 901 important property of co-routed bidirectional LSPs is that their 902 unidirectional component LSPs share fate. In both types of 903 bidirectional LSPs, resource reservations may differ in each 904 direction. The concepts of associated bidirectional flows and co- 905 routed bidirectional flows can also be applied to DetNet IP flows. 907 While the DetNet IP data plane must support bidirectional DetNet 908 flows, there are no special bidirectional features with respect to 909 the data plane other than the need for the two directions of a co- 910 routed bidirectional flow to take the same path. That is to say that 911 bidirectional DetNet flows are solely represented at the management 912 and control plane levels, without specific support or knowledge 913 within the DetNet data plane. Fate sharing and associated or co- 914 routed bidirectional flows, can be managed at the control level. 916 DetNet's use of PREOF may increase the complexity of using co-routing 917 bidirectional flows, since if PREOF is used, then the replication 918 points in one direction would have to match the elimination points in 919 the other direction, and vice versa. In such cases the optimal 920 points for these functions in one direction may not match the optimal 921 points in the other, due to network and traffic constraints. 922 Furthermore, due to the per packet service protection nature, 923 bidirectional forwarding per packet may not be ensured. The first 924 packet of received member flows is selected by the elimination 925 function independently of which path it has taken through the 926 network. 928 Control and management mechanisms need to support bidirectional 929 flows, but the specification of such mechanisms are out of scope of 930 this document. Example control plane solutions for MPLS can be found 931 in [RFC3473] , [RFC6387] and [RFC7551]. These requirements are 932 included in Section 4.1. 934 4.3. Packet Replication, Elimination, and Ordering (PREOF) 936 The controller plane protocol solution required for managing the 937 PREOF processing is outside the scope of this document. That said, 938 it should be noted that the ability to determine, for a particular 939 flow, optimal packet replication and elimination points in the DetNet 940 domain requires explicit support. There may be capabilities that can 941 be used, or extended, for example GMPLS end-to-end recovery [RFC4872] 942 and GMPLS segment recovery [RFC4873]. 944 5. Security Considerations 946 Security considerations for DetNet are described in detail in 947 [I-D.ietf-detnet-security]. General security considerations for 948 DetNet architecture are described in [RFC8655]. This section 949 considers general security considerations applicable to all data 950 planes. 952 Security aspects which are unique to DetNet are those whose aim is to 953 provide the specific quality of service aspects of DetNet, which are 954 primarily to deliver data flows with extremely low packet loss rates 955 and bounded end-to-end delivery latency. 957 The primary consideration for the data plane is to maintain integrity 958 of data and delivery of the associated DetNet service traversing the 959 DetNet network. Application flows can be protected through whatever 960 means is provided by the underlying technology. For example, 961 encryption may be used, such as that provided by IPSec [RFC4301] for 962 IP flows and/or by an underlying sub-net using MACSec 963 [IEEE802.1AE-2018] for Ethernet (Layer-2) flows. 965 At the management and control level DetNet flows are identified on a 966 per-flow basis, which may provide controller plane attackers with 967 additional information about the data flows (when compared to 968 controller planes that do not include per-flow identification). This 969 is an inherent property of DetNet which has security implications 970 that should be considered when determining if DetNet is a suitable 971 technology for any given use case. 973 To provide uninterrupted availability of the DetNet service, 974 provisions can be made against DOS attacks and delay attacks. To 975 protect against DOS attacks, excess traffic due to malicious or 976 malfunctioning devices can be prevented or mitigated, for example 977 through the use of existing mechanisms such as policing and shaping 978 applied at the input of a DetNet domain. To prevent DetNet packets 979 from being delayed by an entity external to a DetNet domain, DetNet 980 technology definition can allow for the mitigation of Man-In-The- 981 Middle attacks, for example through use of authentication and 982 authorization of devices within the DetNet domain. 984 In order to prevent or mitigate DetNet attacks on other networks via 985 flow escape, edge devices can for example use existing mechanism such 986 as policing and shaping applied at the output of a DetNet domain. 988 6. IANA Considerations 990 This document makes no IANA requests. 992 7. Acknowledgements 994 The authors wish to thank Pat Thaler, Norman Finn, Loa Anderson, 995 David Black, Rodney Cummings, Ethan Grossman, Tal Mizrahi, David 996 Mozes, Craig Gunther, George Swallow, Yuanlong Jiang and Carlos J. 997 Bernardos for their various contributions to this work. 999 8. Contributors 1001 The following people contributed substantially to the content of this 1002 document: 1004 Don Fedyk 1005 Jouni Korhonen 1007 9. References 1009 9.1. Normative References 1011 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1012 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1013 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1014 . 1016 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 1017 Switching (GMPLS) Signaling Resource ReserVation Protocol- 1018 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 1019 DOI 10.17487/RFC3473, January 2003, 1020 . 1022 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 1023 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 1024 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 1025 February 2006, . 1027 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 1028 "Deterministic Networking Architecture", RFC 8655, 1029 DOI 10.17487/RFC8655, October 2019, 1030 . 1032 9.2. Informative References 1034 [I-D.ietf-detnet-flow-information-model] 1035 Farkas, J., Varga, B., Cummings, R., Jiang, Y., and D. 1036 Fedyk, "DetNet Flow Information Model", draft-ietf-detnet- 1037 flow-information-model-07 (work in progress), March 2020. 1039 [I-D.ietf-detnet-ip] 1040 Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A., 1041 and S. Bryant, "DetNet Data Plane: IP", draft-ietf-detnet- 1042 ip-05 (work in progress), February 2020. 1044 [I-D.ietf-detnet-mpls] 1045 Varga, B., Farkas, J., Berger, L., Fedyk, D., Malis, A., 1046 Bryant, S., and J. Korhonen, "DetNet Data Plane: MPLS", 1047 draft-ietf-detnet-mpls-05 (work in progress), February 1048 2020. 1050 [I-D.ietf-detnet-mpls-over-tsn] 1051 Varga, B., Farkas, J., Malis, A., and S. Bryant, "DetNet 1052 Data Plane: MPLS over IEEE 802.1 Time Sensitive Networking 1053 (TSN)", draft-ietf-detnet-mpls-over-tsn-02 (work in 1054 progress), March 2020. 1056 [I-D.ietf-detnet-mpls-over-udp-ip] 1057 Varga, B., Farkas, J., Berger, L., Malis, A., and S. 1058 Bryant, "DetNet Data Plane: MPLS over UDP/IP", draft-ietf- 1059 detnet-mpls-over-udp-ip-05 (work in progress), February 1060 2020. 1062 [I-D.ietf-detnet-security] 1063 Mizrahi, T. and E. Grossman, "Deterministic Networking 1064 (DetNet) Security Considerations", draft-ietf-detnet- 1065 security-09 (work in progress), March 2020. 1067 [I-D.ietf-pce-pcep-extension-for-pce-controller] 1068 Zhao, Q., Li, Z., Negi, M., Peng, S., and C. Zhou, "PCEP 1069 Procedures and Protocol Extensions for Using PCE as a 1070 Central Controller (PCECC) of LSPs", draft-ietf-pce-pcep- 1071 extension-for-pce-controller-04 (work in progress), March 1072 2020. 1074 [I-D.ietf-spring-srv6-network-programming] 1075 Filsfils, C., Camarillo, P., Leddy, J., Voyer, D., 1076 Matsushima, S., and Z. Li, "SRv6 Network Programming", 1077 draft-ietf-spring-srv6-network-programming-15 (work in 1078 progress), March 2020. 1080 [IEEE802.1AE-2018] 1081 IEEE Standards Association, "IEEE Std 802.1AE-2018 MAC 1082 Security (MACsec)", 2018, 1083 . 1085 [IEEE802.1TSNTG] 1086 IEEE Standards Association, "IEEE 802.1 Time-Sensitive 1087 Networking Task Group", . 1089 [nwcrg] IRTF, "Coding for efficient NetWork Communications 1090 Research Group (nwcrg)", 1091 . 1093 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1094 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1095 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1096 September 1997, . 1098 [RFC2386] Crawley, E., Nair, R., Rajagopalan, B., and H. Sandick, "A 1099 Framework for QoS-based Routing in the Internet", 1100 RFC 2386, DOI 10.17487/RFC2386, August 1998, 1101 . 1103 [RFC3670] Moore, B., Durham, D., Strassner, J., Westerinen, A., and 1104 W. Weiss, "Information Model for Describing Network Device 1105 QoS Datapath Mechanisms", RFC 3670, DOI 10.17487/RFC3670, 1106 January 2004, . 1108 [RFC4301] Kent, S. and K. Seo, "Security Architecture for the 1109 Internet Protocol", RFC 4301, DOI 10.17487/RFC4301, 1110 December 2005, . 1112 [RFC4872] Lang, J., Ed., Rekhter, Y., Ed., and D. Papadimitriou, 1113 Ed., "RSVP-TE Extensions in Support of End-to-End 1114 Generalized Multi-Protocol Label Switching (GMPLS) 1115 Recovery", RFC 4872, DOI 10.17487/RFC4872, May 2007, 1116 . 1118 [RFC4873] Berger, L., Bryskin, I., Papadimitriou, D., and A. Farrel, 1119 "GMPLS Segment Recovery", RFC 4873, DOI 10.17487/RFC4873, 1120 May 2007, . 1122 [RFC5575] Marques, P., Sheth, N., Raszuk, R., Greene, B., Mauch, J., 1123 and D. McPherson, "Dissemination of Flow Specification 1124 Rules", RFC 5575, DOI 10.17487/RFC5575, August 2009, 1125 . 1127 [RFC5654] Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed., 1128 Sprecher, N., and S. Ueno, "Requirements of an MPLS 1129 Transport Profile", RFC 5654, DOI 10.17487/RFC5654, 1130 September 2009, . 1132 [RFC6387] Takacs, A., Berger, L., Caviglia, D., Fedyk, D., and J. 1133 Meuric, "GMPLS Asymmetric Bandwidth Bidirectional Label 1134 Switched Paths (LSPs)", RFC 6387, DOI 10.17487/RFC6387, 1135 September 2011, . 1137 [RFC7426] Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S., 1138 Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software- 1139 Defined Networking (SDN): Layers and Architecture 1140 Terminology", RFC 7426, DOI 10.17487/RFC7426, January 1141 2015, . 1143 [RFC7551] Zhang, F., Ed., Jing, R., and R. Gandhi, Ed., "RSVP-TE 1144 Extensions for Associated Bidirectional Label Switched 1145 Paths (LSPs)", RFC 7551, DOI 10.17487/RFC7551, May 2015, 1146 . 1148 [RFC7657] Black, D., Ed. and P. Jones, "Differentiated Services 1149 (Diffserv) and Real-Time Communication", RFC 7657, 1150 DOI 10.17487/RFC7657, November 2015, 1151 . 1153 [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", 1154 RFC 7950, DOI 10.17487/RFC7950, August 2016, 1155 . 1157 [RFC8040] Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF 1158 Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017, 1159 . 1161 [RFC8227] Cheng, W., Wang, L., Li, H., van Helvoort, H., and J. 1162 Dong, "MPLS-TP Shared-Ring Protection (MSRP) Mechanism for 1163 Ring Topology", RFC 8227, DOI 10.17487/RFC8227, August 1164 2017, . 1166 Authors' Addresses 1168 Balazs Varga (editor) 1169 Ericsson 1170 Magyar Tudosok krt. 11. 1171 Budapest 1117 1172 Hungary 1174 Email: balazs.a.varga@ericsson.com 1176 Janos Farkas 1177 Ericsson 1178 Magyar Tudosok krt. 11. 1179 Budapest 1117 1180 Hungary 1182 Email: janos.farkas@ericsson.com 1183 Lou Berger 1184 LabN Consulting, L.L.C. 1186 Email: lberger@labn.net 1188 Andrew G. Malis 1189 Malis Consulting 1191 Email: agmalis@gmail.com 1193 Stewart Bryant 1194 Futurewei Technologies 1196 Email: stewart.bryant@gmail.com