idnits 2.17.00 (12 Aug 2021) /tmp/idnits46790/draft-ietf-detnet-mpls-09.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 12, 2020) is 678 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'Network' is mentioned on line 271, but not defined == Outdated reference: draft-ietf-detnet-data-plane-framework has been published as RFC 8938 == Outdated reference: draft-ietf-detnet-ip has been published as RFC 8939 == Outdated reference: draft-ietf-detnet-ip-over-mpls has been published as RFC 9056 == Outdated reference: draft-ietf-detnet-mpls-over-tsn has been published as RFC 9037 == Outdated reference: draft-ietf-detnet-security has been published as RFC 9055 Summary: 0 errors (**), 0 flaws (~~), 7 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 DetNet B. Varga, Ed. 3 Internet-Draft J. Farkas 4 Intended status: Standards Track Ericsson 5 Expires: January 13, 2021 L. Berger 6 LabN Consulting, L.L.C. 7 A. Malis 8 Malis Consulting 9 S. Bryant 10 Futurewei Technologies 11 J. Korhonen 12 July 12, 2020 14 DetNet Data Plane: MPLS 15 draft-ietf-detnet-mpls-09 17 Abstract 19 This document specifies the Deterministic Networking data plane when 20 operating over an MPLS Packet Switched Networks. 22 Status of This Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at https://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on January 13, 2021. 39 Copyright Notice 41 Copyright (c) 2020 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (https://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 57 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 2.1. Terms Used in This Document . . . . . . . . . . . . . . . 3 59 2.2. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 4 60 2.3. Requirements Language . . . . . . . . . . . . . . . . . . 5 61 3. DetNet MPLS Data Plane Overview . . . . . . . . . . . . . . . 5 62 3.1. Layers of DetNet Data Plane . . . . . . . . . . . . . . . 5 63 3.2. DetNet MPLS Data Plane Scenarios . . . . . . . . . . . . 6 64 4. MPLS-Based DetNet Data Plane Solution . . . . . . . . . . . . 8 65 4.1. DetNet Over MPLS Encapsulation Components . . . . . . . . 8 66 4.2. MPLS Data Plane Encapsulation . . . . . . . . . . . . . . 9 67 4.2.1. DetNet Control Word and the DetNet Sequence Number . 10 68 4.2.2. S-Labels . . . . . . . . . . . . . . . . . . . . . . 11 69 4.2.3. F-Labels . . . . . . . . . . . . . . . . . . . . . . 14 70 4.3. OAM Indication . . . . . . . . . . . . . . . . . . . . . 17 71 4.4. Flow Aggregation . . . . . . . . . . . . . . . . . . . . 17 72 4.4.1. Aggregation Via LSP Hierarchy . . . . . . . . . . . . 17 73 4.4.2. Aggregating DetNet Flows as a new DetNet flow . . . . 18 74 4.5. Service Sub-Layer Considerations . . . . . . . . . . . . 19 75 4.5.1. Edge Node Processing . . . . . . . . . . . . . . . . 19 76 4.5.2. Relay Node Processing . . . . . . . . . . . . . . . . 19 77 4.6. Forwarding Sub-Layer Considerations . . . . . . . . . . . 20 78 4.6.1. Class of Service . . . . . . . . . . . . . . . . . . 20 79 4.6.2. Quality of Service . . . . . . . . . . . . . . . . . 21 80 5. Management and Control Information Summary . . . . . . . . . 21 81 5.1. Service Sub-Layer Information Summary . . . . . . . . . . 22 82 5.1.1. Service Aggregation Information Summary . . . . . . . 23 83 5.2. Forwarding Sub-Layer Information Summary . . . . . . . . 23 84 6. Security Considerations . . . . . . . . . . . . . . . . . . . 24 85 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25 86 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 25 87 9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 25 88 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 25 89 10.1. Normative References . . . . . . . . . . . . . . . . . . 25 90 10.2. Informative References . . . . . . . . . . . . . . . . . 27 91 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 30 93 1. Introduction 95 Deterministic Networking (DetNet) is a service that can be offered by 96 a network to DetNet flows. DetNet provides these flows extremely low 97 packet loss rates and assured maximum end-to-end delivery latency. 98 General background and concepts of DetNet can be found in [RFC8655]. 100 The DetNet Architecture models the DetNet related data plane 101 functions decomposed into two sub-layers: a service sub-layer and a 102 forwarding sub-layer. The service sub-layer is used to provide 103 DetNet service functions such as protection and reordering. The 104 forwarding sub-layer is used to provide forwarding assurance (low 105 loss, assured latency, and limited reordering). 107 This document specifies the DetNet data plane operation and the on- 108 wire encapsulation of DetNet flows over an MPLS-based Packet Switched 109 Network (PSN) using the service sub-layer reference model. MPLS 110 encapsulation already provides a solid foundation of building blocks 111 to enable the DetNet service and forwarding sub-layer functions. 112 MPLS encapsulated DetNet can be carried over a variety of different 113 network technologies that can provide the DetNet required level of 114 service. However, the specific details of how DetNet MPLS is carried 115 over different network technologies is out of scope of this document. 117 MPLS encapsulated DetNet flows can carry different types of traffic. 118 The details of the types of traffic that are carried in DetNet are 119 also out of scope of this document. An example of IP using DetNet 120 MPLS sub-networks can be found in [I-D.ietf-detnet-ip]. DetNet MPLS 121 may use an associated controller and Operations, Administration, and 122 Maintenance (OAM) functions that are defined outside of this 123 document. 125 Background information common to all data planes for DetNet can be 126 found in the DetNet Data Plane Framework 127 [I-D.ietf-detnet-data-plane-framework]. 129 2. Terminology 131 2.1. Terms Used in This Document 133 This document uses the terminology established in the DetNet 134 architecture [RFC8655] and the DetNet Data Plane Framework 135 [I-D.ietf-detnet-data-plane-framework]. The reader is assumed to be 136 familiar with these documents, any terminology defined therein and 137 basic MPLS related terminologies in [RFC3031]. 139 The following terminology is introduced in this document: 141 F-Label A Detnet "forwarding" label that identifies the LSP 142 used to forward a DetNet flow across an MPLS PSN, e.g., 143 a hop-by-hop label used between label switching routers 144 (LSR). 146 S-Label A DetNet "service" label that is used between DetNet 147 nodes that implement also the DetNet service sub-layer 148 functions. An S-Label is also used to identify a 149 DetNet flow at DetNet service sub-layer at a receiving 150 DetNet node. 152 A-Label A special case of an S-Label, whose aggregation 153 properties are known only at the aggregation and 154 deaggregation end-points. 156 d-CW A DetNet Control Word (d-CW) is used for sequencing 157 information of a DetNet flow at the DetNet service sub- 158 layer. 160 2.2. Abbreviations 162 The following abbreviations are used in this document: 164 CoS Class of Service. 166 CW Control Word. 168 DetNet Deterministic Networking. 170 LSR Label Switching Router. 172 MPLS Multiprotocol Label Switching. 174 MPLS-TE Multiprotocol Label Switching - Traffic Engineering. 176 MPLS-TP Multiprotocol Label Switching - Transport Profile. 178 OAM Operations, Administration, and Maintenance. 180 PE Provider Edge. 182 PEF Packet Elimination Function. 184 PRF Packet Replication Function. 186 PREOF Packet Replication, Elimination and Ordering Functions. 188 POF Packet Ordering Function. 190 PSN Packet Switched Network. 192 PW PseudoWire. 194 QoS Quality of Service. 196 S-PE Switching Provider Edge. 198 T-PE Terminating Provider Edge. 200 TSN Time-Sensitive Network. 202 2.3. Requirements Language 204 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 205 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 206 "OPTIONAL" in this document are to be interpreted as described in BCP 207 14 [RFC2119] [RFC8174] when, and only when, they appear in all 208 capitals, as shown here. 210 3. DetNet MPLS Data Plane Overview 212 3.1. Layers of DetNet Data Plane 214 MPLS provides a wide range of capabilities that can be utilised by 215 DetNet. A straight forward approach utilizing MPLS for a DetNet 216 service sub-layer is based on the existing pseudowire (PW) 217 encapsulations and by utilizing existing MPLS Traffic Engineering 218 encapsulations and mechanisms. Background on PWs can be found in 219 [RFC3985] and [RFC3031]. Background on MPLS Traffic Engineering can 220 be found in [RFC3272] and [RFC3209]. 222 DetNet MPLS 223 . 224 Bottom of Stack . 225 (inner label) +------------+ 226 | Service | d-CW, S-Label (A-Label) 227 +------------+ 228 | Forwarding | F-Label(s) 229 +------------+ 230 Top of Stack . 231 (outer label) . 233 Figure 1: DetNet Adaptation to MPLS Data Plane 235 The DetNet MPLS data plane representation is illustrated in Figure 1. 236 The service sub-layer includes a DetNet control word (d-CW) and an 237 identifying service label (S-Label). The DetNet control word (d-CW) 238 conforms to the Generic PW MPLS Control Word (PWMCW) defined in 239 [RFC4385]. An aggregation label (A-Label) is a special case of 240 S-Label used for aggregation. 242 A node operating on a received DetNet flow at the Detnet service sub- 243 layer uses the local context associated with a received S-Label, 244 i.e., a received F-Label, to determine which local DetNet 245 operation(s) are applied to that packet. An S-Label may be taken 246 from the platform label space [RFC3031], making it unique, enabling 247 DetNet flow identification regardless of which input interface or LSP 248 the packet arrives on. It is important to note that S-Label values 249 are driven by the receiver, not the sender. 251 The DetNet forwarding sub-layer is supported by zero or more 252 forwarding labels (F-Labels). MPLS Traffic Engineering 253 encapsulations and mechanisms can be utilized to provide a forwarding 254 sub-layer that is responsible for providing resource allocation and 255 explicit routes. 257 3.2. DetNet MPLS Data Plane Scenarios 259 DetNet MPLS Relay Transit Relay DetNet MPLS 260 End System Node Node Node End System 261 (T-PE) (S-PE) (LSR) (S-PE) (T-PE) 262 +----------+ +----------+ 263 | Appl. |<------------ End to End Service ----------->| Appl. | 264 +----------+ +---------+ +---------+ +----------+ 265 | Service |<--| Service |-- DetNet flow --| Service |-->| Service | 266 +----------+ +---------+ +----------+ +---------+ +----------+ 267 |Forwarding| |Fwd| |Fwd| |Forwarding| |Fwd| |Fwd| |Forwarding| 268 +-------.--+ +-.-+ +-.-+ +----.---.-+ +-.-+ +-.-+ +---.------+ 269 : Link : / ,-----. \ : Link : / ,-----. \ 270 +........+ +-[ Sub ]-+ +......+ +-[ Sub ]-+ 271 [Network] [Network] 272 `-----' `-----' 273 |<- LSP -->| |<-------- LSP -----------| |<--- LSP -->| 275 |<----------------- DetNet MPLS --------------------->| 277 Figure 2: A DetNet MPLS Network 279 Figure 2 illustrates a hypothetical DetNet MPLS-only network composed 280 of DetNet aware MPLS enabled end systems, operating over a DetNet 281 aware MPLS network. In this figure, the relay nodes are PE devices 282 that define the MPLS LSP boundaries and the transit nodes are LSRs. 284 DetNet end systems and relay nodes understand the particular needs of 285 DetNet flows and provide both DetNet service and forwarding sub-layer 286 functions. In the case of MPLS, DetNet service-aware nodes add, 287 remove and process d-CWs, S-Labels and F-labels as needed. DetNet 288 MPLS nodes provide functionality analogous to T-PEs when they sit at 289 the edge of an MPLS domain, and S-PEs when they are in the middle of 290 an MPLS domain, see [RFC6073]. 292 In a DetNet MPLS network, transit nodes may be DetNet service aware 293 or may be DetNet unaware MPLS Label Switching Routers (LSRs). In 294 this latter case, such LSRs would be unaware of the special 295 requirements of the DetNet service sub-layer, but would still provide 296 traffic engineering functions and the QoS capabilities needed to 297 ensure that the (TE) LSPs meet the service requirements of the 298 carried DetNet flows. 300 The application of DetNet using MPLS supports a number of control 301 plane/management plane types. These types support certain MPLS data 302 plane deployments. For example RSVP-TE, MPLS-TP, or MPLS Segment 303 Routing (when extended to support resource allocation) are all valid 304 MPLS deployment cases. 306 Figure 3 illustrates how an end-to-end MPLS-based DetNet service is 307 provided in a more detail. In this figure, the customer end systems, 308 CE1 and CE2, are able to send and receive MPLS encapsulated DetNet 309 flows, and R1, R2 and R3 are relay nodes in the middle of a DetNet 310 network. The 'X' in the end systems, and relay nodes represents 311 potential DetNet compound flow packet replication and elimination 312 points. In this example, service protection is supported utilizing 313 at least two DetNet member flows and TE LSPs. For a unidirectional 314 flow, R1 supports PRF and R3 supports PEF and POF. Note that the 315 relay nodes may change the underlying forwarding sub-layer, for 316 example tunneling MPLS over IEEE 802.1 TSN 317 [I-D.ietf-detnet-mpls-over-tsn], or simply over interconnect network 318 links. 320 DetNet DetNet 321 MPLS Service Transit Transit Service MPLS 322 DetNet | |<-Tnl->| |<-Tnl->| | DetNet 323 End | V 1 V V 2 V | End 324 System | +--------+ +--------+ +--------+ | System 325 +---+ | | R1 |=======| R2 |=======| R3 | | +---+ 326 | X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X | 327 |CE1|========| \ | | X | | / |======|CE2| 328 | | | | \_.|..DF2..|._/ \__.|..DF4..|._/ | | | | 329 +---+ | |=======| |=======| | +---+ 330 ^ +--------+ +--------+ +--------+ ^ 331 | Relay Node Relay Node Relay Node | 332 | (S-PE) (S-PE) (S-PE) | 333 | | 334 |<---------------------- DetNet MPLS --------------------->| 335 | | 336 |<--------------- End to End DetNet Service -------------->| 338 -------------------------- Data Flow -------------------------> 340 X = Optional service protection (none, PRF, PREOF, PEF/POF) 341 DFx = DetNet member flow x over a TE LSP 343 Figure 3: MPLS-Based Native DetNet 345 4. MPLS-Based DetNet Data Plane Solution 347 4.1. DetNet Over MPLS Encapsulation Components 349 To carry DetNet over MPLS the following is required: 351 1. A method of identifying the MPLS payload type. 353 2. A method of identifying the DetNet flow group to the processing 354 element. 356 3. A method of distinguishing DetNet OAM packets from DetNet data 357 packets. 359 4. A method of carrying the DetNet sequence number. 361 5. A suitable LSP to deliver the packet to the egress PE. 363 6. A method of carrying queuing and forwarding indication. 365 In this design an MPLS service label (the S-Label), is similar to a 366 pseudowire (PW) label [RFC3985], and is used to identify both the 367 DetNet flow identity and the payload MPLS payload type satisfying (1) 368 and (2) in the list above. OAM traffic discrimination happens 369 through the use of the Associated Channel method described in 370 [RFC4385]. The DetNet sequence number is carried in the DetNet 371 Control word which carries the Data/OAM discriminator. To simplify 372 implementation and to maximize interoperability two sequence number 373 sizes are supported: a 16 bit sequence number and a 28 bit sequence 374 number. The 16 bit sequence number is needed to support some types 375 of legacy clients. The 28 bit sequence number is used in situations 376 where it is necessary ensure that in high speed networks the sequence 377 number space does not wrap whilst packets are in flight. 379 The LSP used to forward the DetNet packet is not restricted regarding 380 any method used for establishing that LSP (for example, MPLS-LDP, 381 MPLS-TE, MPLS-TP [RFC5921], MPLS-SR [RFC8660], etc.). The LSP 382 (F-Label) label(s) and/or the S-Label may be used to indicate the 383 required queue processing as well as the forwarding parameters. Note 384 that the possible use of Penultimate Hop Popping (PHP) means that the 385 S-Label may be the only label received at the terminating DetNet 386 service. 388 4.2. MPLS Data Plane Encapsulation 390 Figure 4 illustrates a DetNet data plane MPLS encapsulation. The 391 MPLS-based encapsulation of the DetNet flows is well suited for the 392 scenarios described in [I-D.ietf-detnet-data-plane-framework]. 393 Furthermore, an end to end DetNet service i.e., native DetNet 394 deployment (see Section 3.2) is also possible if DetNet end systems 395 are capable of initiating and termination MPLS encapsulated packets. 397 The MPLS-based DetNet data plane encapsulation consists of: 399 o DetNet control word (d-CW) containing sequencing information for 400 packet replication and duplicate elimination purposes, and the OAM 401 indicator. 403 o DetNet service Label (S-Label) that identifies a DetNet flow at 404 the receiving DetNet service sub-layer processing node. 406 o Zero or more Detnet MPLS Forwarding label(s) (F-Label) used to 407 direct the packet along the label switched path (LSP) to the next 408 DetNet service sub-layer processing node along the path. When 409 Penultimate Hop Popping is in use there may be no label F-Label in 410 the protocol stack on the final hop. 412 o The necessary data-link encapsulation is then applied prior to 413 transmission over the physical media. 415 DetNet MPLS-based encapsulation 417 +---------------------------------+ 418 | | 419 | DetNet App-Flow | 420 | Payload Packet | 421 | | 422 +---------------------------------+ <--\ 423 | DetNet Control Word | | 424 +---------------------------------+ +--> DetNet data plane 425 | S-Label | | MPLS encapsulation 426 +---------------------------------+ | 427 | [ F-Label(s) ] | | 428 +---------------------------------+ <--/ 429 | Data-Link | 430 +---------------------------------+ 431 | Physical | 432 +---------------------------------+ 434 Figure 4: Encapsulation of a DetNet App-Flow in an MPLS PSN 436 4.2.1. DetNet Control Word and the DetNet Sequence Number 438 A DetNet control word (d-CW) conforms to the Generic PW MPLS Control 439 Word (PWMCW) defined in [RFC4385]. The d-CW formatted as shown in 440 Figure 5 MUST be present in all DetNet packets containing app-flow 441 data. This format of the d-CW was created in order (1) to allow 442 larger S/N space to avoid S/N rollover frequency in some applications 443 and (2) to allow non-skip zero S/N what simplifies implementation. 445 0 1 2 3 446 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 447 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 448 |0 0 0 0| Sequence Number | 449 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 451 Figure 5: DetNet Control Word 453 (bits 0 to 3) 455 Per [RFC4385], MUST be set to zero (0). 457 Sequence Number (bits 4 to 31) 459 An unsigned value implementing the DetNet sequence number. The 460 sequence number space is a circular one. 462 A separate sequence number space MUST be maintained by the node that 463 adds the d-CW for each DetNet app-flow, i.e., DetNet service. The 464 following sequence number field lengths MUST be supported: 466 0 bits 468 16 bits 470 28 bits 472 The sequence number length MUST be provisioned on a per Detnet 473 service basis via configuration, i.e., via the controller plane 474 described in [I-D.ietf-detnet-data-plane-framework]. 476 A 0 bit sequence number field length indicates that there is no 477 DetNet sequence number used for the flow. When the length is zero, 478 the sequence number field MUST be set to zero (0) on all packets sent 479 for the flow. 481 When the sequence number field length is 16 or 28 bits for a flow, 482 the sequence number MUST be incremented by one for each new app-flow 483 packet sent. When the field length is 16 bits, d-CW bits 4 to 15 484 MUST be set to zero (0). The values carried in this field can wrap 485 and it is important to note that zero (0) is a valid field value. 486 For example, were the sequence number size is 16 bits, the sequence 487 will contain: 65535, 0, 1, where zero (0) is an ordinary sequence 488 number. 490 It is important to note that this document differs from [RFC4448] 491 where a sequence number of zero (0) is used to indicate that the 492 sequence number check algorithm is not used. 494 The sequence number is optionally used during receive processing as 495 described below in Section 4.2.2.2 and Section 4.2.2.3. 497 4.2.2. S-Labels 499 A DetNet flow at the DetNet service sub-layer is identified by an 500 S-Label. MPLS-aware DetNet end systems and edge nodes, which are by 501 definition MPLS ingress and egress nodes, MUST add (push) and remove 502 (pop) a DetNet service-specific d-CW and S-Label. Relay nodes MAY 503 swap S-Label values when processing a DetNet flow, i.e., incoming and 504 outgoing S-Labels of a DetNet flow can be different. 506 S-Label values MUST be provisioned per DetNet service via 507 configuration, e.g., via the controller plane described in 508 [I-D.ietf-detnet-data-plane-framework]. Note that S-Labels provide 509 identification at the downstream DetNet service sub-layer receiver, 510 not the sender. As such, S-Labels MUST be allocated by the entity 511 that controls the service sub-layer receiving node's label space, and 512 MAY be allocated from the platform label space [RFC3031]. Because 513 S-Labels are local to each node rather than being a global identifier 514 within a domain, they must be advertised to their upstream DetNet 515 service-aware peer nodes (e.g., a DetNet MPLS End System or a DetNet 516 Relay or Edge Node) and interpreted in the context of their received 517 F-Label(s). In some PREOF topologies, the node performing 518 replication will be sending to multiple nodes performing PEF or POF, 519 and may need to send different S-Label values for the different 520 member flows for the same DetNet service. 522 An S-Label will normally be at the bottom of the label stack once the 523 last F-Label is removed, immediately preceding the d-CW. To support 524 service sub-layer level OAM, an OAM Associated Channel Header (ACH) 525 [RFC4385] together with a Generic Associated Channel Label (GAL) 526 [RFC5586] MAY be used in place of a d-CW. 528 Similarly, an Entropy Label Indicator/Entropy Label (ELI/EL) 529 [RFC6790] MAY be carried below the S-Label in the label stack in 530 networks where DetNet flows would otherwise received ECMP treatment. 531 When ELs are used, the same EL value SHOULD be used for all of the 532 packets sent using a specific S-Label to force the flow to follow the 533 same path. However, as outlines in 534 [I-D.ietf-detnet-data-plane-framework] the use of ECMP for DetNet 535 flows is NOT RECOMMENDED. ECMP MAY be used for non-DetNet flows 536 within a DetNet domain. 538 When receiving a DetNet MPLS packet, an implementation MUST identify 539 the DetNet service associated with the incoming packet based on the 540 S-Label. When a node is using platform labels for S-Labels, no 541 additional information is needed as the S-label uniquely identifies 542 the DetNet service. In the case where platform labels are not used, 543 zero or more F-Labels and optionally, the incoming interface, 544 proceeding the S-Label MUST be used together with the S-Label to 545 uniquely identify the DetNet service associated with a received 546 packet. The incoming interface MAY also be used together with any 547 present F-Label(s) and the S-Label to uniquely identify an incoming 548 DetNet service, for example, in the case where PHP is used. Note 549 that choice to use platform label space for S-Label or S-Label plus 550 one or more F-Labels to identify DetNet services is a local 551 implementation choice, with one caveat. When one or more F-labels, 552 or incoming interface, is needed together with an S-Label to uniquely 553 identify a service, the controller plane must ensure that incoming 554 DetNet MPLS packets arrive with the needed information (F-label(s) 555 and/or incoming interface) and provision the needed information. The 556 provisioned information MUST then be used to identify incoming DetNet 557 service based on the combination of S-Label and F-Label(s) or 558 incoming interface. 560 The use of platform labels for S-Labels matches other pseudowire 561 encapsulations for consistency but there is no hard requirement in 562 this regard. 564 4.2.2.1. Packet Replication Function Processing 566 The Packet Replication Function (PRF) function MAY be supported by an 567 implementation for outgoing DetNet flows. The use of the PRF for a 568 particular DetNet service MUST be provisioned via configuration, 569 e.g., via the controller plane described in 570 [I-D.ietf-detnet-data-plane-framework]. When replication is 571 configure, the same app-flow data will be sent over multiple outgoing 572 DetNet member flows using forwarding sub-layer LSPs. An S-Label 573 value MUST be configured per outgoing member flow. The same d-CW 574 field value MUST be used on all outgoing member flows for each 575 replicated MPLS packet. 577 4.2.2.2. Packet Elimination Function Processing 579 Implementations MAY support the Packet Elimination Function (PEF) for 580 received DetNet MPLS flows. When supported, use of the PEF for a 581 particular DetNet service MUST be provisioned via configuration, 582 e.g., via the controller plane described in 583 [I-D.ietf-detnet-data-plane-framework]. 585 After a DetNet service is identified for a received DetNet MPLS 586 packet, as described above, an implementation MUST check if PEF is 587 configured for that DetNet service. When configured, the 588 implementation MUST track the sequence number contained in received 589 d-CWs and MUST ensure that duplicate (replicated) instances of a 590 particular sequence number are discarded. The specific mechanisms 591 used for an implementation to identify which received packets are 592 duplicates and which are new is an implementation choice. Note that 593 per Section 4.2.1 the sequence number field length may be 16 or 28 594 bits, and the field value can wrap. PEF MUST NOT be used with DetNet 595 flows configured with a d-CW sequence number field length of 0 bits. 597 Note that an implementation MAY wish to constrain the maximum number 598 sequence numbers that are tracked, on platform-wide or per flow 599 basis. Some implementations MAY support the provisioning of the 600 maximum number sequence numbers that are tracked number on either a 601 platform-wide or per flow basis. 603 4.2.2.3. Packet Ordering Function Processing 605 A function that is related to in-order delivery is the Packet 606 Ordering Function (POF). Implementations MAY support POF. When 607 supported, use of the POF for a particular DetNet service MUST be 608 provisioned via configuration, e.g., via the controller plane 609 described by [I-D.ietf-detnet-data-plane-framework]. Implementations 610 MAY required that PEF and POF be used in combination. There is no 611 requirement related to the order of execution of the Packet 612 Elimination and Ordering Functions in an implementation. 614 After a DetNet service is identified for a received DetNet MPLS 615 packet, as described above, an implementation MUST check if POF is 616 configured for that DetNet service. When configured, the 617 implementation MUST track the sequence number contained in received 618 d-CWs and MUST ensure that packets are processed in the order 619 indicated in the received d-CW sequence number field, which may not 620 be in the order the packets are received. As defined in 621 Section 4.2.1 the sequence number field length may be 16 or 28 bits, 622 is incremented by one (1) for each new MPLS packet sent for a 623 particular DetNet service, and the field value can wrap. The 624 specific mechanisms used for an implementation to identify the order 625 of received packets is an implementation choice. 627 Note that an implementation MAY wish to constrain the maximum number 628 of out of order packets that can be processed, on platform-wide or 629 per flow basis. Some implementations MAY support the provisioning of 630 this number on either a platform-wide or per flow basis. The number 631 of out of order packets that can be processed also impacts the 632 latency of a flow. 634 4.2.3. F-Labels 636 F-Labels are supported the DetNet forwarding sub-layer. F-Labels are 637 used to provide LSP-based connectivity between DetNet service sub- 638 layer processing nodes. 640 4.2.3.1. Service Sub-Layer Related Processing 642 DetNet MPLS end systems, edge nodes and relay nodes may operate at 643 the DetNet service sub-layer with understanding of DetNet services 644 and their requirements. As mentioned earlier, when operating at this 645 layer such nodes can push, pop or swap (pop then push) S-Labels. In 646 all cases, the F-Label(s) used for a DetNet service are always 647 replaced and the following procedures apply. 649 When sending a DetNet flow, zero or more F-Labels MAY be pushed on 650 top of an S-Label by the node pushing an S-Label. The F-Label(s) to 651 be pushed when sending a particular DetNet service MUST be 652 provisioned per outgoing S-Label via configuration, e.g., via the 653 controller plane discussed in [I-D.ietf-detnet-data-plane-framework]. 654 F-Label(s) can also provide context for an S-Label. To allow for the 655 omission of F-Label(s), an implementation SHOULD also allow an 656 outgoing interface to be configured per S-Label. 658 Note, when PRF is supported, the same app-flow data will be sent over 659 multiple outgoing DetNet member flows using forwarding sub-layer 660 LSPs. This means that implementation may be sending different sets 661 of F-Labels per DetNet member flow, each with a different S-Label. 663 When a single set of F-Labels is provisioned for a particular 664 outgoing S-Label, that set of F-labels MUST be pushed after the 665 S-Label is pushed. The outgoing packet is then forwarded as 666 described below in Section 4.2.3.2. When a single outgoing interface 667 is provisioned, the outgoing packet is then forwarded as described 668 below in Section 4.2.3.2. 670 When multiple sets of outgoing F-Labels or interfaces are provisioned 671 for a particular DetNet service, a copy of the outgoing packet, 672 including the pushed member flow-specific S-Label, MUST be made per 673 F-label set and outgoing interface. Each set of provisioned F-Labels 674 are then pushed onto a copy of the packet. Each copy is then 675 forwarded as described below in Section 4.2.3.2. 677 As described in the previous section, when receiving a DetNet MPLS 678 flow, an implementation identifies the DetNet service associated with 679 the incoming packet based on the S-Label. When a node is using 680 platform labels for S-Labels, any F-Labels can be popped and the 681 S-label uniquely identifies the DetNet service. In the case where 682 platform labels are not used, incoming F-Label(s) and, optionally, 683 the incoming interface MUST also be provisioned for a DetNet service. 685 4.2.3.2. Common F-Label Processing 687 All DetNet aware MPLS nodes process F-Labels as needed to meet the 688 service requirements of the DetNet flow or flows carried in the LSPs 689 represented by the F-Labels. This includes normal push, pop and swap 690 operations. Such processing is essentially the same type of 691 processing provided for TE LSPs, although the specific service 692 parameters, or traffic specification, can differ. When the DetNet 693 service parameters of the DetNet flow or flows carried in an LSP 694 represented by an F-Label can be met by an existing TE mechanism, the 695 forwarding sub-layer processing node MAY be a DetNet unaware, i.e., 696 standard, MPLS LSR. Such TE LSPs may provide LSP forwarding service 697 as defined in, but not limited to, [RFC3209], [RFC3270], [RFC3272], 698 [RFC3473], [RFC4875], [RFC5440], and [RFC8306]. 700 More specifically, as mentioned above, the DetNet forwarding sub- 701 layer provides explicit routes and allocated resources, and F-Labels 702 are used to map to each. Explicit routes are supported based on the 703 topmost (outermost) F-Label that is pushed or swapped and the LSP 704 that corresponds to this label. This topmost (outgoing) label MUST 705 be associated with a provisioned outgoing interface and, for non- 706 point-to-point outgoing interfaces, a next hop LSR. Note that this 707 information MUST be provisioned via configuration or the controller 708 plane. In the previously mentioned special case where there are no 709 added F-labels and the outgoing interface is not a point-to-point 710 interface, the outgoing interface MUST also be associated with a next 711 hop LSR. 713 Resources may be allocated in a hierarchical fashion per LSP that is 714 represented by each F-Label. Each LSP MAY be provisioned with a 715 service parameters that dictates the specific traffic treatment to be 716 received by the traffic carried over that LSP. Implementations of 717 this document MUST ensure that traffic carried over each LSP 718 represented by one or more F-Labels receives the traffic treatment 719 provisioned for that LSP. Typical mechanisms used to provide 720 different treatment to different flows includes the allocation of 721 system resources (such as queues and buffers) and provisioning or 722 related parameters (such as shaping, and policing). Support can also 723 be provided via an underlying network technology such IEEE802.1 TSN 724 [I-D.ietf-detnet-mpls-over-tsn]. The specific mechanisms used by a 725 DetNet node to ensure DetNet service delivery requirements are met 726 for supported DetNet flows is outside the scope of this document. 728 Packets that are marked in a way that do not correspond to allocated 729 resources, e.g., lack a provisioned F-Label, can disrupt the QoS 730 offered to properly reserved DetNet flows by using resources 731 allocated to the reserved flows. Therefore, the network nodes of a 732 DetNet network: 734 o MUST defend the DetNet QoS by discarding or remarking (to an 735 allocated DetNet flow or non-competing non-DetNet flow) packets 736 received that are not associated with a completed resource 737 allocation. 739 o MUST NOT use a DetNet allocated resource, e.g. a queue or shaper 740 reserved for DetNet flows, for any packet that does match the 741 corresponding DetNet flow. 743 o MUST ensure a QoS flow does not exceed its allocated resources or 744 provisioned service level, 746 o MUST ensure a CoS flow or service class does not impact the 747 service delivered to other flows. This requirement is similar to 748 requirement for MPLS LSRs to that CoS LSPs do not impact the 749 resources allocated to TE LSPs, e.g., via [RFC3473]. 751 Subsequent sections provide additional considerations related to CoS 752 (Section 4.6.1), QoS (Section 4.6.2) and aggregation (Section 4.4). 754 4.3. OAM Indication 756 OAM follows the procedures set out in [RFC5085] with the restriction 757 that only Virtual Circuit Connectivity Verification (VCCV) type 1 is 758 supported. 760 As shown in Figure 3 of [RFC5085] when the first nibble of the d-CW 761 is 0x0 the payload following the d-CW is normal user data. However, 762 when the first nibble of the d-CW is 0X1, the payload that follows 763 the d-CW is an OAM payload with the OAM type indicated by the value 764 in the d-CW Channel Type field. 766 The reader is referred to [RFC5085] for a more detailed description 767 of the Associated Channel mechanism, and to the DetNet work on OAM 768 for more information DetNet OAM. 770 Additional considerations on DetNet-specific OAM are subjects for 771 further study. 773 4.4. Flow Aggregation 775 The ability to aggregate individual flows, and their associated 776 resource control, into a larger aggregate is an important technique 777 for improving scaling of control in the data, management and control 778 planes. The DetNet data plane allows for the aggregation of DetNet 779 flows, to improved scaling. There are two methods of supporting flow 780 aggregation covered in this section. 782 The resource control and management aspects of aggregation (including 783 the configuration of queuing, shaping, and policing) are the 784 responsibility of the DetNet controller plane and is out of scope of 785 this documents. It is also the responsibility of the controller 786 plane to ensure that consistent aggregation methods are used. 788 4.4.1. Aggregation Via LSP Hierarchy 790 DetNet flows forwarded via MPLS can leverage MPLS-TE's existing 791 support for hierarchical LSPs (H-LSPs), see [RFC4206]. H-LSPs are 792 typically used to aggregate control and resources, they may also be 793 used to provide OAM or protection for the aggregated LSPs. Arbitrary 794 levels of aggregation naturally falls out of the definition for 795 hierarchy and the MPLS label stack [RFC3032]. DetNet nodes which 796 support aggregation (LSP hierarchy) map one or more LSPs (labels) 797 into and from an H-LSP. Both carried LSPs and H-LSPs may or may not 798 use the TC field, i.e., L-LSPs or E-LSPs. Such nodes will need to 799 ensure that individual LSPs and H-LSPs receive the traffic treatment 800 required to ensure the required DetNet service is preserved. 802 Additional details of the traffic control capabilities needed at a 803 DetNet-aware node may be covered in the new service definitions 804 mentioned above or in separate future documents. Controller plane 805 mechanisms will also need to ensure that the service required on the 806 aggregate flow are provided, which may include the discarding or 807 remarking mentioned in the previous sections. 809 4.4.2. Aggregating DetNet Flows as a new DetNet flow 811 An aggregate can be built by layering DetNet flows, including both 812 their S-Label and, when present, F-Labels as shown below: 814 +---------------------------------+ 815 | | 816 | DetNet Flow | 817 | Payload Packet | 818 | | 819 +---------------------------------+ <--\ 820 | DetNet Control Word | | 821 +=================================+ | 822 | S-Label | | 823 +---------------------------------+ | 824 | [ F-Label(s) ] | +----DetNet data plane 825 +---------------------------------+ | 826 | DetNet Control Word | | 827 +=================================+ | 828 | A-Label | | 829 +---------------------------------+ | 830 | F-Label(s) | <--/ 831 +---------------------------------+ 832 | Data-Link | 833 +---------------------------------+ 834 | Physical | 835 +---------------------------------+ 837 Figure 6: DetNet Aggregation as a new DetNet Flow 839 Both the aggregation label, which is referred to as an A-Label, and 840 the individual flow's S-Label have their MPLS S bit set indicating 841 bottom of stack, and the d-CW allows the PREOF to work. An A-Label 842 is a special case of an S-Label, whose properties are known only at 843 the aggregation and deaggregation end-points. 845 It is a property of the A-Label that what follows is a d-CW followed 846 by an MPLS label stack. A relay node processing the A-Label would 847 not know the underlying payload type, and the A-Label would be 848 processed as a normal S-Label. This would only be known to a node 849 that was a peer of the node imposing the S-Label. However there is 850 no real need for it to know the payload type during aggregation 851 processing. 853 As in the previous section, nodes supporting this type of aggregation 854 will need to ensure that individual and aggregated flows receive the 855 traffic treatment required to ensure the required DetNet service is 856 preserved. Also, it is the controller plane's responsibility to to 857 ensure that the service required on the aggregate flow are properly 858 provisioned. 860 4.5. Service Sub-Layer Considerations 862 The edge and relay node internal procedures related to PREOF are 863 implementation specific. The order of a packet elimination or 864 replication is out of scope in this specification. 866 It is important that the DetNet layer is configured such that a 867 DetNet node never receives its own replicated packets. If it were to 868 receive such packets the replication function would make the loop 869 more destructive of bandwidth than a conventional unicast loop. 870 Ultimately the TTL in the S-Label will cause the packet to die during 871 a transient loop, but given the sensitivity of applications to packet 872 latency the impact on the DetNet application would be severe. To 873 avoid the problem of a transient forwarding loop, changes to an LSP 874 supporting DetNet MUST be loop-free. 876 4.5.1. Edge Node Processing 878 A DetNet Edge node operates in the DetNet forwarding sub-layer and 879 service sub-layer. An edge node is responsible for matching incoming 880 packets to the service they require and encapsulating them 881 accordingly. An edge node may perform PRF, PEF, and or POF. Details 882 on encapsulation can be found in Section 4.2; details on PRF can be 883 found in Section 4.2.2.1; details on PEF can be found in 884 Section 4.2.2.2; and details on POF can be found in Section 4.2.2.3. 886 4.5.2. Relay Node Processing 888 A DetNet Relay node operates in the DetNet forwarding sub-layer and 889 service sub-layer. For DetNet using MPLS forwarding related 890 processing is performed on the F-Label. This processing is done 891 within an extended forwarder function. Whether an incoming DetNet 892 flow receives DetNet specific processing depends on how the 893 forwarding is programmed. Some relay nodes may be DetNet service 894 aware for certain DetNet services, while for other DetNet services 895 these nodes may perform as unmodified LSRs that only understand how 896 to switch MPLS-TE LSPs, i.e., as a transit node, see Section 4.4. 897 Again, this is entirely up to how the forwarding has been programmed. 899 During the elimination and replication process the sequence number of 900 an incoming DetNet packet MUST be preserved and carried in the 901 corresponding outgoing DetNet packet. For example, a relay node that 902 performs both PEF and PRF first performs PEF on incoming packets to 903 create a compound flow. It then performs PRF and copies the app-flow 904 data and the d-CW into packets for each outgoing DetNet member flow. 906 The internal design of a relay node is out of scope of this document. 907 However the reader's attention is drawn to the need to make any PREOF 908 state available to the packet processor(s) dealing with packets to 909 which the PREOF functions must be applied, and to maintain that state 910 is such a way that it is available to the packet processor operation 911 on the next packet in the DetNet flow (which may be a duplicate, a 912 late packet, or the next packet in sequence). 914 4.6. Forwarding Sub-Layer Considerations 916 4.6.1. Class of Service 918 Class and quality of service, i.e., CoS and QoS, are terms that are 919 often used interchangeably and confused with each other. In the 920 context of DetNet, CoS is used to refer to mechanisms that provide 921 traffic forwarding treatment based on aggregate group basis and QoS 922 is used to refer to mechanisms that provide traffic forwarding 923 treatment based on a specific DetNet flow basis. Examples of 924 existing network level CoS mechanisms include DiffServ which is 925 enabled by IP header differentiated services code point (DSCP) field 926 [RFC2474] and MPLS label traffic class field [RFC5462], and at Layer- 927 2, by IEEE 802.1p priority code point (PCP). 929 CoS for DetNet flows carried in PWs and MPLS is provided using the 930 existing MPLS Differentiated Services (DiffServ) architecture 931 [RFC3270]. Both E-LSP and L-LSP MPLS DiffServ modes MAY be used to 932 support DetNet flows. The Traffic Class field (formerly the EXP 933 field) of an MPLS label follows the definition of [RFC5462] and 934 [RFC3270]. The Uniform, Pipe, and Short Pipe DiffServ tunneling and 935 TTL processing models are described in [RFC3270] and [RFC3443] and 936 MAY be used for MPLS LSPs supporting DetNet flows. MPLS ECN MAY also 937 be used as defined in ECN [RFC5129] and updated by [RFC5462]. 939 4.6.2. Quality of Service 941 In addition to explicit routes, and packet replication and 942 elimination, described in Section 4 above, DetNet provides zero 943 congestion loss and bounded latency and jitter. As described in 944 [RFC8655], there are different mechanisms that maybe used separately 945 or in combination to deliver a zero congestion loss service. This 946 includes Quality of Service (QoS) mechanisms at the MPLS layer, that 947 may be combined with the mechanisms defined by the underlying network 948 layer such as 802.1TSN. 950 Quality of Service (QoS) mechanisms for flow specific traffic 951 treatment typically includes a guarantee/agreement for the service, 952 and allocation of resources to support the service. Example QoS 953 mechanisms include discrete resource allocation, admission control, 954 flow identification and isolation, and sometimes path control, 955 traffic protection, shaping, policing and remarking. Example 956 protocols that support QoS control include Resource ReSerVation 957 Protocol (RSVP) [RFC2205] (RSVP) and RSVP-TE [RFC3209] and [RFC3473]. 958 The existing MPLS mechanisms defined to support CoS [RFC3270] can 959 also be used to reserve resources for specific traffic classes. 961 A baseline set of QoS capabilities for DetNet flows carried in PWs 962 and MPLS can provided by MPLS with Traffic Engineering (MPLS-TE) 963 [RFC3209] and [RFC3473]. TE LSPs can also support explicit routes 964 (path pinning). Current service definitions for packet TE LSPs can 965 be found in "Specification of the Controlled Load Quality of 966 Service", [RFC2211], "Specification of Guaranteed Quality of 967 Service", [RFC2212], and "Ethernet Traffic Parameters", [RFC6003]. 968 Additional service definitions are expected in future documents to 969 support the full range of DetNet services. In all cases, the 970 existing label-based marking mechanisms defined for TE-LSPs and even 971 E-LSPs are use to support the identification of flows requiring 972 DetNet QoS. 974 5. Management and Control Information Summary 976 The specific information needed for the processing of each DetNet 977 service depends on the DetNet node type and the functions being 978 provided on the node. This section summarizes based on the DetNet 979 sub-layer and if the DetNet traffic is being sent or received. All 980 DetNet node types are DetNet forwarding sub-layer aware, while all 981 but transit nodes are service sub-layer aware. This is shown in 982 Figure 2. 984 Much like other MPLS labels, there are a number of alternatives 985 available for DetNet S-Label and F-Label advertisement to an upstream 986 peer node. These include distributed signaling protocols such as 987 RSVP-TE, centralized label distribution via a controller that manages 988 both the sender and the receiver using NETCONF/YANG, BGP, PCEP, etc., 989 and hybrid combinations of the two. The details of the controller 990 plane solution required for the label distribution and the management 991 of the label number space are out of scope of this document. There 992 are particular DetNet considerations and requirements that are 993 discussed in [I-D.ietf-detnet-data-plane-framework]. 995 5.1. Service Sub-Layer Information Summary 997 The following summarizes the information that is needed on service 998 sub-layer aware nodes that transmit DetNet MPLS traffic, on a per 999 service basis: 1001 o App-Flow identification information, e.g., IP information as 1002 defined in [I-D.ietf-detnet-ip-over-mpls]. Note, this information 1003 is not needed on DetNet relay nodes. 1005 o The sequence number length to be used for the service. Valid 1006 values included 0, 16 and 28 bits. 0 bits cannot be used when PEF 1007 or POF is configured for the service. 1009 o If PRF is to be provided for the service. 1011 o The outgoing S-Label for each of the service's outgoing DetNet 1012 (member) flow. 1014 o The forwarding sub-layer information associated with the output of 1015 the service sub-layer. Note that when the PRF function is 1016 provisioned, this information is per DetNet member flow. 1017 Logically the forwarding sub-layer information is a pointer to 1018 further details of transmission of Detnet flows at the forwarding 1019 sub-layer. 1021 The following summarizes the information that is needed on service 1022 sub-layer aware nodes that receive DetNet MPLS traffic, on a per 1023 service basis: 1025 o The forwarding sub-layer information associated with the input of 1026 the service sub-layer. Note that when the PEF function is 1027 provisioned, this information is per DetNet member flow. 1028 Logically the forwarding sub-layer information is a pointer to 1029 further details of the reception of Detnet flows at the forwarding 1030 sub-layer or A-Label. 1032 o The incoming S-Label for the service. 1034 o If PEF or POF is to be provided for the service. 1036 o The sequence number length to be used for the service. Valid 1037 values included 0, 16 and 28 bits. 0 bits cannot be used when PEF 1038 or POF are configured for the service. 1040 o App-Flow identification information, e.g., IP information as 1041 defined in [I-D.ietf-detnet-ip-over-mpls]. Note, this information 1042 is not needed on DetNet relay nodes. 1044 5.1.1. Service Aggregation Information Summary 1046 Nodes performing aggregation using A-Labels, per 1047 Section Section 4.4.2, require the additional information summarized 1048 in this section. 1050 The following summarizes the additional information that is needed on 1051 a node that sends aggregated flows using A-Labels: 1053 o The S-Labels or F-Labels that are to be carried over each 1054 aggregated service. 1056 o The A-Label associated with each aggregated service. 1058 o The other S-Label information summarized above. 1060 On the receiving node, the A-Label provides the forwarding context of 1061 an incoming interface or an F-Label and is used in subsequent service 1062 or forwarding sub-layer receive processing, as appropriated. The 1063 related addition configuration that may be provided discussed 1064 elsewhere in this section. 1066 5.2. Forwarding Sub-Layer Information Summary 1068 The following summarizes the information that is needed on forwarding 1069 sub-layer aware nodes that send DetNet MPLS traffic, on a per 1070 forwarding sub-layer flow basis: 1072 o The outgoing F-Label stack to be pushed. The stack may include 1073 H-LSP labels. 1075 o The traffic parameters associated with a specific label in the 1076 stack. Note that there may be multiple sets of traffic paramters 1077 associated with specific labels in the stack, e.g., when H-LSPs 1078 are used. 1080 o Outgoing interface and, for unicast traffic, the next hop 1081 information. 1083 o Sub-network specific parameters on a technology specific basis. 1084 For example, see [I-D.ietf-detnet-mpls-over-tsn]. 1086 The following summarizes the information that is needed on forwarding 1087 sub-layer aware nodes that receive DetNet MPLS traffic, on a per 1088 forwarding sub-layer flow basis: 1090 o The incoming interface. For some implementations and deployment 1091 scenarios, this information may not be needed. 1093 o The incoming F-Label stack to be popped. The stack may include 1094 H-LSP labels. 1096 o How the incoming forwarding sub-layer flow is to be handled, i.e., 1097 forwarded as a transit node, or provided to the service sub-layer. 1099 It is the responsibility of the DetNet controller plane to properly 1100 provision both flow identification information and the flow specific 1101 resources needed to provided the traffic treatment needed to meet 1102 each flow's service requirements. This applies for aggregated and 1103 individual flows. 1105 6. Security Considerations 1107 General security considerations are described in [RFC8655]. 1108 Additionally, security considerations and a threat analysis are 1109 described in [I-D.ietf-detnet-security]. This section considers 1110 security considerations which are specific to the DetNet MPLS data 1111 plane. The considerations raised related to MPLS networks in general 1112 in [RFC5920] are equally applicable to the the DetNet MPLS data 1113 plane. 1115 Security aspects which are unique to DetNet are those whose aim is to 1116 provide the specific quality of service aspects of DetNet, which are 1117 primarily to deliver data flows with extremely low packet loss rates 1118 and bounded end-to-end delivery latency. 1120 The primary considerations for the data plane is to maintain 1121 integrity of data and delivery of the associated DetNet service 1122 traversing the DetNet network. Application flows can be protected 1123 through whatever means is provided by the underlying technology. For 1124 example, encryption may be used, such as that provided by IPSec 1125 [RFC4301] for IP flows and/or by an underlying sub-net using MACSec 1126 [IEEE802.1AE-2018] for IP over Ethernet (Layer-2) flows. 1128 From a data plane perspective this document does not add or modify 1129 any header information. 1131 At the management and control level DetNet flows are identified on a 1132 per-flow basis, which may provide controller plane attackers with 1133 additional information about the data flows (when compared to 1134 controller planes that do not include per-flow identification). This 1135 is an inherent property of DetNet which has security implications 1136 that should be considered when determining if DetNet is a suitable 1137 technology for any given use case. 1139 To provide uninterrupted availability of the DetNet service, 1140 provisions can be made against DOS attacks and delay attacks. To 1141 protect against DOS attacks, excess traffic due to malicious or 1142 malfunctioning devices is prevented or mitigated through the use of 1143 existing mechanisms, for example by policing and shaping incoming 1144 traffic. To prevent DetNet packets from being delayed by an entity 1145 external to a DetNet domain, DetNet technology definition can allow 1146 for the mitigation of Man-In-The-Middle attacks, for example through 1147 use of authentication and authorization of devices within the DetNet 1148 domain. 1150 7. IANA Considerations 1152 This document makes no IANA requests. 1154 8. Acknowledgements 1156 The authors wish to thank Pat Thaler, Norman Finn, Loa Anderson, 1157 David Black, Rodney Cummings, Ethan Grossman, Tal Mizrahi, David 1158 Mozes, Craig Gunther, George Swallow, Yuanlong Jiang, Jeong-dong Ryoo 1159 and Carlos J. Bernardos for their various contributions to this 1160 work. 1162 9. Contributors 1164 RFC7322 limits the number of authors listed on the front page of a 1165 draft to a maximum of 5. The editor wishes to thank and acknowledge 1166 the follow author for contributing text to this draft. 1168 Don Fedyk 1169 LabN Consulting, L.L.C. 1170 Email: dfedyk@labn.net 1172 10. References 1174 10.1. Normative References 1176 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1177 Requirement Levels", BCP 14, RFC 2119, 1178 DOI 10.17487/RFC2119, March 1997, 1179 . 1181 [RFC2211] Wroclawski, J., "Specification of the Controlled-Load 1182 Network Element Service", RFC 2211, DOI 10.17487/RFC2211, 1183 September 1997, . 1185 [RFC2212] Shenker, S., Partridge, C., and R. Guerin, "Specification 1186 of Guaranteed Quality of Service", RFC 2212, 1187 DOI 10.17487/RFC2212, September 1997, 1188 . 1190 [RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol 1191 Label Switching Architecture", RFC 3031, 1192 DOI 10.17487/RFC3031, January 2001, 1193 . 1195 [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., 1196 Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack 1197 Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001, 1198 . 1200 [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., 1201 and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP 1202 Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, 1203 . 1205 [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, 1206 P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi- 1207 Protocol Label Switching (MPLS) Support of Differentiated 1208 Services", RFC 3270, DOI 10.17487/RFC3270, May 2002, 1209 . 1211 [RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing 1212 in Multi-Protocol Label Switching (MPLS) Networks", 1213 RFC 3443, DOI 10.17487/RFC3443, January 2003, 1214 . 1216 [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label 1217 Switching (GMPLS) Signaling Resource ReserVation Protocol- 1218 Traffic Engineering (RSVP-TE) Extensions", RFC 3473, 1219 DOI 10.17487/RFC3473, January 2003, 1220 . 1222 [RFC4206] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) 1223 Hierarchy with Generalized Multi-Protocol Label Switching 1224 (GMPLS) Traffic Engineering (TE)", RFC 4206, 1225 DOI 10.17487/RFC4206, October 2005, 1226 . 1228 [RFC4385] Bryant, S., Swallow, G., Martini, L., and D. McPherson, 1229 "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for 1230 Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, 1231 February 2006, . 1233 [RFC5085] Nadeau, T., Ed. and C. Pignataro, Ed., "Pseudowire Virtual 1234 Circuit Connectivity Verification (VCCV): A Control 1235 Channel for Pseudowires", RFC 5085, DOI 10.17487/RFC5085, 1236 December 2007, . 1238 [RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion 1239 Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January 1240 2008, . 1242 [RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching 1243 (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic 1244 Class" Field", RFC 5462, DOI 10.17487/RFC5462, February 1245 2009, . 1247 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1248 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1249 May 2017, . 1251 [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, 1252 "Deterministic Networking Architecture", RFC 8655, 1253 DOI 10.17487/RFC8655, October 2019, 1254 . 1256 10.2. Informative References 1258 [I-D.ietf-detnet-data-plane-framework] 1259 Varga, B., Farkas, J., Berger, L., Malis, A., and S. 1260 Bryant, "DetNet Data Plane Framework", draft-ietf-detnet- 1261 data-plane-framework-06 (work in progress), May 2020. 1263 [I-D.ietf-detnet-ip] 1264 Varga, B., Farkas, J., Berger, L., Fedyk, D., and S. 1265 Bryant, "DetNet Data Plane: IP", draft-ietf-detnet-ip-07 1266 (work in progress), July 2020. 1268 [I-D.ietf-detnet-ip-over-mpls] 1269 Varga, B., Berger, L., Fedyk, D., Bryant, S., and J. 1270 Korhonen, "DetNet Data Plane: IP over MPLS", draft-ietf- 1271 detnet-ip-over-mpls-06 (work in progress), May 2020. 1273 [I-D.ietf-detnet-mpls-over-tsn] 1274 Varga, B., Farkas, J., Malis, A., and S. Bryant, "DetNet 1275 Data Plane: MPLS over IEEE 802.1 Time Sensitive Networking 1276 (TSN)", draft-ietf-detnet-mpls-over-tsn-03 (work in 1277 progress), June 2020. 1279 [I-D.ietf-detnet-security] 1280 Mizrahi, T. and E. Grossman, "Deterministic Networking 1281 (DetNet) Security Considerations", draft-ietf-detnet- 1282 security-10 (work in progress), May 2020. 1284 [IEEE802.1AE-2018] 1285 IEEE Standards Association, "IEEE Std 802.1AE-2018 MAC 1286 Security (MACsec)", 2018, 1287 . 1289 [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. 1290 Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 1291 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, 1292 September 1997, . 1294 [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, 1295 "Definition of the Differentiated Services Field (DS 1296 Field) in the IPv4 and IPv6 Headers", RFC 2474, 1297 DOI 10.17487/RFC2474, December 1998, 1298 . 1300 [RFC3272] Awduche, D., Chiu, A., Elwalid, A., Widjaja, I., and X. 1301 Xiao, "Overview and Principles of Internet Traffic 1302 Engineering", RFC 3272, DOI 10.17487/RFC3272, May 2002, 1303 . 1305 [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation 1306 Edge-to-Edge (PWE3) Architecture", RFC 3985, 1307 DOI 10.17487/RFC3985, March 2005, 1308 . 1310 [RFC4301] Kent, S. and K. Seo, "Security Architecture for the 1311 Internet Protocol", RFC 4301, DOI 10.17487/RFC4301, 1312 December 2005, . 1314 [RFC4448] Martini, L., Ed., Rosen, E., El-Aawar, N., and G. Heron, 1315 "Encapsulation Methods for Transport of Ethernet over MPLS 1316 Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006, 1317 . 1319 [RFC4875] Aggarwal, R., Ed., Papadimitriou, D., Ed., and S. 1320 Yasukawa, Ed., "Extensions to Resource Reservation 1321 Protocol - Traffic Engineering (RSVP-TE) for Point-to- 1322 Multipoint TE Label Switched Paths (LSPs)", RFC 4875, 1323 DOI 10.17487/RFC4875, May 2007, 1324 . 1326 [RFC5440] Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation 1327 Element (PCE) Communication Protocol (PCEP)", RFC 5440, 1328 DOI 10.17487/RFC5440, March 2009, 1329 . 1331 [RFC5586] Bocci, M., Ed., Vigoureux, M., Ed., and S. Bryant, Ed., 1332 "MPLS Generic Associated Channel", RFC 5586, 1333 DOI 10.17487/RFC5586, June 2009, 1334 . 1336 [RFC5920] Fang, L., Ed., "Security Framework for MPLS and GMPLS 1337 Networks", RFC 5920, DOI 10.17487/RFC5920, July 2010, 1338 . 1340 [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau, 1341 L., and L. Berger, "A Framework for MPLS in Transport 1342 Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010, 1343 . 1345 [RFC6003] Papadimitriou, D., "Ethernet Traffic Parameters", 1346 RFC 6003, DOI 10.17487/RFC6003, October 2010, 1347 . 1349 [RFC6073] Martini, L., Metz, C., Nadeau, T., Bocci, M., and M. 1350 Aissaoui, "Segmented Pseudowire", RFC 6073, 1351 DOI 10.17487/RFC6073, January 2011, 1352 . 1354 [RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W., and 1355 L. Yong, "The Use of Entropy Labels in MPLS Forwarding", 1356 RFC 6790, DOI 10.17487/RFC6790, November 2012, 1357 . 1359 [RFC8306] Zhao, Q., Dhody, D., Ed., Palleti, R., and D. King, 1360 "Extensions to the Path Computation Element Communication 1361 Protocol (PCEP) for Point-to-Multipoint Traffic 1362 Engineering Label Switched Paths", RFC 8306, 1363 DOI 10.17487/RFC8306, November 2017, 1364 . 1366 [RFC8660] Bashandy, A., Ed., Filsfils, C., Ed., Previdi, S., 1367 Decraene, B., Litkowski, S., and R. Shakir, "Segment 1368 Routing with the MPLS Data Plane", RFC 8660, 1369 DOI 10.17487/RFC8660, December 2019, 1370 . 1372 Authors' Addresses 1374 Balazs Varga (editor) 1375 Ericsson 1376 Magyar Tudosok krt. 11. 1377 Budapest 1117 1378 Hungary 1380 Email: balazs.a.varga@ericsson.com 1382 Janos Farkas 1383 Ericsson 1384 Magyar Tudosok krt. 11. 1385 Budapest 1117 1386 Hungary 1388 Email: janos.farkas@ericsson.com 1390 Lou Berger 1391 LabN Consulting, L.L.C. 1393 Email: lberger@labn.net 1395 Andrew G. Malis 1396 Malis Consulting 1398 Email: agmalis@gmail.com 1399 Stewart Bryant 1400 Futurewei Technologies 1402 Email: stewart.bryant@gmail.com 1404 Jouni Korhonen 1406 Email: jouni.nospam@gmail.com