idnits 2.17.00 (12 Aug 2021) /tmp/idnits9843/draft-filsfils-6man-structured-flow-label-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The abstract seems to contain references ([RFC6437]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 440: '... * The value SHOULD be within the e...' -- The draft header indicates that this document updates RFC6437, but the abstract doesn't seem to directly say this. It does mention RFC6437 though, so this could be OK. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document date (7 March 2022) is 68 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '19' on line 127 -- Looks like a reference, but probably isn't: '16' on line 127 -- Looks like a reference, but probably isn't: '15' on line 133 -- Looks like a reference, but probably isn't: '0' on line 133 == Outdated reference: A later version (-12) exists of draft-song-ippm-postcard-based-telemetry-11 Summary: 4 errors (**), 0 flaws (~~), 1 warning (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 6man C. Filsfils, Ed. 3 Internet-Draft A. Abdelsalam, Ed. 4 Updates: 6437 (if approved) Cisco Systems, Inc. 5 Intended status: Standards Track S. Zadok 6 Expires: 8 September 2022 Broadcom 7 X. Xu 8 Capitalonline 9 W. Cheng 10 China Mobile 11 D. Voyer 12 Bell Canada 13 P. Camarillo 14 Cisco Systems, Inc. 15 7 March 2022 17 Structured Flow Label 18 draft-filsfils-6man-structured-flow-label-02 20 Abstract 22 This document defines the IPv6 Structured Flow Label. The seamless 23 nature of the change to [RFC6437] is demonstrated. Benefits of the 24 solution are explained. Use-cases are illustrated. 26 Status of This Memo 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF). Note that other groups may also distribute 33 working documents as Internet-Drafts. The list of current Internet- 34 Drafts is at https://datatracker.ietf.org/drafts/current/. 36 Internet-Drafts are draft documents valid for a maximum of six months 37 and may be updated, replaced, or obsoleted by other documents at any 38 time. It is inappropriate to use Internet-Drafts as reference 39 material or to cite them other than as "work in progress." 41 This Internet-Draft will expire on 8 September 2022. 43 Copyright Notice 45 Copyright (c) 2022 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 50 license-info) in effect on the date of publication of this document. 51 Please review these documents carefully, as they describe your rights 52 and restrictions with respect to this document. Code Components 53 extracted from this document must include Revised BSD License text as 54 described in Section 4.e of the Trust Legal Provisions and are 55 provided without warranty as described in the Revised BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 60 2. Structured Flow Label Format . . . . . . . . . . . . . . . . 3 61 3. Recommended Design . . . . . . . . . . . . . . . . . . . . . 4 62 4. Seamless Migration from RFC6437 . . . . . . . . . . . . . . . 4 63 5. Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . 5 64 6. IPv6 End-to-End Absolute Loss Measurements . . . . . . . . . 6 65 7. Programmed Sampling of packets . . . . . . . . . . . . . . . 7 66 8. Postcard-based Telemetry using packet Marking (PBT-M) . . . . 8 67 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 9 68 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 9 69 10.1. Normative References . . . . . . . . . . . . . . . . . . 9 70 10.2. Informative References . . . . . . . . . . . . . . . . . 9 71 Appendix A. Entropy . . . . . . . . . . . . . . . . . . . . . . 10 72 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 10 74 1. Introduction 76 The IPv6 header [RFC8200] contains a 20-bit field called "Flow Label" 77 (FL) where the left-most bit is number 19 and the right-most bit is 78 number0. 80 1 0 81 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 82 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 83 | Flow Label | 84 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 85 Figure 1: IPv6 Flow Label 87 The FL usage is specified in [RFC6437], briefly for entropy purpose. 89 Instead of using FL as a single 20-bit entropy structure, this 90 document updates [RFC6437] and defines the 20-bit FL field as a 91 structure of two fields: 93 * FLC: 4-bit per-packet control bits for generic application marking 94 (e.g., group-based policy) 96 * FLE: 16-bit per-flow entropy (equivalent to [RFC6437] definition) 98 This document shows that updating [RFC6437] is a seamless migration 99 operation, simply because a majority of chipsets deployed in the 100 Internet and private domains do not use FL as documented in 101 [RFC6437]: they use a subset of the 20 bits of the FL as entropy, 102 i.e. as documented in this document. 104 This document further shows that even if a chipset were to use the 105 full FL as a 20-bit entropy input for ECMP hash, then the change 106 proposed in this document would not cause any significant backward 107 incompatibility. 109 The seamless nature of the change being explained, the document then 110 explains the benefits of the Structured Flow Label definition. Three 111 use-cases are provided. Several more are expected in the future in 112 separate documents. 114 2. Structured Flow Label Format 116 We define the Structured Flow Label as shown in Figure 2 118 1 0 119 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 120 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 121 | FLC | FLE | 122 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 123 Figure 2: Structured Flow Label Format 125 Where: 127 * FLC: 4-bit [19, 16]: per-packet Control not included in ECMP hash 129 * FLE: 16-bit [15, 0]: per-flow Entropy included in ECMP hash 131 FLE is defined as per [RFC6437]: i.e. [RFC6437] is strictly 132 preserved, the only change is that it defines the usage of the 16 133 low-order bits [15, 0] instead of the full 20-bit of the Flow Label. 135 FLC is defined as follows: the 4-bit FLC field in the IPv6 header is 136 used by the network for group-based policy marking. The value of the 137 FLC bits in a received packet or fragment might be different from the 138 value sent by the packet's source. FLC is not included in the ECMP 139 hash computation. The definition of FLC is modeled on the definition 140 of the "Traffic Class" [RFC8200]. 142 In the same way that "Traffic Class" is not an input for ECMP hash, 143 FLC is not an input for ECMP hash. 145 3. Recommended Design 147 This section provides design recommendation of how customer packets 148 are being forwarded within an operator network. 150 All customer packets are typically encapsulated at the edge of the 151 operator network as illustrated in Figure 3. 153 +------------------------------------------+ 154 | Operator IPv6 network | 155 | | 156 +---+ +-+--+ +------+ +-----+ +-+--+ +---+ 157 | A |<>-<>| R1 |<>-----<>| R2 |<>---<>| R3 |<>--<>| R4 |<>-<>| Z | 158 +---+ +-+--+ +------+ +-----+ +-+--+ +---+ 159 | +--------+ +--------+ +--------+ | 160 | | IP6(R1,| | IP6(R1,| | IP6(R1,| | 161 | | R4,| | R4,| | R4,| | 162 | | FL)| | FL)| | FL)| | 163 +--------+ | +--------+ +--------+ +--------+ | +--------+ 164 |Pkt(A,Z)| | |Pkt(A,Z)| |Pkt(A,Z)| |Pkt(A,Z)| | |Pkt(A,Z)| 165 +--------+ | +--------+ +--------+ +--------+ | +--------+ 166 | | 167 +------------------------------------------+ 169 Figure 3: Packet forwarding within operator IPv6 network 171 When a customer packet is received at the edge router (R1) of 172 operator IPv6 network, the packet is encapsulated using an outer IPv6 173 header. The outer IPv6 header defines the source edge router that 174 encapsulates the packet (R1) and the destination edge router (R4) 175 which has to decapsulate the packet before being forwarded towards 176 its final destination. 178 R1 also sets the Flow Label (FL) of the outer IPv6 header which is 179 computed based on the hash of the customer packet. Every subsequent 180 router (R2 and R3) within the operator network forwards the packet 181 based on the information of the outer IPv6 header. 183 For example, ECMP hash calculation on routers R2 and R3 is performed 184 using the outer IPv6 header information (R1, R4, FL). 186 4. Seamless Migration from RFC6437 188 Cisco and Broadcom report that the norm for their products: 190 * do not set entropy in the 4 most-specific bits of the FL field 191 * do not use the 4 most-specific bits of the FL as input for ECMP 192 hash 194 The authors believe that the same is likely for other vendors and are 195 gathering data for future versions of this document. 197 Even if a chipset were to use the 4 most-specific bits of the FL 198 field as input for ECMP hash while the 4-most specific bits of the FL 199 field were used by other nodes in the network as FLC semantics 200 (hence, per-packet marking, potentially not per-flow constant), still 201 the impact would not be significant. Let us take an example to 202 illustrate this. 204 Let us assume that: 206 * Flow F is to be routed across an operator network. 208 * Using the Structured FL format, all the packets of F have an FLE 209 value of 1010 1111 0100 0101. 211 * The operator leverages the FLC to mark the packets of F into two 212 subsets S1 and S2. 214 * S1 has FLC value of 0000. 216 * S2 has FLC value of 0010. 218 We can have the following two scenarios: 220 *Scenario-1: Routers compliant to this document* 222 These routers will only use FLE for ECMP decision and hence all 223 packets of flow F will be routed on the same path. 225 *Scenario-2: Routers implementing RFC6437* 227 These routers will use the full 20-bit (FLC+FLE) for ECMP decision. 228 This could (but not always) lead to having S1 packets taking a 229 different path from the ones of S2. 231 However, the scenario-2 is unlikely as per the chipset implementation 232 reported in this doc. 234 5. Benefits 236 * Seamless migration from RFC6437 237 * FLE of 16 bits is excellent to drive ECMP hash. [RFC8085] stated 238 that 14 bits are sufficient Appendix A 240 * FLC of 4 bits provides up to 200 to 400% improvement packet 241 marking capability for operator controlled use-cases 243 - Without FLC, operators can only mark 6 bits of the IPv6 header 244 (Traffic Class) 246 - Many deployments consume 4 to 5 of these bits, leaving only 1 247 or 2 available 249 - 4 new bits is a significant richness offered to operators to 250 mark packets 252 * Several use-cases will illustrate the usage of these FLC bits: 254 - IPv6 End-to-End absolute loss measurement 256 - Programmed sampling of packets 258 - Postcard-based Telemetry using packet Marking (PBT-M) 260 6. IPv6 End-to-End Absolute Loss Measurements 262 This section describes the usage of FLC bits to enable packet loss 263 measurements [RFC8321] for IPv6 networks. We re-use the same 264 reference topology from RFC8321 for our illustration (Figure 4). 266 +-------+ +------+ +-------+ +-------+ 267 --<>| R1 |<>----<>| R2 |<>----<>| R3 |<>----<>| R4 |<>--- 268 +-------+ +------+ +-------+ +-------+ 269 . . 270 . End-to-End Loss . 271 <-----------------------------------------> 273 Figure 4: End-to-End Absolute Loss Measurement 275 In order for an operator to enable End-to-End packet loss 276 measurements between routers R1 and R4 for a given flow: 278 * The operator allocates one bit (C-bit) out of the FLC field to be 279 used for packet coloring. 281 * The operator configures R1 to use the C-bit to color packets of 282 the flow of interest. 284 * The operator configures R1 and R4 to match the C-bit and perform 285 packet counting. 287 * The operator configures R4 to clear the C-bit before forwarding 288 the packet. 290 * An SDN controller is used to collect the counters from R1 and R4 291 to perform End-to-End packet loss measurements. 293 The flow selection, flow identification, counters update, counters 294 collection and counters correlation considerations are out of the 295 scope of this doc. They can be realized using the techniques 296 described in [RFC8321]. 298 7. Programmed Sampling of packets 300 An operator can detect End-to-End packet loss by deploying the 301 solution described in Section 6}. 303 In some cases, the operator needs to identify the node(s) or the 304 link(s) where the packet loss happens. In order to so, the operator 305 would need to collect packet loss measurement from each hop on the 306 packet path. Figure 4 shows the combination of End-to-End and per- 307 hop measurements. 309 +------+ +-------+ +-------+ +-------+ 310 --<>| R1 |<>----<>| R2 |<>----<>| R3 |<>-----<>| R4 |<>--- 311 +------+ +-------+ +-------+ +-------+ 312 . . . . . 313 . . . . . 314 . <-------> <---------> 315 . Node Loss Link Loss 316 . . 317 . End-to-End Loss . 318 <-------------------------------------------> 320 Figure 5: End-to-End and Per-Hop Loss Measurements 322 If the operator detects End-to-End packet loss, it will do the 323 following: 325 * The operator allocates another bit (H-bit) out of the FLC field to 326 trigger per-hop measurements. 328 * The operator configures R1 to enable H-bit for sample of the flow 329 that experience End-to-End packet loss. 331 * The operator configures each router on the packet path (R2 and R3 332 in Figure 5) to match the H-bit and perform packet counting 334 * An SDN controller is used to collect the counters, perform End-to- 335 End and per-hop loss measurements, and identify the node(s) or 336 link(s) where the packet loss happens. 338 The SDN controller aspects, flow sampling, flow selection, flow 339 identification, counters update, counters collection and counters 340 correlation considerations are out of the scope of this doc. Some of 341 these considerations can be realized using the techniques described 342 in [RFC8321]. 344 8. Postcard-based Telemetry using packet Marking (PBT-M) 346 This section describes the usage of FLC bits to enable packet marking 347 for PBT-M [I-D.song-ippm-postcard-based-telemetry]. 349 PBT-M enables each router along the packet path exports its telemetry 350 data to the telemetry collector in the form of postcards as 351 illustrated in Figure 6. In PBT-M a single bit is needed to mark the 352 packet which then matched by each node to trigger telemetry export 353 from intermediate routers. 355 +-----------------------+ 356 +------------>| Telemetry Collector |<--------------+ 357 | +-----------------------+ | 358 | ^ ^ | 359 | Postcard | Postcard | Postcard | Postcard 360 | | | | 361 +------+ +------+ +------+ +------+ 362 --<>| R1 |<>----<>| R2 |<>-----<>| R3 |<>-----<>| R4 |<>--- 363 +------+ +------+ +------+ +------+ 365 Figure 6: Postcard-Based Telemetry using packet Marking (PBT-M) 367 An operator that wants to deploy PBT-M, can do the following: 369 * Allocates one bit (T-bit) out of the FLC field to be used for PBT 370 packet marking. 372 * Configures R1 to enable T-bit for sample of the flow of interest 374 * Configures each router to match the T-bit and perform packet 375 counting and send a postcard with its telemetry data to the 376 Telemetry collector. 378 * An SDN controller is used to the collected postcards and analyze 379 them. 381 The SDN controller aspects, flow sampling, flow selection, flow 382 identification, postcard generation, postcard collection and postcard 383 correlation and postcard processing considerations are out of the 384 scope of this doc. Some of these considerations are defined in 385 [I-D.song-ippm-postcard-based-telemetry]. 387 9. Acknowledgements 389 TBD 391 10. References 393 10.1. Normative References 395 [RFC6437] Amante, S., Carpenter, B., Jiang, S., and J. Rajahalme, 396 "IPv6 Flow Label Specification", RFC 6437, 397 DOI 10.17487/RFC6437, November 2011, 398 . 400 10.2. Informative References 402 [I-D.song-ippm-postcard-based-telemetry] 403 Song, H., Mirsky, G., Filsfils, C., Abdelsalam, A., Zhou, 404 T., Li, Z., Shin, J., and K. Lee, "In-Situ OAM Marking- 405 based Direct Export", Work in Progress, Internet-Draft, 406 draft-song-ippm-postcard-based-telemetry-11, 15 November 407 2021, . 410 [RFC8085] Eggert, L., Fairhurst, G., and G. Shepherd, "UDP Usage 411 Guidelines", BCP 145, RFC 8085, DOI 10.17487/RFC8085, 412 March 2017, . 414 [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 415 (IPv6) Specification", STD 86, RFC 8200, 416 DOI 10.17487/RFC8200, July 2017, 417 . 419 [RFC8321] Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli, 420 L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi, 421 "Alternate-Marking Method for Passive and Hybrid 422 Performance Monitoring", RFC 8321, DOI 10.17487/RFC8321, 423 January 2018, . 425 Appendix A. Entropy 427 Section 5.1.1 of [RFC8085] discusses the usage of UDP for Source Port 428 Entropy and states that 14 bits of Entropy are sufficient for most 429 ECMP applications: 431 * In IPv6 UDP tunnel, the BCP suggests the usage of UDP source port 432 for ECMP hash calculation. 434 * A sending tunnel endpoint selects a source port value in the UDP 435 header that is computed from the inner packet information. 437 * To provide sufficient entropy, the sending tunnel endpoint maps 438 the encapsulated traffic to one of a range of UDP source values. 440 * The value SHOULD be within the ephemeral port range, i.e., 49152 441 to 65535, where the high order two bits of the port are set to 442 one. 444 * The available source port entropy of 14 bits (using the ephemeral 445 port range) plus the outer IP addresses seems sufficient for 446 entropy for most ECMP applications. 448 Authors' Addresses 450 Clarence Filsfils (editor) 451 Cisco Systems, Inc. 452 Belgium 453 Email: cf@cisco.com 455 Ahmed Abdelsalam (editor) 456 Cisco Systems, Inc. 457 Italy 458 Email: ahabdels@cisco.com 460 Shay Zadok 461 Broadcom 462 Israel 463 Email: shay.zadok@broadcom.com 465 Xiaohu Xu 466 Capitalonline 467 China 468 Email: Xiaohu.xu@capitalonline.net 469 Weiqiang Cheng 470 China Mobile 471 China 472 Email: chengweiqiang@chinamobile.com 474 Daniel Voyer 475 Bell Canada 476 Canada 477 Email: daniel.voyer@bell.ca 479 Pablo Camarillo Garvia 480 Cisco Systems, Inc. 481 Spain 482 Email: pcamaril@cisco.com