idnits 2.17.00 (12 Aug 2021) /tmp/idnits56024/draft-ietf-ippm-rfc8321bis-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC8321]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document date (April 28, 2022) is 16 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'IEEE-1588' Summary: 1 error (**), 0 flaws (~~), 0 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group G. Fioccola, Ed. 3 Internet-Draft Huawei Technologies 4 Obsoletes: 8321 (if approved) M. Cociglio 5 Intended status: Standards Track Telecom Italia 6 Expires: October 30, 2022 G. Mirsky 7 Ericsson 8 T. Mizrahi 9 T. Zhou 10 Huawei Technologies 11 April 28, 2022 13 Alternate-Marking Method 14 draft-ietf-ippm-rfc8321bis-01 16 Abstract 18 This document describes the Alternate-Marking technique to perform 19 packet loss, delay, and jitter measurements on live traffic. This 20 technology can be applied in various situations and for different 21 protocols. It could be considered Passive or Hybrid depending on the 22 application. This document obsoletes [RFC8321]. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on October 30, 2022. 41 Copyright Notice 43 Copyright (c) 2022 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (https://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 59 1.1. Summary of Changes from RFC 8321 . . . . . . . . . . . . 3 60 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 4 61 2. Overview of the Method . . . . . . . . . . . . . . . . . . . 4 62 3. Detailed Description of the Method . . . . . . . . . . . . . 5 63 3.1. Packet Loss Measurement . . . . . . . . . . . . . . . . . 6 64 3.2. One-Way Delay Measurement . . . . . . . . . . . . . . . . 9 65 3.2.1. Single-Marking Methodology . . . . . . . . . . . . . 9 66 3.2.2. Double-Marking Methodology . . . . . . . . . . . . . 10 67 3.3. Delay Variation Measurement . . . . . . . . . . . . . . . 11 68 4. Alternate Marking Functions . . . . . . . . . . . . . . . . . 12 69 4.1. Marking the Packets . . . . . . . . . . . . . . . . . . . 12 70 4.2. Counting and Timestamping Packets . . . . . . . . . . . . 13 71 4.3. Data Collection and Correlation . . . . . . . . . . . . . 13 72 5. Synchronization and Timing . . . . . . . . . . . . . . . . . 14 73 6. Packet Fragmentation . . . . . . . . . . . . . . . . . . . . 16 74 7. Results of the Alternate Marking Experiment . . . . . . . . . 17 75 7.1. Controlled Domain requirement . . . . . . . . . . . . . . 18 76 8. Compliance with Guidelines from RFC 6390 . . . . . . . . . . 19 77 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 78 10. Security Considerations . . . . . . . . . . . . . . . . . . . 20 79 11. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 22 80 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 22 81 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 82 13.1. Normative References . . . . . . . . . . . . . . . . . . 23 83 13.2. Informative References . . . . . . . . . . . . . . . . . 23 84 Appendix A. Changes Log . . . . . . . . . . . . . . . . . . . . 25 85 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27 87 1. Introduction 89 Most Service Providers' networks carry traffic with contents that are 90 highly sensitive to packet loss [RFC7680], delay [RFC7679], and 91 jitter [RFC3393]. 93 Service Providers need methodologies and tools to monitor and 94 accurately measure network performance, in order to constantly 95 control the quality of experience perceived by their customers. 96 Performance monitoring also provides useful information for improving 97 network management (e.g., isolation of network problems, 98 troubleshooting, etc.). 100 RFC 7799 [RFC7799] defines Passive and Hybrid Methods of Measurement. 101 In particular, Passive Methods of Measurement are based solely on 102 observations of an undisturbed and unmodified packet stream of 103 interest; Hybrid Methods are Methods of Measurement that use a 104 combination of Active Methods and Passive Methods. 106 [RFC7276] provides a good overview of existing Operations, 107 Administration, and Maintenance (OAM) mechanisms defined in the IETF, 108 ITU-T, and IEEE. In the IETF, a lot of work has been done on fault 109 detection and connectivity verification, while little has been thus 110 far dedicated to performance monitoring. The IETF has defined 111 standard metrics to measure network performance; however, its methods 112 mainly focus on Active measurement techniques.For example, [RFC6374] 113 defines mechanisms for measuring packet loss, one-way and two-way 114 delay, and delay variation in MPLS networks, but its applicability to 115 Passive measurements has some limitations, especially for connection- 116 less networks. 118 This document proposes a Passive performance monitoring technique, 119 potentially applicable to any kind of packet-based traffic, including 120 Ethernet, IP, and MPLS, both unicast and multicast. The method 121 addresses primarily packet loss measurement, but it can be easily 122 extended to one-way or two-way delay and delay variation measurements 123 as well. 125 The method has been explicitly designed for Passive measurements, but 126 it can also be used with Active probes. Passive measurements are 127 usually more easily understood by customers and provide much better 128 accuracy, especially for packet loss measurements. 130 Therefore, the Alternate-Marking Method could be considered Hybrid or 131 Passive, depending on the case. In the case where the marking method 132 is obtained by changing existing field values of the packets the 133 technique is Hybrid. In the case where the marking field is 134 dedicated, reserved, and included in the protocol specification, the 135 Alternate-Marking technique can be considered as Passive. 137 1.1. Summary of Changes from RFC 8321 139 This document defines the Alternate-Marking Method, addressing 140 ambiguities and overtaking its experimental phase in the original 141 specification [RFC8321]. 143 The relevant changes are: 145 o Added the recommendations about the methods to employ in case one 146 or two flag bits are available for marking (Section 7). 148 o Changed the structure to improve the readability. 150 o Removed the wording about the initial experiments of the method 151 and considerations that no longer apply. 153 o Extended the description of detailed aspects of the methodology, 154 e.g. synchronization, timing, packet fragmentation, marked and 155 unmarked traffic handling. 157 It is important to note that all the changes are totally backward 158 compatible with [RFC8321] and no new additional technique has been 159 introduced in this document compared to [RFC8321]. 161 1.2. Requirements Language 163 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 164 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 165 "OPTIONAL" in this document are to be interpreted as described in BCP 166 14 [RFC2119] [RFC8174] when, and only when, they appear in all 167 capitals, as shown here. 169 2. Overview of the Method 171 In order to perform packet loss measurements on a production traffic 172 flow, different approaches exist. The most intuitive one consists in 173 numbering the packets so that each router that receives the flow can 174 immediately detect a packet that is missing. This approach, though 175 very simple in theory, is not simple to achieve: it requires the 176 insertion of a sequence number into each packet, and the devices must 177 be able to extract the number and check it in real time. Such a task 178 can be difficult to implement on live traffic: if UDP is used as the 179 transport protocol, the sequence number is not available; on the 180 other hand, if a higher-layer sequence number (e.g., in the RTP 181 header) is used, extracting that information from each packet and 182 processing it in real time could overload the device. 184 An alternate approach is to count the number of packets sent on one 185 end, count the number of packets received on the other end, and 186 compare the two values. This operation is much simpler to implement, 187 but it requires the devices performing the measurement to be in sync: 188 in order to compare two counters, it is required that they refer 189 exactly to the same set of packets. Since a flow is continuous and 190 cannot be stopped when a counter has to be read, it can be difficult 191 to determine exactly when to read the counter. A possible solution 192 to overcome this problem is to virtually split the flow in 193 consecutive blocks by periodically inserting a delimiter so that each 194 counter refers exactly to the same block of packets. The delimiter 195 could be, for example, a special packet inserted artificially into 196 the flow. However, delimiting the flow using specific packets has 197 some limitations. First, it requires generating additional packets 198 within the flow and requires the equipment to be able to process 199 those packets. In addition, the method is vulnerable to out-of-order 200 reception of delimiting packets and, to a lesser extent, to their 201 loss. 203 The method proposed in this document follows the second approach, but 204 it doesn't use additional packets to virtually split the flow in 205 blocks. Instead, it "marks" the packets so that the packets 206 belonging to the same block will have the same color, whilst 207 consecutive blocks will have different colors. Each change of color 208 represents a sort of auto-synchronization signal that guarantees the 209 consistency of measurements taken by different devices along the 210 path. 212 Figure 1 represents a very simple network and shows how the method 213 can be used to measure packet loss on different network segments: by 214 enabling the measurement on several interfaces along the path, it is 215 possible to perform link monitoring, node monitoring, or end-to-end 216 monitoring. The method is flexible enough to measure packet loss on 217 any segment of the network and can be used to isolate the faulty 218 element. 220 Traffic Flow 221 ========================================================> 222 +------+ +------+ +------+ +------+ 223 ---<> R1 <>-----<> R2 <>-----<> R3 <>-----<> R4 <>--- 224 +------+ +------+ +------+ +------+ 225 . . . . . . 226 . . . . . . 227 . <------> <-------> . 228 . Node Packet Loss Link Packet Loss . 229 . . 230 <---------------------------------------------------> 231 End-to-End Packet Loss 233 Figure 1: Available Measurements 235 3. Detailed Description of the Method 237 This section describes, in detail, how the method operates. A 238 special emphasis is given to the measurement of packet loss, which 239 represents the core application of the method, but applicability to 240 delay and jitter measurements is also considered. 242 3.1. Packet Loss Measurement 244 The basic idea is to virtually split traffic flows into consecutive 245 blocks: each block represents a measurable entity unambiguously 246 recognizable by all network devices along the path. By counting the 247 number of packets in each block and comparing the values measured by 248 different network devices along the path, it is possible to measure 249 if packet loss occurred in any single block between any two points. 251 As discussed in the previous section, a simple way to create the 252 blocks is to "color" the traffic (two colors are sufficient), so that 253 packets belonging to different consecutive blocks will have different 254 colors. Whenever the color changes, the previous block terminates 255 and the new one begins. Hence, all the packets belonging to the same 256 block will have the same color and packets of different consecutive 257 blocks will have different colors. The number of packets in each 258 block depends on the criterion used to create the blocks: 260 o if the color is switched after a fixed number of packets, then 261 each block will contain the same number of packets (except for any 262 losses); and 264 o if the color is switched according to a fixed timer, then the 265 number of packets may be different in each block depending on the 266 packet rate. 268 The rest of the document assumes that the blocks are created 269 according to a fixed timer. The switching after a fixed number of 270 packets is an additional possibility but its detailed specification 271 is out of scope. 273 The following figure shows how a flow looks like when it is split in 274 traffic blocks with colored packets. 276 A: packet with A coloring 277 B: packet with B coloring 279 | | | | | 280 | | Traffic Flow | | 281 -------------------------------------------------------------------> 282 BBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA 283 -------------------------------------------------------------------> 284 ... | Block 5 | Block 4 | Block 3 | Block 2 | Block 1 285 | | | | | 287 Figure 2: Traffic Coloring 289 Figure 3 shows how the method can be used to measure link packet loss 290 between two adjacent nodes. 292 Referring to the figure, let's assume we want to monitor the packet 293 loss on the link between two routers: router R1 and router R2. 294 According to the method, the traffic is colored alternatively with 295 two different colors: A and B. Whenever the color changes, the 296 transition generates a sort of square-wave signal, as depicted in the 297 following figure. 299 Color A ----------+ +-----------+ +---------- 300 | | | | 301 Color B +-----------+ +-----------+ 302 Block n ... Block 3 Block 2 Block 1 303 <---------> <---------> <---------> <---------> <---------> 305 Traffic Flow 306 ===========================================================> 307 Color ...AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA... 308 ===========================================================> 310 Figure 3: Computation of Link Packet Loss 312 Traffic coloring can be done by R1 itself if the traffic is not 313 already colored. R1 needs two counters, C(A)R1 and C(B)R1, on its 314 egress interface: C(A)R1 counts the packets with color A and C(B)R1 315 counts those with color B. As long as traffic is colored as A, only 316 counter C(A)R1 will be incremented, while C(B)R1 is not incremented; 317 conversely, when the traffic is colored as B, only C(B)R1 is 318 incremented. C(A)R1 and C(B)R1 can be used as reference values to 319 determine the packet loss from R1 to any other measurement point down 320 the path. Router R2, similarly, will need two counters on its 321 ingress interface, C(A)R2 and C(B)R2, to count the packets received 322 on that interface and colored with A and B, respectively. When an A 323 block ends, it is possible to compare C(A)R1 and C(A)R2 and calculate 324 the packet loss within the block; similarly, when the successive B 325 block terminates, it is possible to compare C(B)R1 with C(B)R2, and 326 so on, for every successive block. 328 Likewise, by using two counters on the R2 egress interface, it is 329 possible to count the packets sent out of the R2 interface and use 330 them as reference values to calculate the packet loss from R2 to any 331 measurement point downstream from R2. 333 Using a fixed timer for color switching offers better control over 334 the method: the (time) length of the blocks can be chosen large 335 enough to simplify the collection and the comparison of measures 336 taken by different network devices. It's preferable to read the 337 value of the counters not immediately after the color switch: some 338 packets could arrive out of order and increment the counter 339 associated with the previous block (color), so it is worth waiting 340 for some time. A safe choice is to wait L/2 time units (where L is 341 the duration for each block) after the color switch, to read the 342 counter of the previous color. The drawback is that the longer the 343 duration of the block, the less frequently the measurement can be 344 taken. 346 Two different strategies that can be used when implementing the 347 method: 349 o flow-based: the flow-based strategy is used when only a limited 350 number of traffic flows need to be monitored. According to this 351 strategy, only a subset of the flows is colored. Counters for 352 packet loss measurements can be instantiated for each single flow, 353 or for the set as a whole, depending on the desired granularity. 354 A relevant problem with this approach is the necessity to know in 355 advance the path followed by flows that are subject to 356 measurement. Path rerouting and traffic load-balancing increase 357 the issue complexity, especially for unicast traffic. The problem 358 is easier to solve for multicast traffic, where load-balancing is 359 seldom used and static joins are frequently used to force traffic 360 forwarding and replication. 362 o link-based: measurements are performed on all the traffic on a 363 link-by-link basis. The link could be a physical link or a 364 logical link. Counters could be instantiated for the traffic as a 365 whole or for each traffic class (in case it is desired to monitor 366 each class separately), but in the second case, two counters are 367 needed for each class. 369 As mentioned, the flow-based measurement requires the identification 370 of the flow to be monitored and the discovery of the path followed by 371 the selected flow. It is possible to monitor a single flow or 372 multiple flows grouped together, but in this case, measurement is 373 consistent only if all the flows in the group follow the same path. 374 Moreover, if a measurement is performed by grouping many flows, it is 375 not possible to determine exactly which flow was affected by packet 376 loss. In order to have measures per single flow, it is necessary to 377 configure counters for each specific flow. Once the flow(s) to be 378 monitored has been identified, it is necessary to configure the 379 monitoring on the proper nodes. Configuring the monitoring means 380 configuring the rule to intercept the traffic and configuring the 381 counters to count the packets. To have just an end-to-end 382 monitoring, it is sufficient to enable the monitoring on the first- 383 and last-hop routers of the path: the mechanism is completely 384 transparent to intermediate nodes and independent from the path 385 followed by traffic flows. On the contrary, to monitor the flow on a 386 hop-by-hop basis along its whole path, it is necessary to enable the 387 monitoring on every node from the source to the destination. In case 388 the exact path followed by the flow is not known a priori (i.e., the 389 flow has multiple paths to reach the destination), it is necessary to 390 enable the monitoring system on every path: counters on interfaces 391 traversed by the flow will report packet count, whereas counters on 392 other interfaces will be null. 394 3.2. One-Way Delay Measurement 396 The same principle used to measure packet loss can be applied also to 397 one-way delay measurement. There are three alternatives, as 398 described hereinafter. 400 Note that, for all the one-way delay alternatives described in the 401 next sections, by summing the one-way delays of the two directions of 402 a path, it is always possible to measure the two-way delay (round- 403 trip "virtual" delay). 405 3.2.1. Single-Marking Methodology 407 The alternation of colors can be used as a time reference to 408 calculate the delay. Whenever the color changes (which means that a 409 new block has started), a network device can store the timestamp of 410 the first packet of the new block; that timestamp can be compared 411 with the timestamp of the same packet on a second router to compute 412 packet delay. When looking at Figure 2, R1 stores the timestamp 413 TS(A1)R1 when it sends the first packet of block 1 (A-colored), the 414 timestamp TS(B2)R1 when it sends the first packet of block 2 415 (B-colored), and so on for every other block. R2 performs the same 416 operation on the receiving side, recording TS(A1)R2, TS(B2)R2, and so 417 on. Since the timestamps refer to specific packets (the first packet 418 of each block), we are sure that timestamps compared to compute delay 419 refer to the same packets. By comparing TS(A1)R1 with TS(A1)R2 (and 420 similarly TS(B2)R1 with TS(B2)R2, and so on), it is possible to 421 measure the delay between R1 and R2. In order to have more 422 measurements, it is possible to take and store more timestamps, 423 referring to other packets within each block. The number of 424 measurements could be increased by considering multiple packets in 425 the block: for instance, a timestamp could be taken every N packets, 426 thus generating multiple delay measurements. Taking this to the 427 limit, in principle, the delay could be measured for each packet by 428 taking and comparing the corresponding timestamps (possible but 429 impractical from an implementation point of view). 431 In order to coherently compare timestamps collected on different 432 routers, the clocks on the network nodes must be in sync. 433 Furthermore, a measurement is valid only if no packet loss occurs and 434 if packet misordering can be avoided; otherwise, the first packet of 435 a block on R1 could be different from the first packet of the same 436 block on R2 (for instance, if that packet is lost between R1 and R2 437 or it arrives after the next one). Since packet misordering is 438 generally undetectable it is not possible to check whether the first 439 packet on R1 is the same on R2 and this is part of the intrinsic 440 error in this measurement. 442 3.2.1.1. Mean Delay 444 The method previously exposed for measuring the delay is sensitive to 445 out-of-order reception of packets. In order to overcome this 446 problem, an approach based on the concept of mean delay can be 447 considered. The mean delay is calculated by considering the average 448 arrival time of the packets within a single block. The network 449 device locally stores a timestamp for each packet received within a 450 single block: summing all the timestamps and dividing by the total 451 number of packets received, the average arrival time for that block 452 of packets can be calculated. By subtracting the average arrival 453 times of two adjacent devices, it is possible to calculate the mean 454 delay between those nodes. This method greatly reduces the number of 455 timestamps that have to be collected (only one per block for each 456 network device) and it is robust to out-of-order packets with only a 457 small error introduced in case of packet loss. But, when computing 458 the mean delay, the measurement error could be augmented by 459 accumulating the measurement error of a lot of packets. 460 Additionally, it only gives one measure for the duration of the 461 block, and it doesn't give the minimum, maximum, and median delay 462 values [RFC6703]. This limitation could be overcome by reducing the 463 duration of the block (for instance, from minutes to seconds), which 464 implies a highly optimized implementation of the method. For this 465 reason, the mean delay calculation may not be so viable in some 466 cases. 468 3.2.2. Double-Marking Methodology 470 As mentioned above, the Single-Marking methodology for one-way delay 471 measurement has some limitations, since it is sensitive to out-of- 472 order reception of packets and even the mean delay calculation is 473 limited because it doesn't give information about the delay value's 474 distribution for the duration of the block. Actually, it may be 475 useful to have not only the mean delay but also the minimum, maximum, 476 and median delay values and, in wider terms, to know more about the 477 statistic distribution of delay values. So, in order to have more 478 information about the delay and to overcome out-of-order issues, a 479 different approach can be introduced and it is based on a Double- 480 Marking methodology. 482 Basically, the idea is to use the first marking to create the 483 alternate flow and, within this colored flow, a second marking to 484 select the packets for measuring delay/jitter. The first marking is 485 needed for packet loss and may be used for mean delay measurement. 486 The second marking creates a new set of marked packets that are fully 487 identified over the network, so that a network device can store the 488 timestamps of these packets; these timestamps can be compared with 489 the timestamps of the same packets on a second router to compute 490 packet delay values for each packet. The number of measurements can 491 be easily increased by changing the frequency of the second marking. 492 But the frequency of the second marking must not be too high in order 493 to avoid out-of-order issues. Between packets with the second 494 marking, there should be a security time gap (e.g., this gap could 495 be, at the minimum, the mean network delay calculated with the 496 previous methodology) to avoid out-of-order issues and also to have a 497 number of measurement packets that are rate independent. If a 498 second-marking packet is lost, the delay measurement for the 499 considered block is corrupted and should be discarded. 501 An efficient and robust mode is to select a single packet with the 502 second marking for each block, in this way there is no time gap to 503 consider between the double-marked packets to avoid their reorder. 505 The Double-Marking methodology can also be used to get more 506 statistics of delay extent data, e.g., percentiles, variance, and 507 median delay values. Indeed, a subset of batch packets is selected 508 for extensive delay calculation by using the second marking and it is 509 possible to perform a detailed analysis on these double-marked 510 packets. It is worth noting that there are classic algorithms for 511 median and variance calculation, but they are out of the scope of 512 this document. The conventional range (maximum-minimum) should be 513 avoided for several reasons, including stability of the maximum delay 514 due to the influence by outliers. In this regard, RFC 5481 515 [RFC5481], Section 6.5 highlights how the 99.9th percentile of delay 516 and delay variation is more helpful to performance planners. 518 3.3. Delay Variation Measurement 520 Similar to one-way delay measurement (both for Single Marking and 521 Double Marking), the method can also be used to measure the inter- 522 arrival jitter. We refer to the definition in RFC 3393 [RFC3393]. 523 The alternation of colors, for a Single-Marking Method, can be used 524 as a time reference to measure delay variations. In case of Double 525 Marking, the time reference is given by the second-marked packets. 526 Considering the example depicted in Figure 2, R1 stores the timestamp 527 TS(A)R1 whenever it sends the first packet of a block, and R2 stores 528 the timestamp TS(B)R2 whenever it receives the first packet of a 529 block. The inter-arrival jitter can be easily derived from one-way 530 delay measurement, by evaluating the delay variation of consecutive 531 samples. 533 The concept of mean delay can also be applied to delay variation, by 534 evaluating the average variation of the interval between consecutive 535 packets of the flow from R1 to R2. 537 4. Alternate Marking Functions 539 4.1. Marking the Packets 541 The coloring operation is fundamental in order to create packet 542 blocks and marked packets. This implies choosing where to activate 543 the coloring and how to color the packets. 545 In case of flow-based measurements, the flow to monitor can be 546 defined by a set of selection rules (e.g., header fields) used to 547 match a subset of the packets; in this way, it is possible to control 548 the number of involved nodes, the path followed by the packets, and 549 the size of the flows. It is possible, in general, to have multiple 550 coloring nodes or a single coloring node that is easier to manage and 551 doesn't raise any risk of conflict. Coloring in multiple nodes can 552 be done, and the requirement is that the coloring must change 553 periodically between the nodes according to the timing considerations 554 in Section 5; so every node that is designated as a measurement point 555 along the path should be able to identify unambiguously the colored 556 packets. Furthermore, [I-D.fioccola-rfc8889bis] generalizes the 557 coloring for multipoint-to-multipoint flow. In addition, it can be 558 advantageous to color the flow as close as possible to the source 559 because it allows an end-to-end measure if a measurement point is 560 enabled on the last-hop router as well. 562 For link-based measurements, all traffic needs to be colored when 563 transmitted on the link. If the traffic had already been colored, 564 then it has to be re-colored because the color must be consistent on 565 the link. This means that each hop along the path must (re-)color 566 the traffic; the color is not required to be consistent along 567 different links. 569 Traffic coloring can be implemented by setting specific flags in the 570 packet header and changing the value of that bit periodically. How 571 to choose the marking field depends on the application and is out of 572 scope here. 574 4.2. Counting and Timestamping Packets 576 For flow-based measurements, assuming that the coloring of the 577 packets is performed only by the source nodes, the nodes between 578 source and destination (included) have to count and timestamp the 579 colored packets that they receive and forward: this operation can be 580 enabled on every router along the path or only on a subset, depending 581 on which network segment is being monitored (a single link, a 582 particular metro area, the backbone, or the whole path). Since the 583 color switches periodically between two values, two counters (one for 584 each value) are needed for each flow and for every interface being 585 monitored. The number of timestamps to be stored depends on the 586 method for delay measurement that is applied. Furthermore, 587 [I-D.fioccola-rfc8889bis] generalizes the counting for multipoint-to- 588 multipoint flow. 590 In case of link-based measurements, the behavior is similar except 591 that coloring, counting and timestamping operations are performed on 592 a link-by-link basis at each endpoint of the link. 594 Another important aspect to take into consideration is when to read 595 the counters or when to select the packets to be double-marked for 596 delay measurement. It involves timing aspects to consider that are 597 further described in Section 5. 599 4.3. Data Collection and Correlation 601 The nodes enabled to perform performance monitoring collect the value 602 of the counters and timestamps, but they are not able to directly use 603 this information to measure packet loss and delay, because they only 604 have their own samples. 606 Data collection enables the transmission of the counters and 607 timestamps as soon as it has been read. While, data correlation is 608 the mechanism to compare counters and timestamps for packet loss, 609 delay, and delay variation calculation. 611 There are two main possibilities to perform both data collection and 612 correlation depending on the Alternate-Marking application and use 613 case: 615 o Use of a centralized solution using Network Management System 616 (NMS) to correlate data. This can be done in Push Mode or Polling 617 Mode. In the first case, each router periodically sends the 618 information to the NMS; in the latter case, it is the NMS that 619 periodically polls routers to collect information. In any case, 620 the NMS has to collect all the relevant values from all the 621 routers within one cycle of the timer. 623 o Definition of a protocol-based distributed solution to exchange 624 values of counters and timestamps between the endpoints. This can 625 be done by introducing a new protocol or by extending the existing 626 protocols (e.g., the Two-Way Active Measurement Protocol (TWAMP) 627 as defined in RFC 5357 [RFC5357] or the One-Way Active Measurement 628 Protocol (OWAMP) as defined in RFC 4656 [RFC4656]) in order to 629 communicate the counters and timestamps between nodes. 631 In the following paragraphs, an example data correlation mechanism is 632 explained and could be used independently of the adopted solutions. 634 When data is collected on the upstream and downstream nodes, e.g., 635 packet counts for packet loss measurement or timestamps for packet 636 delay measurement, and is periodically reported to or pulled by other 637 nodes or an NMS, a certain data correlation mechanism SHOULD be in 638 use to help the nodes or NMS tell whether any two or more packet 639 counts are related to the same block of markers or if any two 640 timestamps are related to the same marked packet. 642 The Alternate-Marking Method described in this document literally 643 splits the packets of the measured flow into different measurement 644 blocks. An implementation MAY use a Block Number (BN) for data 645 correlation. The BN MAY be assigned to each measurement block and 646 associated with each packet count and timestamp reported to or pulled 647 by other nodes or NMSs. When the nodes or NMS see, for example, the 648 same BNs associated with two packet counts from an upstream and a 649 downstream node, respectively, it considers that these two packet 650 counts correspond to the same block. The assumption of this BN 651 mechanism is that the measurement nodes are time synchronized. This 652 requires the measurement nodes to have a certain time synchronization 653 capability (e.g., the Network Time Protocol (NTP) [RFC5905] or the 654 IEEE 1588 Precision Time Protocol (PTP) [IEEE-1588]). 656 5. Synchronization and Timing 658 This document introduces two color-switching methods: one is based on 659 a fixed number of packets, and the other is based on a fixed timer. 660 But the method based on a fixed timer is preferable because it is 661 more deterministic, and it is considered in the document. 663 Color switching is the reference for all the network devices, and the 664 only requirement to be achieved is that all network devices have to 665 recognize the right batch along the path. 667 In general, clocks in network devices are not accurate and for this 668 reason, there is a clock error between the measurement points R1 and 669 R2. And, to implement the methodology, they must be synchronized to 670 the same clock reference with an adequate accuracy in order to 671 guarantee that all network devices consistently match the marking bit 672 to the correct block. Additionally, in practice, besides clock 673 errors, packet reordering is also very common in a packet network due 674 to equal-cost multipath (ECMP). In particular, the delay between 675 measurement points is the main cause of out of order because each 676 packet can be delayed differently. If the block is sufficiently 677 large, packet reordering occurs only at the edge of adjacent blocks 678 and it can be easy to assign reordered packets to the right interval 679 blocks. 681 In summary, we need to take into account two contributions: clock 682 error between network devices and the interval we need to wait to 683 avoid packets being out of order because of network delay. 685 The following figure explains both issues. 687 ...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB... 688 |<======================================>| 689 | L | 690 ...=========>|<==================><==================>|<==========... 691 | L/2 L/2 | 692 |<===>| |<===>| 693 d | | d 694 |<==========================>| 695 available counting interval 697 Figure 4: Timing Aspects 699 Where L is the time duration of each block. 701 It is assumed that all network devices are synchronized to a common 702 reference time with an accuracy of +/- A/2. Thus, the difference 703 between the clock values of any two network devices is bounded by A. 705 The network delay between the network devices can be represented as a 706 data set and 99.7% of the samples are within 3 standard deviation of 707 the average. 709 The guard band d is given by: 711 d = A + D_avg + 3*D_stddev, 713 where A is the clock accuracy, D_avg is the average value of the 714 network delay between the network devices, and D_stddev is the 715 standard deviation of the delay. 717 The available counting interval is L - 2d that must be > 0. 719 The condition that must be satisfied and is a requirement on the 720 synchronization accuracy is: 722 d < L/2. 724 This is the fundamental rule for deciding when to read the counters 725 and when to select the packets to be double-marked, indeed packet 726 counter and double-marked packets MUST respectively be taken and 727 chosen within the available counting interval that is not affected by 728 error factors. 730 It is worth mentioning that, if the time duration L of each block is 731 not so small, the synchronization requirement could be satisfied even 732 with a relatively inaccurate synchronization method. This is true 733 for packet loss and two-way delay measurement, but not for one-way 734 delay measurement, where clock synchronization must be accurate. 735 Therefore, a system that uses only packet loss and two-way delay 736 measurement may not require a very precise synchronization. This is 737 because the value of the clocks of network devices does not affect 738 the computation of the two-way delay measurement. 740 6. Packet Fragmentation 742 Fragmentation can be managed with the Alternate-Marking Method and in 743 particular it is possible to give the following guidance: 745 Marking nodes MUST mark all fragments if there are flag bits to 746 use (i.e. it is in the specific encapsulation), as if they were 747 separate packets. 749 Nodes that fragment packets within the measurement domain SHOULD, 750 if they have the capability to do so, ensure that only one 751 resulting fragment carries the marking bit(s) of the original 752 packet. Failure to do so can introduce errors into the 753 measurement. 755 Measurement points MAY simply ignore unmarked fragments and count 756 marked fragments as full packets. However, if resources allow, 757 measurement points MAY make note of both marked and unmarked 758 initial fragments and only increment the corresponding counter if 759 (a) other fragments are also marked, or (b) it observes all other 760 fragments and they are unmarked. 762 The proposed approach allows the marking node to mark all the 763 fragments except in the case of fragmentation within the network 764 domain, in that event it is suggested to mark only the first 765 fragment. 767 7. Results of the Alternate Marking Experiment 769 The methodology described in the previous sections can be applied to 770 various performance measurement problems, as explained in [RFC8321]. 771 The only requirement is to select and mark the flow to be monitored; 772 in this way, packets are batched by the sender, and each batch is 773 alternately marked such that it can be easily recognized by the 774 receiver. 776 Either one or two flag bits might be available for marking in 777 different deployments: 779 One flag: packet loss measurement SHOULD be done as described in 780 Section 3.1, while delay measurement MAY be done according to the 781 single-marking method described in Section 3.2.1. Mean delay 782 (Section 3.2.1.1) is NOT RECOMMENDED since it implies more 783 computational load. 785 Two flags: packet loss measurement SHOULD be done as described in 786 Section 3.1, while delay measurement SHOULD be done according to 787 double-marking method Section 3.2.2. In this case single-marking 788 MAY also be used in combination with double-marking and the two 789 approaches provide slightly different pieces of information that 790 can be combined to have a more robust data set. 792 The experiment with Alternate Marking methodologies confirmed the 793 following benefits: 795 o easy implementation: it can be implemented by using features 796 already available on major routing platforms, or by applying an 797 optimized implementation of the method for both legacy and newest 798 technologies; 800 o low computational effort: the additional load on processing is 801 negligible; 803 o accurate loss and delay measurements: single packet loss 804 granularity is achieved with a Passive measurement; 806 o potential applicability to any kind of packet-based or frame-based 807 traffic: Ethernet, IP, MPLS, etc., and both unicast and multicast; 809 o robustness: the method can easily tolerate out-of-order packets, 810 and it's not based on "special" packets whose loss could have a 811 negative impact; 813 o flexibility: all the timestamp formats are allowed, because they 814 are managed out of band. The format (the Network Time Protocol 815 (NTP) [RFC5905] or the IEEE 1588 Precision Time Protocol (PTP) 816 [IEEE-1588]) depends on the precision you want; and 818 o no interoperability issues: the features required are available on 819 all current routing platforms. Both a centralized or distributed 820 solution can be used to harvest data from the routers. 822 A deployment of the Alternate-Marking Method SHOULD also take into 823 account how to handle and recognize marked and unmarked traffic 824 depending on whether the technique is applied as Hybrid or Passive. 825 In the case where the marking method is applied by changing existing 826 fields of the packets, it is RECOMMENDED to use an additional flag or 827 some out-of-band signaling to indicate if the measurement is 828 activated or not in order to inform the measurement points. While, 829 in the case where the marking field is dedicated, reserved, and 830 included in a protocol extension, the measurement points can learn 831 whether the measurement is activated or not by checking if the 832 specific extension is included or not within the packets. 834 It is worth mentioning some related work: in particular 835 [IEEE-Network-PNPM] explains the Alternate-Marking method together 836 with new mechanisms based on hashing techniques as also further 837 described in [I-D.mizrahi-ippm-marking]; while 838 [I-D.zhou-ippm-enhanced-alternate-marking] extends the Alternate- 839 Marking Data Fields, to provide enhanced capabilities and allow 840 advanced functionalities. 842 7.1. Controlled Domain requirement 844 The Alternate Marking Method is an example of a solution limited to a 845 controlled domain [RFC8799]. 847 A controlled domain is a managed network that selects, monitors, and 848 controls access by enforcing policies at the domain boundaries, in 849 order to discard undesired external packets entering the domain and 850 check internal packets leaving the domain. It does not necessarily 851 mean that a controlled domain is a single administrative domain or a 852 single organization. A controlled domain can correspond to a single 853 administrative domain or multiple administrative domains under a 854 defined network management. It must be possible to control the 855 domain boundaries, and use specific precautions if traffic traverses 856 the Internet. 858 For security reasons, the Alternate Marking Method is RECOMMENDED 859 only for controlled domains. 861 8. Compliance with Guidelines from RFC 6390 863 RFC 6390 [RFC6390] defines a framework and a process for developing 864 Performance Metrics for protocols above and below the IP layer (such 865 as IP-based applications that operate over reliable or datagram 866 transport protocols). 868 This document doesn't aim to propose a new Performance Metric but 869 rather a new Method of Measurement for a few Performance Metrics that 870 have already been standardized. Nevertheless, it's worth applying 871 guidelines from [RFC6390] to the present document, in order to 872 provide a more complete and coherent description of the proposed 873 method. We used a combination of the Performance Metric Definition 874 template defined in Section 5.4 of [RFC6390] and the Dependencies 875 laid out in Section 5.5 of that document. 877 o Metric Name / Metric Description: as already stated, this document 878 doesn't propose any new Performance Metrics. On the contrary, it 879 describes a novel method for measuring packet loss [RFC7680]. The 880 same concept, with small differences, can also be used to measure 881 delay [RFC7679] and jitter [RFC3393]. The document mainly 882 describes the applicability to packet loss measurement. 884 o Method of Measurement or Calculation: according to the method 885 described in the previous sections, the number of packets lost is 886 calculated by subtracting the value of the counter on the source 887 node from the value of the counter on the destination node. Both 888 counters must refer to the same color. The calculation is 889 performed when the value of the counters is in a steady state. 890 The steady state is an intrinsic characteristic of the marking 891 method counters because the alternation of color makes the counter 892 associated with a color inactive for the duration of a marking 893 period. 895 o Units of Measurement: the method calculates and reports the exact 896 number of packets sent by the source node and not received by the 897 destination node. 899 o Measurement Point(s) with Potential Measurement Domain: the 900 measurement can be performed between adjacent nodes, on a per-link 901 basis, or along a multi-hop path, provided that the traffic under 902 measurement follows that path. In case of a multi-hop path, the 903 measurements can be performed both end-to-end and hop-by-hop. 905 o Measurement Timing: the method has a constraint on the frequency 906 of measurements. This is detailed in Section 5, where it is 907 specified that the marking period and the guard band interval are 908 strictly related each other to avoid out-of-order issues. That is 909 because, in order to perform a measurement, the counter must be in 910 a steady state, and this happens when the traffic is being colored 911 with the alternate color. 913 o Implementation: the method uses one or two marking bits to color 914 the packets; this enables the use of policy configurations on the 915 router to color the packets and accordingly configure the counter 916 for each color. The path followed by traffic being measured 917 should be known in advance in order to configure the counters 918 along the path and be able to compare the correct values. 920 o Verification: both in the lab and in the operational network, the 921 methodology has been tested and experimented for packet loss and 922 delay measurements by using traffic generators together with 923 precision test instruments and network emulators. 925 o Use and Applications: the method can be used to measure packet 926 loss with high precision on live traffic; moreover, by combining 927 end-to-end and per-link measurements, the method is useful to 928 pinpoint the single link that is experiencing loss events. 930 o Reporting Model: the value of the counters has to be sent to a 931 centralized management system that performs the calculations; such 932 samples must contain a reference to the time interval they refer 933 to, so that the management system can perform the correct 934 correlation; the samples have to be sent while the corresponding 935 counter is in a steady state (within a time interval); otherwise, 936 the value of the sample should be stored locally. 938 o Dependencies: the values of the counters have to be correlated to 939 the time interval they refer to. 941 o Organization of Results: the Method of Measurement produces 942 singletons. 944 o Parameters: currently, the main parameter of the method is the 945 time interval used to alternate the colors and read the counters. 947 9. IANA Considerations 949 This document has no IANA actions. 951 10. Security Considerations 953 This document specifies a method to perform measurements in the 954 context of a Service Provider's network and has not been developed to 955 conduct Internet measurements, so it does not directly affect 956 Internet security nor applications that run on the Internet. 958 However, implementation of this method must be mindful of security 959 and privacy concerns. 961 There are two types of security concerns: potential harm caused by 962 the measurements and potential harm to the measurements. 964 o Harm caused by the measurement: the measurements described in this 965 document are Passive, so there are no new packets injected into 966 the network causing potential harm to the network itself and to 967 data traffic. Nevertheless, the method implies modifications on 968 the fly to a header or encapsulation of the data packets: this 969 must be performed in a way that doesn't alter the quality of 970 service experienced by packets subject to measurements and that 971 preserves stability and performance of routers doing the 972 measurements. One of the main security threats in OAM protocols 973 is network reconnaissance; an attacker can gather information 974 about the network performance by passively eavesdropping on OAM 975 messages. The advantage of the methods described in this document 976 is that the marking bits are the only information that is 977 exchanged between the network devices. Therefore, Passive 978 eavesdropping on data-plane traffic does not allow attackers to 979 gain information about the network performance. 981 o Harm to the Measurement: the measurements could be harmed by 982 routers altering the marking of the packets or by an attacker 983 injecting artificial traffic. Authentication techniques, such as 984 digital signatures, may be used where appropriate to guard against 985 injected traffic attacks. Since the measurement itself may be 986 affected by routers (or other network devices) along the path of 987 IP packets intentionally altering the value of marking bits of 988 packets, as mentioned above, the mechanism specified in this 989 document can be applied just in the context of a controlled 990 domain; thus, the routers (or other network devices) are locally 991 administered and this type of attack can be avoided. 993 It is worth highlighting that an attacker can't gain information 994 about network performance from a single monitoring point; it must use 995 synchronized monitoring points at multiple points on the path, 996 because they have to do the same kind of measurement and aggregation 997 that Service Providers using Alternate Marking must do. 999 Attacks on the data collection and reporting of the statistics 1000 between the monitoring points and the network management system can 1001 interfere with the proper functioning of the system. Hence, the 1002 channels used to report back flow statistics MUST be secured. 1004 The privacy concerns of network measurement are limited because the 1005 method only relies on information contained in the header or 1006 encapsulation without any release of user data. Although information 1007 in the header or encapsulation is metadata that can be used to 1008 compromise the privacy of users, the limited marking technique in 1009 this document seems unlikely to substantially increase the existing 1010 privacy risks from header or encapsulation metadata. It might be 1011 theoretically possible to modulate the marking to serve as a covert 1012 channel, but it would have a very low data rate if it is to avoid 1013 adversely affecting the measurement systems that monitor the marking. 1015 Delay attacks are another potential threat in the context of this 1016 document. Delay measurement is performed using a specific packet in 1017 each block, marked by a dedicated color bit. Therefore, a 1018 man-in-the-middle attacker can selectively induce synthetic delay 1019 only to delay-colored packets, causing systematic error in the delay 1020 measurements. As discussed in previous sections, the methods 1021 described in this document rely on an underlying time synchronization 1022 protocol. Thus, by attacking the time protocol, an attacker can 1023 potentially compromise the integrity of the measurement. A detailed 1024 discussion about the threats against time protocols and how to 1025 mitigate them is presented in RFC 7384 [RFC7384]. 1027 11. Contributors 1029 Xiao Min 1030 ZTE Corp. 1031 Email: xiao.min2@zte.com.cn 1033 Mach(Guoyi) Chen 1034 Huawei Technologies 1035 Email: mach.chen@huawei.com 1037 Alessandro Capello 1038 Telecom Italia 1039 Email: alessandro.capello@telecomitalia.it 1041 12. Acknowledgements 1043 The authors would like to thank Alberto Tempia Bonda, Luca 1044 Castaldelli and Lianshu Zheng for their contribution to the 1045 experimentation of the method. 1047 The authors would also thank Martin Duke and Tommy Pauly for their 1048 assistance and their detailed and precious reviews. 1050 13. References 1052 13.1. Normative References 1054 [IEEE-1588] 1055 IEEE, "IEEE Standard for a Precision Clock Synchronization 1056 Protocol for Networked Measurement and Control Systems", 1057 IEEE Std 1588-2008. 1059 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1060 Requirement Levels", BCP 14, RFC 2119, 1061 DOI 10.17487/RFC2119, March 1997, 1062 . 1064 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 1065 "Network Time Protocol Version 4: Protocol and Algorithms 1066 Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, 1067 . 1069 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1070 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1071 May 2017, . 1073 13.2. Informative References 1075 [I-D.fioccola-rfc8889bis] 1076 Fioccola, G., Cociglio, M., Sapio, A., Sisto, R., and T. 1077 Zhou, "Multipoint Alternate-Marking Method", draft- 1078 fioccola-rfc8889bis-04 (work in progress), April 2022. 1080 [I-D.mizrahi-ippm-marking] 1081 Mizrahi, T., Fioccola, G., Cociglio, M., Chen, M., and G. 1082 Mirsky, "Marking Methods for Performance Measurement", 1083 draft-mizrahi-ippm-marking-00 (work in progress), October 1084 2021. 1086 [I-D.zhou-ippm-enhanced-alternate-marking] 1087 Zhou, T., Fioccola, G., Liu, Y., Cociglio, M., Lee, S., 1088 and W. Li, "Enhanced Alternate Marking Method", draft- 1089 zhou-ippm-enhanced-alternate-marking-09 (work in 1090 progress), February 2022. 1092 [IEEE-Network-PNPM] 1093 IEEE Network, "AM-PM: Efficient Network Telemetry using 1094 Alternate Marking", DOI 10.1109/MNET.2019.1800152, 2019. 1096 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 1097 Metric for IP Performance Metrics (IPPM)", RFC 3393, 1098 DOI 10.17487/RFC3393, November 2002, 1099 . 1101 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 1102 Zekauskas, "A One-way Active Measurement Protocol 1103 (OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006, 1104 . 1106 [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. 1107 Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", 1108 RFC 5357, DOI 10.17487/RFC5357, October 2008, 1109 . 1111 [RFC5481] Morton, A. and B. Claise, "Packet Delay Variation 1112 Applicability Statement", RFC 5481, DOI 10.17487/RFC5481, 1113 March 2009, . 1115 [RFC6374] Frost, D. and S. Bryant, "Packet Loss and Delay 1116 Measurement for MPLS Networks", RFC 6374, 1117 DOI 10.17487/RFC6374, September 2011, 1118 . 1120 [RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New 1121 Performance Metric Development", BCP 170, RFC 6390, 1122 DOI 10.17487/RFC6390, October 2011, 1123 . 1125 [RFC6703] Morton, A., Ramachandran, G., and G. Maguluri, "Reporting 1126 IP Network Performance Metrics: Different Points of View", 1127 RFC 6703, DOI 10.17487/RFC6703, August 2012, 1128 . 1130 [RFC7276] Mizrahi, T., Sprecher, N., Bellagamba, E., and Y. 1131 Weingarten, "An Overview of Operations, Administration, 1132 and Maintenance (OAM) Tools", RFC 7276, 1133 DOI 10.17487/RFC7276, June 2014, 1134 . 1136 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in 1137 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384, 1138 October 2014, . 1140 [RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, 1141 Ed., "A One-Way Delay Metric for IP Performance Metrics 1142 (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January 1143 2016, . 1145 [RFC7680] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, 1146 Ed., "A One-Way Loss Metric for IP Performance Metrics 1147 (IPPM)", STD 82, RFC 7680, DOI 10.17487/RFC7680, January 1148 2016, . 1150 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 1151 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 1152 May 2016, . 1154 [RFC8321] Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli, 1155 L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi, 1156 "Alternate-Marking Method for Passive and Hybrid 1157 Performance Monitoring", RFC 8321, DOI 10.17487/RFC8321, 1158 January 2018, . 1160 [RFC8799] Carpenter, B. and B. Liu, "Limited Domains and Internet 1161 Protocols", RFC 8799, DOI 10.17487/RFC8799, July 2020, 1162 . 1164 Appendix A. Changes Log 1166 Changes from RFC 8321 in draft-fioccola-rfc8321bis-00 include: 1168 o Minor editorial changes 1170 o Replacement of the section on "Applications, Implementation, and 1171 Deployment" with "Finding of the Alternate Marking Implementations 1172 and Deployments" 1174 o Moved advantages and benefits of the method from "Introduction" to 1175 the new section on "Finding of the Alternate Marking 1176 Implementations and Deployments" 1178 o Removed section on "Hybrid Measurement" 1180 Changes in draft-fioccola-rfc8321bis-01 include: 1182 o Considerations on the reference: [IEEE-Network-PNPM] 1184 o Clarified that the method based on a fixed timer is specified in 1185 this document while the method based on a fixed number of packets 1186 is only mentioned but not detailed. 1188 o Explanation of the the intrinsic error in section 3.3.1 on 1189 "Single-Marking Methodology" 1191 o Deleted some parts in section 4 "Considerations" that no longer 1192 apply 1194 o New section on "Packet Fragmentation" 1196 Changes in draft-fioccola-rfc8321bis-02 include: 1198 o Considerations on how to handle unmarked traffic in section 5 on 1199 "Results of the Alternate Marking Experiment" 1201 o Minor rewording in section 4.4 on "Packet Fragmentation" 1203 Changes in draft-fioccola-rfc8321bis-03 include: 1205 o Deleted numeric examples in sections on "Packet Loss Measurement" 1206 and on "Single-Marking Methodology" 1208 o New section on "Alternate Marking Functions" 1210 o Moved sections 3.1.1 on "Coloring the Packets", 3.1.2 on "Counting 1211 the Packets" and 3.1.3 on "Collecting Data and Calculating Packet 1212 Loss" into the new section on "Alternate Marking Functions" 1214 o Renamed sections 4.1 as "Marking the Packets", 4.2 as "Counting 1215 and Timestamping Packets" and 4.3 as "Data Collection and 1216 Correlation" 1218 o Merged old section on "Data Correlation" with section 4.3 on "Data 1219 Collection and Correlation" 1221 o Moved and renamed section on "Timing Aspects" as "Synchronization 1222 and Timing" 1224 o Merged old section on "Synchronization" with section on 1225 "Synchronization and Timing" 1227 o Merged old section on "Packet Reordering" with section on 1228 "Synchronization and Timing" 1230 Changes in draft-fioccola-rfc8321bis-04/draft-ietf-ippm-rfc8321bis-00 1231 include: 1233 o Revised "Introduction" section 1235 o Revised sections 4.2 "Counting and Timestamping Packets" and 4.3 1236 on "Data Collection and Correlation" 1238 o Revised section 5 on "Synchronization and Timing" 1240 Changes in draft-ietf-ippm-rfc8321bis-01 include: 1242 o New section on "Summary of Changes from RFC 8321" 1244 o Revised sections on "Single-Marking Methodology" and "Double- 1245 Marking Methodology" 1247 Authors' Addresses 1249 Giuseppe Fioccola (editor) 1250 Huawei Technologies 1251 Riesstrasse, 25 1252 Munich 80992 1253 Germany 1255 Email: giuseppe.fioccola@huawei.com 1257 Mauro Cociglio 1258 Telecom Italia 1259 Via Reiss Romoli, 274 1260 Torino 10148 1261 Italy 1263 Email: mauro.cociglio@telecomitalia.it 1265 Greg Mirsky 1266 Ericsson 1268 Email: gregimirsky@gmail.com 1270 Tal Mizrahi 1271 Huawei Technologies 1273 Email: tal.mizrahi.phd@gmail.com 1275 Tianran Zhou 1276 Huawei Technologies 1277 156 Beiqing Rd. 1278 Beijing 100095 1279 China 1281 Email: zhoutianran@huawei.com