idnits 2.17.00 (12 Aug 2021) /tmp/idnits18487/draft-ietf-ippm-rfc8321bis-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC8321]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document date (April 26, 2022) is 25 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'IEEE-1588' Summary: 1 error (**), 0 flaws (~~), 0 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group G. Fioccola, Ed. 3 Internet-Draft Huawei Technologies 4 Obsoletes: 8321 (if approved) M. Cociglio 5 Intended status: Standards Track Telecom Italia 6 Expires: October 28, 2022 G. Mirsky 7 Ericsson 8 T. Mizrahi 9 T. Zhou 10 Huawei Technologies 11 April 26, 2022 13 Alternate-Marking Method 14 draft-ietf-ippm-rfc8321bis-00 16 Abstract 18 This document describes the Alternate-Marking technique to perform 19 packet loss, delay, and jitter measurements on live traffic. This 20 technology can be applied in various situations and for different 21 protocols. It could be considered Passive or Hybrid depending on the 22 application. This document obsoletes [RFC8321]. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on October 28, 2022. 41 Copyright Notice 43 Copyright (c) 2022 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (https://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 59 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 60 2. Overview of the Method . . . . . . . . . . . . . . . . . . . 4 61 3. Detailed Description of the Method . . . . . . . . . . . . . 5 62 3.1. Packet Loss Measurement . . . . . . . . . . . . . . . . . 5 63 3.2. One-Way Delay Measurement . . . . . . . . . . . . . . . . 9 64 3.2.1. Single-Marking Methodology . . . . . . . . . . . . . 9 65 3.2.2. Double-Marking Methodology . . . . . . . . . . . . . 10 66 3.3. Delay Variation Measurement . . . . . . . . . . . . . . . 11 67 4. Alternate Marking Functions . . . . . . . . . . . . . . . . . 12 68 4.1. Marking the Packets . . . . . . . . . . . . . . . . . . . 12 69 4.2. Counting and Timestamping Packets . . . . . . . . . . . . 12 70 4.3. Data Collection and Correlation . . . . . . . . . . . . . 13 71 5. Synchronization and Timing . . . . . . . . . . . . . . . . . 14 72 6. Packet Fragmentation . . . . . . . . . . . . . . . . . . . . 16 73 7. Results of the Alternate Marking Experiment . . . . . . . . . 16 74 7.1. Controlled Domain requirement . . . . . . . . . . . . . . 18 75 8. Compliance with Guidelines from RFC 6390 . . . . . . . . . . 18 76 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 77 10. Security Considerations . . . . . . . . . . . . . . . . . . . 20 78 11. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 22 79 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 22 80 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 22 81 13.1. Normative References . . . . . . . . . . . . . . . . . . 22 82 13.2. Informative References . . . . . . . . . . . . . . . . . 23 83 Appendix A. Changes Log . . . . . . . . . . . . . . . . . . . . 25 84 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 86 1. Introduction 88 Most Service Providers' networks carry traffic with contents that are 89 highly sensitive to packet loss [RFC7680], delay [RFC7679], and 90 jitter [RFC3393]. 92 Service Providers need methodologies and tools to monitor and 93 accurately measure network performance, in order to constantly 94 control the quality of experience perceived by their customers. 95 Performance monitoring also provides useful information for improving 96 network management (e.g., isolation of network problems, 97 troubleshooting, etc.). 99 RFC 7799 [RFC7799] defines Passive and Hybrid Methods of Measurement. 100 In particular, Passive Methods of Measurement are based solely on 101 observations of an undisturbed and unmodified packet stream of 102 interest; Hybrid Methods are Methods of Measurement that use a 103 combination of Active Methods and Passive Methods. 105 [RFC7276] provides a good overview of existing Operations, 106 Administration, and Maintenance (OAM) mechanisms defined in the IETF, 107 ITU-T, and IEEE. In the IETF, a lot of work has been done on fault 108 detection and connectivity verification, while little has been thus 109 far dedicated to performance monitoring. The IETF has defined 110 standard metrics to measure network performance; however, its methods 111 mainly focus on Active measurement techniques.For example, [RFC6374] 112 defines mechanisms for measuring packet loss, one-way and two-way 113 delay, and delay variation in MPLS networks, but its applicability to 114 Passive measurements has some limitations, especially for connection- 115 less networks. 117 This document proposes a Passive performance monitoring technique, 118 potentially applicable to any kind of packet-based traffic, including 119 Ethernet, IP, and MPLS, both unicast and multicast. The method 120 addresses primarily packet loss measurement, but it can be easily 121 extended to one-way or two-way delay and delay variation measurements 122 as well. 124 The method has been explicitly designed for Passive measurements, but 125 it can also be used with Active probes. Passive measurements are 126 usually more easily understood by customers and provide much better 127 accuracy, especially for packet loss measurements. 129 Therefore, the Alternate-Marking Method could be considered Hybrid or 130 Passive, depending on the case. In the case where the marking method 131 is obtained by changing existing field values of the packets the 132 technique is Hybrid. In the case where the marking field is 133 dedicated, reserved, and included in the protocol specification, the 134 Alternate-Marking technique can be considered as Passive. 136 1.1. Requirements Language 138 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 139 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 140 "OPTIONAL" in this document are to be interpreted as described in BCP 141 14 [RFC2119] [RFC8174] when, and only when, they appear in all 142 capitals, as shown here. 144 2. Overview of the Method 146 In order to perform packet loss measurements on a production traffic 147 flow, different approaches exist. The most intuitive one consists in 148 numbering the packets so that each router that receives the flow can 149 immediately detect a packet that is missing. This approach, though 150 very simple in theory, is not simple to achieve: it requires the 151 insertion of a sequence number into each packet, and the devices must 152 be able to extract the number and check it in real time. Such a task 153 can be difficult to implement on live traffic: if UDP is used as the 154 transport protocol, the sequence number is not available; on the 155 other hand, if a higher-layer sequence number (e.g., in the RTP 156 header) is used, extracting that information from each packet and 157 processing it in real time could overload the device. 159 An alternate approach is to count the number of packets sent on one 160 end, count the number of packets received on the other end, and 161 compare the two values. This operation is much simpler to implement, 162 but it requires the devices performing the measurement to be in sync: 163 in order to compare two counters, it is required that they refer 164 exactly to the same set of packets. Since a flow is continuous and 165 cannot be stopped when a counter has to be read, it can be difficult 166 to determine exactly when to read the counter. A possible solution 167 to overcome this problem is to virtually split the flow in 168 consecutive blocks by periodically inserting a delimiter so that each 169 counter refers exactly to the same block of packets. The delimiter 170 could be, for example, a special packet inserted artificially into 171 the flow. However, delimiting the flow using specific packets has 172 some limitations. First, it requires generating additional packets 173 within the flow and requires the equipment to be able to process 174 those packets. In addition, the method is vulnerable to out-of-order 175 reception of delimiting packets and, to a lesser extent, to their 176 loss. 178 The method proposed in this document follows the second approach, but 179 it doesn't use additional packets to virtually split the flow in 180 blocks. Instead, it "marks" the packets so that the packets 181 belonging to the same block will have the same color, whilst 182 consecutive blocks will have different colors. Each change of color 183 represents a sort of auto-synchronization signal that guarantees the 184 consistency of measurements taken by different devices along the 185 path. 187 Figure 1 represents a very simple network and shows how the method 188 can be used to measure packet loss on different network segments: by 189 enabling the measurement on several interfaces along the path, it is 190 possible to perform link monitoring, node monitoring, or end-to-end 191 monitoring. The method is flexible enough to measure packet loss on 192 any segment of the network and can be used to isolate the faulty 193 element. 195 Traffic Flow 196 ========================================================> 197 +------+ +------+ +------+ +------+ 198 ---<> R1 <>-----<> R2 <>-----<> R3 <>-----<> R4 <>--- 199 +------+ +------+ +------+ +------+ 200 . . . . . . 201 . . . . . . 202 . <------> <-------> . 203 . Node Packet Loss Link Packet Loss . 204 . . 205 <---------------------------------------------------> 206 End-to-End Packet Loss 208 Figure 1: Available Measurements 210 3. Detailed Description of the Method 212 This section describes, in detail, how the method operates. A 213 special emphasis is given to the measurement of packet loss, which 214 represents the core application of the method, but applicability to 215 delay and jitter measurements is also considered. 217 3.1. Packet Loss Measurement 219 The basic idea is to virtually split traffic flows into consecutive 220 blocks: each block represents a measurable entity unambiguously 221 recognizable by all network devices along the path. By counting the 222 number of packets in each block and comparing the values measured by 223 different network devices along the path, it is possible to measure 224 if packet loss occurred in any single block between any two points. 226 As discussed in the previous section, a simple way to create the 227 blocks is to "color" the traffic (two colors are sufficient), so that 228 packets belonging to different consecutive blocks will have different 229 colors. Whenever the color changes, the previous block terminates 230 and the new one begins. Hence, all the packets belonging to the same 231 block will have the same color and packets of different consecutive 232 blocks will have different colors. The number of packets in each 233 block depends on the criterion used to create the blocks: 235 o if the color is switched after a fixed number of packets, then 236 each block will contain the same number of packets (except for any 237 losses); and 239 o if the color is switched according to a fixed timer, then the 240 number of packets may be different in each block depending on the 241 packet rate. 243 The rest of the document assumes that the blocks are created 244 according to a fixed timer. The switching after a fixed number of 245 packets is an additional possibility but its detailed specification 246 is out of scope. 248 The following figure shows how a flow looks like when it is split in 249 traffic blocks with colored packets. 251 A: packet with A coloring 252 B: packet with B coloring 254 | | | | | 255 | | Traffic Flow | | 256 -------------------------------------------------------------------> 257 BBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA 258 -------------------------------------------------------------------> 259 ... | Block 5 | Block 4 | Block 3 | Block 2 | Block 1 260 | | | | | 262 Figure 2: Traffic Coloring 264 Figure 3 shows how the method can be used to measure link packet loss 265 between two adjacent nodes. 267 Referring to the figure, let's assume we want to monitor the packet 268 loss on the link between two routers: router R1 and router R2. 269 According to the method, the traffic is colored alternatively with 270 two different colors: A and B. Whenever the color changes, the 271 transition generates a sort of square-wave signal, as depicted in the 272 following figure. 274 Color A ----------+ +-----------+ +---------- 275 | | | | 276 Color B +-----------+ +-----------+ 277 Block n ... Block 3 Block 2 Block 1 278 <---------> <---------> <---------> <---------> <---------> 280 Traffic Flow 281 ===========================================================> 282 Color ...AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA... 283 ===========================================================> 285 Figure 3: Computation of Link Packet Loss 287 Traffic coloring can be done by R1 itself if the traffic is not 288 already colored. R1 needs two counters, C(A)R1 and C(B)R1, on its 289 egress interface: C(A)R1 counts the packets with color A and C(B)R1 290 counts those with color B. As long as traffic is colored as A, only 291 counter C(A)R1 will be incremented, while C(B)R1 is not incremented; 292 conversely, when the traffic is colored as B, only C(B)R1 is 293 incremented. C(A)R1 and C(B)R1 can be used as reference values to 294 determine the packet loss from R1 to any other measurement point down 295 the path. Router R2, similarly, will need two counters on its 296 ingress interface, C(A)R2 and C(B)R2, to count the packets received 297 on that interface and colored with A and B, respectively. When an A 298 block ends, it is possible to compare C(A)R1 and C(A)R2 and calculate 299 the packet loss within the block; similarly, when the successive B 300 block terminates, it is possible to compare C(B)R1 with C(B)R2, and 301 so on, for every successive block. 303 Likewise, by using two counters on the R2 egress interface, it is 304 possible to count the packets sent out of the R2 interface and use 305 them as reference values to calculate the packet loss from R2 to any 306 measurement point downstream from R2. 308 Using a fixed timer for color switching offers better control over 309 the method: the (time) length of the blocks can be chosen large 310 enough to simplify the collection and the comparison of measures 311 taken by different network devices. It's preferable to read the 312 value of the counters not immediately after the color switch: some 313 packets could arrive out of order and increment the counter 314 associated with the previous block (color), so it is worth waiting 315 for some time. A safe choice is to wait L/2 time units (where L is 316 the duration for each block) after the color switch, to read the 317 counter of the previous color. The drawback is that the longer the 318 duration of the block, the less frequently the measurement can be 319 taken. 321 Two different strategies that can be used when implementing the 322 method: 324 o flow-based: the flow-based strategy is used when only a limited 325 number of traffic flows need to be monitored. According to this 326 strategy, only a subset of the flows is colored. Counters for 327 packet loss measurements can be instantiated for each single flow, 328 or for the set as a whole, depending on the desired granularity. 329 A relevant problem with this approach is the necessity to know in 330 advance the path followed by flows that are subject to 331 measurement. Path rerouting and traffic load-balancing increase 332 the issue complexity, especially for unicast traffic. The problem 333 is easier to solve for multicast traffic, where load-balancing is 334 seldom used and static joins are frequently used to force traffic 335 forwarding and replication. 337 o link-based: measurements are performed on all the traffic on a 338 link-by-link basis. The link could be a physical link or a 339 logical link. Counters could be instantiated for the traffic as a 340 whole or for each traffic class (in case it is desired to monitor 341 each class separately), but in the second case, two counters are 342 needed for each class. 344 As mentioned, the flow-based measurement requires the identification 345 of the flow to be monitored and the discovery of the path followed by 346 the selected flow. It is possible to monitor a single flow or 347 multiple flows grouped together, but in this case, measurement is 348 consistent only if all the flows in the group follow the same path. 349 Moreover, if a measurement is performed by grouping many flows, it is 350 not possible to determine exactly which flow was affected by packet 351 loss. In order to have measures per single flow, it is necessary to 352 configure counters for each specific flow. Once the flow(s) to be 353 monitored has been identified, it is necessary to configure the 354 monitoring on the proper nodes. Configuring the monitoring means 355 configuring the rule to intercept the traffic and configuring the 356 counters to count the packets. To have just an end-to-end 357 monitoring, it is sufficient to enable the monitoring on the first- 358 and last-hop routers of the path: the mechanism is completely 359 transparent to intermediate nodes and independent from the path 360 followed by traffic flows. On the contrary, to monitor the flow on a 361 hop-by-hop basis along its whole path, it is necessary to enable the 362 monitoring on every node from the source to the destination. In case 363 the exact path followed by the flow is not known a priori (i.e., the 364 flow has multiple paths to reach the destination), it is necessary to 365 enable the monitoring system on every path: counters on interfaces 366 traversed by the flow will report packet count, whereas counters on 367 other interfaces will be null. 369 3.2. One-Way Delay Measurement 371 The same principle used to measure packet loss can be applied also to 372 one-way delay measurement. There are three alternatives, as 373 described hereinafter. 375 Note that, for all the one-way delay alternatives described in the 376 next sections, by summing the one-way delays of the two directions of 377 a path, it is always possible to measure the two-way delay (round- 378 trip "virtual" delay). 380 3.2.1. Single-Marking Methodology 382 The alternation of colors can be used as a time reference to 383 calculate the delay. Whenever the color changes (which means that a 384 new block has started), a network device can store the timestamp of 385 the first packet of the new block; that timestamp can be compared 386 with the timestamp of the same packet on a second router to compute 387 packet delay. When looking at Figure 2, R1 stores the timestamp 388 TS(A1)R1 when it sends the first packet of block 1 (A-colored), the 389 timestamp TS(B2)R1 when it sends the first packet of block 2 390 (B-colored), and so on for every other block. R2 performs the same 391 operation on the receiving side, recording TS(A1)R2, TS(B2)R2, and so 392 on. Since the timestamps refer to specific packets (the first packet 393 of each block), we are sure that timestamps compared to compute delay 394 refer to the same packets. By comparing TS(A1)R1 with TS(A1)R2 (and 395 similarly TS(B2)R1 with TS(B2)R2, and so on), it is possible to 396 measure the delay between R1 and R2. In order to have more 397 measurements, it is possible to take and store more timestamps, 398 referring to other packets within each block. The number of 399 measurements could be increased by considering multiple packets in 400 the block: for instance, a timestamp could be taken every N packets, 401 thus generating multiple delay measurements. Taking this to the 402 limit, in principle, the delay could be measured for each packet by 403 taking and comparing the corresponding timestamps (possible but 404 impractical from an implementation point of view). 406 In order to coherently compare timestamps collected on different 407 routers, the clocks on the network nodes must be in sync. 408 Furthermore, a measurement is valid only if no packet loss occurs and 409 if packet misordering can be avoided; otherwise, the first packet of 410 a block on R1 could be different from the first packet of the same 411 block on R2 (for instance, if that packet is lost between R1 and R2 412 or it arrives after the next one). Since packet misordering is 413 generally undetectable it is not possible to check whether the first 414 packet on R1 is the same on R2 and this is part of the intrinsic 415 error in this measurement. 417 3.2.1.1. Mean Delay 419 As mentioned before, the method previously exposed for measuring the 420 delay is sensitive to out-of-order reception of packets. In order to 421 overcome this problem, a different approach has been considered: it 422 is based on the concept of mean delay. The mean delay is calculated 423 by considering the average arrival time of the packets within a 424 single block. The network device locally stores a timestamp for each 425 packet received within a single block: summing all the timestamps and 426 dividing by the total number of packets received, the average arrival 427 time for that block of packets can be calculated. By subtracting the 428 average arrival times of two adjacent devices, it is possible to 429 calculate the mean delay between those nodes. When computing the 430 mean delay, the measurement error could be augmented by accumulating 431 the measurement error of a lot of packets. This method is robust to 432 out-of-order packets and also to packet loss (only a small error is 433 introduced). Moreover, it greatly reduces the number of timestamps 434 (only one per block for each network device) that have to be 435 collected by the management system. On the other hand, it only gives 436 one measure for the duration of the block, and it doesn't give the 437 minimum, maximum, and median delay values [RFC6703]. This limitation 438 could be overcome by reducing the duration of the block (for 439 instance, from 5 minutes to a few seconds), which implies a highly 440 optimized implementation of the method. 442 3.2.2. Double-Marking Methodology 444 The Single-Marking methodology for one-way delay measurement is 445 sensitive to out-of-order reception of packets. The first approach 446 to overcome this problem has been described before and is based on 447 the concept of mean delay. But the limitation of mean delay is that 448 it doesn't give information about the delay value's distribution for 449 the duration of the block. Additionally, it may be useful to have 450 not only the mean delay but also the minimum, maximum, and median 451 delay values and, in wider terms, to know more about the statistic 452 distribution of delay values. So, in order to have more information 453 about the delay and to overcome out-of-order issues, a different 454 approach can be introduced; it is based on a Double-Marking 455 methodology. 457 Basically, the idea is to use the first marking to create the 458 alternate flow and, within this colored flow, a second marking to 459 select the packets for measuring delay/jitter. The first marking is 460 needed for packet loss and mean delay measurement. The second 461 marking creates a new set of marked packets that are fully identified 462 over the network, so that a network device can store the timestamps 463 of these packets; these timestamps can be compared with the 464 timestamps of the same packets on a second router to compute packet 465 delay values for each packet. The number of measurements can be 466 easily increased by changing the frequency of the second marking. 467 But the frequency of the second marking must not be too high in order 468 to avoid out-of-order issues. Between packets with the second 469 marking, there should be a security time gap (e.g., this gap could 470 be, at the minimum, the mean network delay calculated with the 471 previous methodology) to avoid out-of-order issues and also to have a 472 number of measurement packets that are rate independent. If a 473 second-marking packet is lost, the delay measurement for the 474 considered block is corrupted and should be discarded. 476 Mean delay is calculated on all the packets of a sample and is a 477 simple computation to be performed for a Single-Marking Method. In 478 some cases, the mean delay measure is not sufficient to characterize 479 the sample, and more statistics of delay extent data are needed, 480 e.g., percentiles, variance, and median delay values. The 481 conventional range (maximum-minimum) should be avoided for several 482 reasons, including stability of the maximum delay due to the 483 influence by outliers. RFC 5481 [RFC5481], Section 6.5 highlights 484 how the 99.9th percentile of delay and delay variation is more 485 helpful to performance planners. To overcome this drawback, the idea 486 is to couple the mean delay measure for the entire batch with a 487 Double-Marking Method, where a subset of batch packets is selected 488 for extensive delay calculation by using a second marking. In this 489 way, it is possible to perform a detailed analysis on these double- 490 marked packets. Please note that there are classic algorithms for 491 median and variance calculation, but they are out of the scope of 492 this document. The comparison between the mean delay for the entire 493 batch and the mean delay on these double-marked packets gives useful 494 information since it is possible to understand if the Double-Marking 495 measurements are actually representative of the delay trends. 497 3.3. Delay Variation Measurement 499 Similar to one-way delay measurement (both for Single Marking and 500 Double Marking), the method can also be used to measure the inter- 501 arrival jitter. We refer to the definition in RFC 3393 [RFC3393]. 502 The alternation of colors, for a Single-Marking Method, can be used 503 as a time reference to measure delay variations. In case of Double 504 Marking, the time reference is given by the second-marked packets. 505 Considering the example depicted in Figure 2, R1 stores the timestamp 506 TS(A)R1 whenever it sends the first packet of a block, and R2 stores 507 the timestamp TS(B)R2 whenever it receives the first packet of a 508 block. The inter-arrival jitter can be easily derived from one-way 509 delay measurement, by evaluating the delay variation of consecutive 510 samples. 512 The concept of mean delay can also be applied to delay variation, by 513 evaluating the average variation of the interval between consecutive 514 packets of the flow from R1 to R2. 516 4. Alternate Marking Functions 518 4.1. Marking the Packets 520 The coloring operation is fundamental in order to create packet 521 blocks and marked packets. This implies choosing where to activate 522 the coloring and how to color the packets. 524 In case of flow-based measurements, the flow to monitor can be 525 defined by a set of selection rules (e.g., header fields) used to 526 match a subset of the packets; in this way, it is possible to control 527 the number of involved nodes, the path followed by the packets, and 528 the size of the flows. It is possible, in general, to have multiple 529 coloring nodes or a single coloring node that is easier to manage and 530 doesn't raise any risk of conflict. Coloring in multiple nodes can 531 be done, and the requirement is that the coloring must change 532 periodically between the nodes according to the timing considerations 533 in Section 5; so every node that is designated as a measurement point 534 along the path should be able to identify unambiguously the colored 535 packets. Furthermore, [I-D.fioccola-rfc8889bis] generalizes the 536 coloring for multipoint-to-multipoint flow. In addition, it can be 537 advantageous to color the flow as close as possible to the source 538 because it allows an end-to-end measure if a measurement point is 539 enabled on the last-hop router as well. 541 For link-based measurements, all traffic needs to be colored when 542 transmitted on the link. If the traffic had already been colored, 543 then it has to be re-colored because the color must be consistent on 544 the link. This means that each hop along the path must (re-)color 545 the traffic; the color is not required to be consistent along 546 different links. 548 Traffic coloring can be implemented by setting specific flags in the 549 packet header and changing the value of that bit periodically. How 550 to choose the marking field depends on the application and is out of 551 scope here. 553 4.2. Counting and Timestamping Packets 555 For flow-based measurements, assuming that the coloring of the 556 packets is performed only by the source nodes, the nodes between 557 source and destination (included) have to count and timestamp the 558 colored packets that they receive and forward: this operation can be 559 enabled on every router along the path or only on a subset, depending 560 on which network segment is being monitored (a single link, a 561 particular metro area, the backbone, or the whole path). Since the 562 color switches periodically between two values, two counters (one for 563 each value) are needed for each flow and for every interface being 564 monitored. The number of timestamps to be stored depends on the 565 method for delay measurement that is applied. Furthermore, 566 [I-D.fioccola-rfc8889bis] generalizes the counting for multipoint-to- 567 multipoint flow. 569 In case of link-based measurements, the behavior is similar except 570 that coloring, counting and timestamping operations are performed on 571 a link-by-link basis at each endpoint of the link. 573 Another important aspect to take into consideration is when to read 574 the counters or when to select the packets to be double-marked for 575 delay measurement. It involves timing aspects to consider that are 576 further described in Section 5. 578 4.3. Data Collection and Correlation 580 The nodes enabled to perform performance monitoring collect the value 581 of the counters and timestamps, but they are not able to directly use 582 this information to measure packet loss and delay, because they only 583 have their own samples. 585 Data collection enables the transmission of the counters and 586 timestamps as soon as it has been read. While, data correlation is 587 the mechanism to compare counters and timestamps for packet loss, 588 delay, and delay variation calculation. 590 There are two main possibilities to perform both data collection and 591 correlation depending on the Alternate-Marking application and use 592 case: 594 o Use of a centralized solution using Network Management System 595 (NMS) to correlate data. This can be done in Push Mode or Polling 596 Mode. In the first case, each router periodically sends the 597 information to the NMS; in the latter case, it is the NMS that 598 periodically polls routers to collect information. In any case, 599 the NMS has to collect all the relevant values from all the 600 routers within one cycle of the timer. 602 o Definition of a protocol-based distributed solution to exchange 603 values of counters and timestamps between the endpoints. This can 604 be done by introducing a new protocol or by extending the existing 605 protocols (e.g., the Two-Way Active Measurement Protocol (TWAMP) 606 as defined in RFC 5357 [RFC5357] or the One-Way Active Measurement 607 Protocol (OWAMP) as defined in RFC 4656 [RFC4656]) in order to 608 communicate the counters and timestamps between nodes. 610 In the following paragraphs, an example data correlation mechanism is 611 explained and could be used independently of the adopted solutions. 613 When data is collected on the upstream and downstream nodes, e.g., 614 packet counts for packet loss measurement or timestamps for packet 615 delay measurement, and is periodically reported to or pulled by other 616 nodes or an NMS, a certain data correlation mechanism SHOULD be in 617 use to help the nodes or NMS tell whether any two or more packet 618 counts are related to the same block of markers or if any two 619 timestamps are related to the same marked packet. 621 The Alternate-Marking Method described in this document literally 622 splits the packets of the measured flow into different measurement 623 blocks. An implementation MAY use a Block Number (BN) for data 624 correlation. The BN MAY be assigned to each measurement block and 625 associated with each packet count and timestamp reported to or pulled 626 by other nodes or NMSs. When the nodes or NMS see, for example, the 627 same BNs associated with two packet counts from an upstream and a 628 downstream node, respectively, it considers that these two packet 629 counts correspond to the same block. The assumption of this BN 630 mechanism is that the measurement nodes are time synchronized. This 631 requires the measurement nodes to have a certain time synchronization 632 capability (e.g., the Network Time Protocol (NTP) [RFC5905] or the 633 IEEE 1588 Precision Time Protocol (PTP) [IEEE-1588]). 635 5. Synchronization and Timing 637 This document introduces two color-switching methods: one is based on 638 a fixed number of packets, and the other is based on a fixed timer. 639 But the method based on a fixed timer is preferable because it is 640 more deterministic, and it is considered in the document. 642 Color switching is the reference for all the network devices, and the 643 only requirement to be achieved is that all network devices have to 644 recognize the right batch along the path. 646 In general, clocks in network devices are not accurate and for this 647 reason, there is a clock error between the measurement points R1 and 648 R2. And, to implement the methodology, they must be synchronized to 649 the same clock reference with an adequate accuracy in order to 650 guarantee that all network devices consistently match the marking bit 651 to the correct block. Additionally, in practice, besides clock 652 errors, packet reordering is also very common in a packet network due 653 to equal-cost multipath (ECMP). In particular, the delay between 654 measurement points is the main cause of out of order because each 655 packet can be delayed differently. If the block is sufficiently 656 large, packet reordering occurs only at the edge of adjacent blocks 657 and it can be easy to assign reordered packets to the right interval 658 blocks. 660 In summary, we need to take into account two contributions: clock 661 error between network devices and the interval we need to wait to 662 avoid packets being out of order because of network delay. 664 The following figure explains both issues. 666 ...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB... 667 |<======================================>| 668 | L | 669 ...=========>|<==================><==================>|<==========... 670 | L/2 L/2 | 671 |<===>| |<===>| 672 d | | d 673 |<==========================>| 674 available counting interval 676 Figure 4: Timing Aspects 678 Where L is the time duration of each block. 680 It is assumed that all network devices are synchronized to a common 681 reference time with an accuracy of +/- A/2. Thus, the difference 682 between the clock values of any two network devices is bounded by A. 684 The network delay between the network devices can be represented as a 685 data set and 99.7% of the samples are within 3 standard deviation of 686 the average. 688 The guard band d is given by: 690 d = A + D_avg + 3*D_stddev, 692 where A is the clock accuracy, D_avg is the average value of the 693 network delay between the network devices, and D_stddev is the 694 standard deviation of the delay. 696 The available counting interval is L - 2d that must be > 0. 698 The condition that must be satisfied and is a requirement on the 699 synchronization accuracy is: 701 d < L/2. 703 It is worth mentioning that, if the time duration L of each block is 704 not so small, the synchronization requirement could be satisfied even 705 with a relatively inaccurate synchronization method. This is true 706 for packet loss and two-way delay measurement, but not for one-way 707 delay measurement, where clock synchronization must be accurate. 708 Therefore, a system that uses only packet loss and two-way delay 709 measurement may not require a very precise synchronization. This is 710 because the value of the clocks of network devices does not affect 711 the computation of the two-way delay measurement. 713 6. Packet Fragmentation 715 Fragmentation can be managed with the Alternate-Marking Method and in 716 particular it is possible to give the following guidance: 718 Marking nodes MUST mark all fragments if there are flag bits to 719 use (i.e. it is in the specific encapsulation), as if they were 720 separate packets. 722 Nodes that fragment packets within the measurement domain SHOULD, 723 if they have the capability to do so, ensure that only one 724 resulting fragment carries the marking bit(s) of the original 725 packet. Failure to do so can introduce errors into the 726 measurement. 728 Measurement points MAY simply ignore unmarked fragments and count 729 marked fragments as full packets. However, if resources allow, 730 measurement points MAY make note of both marked and unmarked 731 initial fragments and only increment the corresponding counter if 732 (a) other fragments are also marked, or (b) it observes all other 733 fragments and they are unmarked. 735 The proposed approach allows the marking node to mark all the 736 fragments except in the case of fragmentation within the network 737 domain, in that event it is suggested to mark only the first 738 fragment. 740 7. Results of the Alternate Marking Experiment 742 The methodology described in the previous sections can be applied to 743 various performance measurement problems, as explained in [RFC8321]. 744 The only requirement is to select and mark the flow to be monitored; 745 in this way, packets are batched by the sender, and each batch is 746 alternately marked such that it can be easily recognized by the 747 receiver. 749 Either one or two flag bits might be available for marking in 750 different deployments: 752 One flag: packet loss measurement SHOULD be done as described in 753 Section 3.1, while delay measurement MAY be done according to the 754 single-marking method described in Section 3.2.1. Mean delay 755 (Section 3.2.1.1) is NOT RECOMMENDED since it implies more 756 computational load. 758 Two flags: packet loss measurement SHOULD be done as described in 759 Section 3.1, while delay measurement SHOULD be done according to 760 double-marking method Section 3.2.2. In this case single-marking 761 MAY also be used in combination with double-marking and the two 762 approaches provide slightly different pieces of information that 763 can be combined to have a more robust data set. 765 The experiment with Alternate Marking methodologies confirmed the 766 following benefits: 768 o easy implementation: it can be implemented by using features 769 already available on major routing platforms, or by applying an 770 optimized implementation of the method for both legacy and newest 771 technologies; 773 o low computational effort: the additional load on processing is 774 negligible; 776 o accurate loss and delay measurements: single packet loss 777 granularity is achieved with a Passive measurement; 779 o potential applicability to any kind of packet-based or frame-based 780 traffic: Ethernet, IP, MPLS, etc., and both unicast and multicast; 782 o robustness: the method can easily tolerate out-of-order packets, 783 and it's not based on "special" packets whose loss could have a 784 negative impact; 786 o flexibility: all the timestamp formats are allowed, because they 787 are managed out of band. The format (the Network Time Protocol 788 (NTP) [RFC5905] or the IEEE 1588 Precision Time Protocol (PTP) 789 [IEEE-1588]) depends on the precision you want; and 791 o no interoperability issues: the features required are available on 792 all current routing platforms. Both a centralized or distributed 793 solution can be used to harvest data from the routers. 795 A deployment of the Alternate-Marking Method SHOULD also take into 796 account how to handle and recognize marked and unmarked traffic 797 depending on whether the technique is applied as Hybrid or Passive. 798 In the case where the marking method is applied by changing existing 799 fields of the packets, it is RECOMMENDED to use an additional flag or 800 some out-of-band signaling to indicate if the measurement is 801 activated or not in order to inform the measurement points. While, 802 in the case where the marking field is dedicated, reserved, and 803 included in a protocol extension, the measurement points can learn 804 whether the measurement is activated or not by checking if the 805 specific extension is included or not within the packets. 807 It is worth mentioning some related work: in particular 808 [IEEE-Network-PNPM] explains the Alternate-Marking method together 809 with new mechanisms based on hashing techniques as also further 810 described in [I-D.mizrahi-ippm-marking]; while 811 [I-D.zhou-ippm-enhanced-alternate-marking] extends the Alternate- 812 Marking Data Fields, to provide enhanced capabilities and allow 813 advanced functionalities. 815 7.1. Controlled Domain requirement 817 The Alternate Marking Method is an example of a solution limited to a 818 controlled domain [RFC8799]. 820 A controlled domain is a managed network that selects, monitors, and 821 controls access by enforcing policies at the domain boundaries, in 822 order to discard undesired external packets entering the domain and 823 check internal packets leaving the domain. It does not necessarily 824 mean that a controlled domain is a single administrative domain or a 825 single organization. A controlled domain can correspond to a single 826 administrative domain or multiple administrative domains under a 827 defined network management. It must be possible to control the 828 domain boundaries, and use specific precautions if traffic traverses 829 the Internet. 831 For security reasons, the Alternate Marking Method is RECOMMENDED 832 only for controlled domains. 834 8. Compliance with Guidelines from RFC 6390 836 RFC 6390 [RFC6390] defines a framework and a process for developing 837 Performance Metrics for protocols above and below the IP layer (such 838 as IP-based applications that operate over reliable or datagram 839 transport protocols). 841 This document doesn't aim to propose a new Performance Metric but 842 rather a new Method of Measurement for a few Performance Metrics that 843 have already been standardized. Nevertheless, it's worth applying 844 guidelines from [RFC6390] to the present document, in order to 845 provide a more complete and coherent description of the proposed 846 method. We used a combination of the Performance Metric Definition 847 template defined in Section 5.4 of [RFC6390] and the Dependencies 848 laid out in Section 5.5 of that document. 850 o Metric Name / Metric Description: as already stated, this document 851 doesn't propose any new Performance Metrics. On the contrary, it 852 describes a novel method for measuring packet loss [RFC7680]. The 853 same concept, with small differences, can also be used to measure 854 delay [RFC7679] and jitter [RFC3393]. The document mainly 855 describes the applicability to packet loss measurement. 857 o Method of Measurement or Calculation: according to the method 858 described in the previous sections, the number of packets lost is 859 calculated by subtracting the value of the counter on the source 860 node from the value of the counter on the destination node. Both 861 counters must refer to the same color. The calculation is 862 performed when the value of the counters is in a steady state. 863 The steady state is an intrinsic characteristic of the marking 864 method counters because the alternation of color makes the counter 865 associated with a color inactive for the duration of a marking 866 period. 868 o Units of Measurement: the method calculates and reports the exact 869 number of packets sent by the source node and not received by the 870 destination node. 872 o Measurement Point(s) with Potential Measurement Domain: the 873 measurement can be performed between adjacent nodes, on a per-link 874 basis, or along a multi-hop path, provided that the traffic under 875 measurement follows that path. In case of a multi-hop path, the 876 measurements can be performed both end-to-end and hop-by-hop. 878 o Measurement Timing: the method has a constraint on the frequency 879 of measurements. This is detailed in Section 5, where it is 880 specified that the marking period and the guard band interval are 881 strictly related each other to avoid out-of-order issues. That is 882 because, in order to perform a measurement, the counter must be in 883 a steady state, and this happens when the traffic is being colored 884 with the alternate color. 886 o Implementation: the method uses one or two marking bits to color 887 the packets; this enables the use of policy configurations on the 888 router to color the packets and accordingly configure the counter 889 for each color. The path followed by traffic being measured 890 should be known in advance in order to configure the counters 891 along the path and be able to compare the correct values. 893 o Verification: both in the lab and in the operational network, the 894 methodology has been tested and experimented for packet loss and 895 delay measurements by using traffic generators together with 896 precision test instruments and network emulators. 898 o Use and Applications: the method can be used to measure packet 899 loss with high precision on live traffic; moreover, by combining 900 end-to-end and per-link measurements, the method is useful to 901 pinpoint the single link that is experiencing loss events. 903 o Reporting Model: the value of the counters has to be sent to a 904 centralized management system that performs the calculations; such 905 samples must contain a reference to the time interval they refer 906 to, so that the management system can perform the correct 907 correlation; the samples have to be sent while the corresponding 908 counter is in a steady state (within a time interval); otherwise, 909 the value of the sample should be stored locally. 911 o Dependencies: the values of the counters have to be correlated to 912 the time interval they refer to. 914 o Organization of Results: the Method of Measurement produces 915 singletons. 917 o Parameters: currently, the main parameter of the method is the 918 time interval used to alternate the colors and read the counters. 920 9. IANA Considerations 922 This document has no IANA actions. 924 10. Security Considerations 926 This document specifies a method to perform measurements in the 927 context of a Service Provider's network and has not been developed to 928 conduct Internet measurements, so it does not directly affect 929 Internet security nor applications that run on the Internet. 930 However, implementation of this method must be mindful of security 931 and privacy concerns. 933 There are two types of security concerns: potential harm caused by 934 the measurements and potential harm to the measurements. 936 o Harm caused by the measurement: the measurements described in this 937 document are Passive, so there are no new packets injected into 938 the network causing potential harm to the network itself and to 939 data traffic. Nevertheless, the method implies modifications on 940 the fly to a header or encapsulation of the data packets: this 941 must be performed in a way that doesn't alter the quality of 942 service experienced by packets subject to measurements and that 943 preserves stability and performance of routers doing the 944 measurements. One of the main security threats in OAM protocols 945 is network reconnaissance; an attacker can gather information 946 about the network performance by passively eavesdropping on OAM 947 messages. The advantage of the methods described in this document 948 is that the marking bits are the only information that is 949 exchanged between the network devices. Therefore, Passive 950 eavesdropping on data-plane traffic does not allow attackers to 951 gain information about the network performance. 953 o Harm to the Measurement: the measurements could be harmed by 954 routers altering the marking of the packets or by an attacker 955 injecting artificial traffic. Authentication techniques, such as 956 digital signatures, may be used where appropriate to guard against 957 injected traffic attacks. Since the measurement itself may be 958 affected by routers (or other network devices) along the path of 959 IP packets intentionally altering the value of marking bits of 960 packets, as mentioned above, the mechanism specified in this 961 document can be applied just in the context of a controlled 962 domain; thus, the routers (or other network devices) are locally 963 administered and this type of attack can be avoided. 965 It is worth highlighting that an attacker can't gain information 966 about network performance from a single monitoring point; it must use 967 synchronized monitoring points at multiple points on the path, 968 because they have to do the same kind of measurement and aggregation 969 that Service Providers using Alternate Marking must do. 971 Attacks on the data collection and reporting of the statistics 972 between the monitoring points and the network management system can 973 interfere with the proper functioning of the system. Hence, the 974 channels used to report back flow statistics MUST be secured. 976 The privacy concerns of network measurement are limited because the 977 method only relies on information contained in the header or 978 encapsulation without any release of user data. Although information 979 in the header or encapsulation is metadata that can be used to 980 compromise the privacy of users, the limited marking technique in 981 this document seems unlikely to substantially increase the existing 982 privacy risks from header or encapsulation metadata. It might be 983 theoretically possible to modulate the marking to serve as a covert 984 channel, but it would have a very low data rate if it is to avoid 985 adversely affecting the measurement systems that monitor the marking. 987 Delay attacks are another potential threat in the context of this 988 document. Delay measurement is performed using a specific packet in 989 each block, marked by a dedicated color bit. Therefore, a 990 man-in-the-middle attacker can selectively induce synthetic delay 991 only to delay-colored packets, causing systematic error in the delay 992 measurements. As discussed in previous sections, the methods 993 described in this document rely on an underlying time synchronization 994 protocol. Thus, by attacking the time protocol, an attacker can 995 potentially compromise the integrity of the measurement. A detailed 996 discussion about the threats against time protocols and how to 997 mitigate them is presented in RFC 7384 [RFC7384]. 999 11. Contributors 1001 Xiao Min 1002 ZTE Corp. 1003 Email: xiao.min2@zte.com.cn 1005 Mach(Guoyi) Chen 1006 Huawei Technologies 1007 Email: mach.chen@huawei.com 1009 Alessandro Capello 1010 Telecom Italia 1011 Email: alessandro.capello@telecomitalia.it 1013 12. Acknowledgements 1015 The authors would like to thank Alberto Tempia Bonda, Luca 1016 Castaldelli and Lianshu Zheng for their contribution to the 1017 experimentation of the method. 1019 The authors would also thank Martin Duke and Tommy Pauly for their 1020 assistance and their detailed and precious reviews. 1022 13. References 1024 13.1. Normative References 1026 [IEEE-1588] 1027 IEEE, "IEEE Standard for a Precision Clock Synchronization 1028 Protocol for Networked Measurement and Control Systems", 1029 IEEE Std 1588-2008. 1031 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1032 Requirement Levels", BCP 14, RFC 2119, 1033 DOI 10.17487/RFC2119, March 1997, 1034 . 1036 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 1037 "Network Time Protocol Version 4: Protocol and Algorithms 1038 Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, 1039 . 1041 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1042 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1043 May 2017, . 1045 13.2. Informative References 1047 [I-D.fioccola-rfc8889bis] 1048 Fioccola, G., Cociglio, M., Sapio, A., Sisto, R., and T. 1049 Zhou, "Multipoint Alternate-Marking Method", draft- 1050 fioccola-rfc8889bis-04 (work in progress), April 2022. 1052 [I-D.mizrahi-ippm-marking] 1053 Mizrahi, T., Fioccola, G., Cociglio, M., Chen, M., and G. 1054 Mirsky, "Marking Methods for Performance Measurement", 1055 draft-mizrahi-ippm-marking-00 (work in progress), October 1056 2021. 1058 [I-D.zhou-ippm-enhanced-alternate-marking] 1059 Zhou, T., Fioccola, G., Liu, Y., Cociglio, M., Lee, S., 1060 and W. Li, "Enhanced Alternate Marking Method", draft- 1061 zhou-ippm-enhanced-alternate-marking-09 (work in 1062 progress), February 2022. 1064 [IEEE-Network-PNPM] 1065 IEEE Network, "AM-PM: Efficient Network Telemetry using 1066 Alternate Marking", DOI 10.1109/MNET.2019.1800152, 2019. 1068 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 1069 Metric for IP Performance Metrics (IPPM)", RFC 3393, 1070 DOI 10.17487/RFC3393, November 2002, 1071 . 1073 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 1074 Zekauskas, "A One-way Active Measurement Protocol 1075 (OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006, 1076 . 1078 [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. 1079 Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", 1080 RFC 5357, DOI 10.17487/RFC5357, October 2008, 1081 . 1083 [RFC5481] Morton, A. and B. Claise, "Packet Delay Variation 1084 Applicability Statement", RFC 5481, DOI 10.17487/RFC5481, 1085 March 2009, . 1087 [RFC6374] Frost, D. and S. Bryant, "Packet Loss and Delay 1088 Measurement for MPLS Networks", RFC 6374, 1089 DOI 10.17487/RFC6374, September 2011, 1090 . 1092 [RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New 1093 Performance Metric Development", BCP 170, RFC 6390, 1094 DOI 10.17487/RFC6390, October 2011, 1095 . 1097 [RFC6703] Morton, A., Ramachandran, G., and G. Maguluri, "Reporting 1098 IP Network Performance Metrics: Different Points of View", 1099 RFC 6703, DOI 10.17487/RFC6703, August 2012, 1100 . 1102 [RFC7276] Mizrahi, T., Sprecher, N., Bellagamba, E., and Y. 1103 Weingarten, "An Overview of Operations, Administration, 1104 and Maintenance (OAM) Tools", RFC 7276, 1105 DOI 10.17487/RFC7276, June 2014, 1106 . 1108 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in 1109 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384, 1110 October 2014, . 1112 [RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, 1113 Ed., "A One-Way Delay Metric for IP Performance Metrics 1114 (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January 1115 2016, . 1117 [RFC7680] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, 1118 Ed., "A One-Way Loss Metric for IP Performance Metrics 1119 (IPPM)", STD 82, RFC 7680, DOI 10.17487/RFC7680, January 1120 2016, . 1122 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 1123 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 1124 May 2016, . 1126 [RFC8321] Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli, 1127 L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi, 1128 "Alternate-Marking Method for Passive and Hybrid 1129 Performance Monitoring", RFC 8321, DOI 10.17487/RFC8321, 1130 January 2018, . 1132 [RFC8799] Carpenter, B. and B. Liu, "Limited Domains and Internet 1133 Protocols", RFC 8799, DOI 10.17487/RFC8799, July 2020, 1134 . 1136 Appendix A. Changes Log 1138 Changes from RFC 8321 in draft-fioccola-rfc8321bis-00 include: 1140 o Minor editorial changes 1142 o Replacement of the section on "Applications, Implementation, and 1143 Deployment" with "Finding of the Alternate Marking Implementations 1144 and Deployments" 1146 o Moved advantages and benefits of the method from "Introduction" to 1147 the new section on "Finding of the Alternate Marking 1148 Implementations and Deployments" 1150 o Removed section on "Hybrid Measurement" 1152 Changes in draft-fioccola-rfc8321bis-01 include: 1154 o Considerations on the reference: [IEEE-Network-PNPM] 1156 o Clarified that the method based on a fixed timer is specified in 1157 this document while the method based on a fixed number of packets 1158 is only mentioned but not detailed. 1160 o Explanation of the the intrinsic error in section 3.3.1 on 1161 "Single-Marking Methodology" 1163 o Deleted some parts in section 4 "Considerations" that no longer 1164 apply 1166 o New section on "Packet Fragmentation" 1168 Changes in draft-fioccola-rfc8321bis-02 include: 1170 o Considerations on how to handle unmarked traffic in section 5 on 1171 "Results of the Alternate Marking Experiment" 1173 o Minor rewording in section 4.4 on "Packet Fragmentation" 1175 Changes in draft-fioccola-rfc8321bis-03 include: 1177 o Deleted numeric examples in sections on "Packet Loss Measurement" 1178 and on "Single-Marking Methodology" 1180 o New section on "Alternate Marking Functions" 1182 o Moved sections 3.1.1 on "Coloring the Packets", 3.1.2 on "Counting 1183 the Packets" and 3.1.3 on "Collecting Data and Calculating Packet 1184 Loss" into the new section on "Alternate Marking Functions" 1186 o Renamed sections 4.1 as "Marking the Packets", 4.2 as "Counting 1187 and Timestamping Packets" and 4.3 as "Data Collection and 1188 Correlation" 1190 o Merged old section on "Data Correlation" with section 4.3 on "Data 1191 Collection and Correlation" 1193 o Moved and renamed section on "Timing Aspects" as "Synchronization 1194 and Timing" 1196 o Merged old section on "Synchronization" with section on 1197 "Synchronization and Timing" 1199 o Merged old section on "Packet Reordering" with section on 1200 "Synchronization and Timing" 1202 Changes in draft-fioccola-rfc8321bis-04/draft-ietf-ippm-rfc8321bis-00 1203 include: 1205 o Revised "Introduction" section 1207 o Revised sections 4.2 "Counting and Timestamping Packets" and 4.3 1208 on "Data Collection and Correlation" 1210 o Revised section 5 on "Synchronization and Timing" 1212 Authors' Addresses 1214 Giuseppe Fioccola (editor) 1215 Huawei Technologies 1216 Riesstrasse, 25 1217 Munich 80992 1218 Germany 1220 Email: giuseppe.fioccola@huawei.com 1221 Mauro Cociglio 1222 Telecom Italia 1223 Via Reiss Romoli, 274 1224 Torino 10148 1225 Italy 1227 Email: mauro.cociglio@telecomitalia.it 1229 Greg Mirsky 1230 Ericsson 1232 Email: gregimirsky@gmail.com 1234 Tal Mizrahi 1235 Huawei Technologies 1237 Email: tal.mizrahi.phd@gmail.com 1239 Tianran Zhou 1240 Huawei Technologies 1241 156 Beiqing Rd. 1242 Beijing 100095 1243 China 1245 Email: zhoutianran@huawei.com