idnits 2.17.00 (12 Aug 2021) /tmp/idnits22150/draft-fioccola-rfc8321bis-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC8321]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document date (February 23, 2022) is 87 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'IEEE-1588' == Outdated reference: A later version (-04) exists of draft-fioccola-rfc8889bis-02 == Outdated reference: A later version (-09) exists of draft-zhou-ippm-enhanced-alternate-marking-08 Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group G. Fioccola, Ed. 3 Internet-Draft Huawei Technologies 4 Obsoletes: 8321 (if approved) M. Cociglio 5 Intended status: Standards Track Telecom Italia 6 Expires: August 27, 2022 G. Mirsky 7 Ericsson 8 T. Mizrahi 9 T. Zhou 10 Huawei Technologies 11 X. Min 12 ZTE Corp. 13 February 23, 2022 15 Alternate-Marking Method 16 draft-fioccola-rfc8321bis-03 18 Abstract 20 This document describes the Alternate-Marking technique to perform 21 packet loss, delay, and jitter measurements on live traffic. This 22 technology can be applied in various situations and for different 23 protocols. It could be considered Passive or Hybrid depending on the 24 application. This document obsoletes [RFC8321]. 26 Status of This Memo 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF). Note that other groups may also distribute 33 working documents as Internet-Drafts. The list of current Internet- 34 Drafts is at https://datatracker.ietf.org/drafts/current/. 36 Internet-Drafts are draft documents valid for a maximum of six months 37 and may be updated, replaced, or obsoleted by other documents at any 38 time. It is inappropriate to use Internet-Drafts as reference 39 material or to cite them other than as "work in progress." 41 This Internet-Draft will expire on August 27, 2022. 43 Copyright Notice 45 Copyright (c) 2022 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents 50 (https://trustee.ietf.org/license-info) in effect on the date of 51 publication of this document. Please review these documents 52 carefully, as they describe your rights and restrictions with respect 53 to this document. Code Components extracted from this document must 54 include Simplified BSD License text as described in Section 4.e of 55 the Trust Legal Provisions and are provided without warranty as 56 described in the Simplified BSD License. 58 Table of Contents 60 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 61 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 4 62 2. Overview of the Method . . . . . . . . . . . . . . . . . . . 4 63 3. Detailed Description of the Method . . . . . . . . . . . . . 5 64 3.1. Packet Loss Measurement . . . . . . . . . . . . . . . . . 5 65 3.2. One-Way Delay Measurement . . . . . . . . . . . . . . . . 9 66 3.2.1. Single-Marking Methodology . . . . . . . . . . . . . 9 67 3.2.2. Double-Marking Methodology . . . . . . . . . . . . . 10 68 3.3. Delay Variation Measurement . . . . . . . . . . . . . . . 11 69 4. Alternate Marking Functions . . . . . . . . . . . . . . . . . 12 70 4.1. Marking the Packets . . . . . . . . . . . . . . . . . . . 12 71 4.2. Counting and Timestamping Packets . . . . . . . . . . . . 13 72 4.3. Data Collection and Correlation . . . . . . . . . . . . . 14 73 5. Synchronization and Timing . . . . . . . . . . . . . . . . . 15 74 6. Packet Fragmentation . . . . . . . . . . . . . . . . . . . . 17 75 7. Results of the Alternate Marking Experiment . . . . . . . . . 17 76 7.1. Controlled Domain requirement . . . . . . . . . . . . . . 19 77 8. Compliance with Guidelines from RFC 6390 . . . . . . . . . . 19 78 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 79 10. Security Considerations . . . . . . . . . . . . . . . . . . . 21 80 11. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 23 81 12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23 82 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 83 13.1. Normative References . . . . . . . . . . . . . . . . . . 23 84 13.2. Informative References . . . . . . . . . . . . . . . . . 24 85 Appendix A. Changes Log . . . . . . . . . . . . . . . . . . . . 26 86 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27 88 1. Introduction 90 Nowadays, most Service Providers' networks carry traffic with 91 contents that are highly sensitive to packet loss [RFC7680], delay 92 [RFC7679], and jitter [RFC3393]. 94 In view of this scenario, Service Providers need methodologies and 95 tools to monitor and measure network performance with an adequate 96 accuracy, in order to constantly control the quality of experience 97 perceived by their customers. Performance monitoring also provides 98 useful information for improving network management (e.g., isolation 99 of network problems, troubleshooting, etc.). 101 A lot of work related to Operations, Administration, and Maintenance 102 (OAM), which also includes performance monitoring techniques, has 103 been done by Standards Developing Organizations (SDOs): [RFC7276] 104 provides a good overview of existing OAM mechanisms defined in the 105 IETF, ITU-T, and IEEE. In the IETF, a lot of work has been done on 106 fault detection and connectivity verification, while a minor effort 107 has been thus far dedicated to performance monitoring. The IPPM WG 108 has defined standard metrics to measure network performance; however, 109 the methods developed in this WG mainly refer to focus on Active 110 measurement techniques. More recently, the MPLS WG has defined 111 mechanisms for measuring packet loss, one-way and two-way delay, and 112 delay variation in MPLS networks [RFC6374], but their applicability 113 to Passive measurements has some limitations, especially for pure 114 connection-less networks. 116 The lack of adequate tools to measure packet loss with the desired 117 accuracy drove an effort to design a new method for the performance 118 monitoring of live traffic, which is easy to implement and deploy. 119 The effort led to the method described in this document: basically, 120 it is a Passive performance monitoring technique, potentially 121 applicable to any kind of packet-based traffic, including Ethernet, 122 IP, and MPLS, both unicast and multicast. The method addresses 123 primarily packet loss measurement, but it can be easily extended to 124 one-way or two-way delay and delay variation measurements as well. 126 The method has been explicitly designed for Passive measurements, but 127 it can also be used with Active probes. Passive measurements are 128 usually more easily understood by customers and provide much better 129 accuracy, especially for packet loss measurements. 131 RFC 7799 [RFC7799] defines Passive and Hybrid Methods of Measurement. 132 In particular, Passive Methods of Measurement are based solely on 133 observations of an undisturbed and unmodified packet stream of 134 interest; Hybrid Methods are Methods of Measurement that use a 135 combination of Active Methods and Passive Methods. 137 Taking into consideration these definitions, the Alternate-Marking 138 Method could be considered Hybrid or Passive, depending on the case. 139 In the case where the marking method is obtained by changing existing 140 field values of the packets the technique is Hybrid. In the case 141 where the marking field is dedicated, reserved, and included in the 142 protocol specification, the Alternate-Marking technique can be 143 considered as Passive. 145 1.1. Requirements Language 147 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 148 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 149 "OPTIONAL" in this document are to be interpreted as described in BCP 150 14 [RFC2119] [RFC8174] when, and only when, they appear in all 151 capitals, as shown here. 153 2. Overview of the Method 155 In order to perform packet loss measurements on a production traffic 156 flow, different approaches exist. The most intuitive one consists in 157 numbering the packets so that each router that receives the flow can 158 immediately detect a packet that is missing. This approach, though 159 very simple in theory, is not simple to achieve: it requires the 160 insertion of a sequence number into each packet, and the devices must 161 be able to extract the number and check it in real time. Such a task 162 can be difficult to implement on live traffic: if UDP is used as the 163 transport protocol, the sequence number is not available; on the 164 other hand, if a higher-layer sequence number (e.g., in the RTP 165 header) is used, extracting that information from each packet and 166 processing it in real time could overload the device. 168 An alternate approach is to count the number of packets sent on one 169 end, count the number of packets received on the other end, and 170 compare the two values. This operation is much simpler to implement, 171 but it requires the devices performing the measurement to be in sync: 172 in order to compare two counters, it is required that they refer 173 exactly to the same set of packets. Since a flow is continuous and 174 cannot be stopped when a counter has to be read, it can be difficult 175 to determine exactly when to read the counter. A possible solution 176 to overcome this problem is to virtually split the flow in 177 consecutive blocks by periodically inserting a delimiter so that each 178 counter refers exactly to the same block of packets. The delimiter 179 could be, for example, a special packet inserted artificially into 180 the flow. However, delimiting the flow using specific packets has 181 some limitations. First, it requires generating additional packets 182 within the flow and requires the equipment to be able to process 183 those packets. In addition, the method is vulnerable to out-of-order 184 reception of delimiting packets and, to a lesser extent, to their 185 loss. 187 The method proposed in this document follows the second approach, but 188 it doesn't use additional packets to virtually split the flow in 189 blocks. Instead, it "marks" the packets so that the packets 190 belonging to the same block will have the same color, whilst 191 consecutive blocks will have different colors. Each change of color 192 represents a sort of auto-synchronization signal that guarantees the 193 consistency of measurements taken by different devices along the 194 path. 196 Figure 1 represents a very simple network and shows how the method 197 can be used to measure packet loss on different network segments: by 198 enabling the measurement on several interfaces along the path, it is 199 possible to perform link monitoring, node monitoring, or end-to-end 200 monitoring. The method is flexible enough to measure packet loss on 201 any segment of the network and can be used to isolate the faulty 202 element. 204 Traffic Flow 205 ========================================================> 206 +------+ +------+ +------+ +------+ 207 ---<> R1 <>-----<> R2 <>-----<> R3 <>-----<> R4 <>--- 208 +------+ +------+ +------+ +------+ 209 . . . . . . 210 . . . . . . 211 . <------> <-------> . 212 . Node Packet Loss Link Packet Loss . 213 . . 214 <---------------------------------------------------> 215 End-to-End Packet Loss 217 Figure 1: Available Measurements 219 3. Detailed Description of the Method 221 This section describes, in detail, how the method operates. A 222 special emphasis is given to the measurement of packet loss, which 223 represents the core application of the method, but applicability to 224 delay and jitter measurements is also considered. 226 3.1. Packet Loss Measurement 228 The basic idea is to virtually split traffic flows into consecutive 229 blocks: each block represents a measurable entity unambiguously 230 recognizable by all network devices along the path. By counting the 231 number of packets in each block and comparing the values measured by 232 different network devices along the path, it is possible to measure 233 if packet loss occurred in any single block between any two points. 235 As discussed in the previous section, a simple way to create the 236 blocks is to "color" the traffic (two colors are sufficient), so that 237 packets belonging to different consecutive blocks will have different 238 colors. Whenever the color changes, the previous block terminates 239 and the new one begins. Hence, all the packets belonging to the same 240 block will have the same color and packets of different consecutive 241 blocks will have different colors. The number of packets in each 242 block depends on the criterion used to create the blocks: 244 o if the color is switched after a fixed number of packets, then 245 each block will contain the same number of packets (except for any 246 losses); and 248 o if the color is switched according to a fixed timer, then the 249 number of packets may be different in each block depending on the 250 packet rate. 252 The rest of the document assumes that the blocks are created 253 according to a fixed timer. The switching after a fixed number of 254 packets is an additional possibility but its detailed specification 255 is out of scope. 257 The following figure shows how a flow looks like when it is split in 258 traffic blocks with colored packets. 260 A: packet with A coloring 261 B: packet with B coloring 263 | | | | | 264 | | Traffic Flow | | 265 -------------------------------------------------------------------> 266 BBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA 267 -------------------------------------------------------------------> 268 ... | Block 5 | Block 4 | Block 3 | Block 2 | Block 1 269 | | | | | 271 Figure 2: Traffic Coloring 273 Figure 3 shows how the method can be used to measure link packet loss 274 between two adjacent nodes. 276 Referring to the figure, let's assume we want to monitor the packet 277 loss on the link between two routers: router R1 and router R2. 278 According to the method, the traffic is colored alternatively with 279 two different colors: A and B. Whenever the color changes, the 280 transition generates a sort of square-wave signal, as depicted in the 281 following figure. 283 Color A ----------+ +-----------+ +---------- 284 | | | | 285 Color B +-----------+ +-----------+ 286 Block n ... Block 3 Block 2 Block 1 287 <---------> <---------> <---------> <---------> <---------> 289 Traffic Flow 290 ===========================================================> 291 Color ...AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA... 292 ===========================================================> 294 Figure 3: Computation of Link Packet Loss 296 Traffic coloring can be done by R1 itself if the traffic is not 297 already colored. R1 needs two counters, C(A)R1 and C(B)R1, on its 298 egress interface: C(A)R1 counts the packets with color A and C(B)R1 299 counts those with color B. As long as traffic is colored as A, only 300 counter C(A)R1 will be incremented, while C(B)R1 is not incremented; 301 conversely, when the traffic is colored as B, only C(B)R1 is 302 incremented. C(A)R1 and C(B)R1 can be used as reference values to 303 determine the packet loss from R1 to any other measurement point down 304 the path. Router R2, similarly, will need two counters on its 305 ingress interface, C(A)R2 and C(B)R2, to count the packets received 306 on that interface and colored with A and B, respectively. When an A 307 block ends, it is possible to compare C(A)R1 and C(A)R2 and calculate 308 the packet loss within the block; similarly, when the successive B 309 block terminates, it is possible to compare C(B)R1 with C(B)R2, and 310 so on, for every successive block. 312 Likewise, by using two counters on the R2 egress interface, it is 313 possible to count the packets sent out of the R2 interface and use 314 them as reference values to calculate the packet loss from R2 to any 315 measurement point down R2. 317 Using a fixed timer for color switching offers better control over 318 the method: the (time) length of the blocks can be chosen large 319 enough to simplify the collection and the comparison of measures 320 taken by different network devices. It's preferable to read the 321 value of the counters not immediately after the color switch: some 322 packets could arrive out of order and increment the counter 323 associated with the previous block (color), so it is worth waiting 324 for some time. A safe choice is to wait L/2 time units (where L is 325 the duration for each block) after the color switch, to read the 326 still counter of the previous color, so the possibility of reading a 327 running counter instead of a still one is minimized. The drawback is 328 that the longer the duration of the block, the less frequent the 329 measurement can be taken. 331 The timer-based batches is preferable because it is more 332 deterministic that the counter-based batches and it will be 333 considered hereafter. 335 It's worth mentioning two different strategies that can be used when 336 implementing the method: 338 o flow-based: the flow-based strategy is used when only a limited 339 number of traffic flows need to be monitored. According to this 340 strategy, only a subset of the flows is colored. Counters for 341 packet loss measurements can be instantiated for each single flow, 342 or for the set as a whole, depending on the desired granularity. 343 A relevant problem with this approach is the necessity to know in 344 advance the path followed by flows that are subject to 345 measurement. Path rerouting and traffic load-balancing increase 346 the issue complexity, especially for unicast traffic. The problem 347 is easier to solve for multicast traffic, where load-balancing is 348 seldom used and static joins are frequently used to force traffic 349 forwarding and replication. 351 o link-based: measurements are performed on all the traffic on a 352 link-by-link basis. The link could be a physical link or a 353 logical link. Counters could be instantiated for the traffic as a 354 whole or for each traffic class (in case it is desired to monitor 355 each class separately), but in the second case, two counters are 356 needed for each class. 358 As mentioned, the flow-based measurement requires the identification 359 of the flow to be monitored and the discovery of the path followed by 360 the selected flow. It is possible to monitor a single flow or 361 multiple flows grouped together, but in this case, measurement is 362 consistent only if all the flows in the group follow the same path. 363 Moreover, if a measurement is performed by grouping many flows, it is 364 not possible to determine exactly which flow was affected by packet 365 loss. In order to have measures per single flow, it is necessary to 366 configure counters for each specific flow. Once the flow(s) to be 367 monitored has been identified, it is necessary to configure the 368 monitoring on the proper nodes. Configuring the monitoring means 369 configuring the rule to intercept the traffic and configuring the 370 counters to count the packets. To have just an end-to-end 371 monitoring, it is sufficient to enable the monitoring on the first- 372 and last-hop routers of the path: the mechanism is completely 373 transparent to intermediate nodes and independent from the path 374 followed by traffic flows. On the contrary, to monitor the flow on a 375 hop-by-hop basis along its whole path, it is necessary to enable the 376 monitoring on every node from the source to the destination. In case 377 the exact path followed by the flow is not known a priori (i.e., the 378 flow has multiple paths to reach the destination), it is necessary to 379 enable the monitoring system on every path: counters on interfaces 380 traversed by the flow will report packet count, whereas counters on 381 other interfaces will be null. 383 3.2. One-Way Delay Measurement 385 The same principle used to measure packet loss can be applied also to 386 one-way delay measurement. There are three alternatives, as 387 described hereinafter. 389 Note that, for all the one-way delay alternatives described in the 390 next sections, by summing the one-way delays of the two directions of 391 a path, it is always possible to measure the two-way delay (round- 392 trip "virtual" delay). 394 3.2.1. Single-Marking Methodology 396 The alternation of colors can be used as a time reference to 397 calculate the delay. Whenever the color changes (which means that a 398 new block has started), a network device can store the timestamp of 399 the first packet of the new block; that timestamp can be compared 400 with the timestamp of the same packet on a second router to compute 401 packet delay. When looking at Figure 2, R1 stores the timestamp 402 TS(A1)R1 when it sends the first packet of block 1 (A-colored), the 403 timestamp TS(B2)R1 when it sends the first packet of block 2 404 (B-colored), and so on for every other block. R2 performs the same 405 operation on the receiving side, recording TS(A1)R2, TS(B2)R2, and so 406 on. Since the timestamps refer to specific packets (the first packet 407 of each block), we are sure that timestamps compared to compute delay 408 refer to the same packets. By comparing TS(A1)R1 with TS(A1)R2 (and 409 similarly TS(B2)R1 with TS(B2)R2, and so on), it is possible to 410 measure the delay between R1 and R2. In order to have more 411 measurements, it is possible to take and store more timestamps, 412 referring to other packets within each block. The number of 413 measurements could be increased by considering multiple packets in 414 the block: for instance, a timestamp could be taken every N packets, 415 thus generating multiple delay measurements. Taking this to the 416 limit, in principle, the delay could be measured for each packet by 417 taking and comparing the corresponding timestamps (possible but 418 impractical from an implementation point of view). 420 In order to coherently compare timestamps collected on different 421 routers, the clocks on the network nodes must be in sync. 422 Furthermore, a measurement is valid only if no packet loss occurs and 423 if packet misordering can be avoided; otherwise, the first packet of 424 a block on R1 could be different from the first packet of the same 425 block on R2 (for instance, if that packet is lost between R1 and R2 426 or it arrives after the next one). Since packet misordering is 427 generally undetectable it is not possible to check whether the first 428 packet on R1 is the same on R2 and this is part of the intrinsic 429 error in this measurement. 431 3.2.1.1. Mean Delay 433 As mentioned before, the method previously exposed for measuring the 434 delay is sensitive to out-of-order reception of packets. In order to 435 overcome this problem, a different approach has been considered: it 436 is based on the concept of mean delay. The mean delay is calculated 437 by considering the average arrival time of the packets within a 438 single block. The network device locally stores a timestamp for each 439 packet received within a single block: summing all the timestamps and 440 dividing by the total number of packets received, the average arrival 441 time for that block of packets can be calculated. By subtracting the 442 average arrival times of two adjacent devices, it is possible to 443 calculate the mean delay between those nodes. When computing the 444 mean delay, the measurement error could be augmented by accumulating 445 the measurement error of a lot of packets. This method is robust to 446 out-of-order packets and also to packet loss (only a small error is 447 introduced). Moreover, it greatly reduces the number of timestamps 448 (only one per block for each network device) that have to be 449 collected by the management system. On the other hand, it only gives 450 one measure for the duration of the block, and it doesn't give the 451 minimum, maximum, and median delay values [RFC6703]. This limitation 452 could be overcome by reducing the duration of the block (for 453 instance, from 5 minutes to a few seconds), which implies a highly 454 optimized implementation of the method. 456 3.2.2. Double-Marking Methodology 458 The Single-Marking methodology for one-way delay measurement is 459 sensitive to out-of-order reception of packets. The first approach 460 to overcome this problem has been described before and is based on 461 the concept of mean delay. But the limitation of mean delay is that 462 it doesn't give information about the delay value's distribution for 463 the duration of the block. Additionally, it may be useful to have 464 not only the mean delay but also the minimum, maximum, and median 465 delay values and, in wider terms, to know more about the statistic 466 distribution of delay values. So, in order to have more information 467 about the delay and to overcome out-of-order issues, a different 468 approach can be introduced; it is based on a Double-Marking 469 methodology. 471 Basically, the idea is to use the first marking to create the 472 alternate flow and, within this colored flow, a second marking to 473 select the packets for measuring delay/jitter. The first marking is 474 needed for packet loss and mean delay measurement. The second 475 marking creates a new set of marked packets that are fully identified 476 over the network, so that a network device can store the timestamps 477 of these packets; these timestamps can be compared with the 478 timestamps of the same packets on a second router to compute packet 479 delay values for each packet. The number of measurements can be 480 easily increased by changing the frequency of the second marking. 481 But the frequency of the second marking must not be too high in order 482 to avoid out-of-order issues. Between packets with the second 483 marking, there should be a security time gap (e.g., this gap could 484 be, at the minimum, the mean network delay calculated with the 485 previous methodology) to avoid out-of-order issues and also to have a 486 number of measurement packets that are rate independent. If a 487 second-marking packet is lost, the delay measurement for the 488 considered block is corrupted and should be discarded. 490 Mean delay is calculated on all the packets of a sample and is a 491 simple computation to be performed for a Single-Marking Method. In 492 some cases, the mean delay measure is not sufficient to characterize 493 the sample, and more statistics of delay extent data are needed, 494 e.g., percentiles, variance, and median delay values. The 495 conventional range (maximum-minimum) should be avoided for several 496 reasons, including stability of the maximum delay due to the 497 influence by outliers. RFC 5481 [RFC5481], Section 6.5 highlights 498 how the 99.9th percentile of delay and delay variation is more 499 helpful to performance planners. To overcome this drawback, the idea 500 is to couple the mean delay measure for the entire batch with a 501 Double-Marking Method, where a subset of batch packets is selected 502 for extensive delay calculation by using a second marking. In this 503 way, it is possible to perform a detailed analysis on these double- 504 marked packets. Please note that there are classic algorithms for 505 median and variance calculation, but they are out of the scope of 506 this document. The comparison between the mean delay for the entire 507 batch and the mean delay on these double-marked packets gives useful 508 information since it is possible to understand if the Double-Marking 509 measurements are actually representative of the delay trends. 511 3.3. Delay Variation Measurement 513 Similar to one-way delay measurement (both for Single Marking and 514 Double Marking), the method can also be used to measure the inter- 515 arrival jitter. We refer to the definition in RFC 3393 [RFC3393]. 516 The alternation of colors, for a Single-Marking Method, can be used 517 as a time reference to measure delay variations. In case of Double 518 Marking, the time reference is given by the second-marked packets. 519 Considering the example depicted in Figure 2, R1 stores the timestamp 520 TS(A)R1 whenever it sends the first packet of a block, and R2 stores 521 the timestamp TS(B)R2 whenever it receives the first packet of a 522 block. The inter-arrival jitter can be easily derived from one-way 523 delay measurement, by evaluating the delay variation of consecutive 524 samples. 526 The concept of mean delay can also be applied to delay variation, by 527 evaluating the average variation of the interval between consecutive 528 packets of the flow from R1 to R2. 530 4. Alternate Marking Functions 532 4.1. Marking the Packets 534 The coloring operation is fundamental in order to create packet 535 blocks and marked packets. This implies choosing where to activate 536 the coloring and how to color the packets. 538 In case of flow-based measurements, the flow to monitor can be 539 defined by a set of selection rules (e.g., header fields) used to 540 match a subset of the packets; in this way, it is possible to control 541 the number of involved nodes, the path followed by the packets, and 542 the size of the flows. It is possible, in general, to have multiple 543 coloring nodes or a single coloring node that is easier to manage and 544 doesn't raise any risk of conflict. Coloring in multiple nodes can 545 be done, and the requirement is that the coloring must change 546 periodically between the nodes according to the timing considerations 547 in Section 5; so every node that is designated as a measurement point 548 along the path should be able to identify unambiguously the colored 549 packets. Furthermore, [I-D.fioccola-rfc8889bis] generalizes the 550 coloring for multipoint-to-multipoint flow. In addition, it can be 551 advantageous to color the flow as close as possible to the source 552 because it allows an end-to-end measure if a measurement point is 553 enabled on the last-hop router as well. 555 For link-based measurements, all traffic needs to be colored when 556 transmitted on the link. If the traffic had already been colored, 557 then it has to be re-colored because the color must be consistent on 558 the link. This means that each hop along the path must (re-)color 559 the traffic; the color is not required to be consistent along 560 different links. 562 Traffic coloring can be implemented by setting specific flags in the 563 packet header and changing the value of that bit periodically. How 564 to choose the marking field depends on the application and is out of 565 scope here. 567 4.2. Counting and Timestamping Packets 569 For flow-based measurements, assuming that the coloring of the 570 packets is performed only by the source nodes, the nodes between 571 source and destination (included) have to count and timestamp the 572 colored packets that they receive and forward: this operation can be 573 enabled on every router along the path or only on a subset, depending 574 on which network segment is being monitored (a single link, a 575 particular metro area, the backbone, or the whole path). Since the 576 color switches periodically between two values, two counters (one for 577 each value) are needed: one counter for packets with color A and one 578 counter for packets with color B. For each flow (or group of flows) 579 being monitored and for every interface where the monitoring is 580 Active, two counters are needed. For example, in order to separately 581 monitor three flows on a router with four interfaces involved, 24 582 counters are needed (two counters for each of the three flows on each 583 of the four interfaces). The number of timestamps to be stored 584 depends on the method for delay measurement that is applied. 585 Furthermore, [I-D.fioccola-rfc8889bis] generalizes the counting for 586 multipoint-to-multipoint flow. 588 In case of link-based measurements, the behavior is similar except 589 that coloring, counting and timestamping operations are performed on 590 a link-by-link basis at each endpoint of the link. 592 Another important aspect to take into consideration is when to read 593 the counters: in order to count the exact number of packets of a 594 block, the routers must perform this operation when that block has 595 ended; in other words, the counter for color A must be read when the 596 current block has color B, in order to be sure that the value of the 597 counter is stable. This task can be accomplished in two ways. The 598 general approach suggests reading the counters periodically, many 599 times during a block duration, and comparing these successive 600 readings: when the counter stops incrementing, it means that the 601 current block has ended, and its value can be elaborated safely. 602 Alternatively, if the coloring operation is performed on the basis of 603 a fixed timer, it is possible to configure the reading of the 604 counters according to that timer: for example, reading the counter 605 for color A every period in the middle of the subsequent block with 606 color B is a safe choice. A sufficient margin should be considered 607 between the end of a block and the reading of the counter, in order 608 to take into account any out-of-order packets. Regarding the 609 selection of the packet to be double-marked for delay measurement, 610 the same considerations for packet loss measurement apply also here 611 and it is reasonable to choose the double-marked packet in the middle 612 of the block. The timing aspects are further described in Section 5. 614 4.3. Data Collection and Correlation 616 The nodes enabled to perform performance monitoring collect the value 617 of the counters and timestamps, but they are not able to directly use 618 this information to measure packet loss and delay, because they only 619 have their own samples. 621 Data collection enables the transmission of the counters and 622 timestamps as soon as it has been read. While, data correlation is 623 the mechanism to compare counters and timestamps for packet loss, 624 delay, and delay variation calculation. 626 There are two main possibilities to perform both data collection and 627 correlation depending on the Alternate-Marking application and use 628 case: 630 o Use of a centralized solution using Network Management System 631 (NMS) to correlate data. This can be done in Push Mode or Polling 632 Mode. In the first case, each router periodically sends the 633 information to the NMS; in the latter case, it is the NMS that 634 periodically polls routers to collect information. In any case, 635 the NMS has to collect all the relevant values from all the 636 routers within one cycle of the timer. 638 o Definition of a protocol-based distributed solution to exchange 639 values of counters and timestamps between the endpoints. This can 640 be done by introducing a new protocol or by extending the existing 641 protocols (e.g., the Two-Way Active Measurement Protocol (TWAMP) 642 as defined in RFC 5357 [RFC5357] or the One-Way Active Measurement 643 Protocol (OWAMP) as defined in RFC 4656 [RFC4656]) in order to 644 communicate the counters and timestamps between nodes. 646 In the following paragraphs, an example data correlation mechanism is 647 explained and could be used independently of the adopted solutions. 649 When data is collected on the upstream and downstream nodes, e.g., 650 packet counts for packet loss measurement or timestamps for packet 651 delay measurement, and is periodically reported to or pulled by other 652 nodes or an NMS, a certain data correlation mechanism SHOULD be in 653 use to help the nodes or NMS tell whether any two or more packet 654 counts are related to the same block of markers or if any two 655 timestamps are related to the same marked packet. 657 The Alternate-Marking Method described in this document literally 658 splits the packets of the measured flow into different measurement 659 blocks; in addition, a Block Number (BN) could be assigned to each 660 such measurement block. The BN is generated each time a node reads 661 the data (packet counts or timestamps) and is associated with each 662 packet count and timestamp reported to or pulled by other nodes or 663 NMSs. The value of a BN could be calculated as the modulo of the 664 local time (when the data are read) and the interval of the marking 665 time period. 667 When the nodes or NMS see, for example, the same BNs associated with 668 two packet counts from an upstream and a downstream node, 669 respectively, it considers that these two packet counts correspond to 670 the same block, i.e., these two packet counts belong to the same 671 block of markers from the upstream and downstream nodes. The 672 assumption of this BN mechanism is that the measurement nodes are 673 time synchronized. This requires the measurement nodes to have a 674 certain time synchronization capability (e.g., the Network Time 675 Protocol (NTP) [RFC5905] or the IEEE 1588 Precision Time Protocol 676 (PTP) [IEEE-1588]). 678 5. Synchronization and Timing 680 This document introduces two color-switching methods: one is based on 681 a fixed number of packets, and the other is based on a fixed timer. 682 But the method based on a fixed timer is preferable because it is 683 more deterministic, and it is considered in the document. 685 Color switching is the reference for all the network devices, and the 686 only requirement to be achieved is that all network devices have to 687 recognize the right batch along the path. 689 In general, clocks in network devices are not accurate and for this 690 reason, there is a clock error between the measurement points R1 and 691 R2. But, to implement the methodology, they must be synchronized to 692 the same clock reference with an accuracy of +/- L/2 time units, 693 where L is the fixed time duration of the block. So each colored 694 packet can be assigned to the right batch by each router. This is 695 because the minimum time distance between two packets of the same 696 color but that belong to different batches is L time units. This 697 level of accuracy guarantees that all network devices consistently 698 match the marking bit to the correct block. 700 If the value of L is not too small, this synchronization requirement 701 could be satisfied even with a relatively inaccurate synchronization 702 method. This is true for packet loss and two-way delay measurement, 703 but not for one-way delay measurement, where clock synchronization 704 must be accurate. Therefore, a system that uses only packet loss and 705 two-way delay measurement does not require a very precise 706 synchronization. This is because the value of the clocks of network 707 devices does not affect the computation of the two-way delay 708 measurement. 710 But, in practice, besides clock errors, packet reordering is also 711 very common in a packet network due to equal-cost multipath (ECMP). 712 In particular, the delay between measurement points is the main cause 713 of out of order because each packet can be delayed differently. For 714 this reason, the accuracy of the Alternate-Marking Method, especially 715 for packet loss measurement, is affected by packet reordering. 717 If the time duration L of each block is too small, it may be 718 difficult to determine to which block the reordered packets belong. 719 However, if the value of L is sufficiently large, packet reordering 720 occurs only at the edge of adjacent blocks and it becomes easy to 721 assign reordered packets to the right interval blocks. This means 722 that, without considering clock error, we can wait L/2 after color 723 switching to be sure to take a still counter and mitigate the 724 reordering issues. 726 In summary, we need to take into account two contributions: clock 727 error between network devices and the interval we need to wait to 728 avoid packets being out of order because of network delay. 730 The following figure explains both issues. 732 ...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB... 733 |<======================================>| 734 | L | 735 ...=========>|<==================><==================>|<==========... 736 | L/2 L/2 | 737 |<===>| |<===>| 738 d | | d 739 |<==========================>| 740 available counting interval 742 Figure 4: Timing Aspects 744 It is assumed that all network devices are synchronized to a common 745 reference time with an accuracy of +/- A/2. Thus, the difference 746 between the clock values of any two network devices is bounded by A. 748 The network delay between the network devices can be represented as a 749 data set and 99.7% of the samples are within 3 standard deviation of 750 the average. 752 The guard band d is given by: 754 d = A + D_avg + 3*D_stddev, 755 where A is the clock accuracy, D_avg is the average value of the 756 network delay between the network devices, and D_stddev is the 757 standard deviation of the delay. 759 The available counting interval is L - 2d that must be > 0. 761 The condition that must be satisfied and is a requirement on the 762 synchronization accuracy is: 764 d < L/2. 766 6. Packet Fragmentation 768 Fragmentation can be managed with the Alternate-Marking Method and in 769 particular it is possible to give the following guidance: 771 Marking nodes MUST mark all fragments if there are flag bits to 772 use (i.e. it is in the specific encapsulation), as if they were 773 separate packets. 775 Nodes that fragment packets within the measurement domain SHOULD, 776 if they have the capability to do so, ensure that only one 777 resulting fragment carries the marking bit(s) of the original 778 packet. Failure to do so can introduce errors into the 779 measurement. 781 Measurement points MAY simply ignore unmarked fragments and count 782 marked fragments as full packets. However, if resources allow, 783 measurement points MAY make note of both marked and unmarked 784 initial fragments and only increment the corresponding counter if 785 (a) other fragments are also marked, or (b) it observes all other 786 fragments and they are unmarked. 788 The proposed approach allows the marking node to mark all the 789 fragments except in the case of fragmentation within the network 790 domain, in that event it is suggested to mark only the first 791 fragment. In addition it could be possible to take the counters 792 properly in order to keep track of both marked and unmarked 793 fragments. 795 7. Results of the Alternate Marking Experiment 797 The methodology described in the previous sections can be applied to 798 various performance measurement problems, as explained in [RFC8321]. 799 The only requirement is to select and mark the flow to be monitored; 800 in this way, packets are batched by the sender, and each batch is 801 alternately marked such that it can be easily recognized by the 802 receiver. 804 Either one or two flag bits might be available for marking in 805 different deployments: 807 One flag: packet loss measurement SHOULD be done as described in 808 Section 3.1, while delay measurement MAY be done according to the 809 single-marking method described in Section 3.2.1. Mean delay 810 (Section 3.2.1.1) is NOT RECOMMENDED since it implies more 811 computational load. 813 Two flags: packet loss measurement SHOULD be done as described in 814 Section 3.1, while delay measurement SHOULD be done according to 815 double-marking method Section 3.2.2. In this case single-marking 816 MAY also be used in combination with double-marking and the two 817 approaches provide slightly different pieces of information that 818 can be combined to have a more robust data set. 820 The experiment with Alternate Marking methodologies confirmed the 821 following benefits: 823 o easy implementation: it can be implemented by using features 824 already available on major routing platforms, or by applying an 825 optimized implementation of the method for both legacy and newest 826 technologies; 828 o low computational effort: the additional load on processing is 829 negligible; 831 o accurate loss and delay measurements: single packet loss 832 granularity is achieved with a Passive measurement; 834 o potential applicability to any kind of packet-based or frame-based 835 traffic: Ethernet, IP, MPLS, etc., and both unicast and multicast; 837 o robustness: the method can easily tolerate out-of-order packets, 838 and it's not based on "special" packets whose loss could have a 839 negative impact; 841 o flexibility: all the timestamp formats are allowed, because they 842 are managed out of band. The format (the Network Time Protocol 843 (NTP) [RFC5905] or the IEEE 1588 Precision Time Protocol (PTP) 844 [IEEE-1588]) depends on the precision you want; and 846 o no interoperability issues: the features required are available on 847 all current routing platforms. Both a centralized or distributed 848 solution can be used to harvest data from the routers. 850 A deployment of the Alternate-Marking Method SHOULD also take into 851 account how to handle and recognize marked and unmarked traffic 852 depending on whether the technique is applied as Hybrid or Passive. 853 In the case where the marking method is applied by changing existing 854 fields of the packets, it is RECOMMENDED to use an additional flag or 855 some out-of-band signaling to indicate if the measurement is 856 activated or not in order to inform the measurement points. While, 857 in the case where the marking field is dedicated, reserved, and 858 included in a protocol extension, the measurement points can learn 859 whether the measurement is activated or not by checking if the 860 specific extension is included or not within the packets. 862 It is worth mentioning some related work: in particular 863 [IEEE-Network-PNPM] explains the Alternate-Marking method together 864 with new mechanisms based on hashing techniques as also further 865 described in [I-D.mizrahi-ippm-marking]; while 866 [I-D.zhou-ippm-enhanced-alternate-marking] extends the Alternate- 867 Marking Data Fields, to provide enhanced capabilities and allow 868 advanced functionalities. 870 7.1. Controlled Domain requirement 872 The Alternate Marking Method is an example of a solution limited to a 873 controlled domain [RFC8799]. 875 A controlled domain is a managed network that selects, monitors, and 876 controls access by enforcing policies at the domain boundaries, in 877 order to discard undesired external packets entering the domain and 878 check internal packets leaving the domain. It does not necessarily 879 mean that a controlled domain is a single administrative domain or a 880 single organization. A controlled domain can correspond to a single 881 administrative domain or multiple administrative domains under a 882 defined network management. It must be possible to control the 883 domain boundaries, and use specific precautions if traffic traverses 884 the Internet. 886 For security reasons, the Alternate Marking Method is RECOMMENDED 887 only for controlled domains. 889 8. Compliance with Guidelines from RFC 6390 891 RFC 6390 [RFC6390] defines a framework and a process for developing 892 Performance Metrics for protocols above and below the IP layer (such 893 as IP-based applications that operate over reliable or datagram 894 transport protocols). 896 This document doesn't aim to propose a new Performance Metric but 897 rather a new Method of Measurement for a few Performance Metrics that 898 have already been standardized. Nevertheless, it's worth applying 899 guidelines from [RFC6390] to the present document, in order to 900 provide a more complete and coherent description of the proposed 901 method. We used a combination of the Performance Metric Definition 902 template defined in Section 5.4 of [RFC6390] and the Dependencies 903 laid out in Section 5.5 of that document. 905 o Metric Name / Metric Description: as already stated, this document 906 doesn't propose any new Performance Metrics. On the contrary, it 907 describes a novel method for measuring packet loss [RFC7680]. The 908 same concept, with small differences, can also be used to measure 909 delay [RFC7679] and jitter [RFC3393]. The document mainly 910 describes the applicability to packet loss measurement. 912 o Method of Measurement or Calculation: according to the method 913 described in the previous sections, the number of packets lost is 914 calculated by subtracting the value of the counter on the source 915 node from the value of the counter on the destination node. Both 916 counters must refer to the same color. The calculation is 917 performed when the value of the counters is in a steady state. 918 The steady state is an intrinsic characteristic of the marking 919 method counters because the alternation of color makes the 920 counters associated with each color still one at a time for the 921 duration of a marking period. 923 o Units of Measurement: the method calculates and reports the exact 924 number of packets sent by the source node and not received by the 925 destination node. 927 o Measurement Point(s) with Potential Measurement Domain: the 928 measurement can be performed between adjacent nodes, on a per-link 929 basis, or along a multi-hop path, provided that the traffic under 930 measurement follows that path. In case of a multi-hop path, the 931 measurements can be performed both end-to-end and hop-by-hop. 933 o Measurement Timing: the method has a constraint on the frequency 934 of measurements. This is detailed in Section 5, where it is 935 specified that the marking period and the guard band interval are 936 strictly related each other to avoid out-of-order issues. That is 937 because, in order to perform a measurement, the counter must be in 938 a steady state, and this happens when the traffic is being colored 939 with the alternate color. 941 o Implementation: the method uses one or two marking bits to color 942 the packets; this enables the use of policy configurations on the 943 router to color the packets and accordingly configure the counter 944 for each color. The path followed by traffic being measured 945 should be known in advance in order to configure the counters 946 along the path and be able to compare the correct values. 948 o Verification: both in the lab and in the operational network, the 949 methodology has been tested and experimented for packet loss and 950 delay measurements by using traffic generators together with 951 precision test instruments and network emulators. 953 o Use and Applications: the method can be used to measure packet 954 loss with high precision on live traffic; moreover, by combining 955 end-to-end and per-link measurements, the method is useful to 956 pinpoint the single link that is experiencing loss events. 958 o Reporting Model: the value of the counters has to be sent to a 959 centralized management system that performs the calculations; such 960 samples must contain a reference to the time interval they refer 961 to, so that the management system can perform the correct 962 correlation; the samples have to be sent while the corresponding 963 counter is in a steady state (within a time interval); otherwise, 964 the value of the sample should be stored locally. 966 o Dependencies: the values of the counters have to be correlated to 967 the time interval they refer to. 969 o Organization of Results: the Method of Measurement produces 970 singletons. 972 o Parameters: currently, the main parameter of the method is the 973 time interval used to alternate the colors and read the counters. 975 9. IANA Considerations 977 This document has no IANA actions. 979 10. Security Considerations 981 This document specifies a method to perform measurements in the 982 context of a Service Provider's network and has not been developed to 983 conduct Internet measurements, so it does not directly affect 984 Internet security nor applications that run on the Internet. 985 However, implementation of this method must be mindful of security 986 and privacy concerns. 988 There are two types of security concerns: potential harm caused by 989 the measurements and potential harm to the measurements. 991 o Harm caused by the measurement: the measurements described in this 992 document are Passive, so there are no new packets injected into 993 the network causing potential harm to the network itself and to 994 data traffic. Nevertheless, the method implies modifications on 995 the fly to a header or encapsulation of the data packets: this 996 must be performed in a way that doesn't alter the quality of 997 service experienced by packets subject to measurements and that 998 preserves stability and performance of routers doing the 999 measurements. One of the main security threats in OAM protocols 1000 is network reconnaissance; an attacker can gather information 1001 about the network performance by passively eavesdropping on OAM 1002 messages. The advantage of the methods described in this document 1003 is that the marking bits are the only information that is 1004 exchanged between the network devices. Therefore, Passive 1005 eavesdropping on data-plane traffic does not allow attackers to 1006 gain information about the network performance. 1008 o Harm to the Measurement: the measurements could be harmed by 1009 routers altering the marking of the packets or by an attacker 1010 injecting artificial traffic. Authentication techniques, such as 1011 digital signatures, may be used where appropriate to guard against 1012 injected traffic attacks. Since the measurement itself may be 1013 affected by routers (or other network devices) along the path of 1014 IP packets intentionally altering the value of marking bits of 1015 packets, as mentioned above, the mechanism specified in this 1016 document can be applied just in the context of a controlled 1017 domain; thus, the routers (or other network devices) are locally 1018 administered and this type of attack can be avoided. In addition, 1019 an attacker can't gain information about network performance from 1020 a single monitoring point; it must use synchronized monitoring 1021 points at multiple points on the path, because they have to do the 1022 same kind of measurement and aggregation that Service Providers 1023 using Alternate Marking must do. 1025 Attacks on the data collection and reporting of the statistics 1026 between the monitoring points and the network management system can 1027 interfere with the proper functioning of the system. Hence, the 1028 channels used to report back flow statistics MUST be secured. 1030 The privacy concerns of network measurement are limited because the 1031 method only relies on information contained in the header or 1032 encapsulation without any release of user data. Although information 1033 in the header or encapsulation is metadata that can be used to 1034 compromise the privacy of users, the limited marking technique in 1035 this document seems unlikely to substantially increase the existing 1036 privacy risks from header or encapsulation metadata. It might be 1037 theoretically possible to modulate the marking to serve as a covert 1038 channel, but it would have a very low data rate if it is to avoid 1039 adversely affecting the measurement systems that monitor the marking. 1041 Delay attacks are another potential threat in the context of this 1042 document. Delay measurement is performed using a specific packet in 1043 each block, marked by a dedicated color bit. Therefore, a 1044 man-in-the-middle attacker can selectively induce synthetic delay 1045 only to delay-colored packets, causing systematic error in the delay 1046 measurements. As discussed in previous sections, the methods 1047 described in this document rely on an underlying time synchronization 1048 protocol. Thus, by attacking the time protocol, an attacker can 1049 potentially compromise the integrity of the measurement. A detailed 1050 discussion about the threats against time protocols and how to 1051 mitigate them is presented in RFC 7384 [RFC7384]. 1053 11. Contributors 1055 Mach(Guoyi) Chen 1056 Huawei Technologies 1057 Email: mach.chen@huawei.com 1059 Alessandro Capello 1060 Telecom Italia 1061 Email: alessandro.capello@telecomitalia.it 1063 12. Acknowledgements 1065 The authors would like to thank Alberto Tempia Bonda, Luca 1066 Castaldelli and Lianshu Zheng for their contribution to the 1067 experimentation of the method. 1069 The authors would also thank Martin Duke and Tommy Pauly for their 1070 assistance and their detailed and precious reviews. 1072 13. References 1074 13.1. Normative References 1076 [IEEE-1588] 1077 IEEE, "IEEE Standard for a Precision Clock Synchronization 1078 Protocol for Networked Measurement and Control Systems", 1079 IEEE Std 1588-2008. 1081 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1082 Requirement Levels", BCP 14, RFC 2119, 1083 DOI 10.17487/RFC2119, March 1997, 1084 . 1086 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 1087 "Network Time Protocol Version 4: Protocol and Algorithms 1088 Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, 1089 . 1091 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1092 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1093 May 2017, . 1095 13.2. Informative References 1097 [I-D.fioccola-rfc8889bis] 1098 Fioccola, G., Cociglio, M., Sapio, A., Sisto, R., and T. 1099 Zhou, "Multipoint Alternate-Marking Method", draft- 1100 fioccola-rfc8889bis-02 (work in progress), February 2022. 1102 [I-D.mizrahi-ippm-marking] 1103 Mizrahi, T., Fioccola, G., Cociglio, M., Chen, M., and G. 1104 Mirsky, "Marking Methods for Performance Measurement", 1105 draft-mizrahi-ippm-marking-00 (work in progress), October 1106 2021. 1108 [I-D.zhou-ippm-enhanced-alternate-marking] 1109 Zhou, T., Fioccola, G., Liu, Y., Lee, S., Cociglio, M., 1110 and W. Li, "Enhanced Alternate Marking Method", draft- 1111 zhou-ippm-enhanced-alternate-marking-08 (work in 1112 progress), January 2022. 1114 [IEEE-Network-PNPM] 1115 IEEE Network, "AM-PM: Efficient Network Telemetry using 1116 Alternate Marking", DOI 10.1109/MNET.2019.1800152, 2019. 1118 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 1119 Metric for IP Performance Metrics (IPPM)", RFC 3393, 1120 DOI 10.17487/RFC3393, November 2002, 1121 . 1123 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 1124 Zekauskas, "A One-way Active Measurement Protocol 1125 (OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006, 1126 . 1128 [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. 1129 Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", 1130 RFC 5357, DOI 10.17487/RFC5357, October 2008, 1131 . 1133 [RFC5481] Morton, A. and B. Claise, "Packet Delay Variation 1134 Applicability Statement", RFC 5481, DOI 10.17487/RFC5481, 1135 March 2009, . 1137 [RFC6374] Frost, D. and S. Bryant, "Packet Loss and Delay 1138 Measurement for MPLS Networks", RFC 6374, 1139 DOI 10.17487/RFC6374, September 2011, 1140 . 1142 [RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New 1143 Performance Metric Development", BCP 170, RFC 6390, 1144 DOI 10.17487/RFC6390, October 2011, 1145 . 1147 [RFC6703] Morton, A., Ramachandran, G., and G. Maguluri, "Reporting 1148 IP Network Performance Metrics: Different Points of View", 1149 RFC 6703, DOI 10.17487/RFC6703, August 2012, 1150 . 1152 [RFC7276] Mizrahi, T., Sprecher, N., Bellagamba, E., and Y. 1153 Weingarten, "An Overview of Operations, Administration, 1154 and Maintenance (OAM) Tools", RFC 7276, 1155 DOI 10.17487/RFC7276, June 2014, 1156 . 1158 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in 1159 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384, 1160 October 2014, . 1162 [RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, 1163 Ed., "A One-Way Delay Metric for IP Performance Metrics 1164 (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January 1165 2016, . 1167 [RFC7680] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, 1168 Ed., "A One-Way Loss Metric for IP Performance Metrics 1169 (IPPM)", STD 82, RFC 7680, DOI 10.17487/RFC7680, January 1170 2016, . 1172 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 1173 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 1174 May 2016, . 1176 [RFC8321] Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli, 1177 L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi, 1178 "Alternate-Marking Method for Passive and Hybrid 1179 Performance Monitoring", RFC 8321, DOI 10.17487/RFC8321, 1180 January 2018, . 1182 [RFC8799] Carpenter, B. and B. Liu, "Limited Domains and Internet 1183 Protocols", RFC 8799, DOI 10.17487/RFC8799, July 2020, 1184 . 1186 Appendix A. Changes Log 1188 Changes from RFC 8321 include: 1190 o Minor editorial changes 1192 o Replacement of the section on "Applications, Implementation, and 1193 Deployment" with "Finding of the Alternate Marking Implementations 1194 and Deployments" 1196 o Moved advantages and benefits of the method from "Introduction" to 1197 the new section on "Finding of the Alternate Marking 1198 Implementations and Deployments" 1200 o Removed section on "Hybrid Measurement" 1202 Changes in v-(01) include: 1204 o Considerations on the reference: [IEEE-Network-PNPM] 1206 o Clarified that the method based on a fixed timer is specified in 1207 this document while the method based on a fixed number of packets 1208 is only mentioned but not detailed. 1210 o Explanation of the the intrinsic error in section 3.3.1 on 1211 "Single-Marking Methodology" 1213 o Deleted some parts in section 4 "Considerations" that no longer 1214 apply 1216 o New section on "Packet Fragmentation" 1218 Changes in v-(02) include: 1220 o Considerations on how to handle unmarked traffic in section 5 on 1221 "Results of the Alternate Marking Experiment" 1223 o Minor rewording in section 4.4 on "Packet Fragmentation" 1225 Changes in v-(03) include: 1227 o Deleted numeric examples in sections on "Packet Loss Measurement" 1228 and on "Single-Marking Methodology" 1230 o New section on "Alternate Marking Functions" 1231 o Moved sections 3.1.1 on "Coloring the Packets", 3.1.2 on "Counting 1232 the Packets" and 3.1.3 on "Collecting Data and Calculating Packet 1233 Loss" into the new section on "Alternate Marking Functions" 1235 o Renamed sections 4.1 as "Marking the Packets", 4.2 as "Counting 1236 and Timestamping Packets" and 4.3 as "Data Collection and 1237 Correlation" 1239 o Merged old section on "Data Correlation" with section 4.3 on "Data 1240 Collection and Correlation" 1242 o Moved and renamed section on "Timing Aspects" as "Synchronization 1243 and Timing" 1245 o Merged old section on "Synchronization" with section on 1246 "Synchronization and Timing" 1248 o Merged old section on "Packet Reordering" with section on 1249 "Synchronization and Timing" 1251 Authors' Addresses 1253 Giuseppe Fioccola (editor) 1254 Huawei Technologies 1255 Riesstrasse, 25 1256 Munich 80992 1257 Germany 1259 Email: giuseppe.fioccola@huawei.com 1261 Mauro Cociglio 1262 Telecom Italia 1263 Via Reiss Romoli, 274 1264 Torino 10148 1265 Italy 1267 Email: mauro.cociglio@telecomitalia.it 1269 Greg Mirsky 1270 Ericsson 1272 Email: gregimirsky@gmail.com 1273 Tal Mizrahi 1274 Huawei Technologies 1276 Email: tal.mizrahi.phd@gmail.com 1278 Tianran Zhou 1279 Huawei Technologies 1280 156 Beiqing Rd. 1281 Beijing 100095 1282 China 1284 Email: zhoutianran@huawei.com 1286 Xiao Min 1287 ZTE Corp. 1289 Email: xiao.min2@zte.com.cn