idnits 2.17.00 (12 Aug 2021) /tmp/idnits19903/draft-fioccola-rfc8321bis-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([RFC8321]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 19, 2021) is 183 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'IEEE-1588' == Outdated reference: A later version (-09) exists of draft-zhou-ippm-enhanced-alternate-marking-07 Summary: 1 error (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group G. Fioccola, Ed. 3 Internet-Draft Huawei Technologies 4 Obsoletes: 8321 (if approved) M. Cociglio 5 Intended status: Standards Track Telecom Italia 6 Expires: May 23, 2022 G. Mirsky 7 Ericsson 8 T. Mizrahi 9 Huawei Technologies 10 November 19, 2021 12 Alternate-Marking Method 13 draft-fioccola-rfc8321bis-00 15 Abstract 17 This document describes a method based on an Alternate-Marking 18 technique to perform packet loss, delay, and jitter measurements on 19 live traffic. This technology can be applied in various situations 20 and for different protocols. It could be considered Passive or 21 Hybrid depending on the application. This document obsoletes 22 [RFC8321]. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on May 23, 2022. 41 Copyright Notice 43 Copyright (c) 2021 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (https://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 59 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 4 60 2. Overview of the Method . . . . . . . . . . . . . . . . . . . 4 61 3. Detailed Description of the Method . . . . . . . . . . . . . 5 62 3.1. Packet Loss Measurement . . . . . . . . . . . . . . . . . 5 63 3.1.1. Coloring the Packets . . . . . . . . . . . . . . . . 10 64 3.1.2. Counting the Packets . . . . . . . . . . . . . . . . 10 65 3.1.3. Collecting Data and Calculating Packet Loss . . . . . 11 66 3.2. Timing Aspects . . . . . . . . . . . . . . . . . . . . . 12 67 3.3. One-Way Delay Measurement . . . . . . . . . . . . . . . . 13 68 3.3.1. Single-Marking Methodology . . . . . . . . . . . . . 13 69 3.3.2. Double-Marking Methodology . . . . . . . . . . . . . 15 70 3.4. Delay Variation Measurement . . . . . . . . . . . . . . . 17 71 4. Considerations . . . . . . . . . . . . . . . . . . . . . . . 17 72 4.1. Synchronization . . . . . . . . . . . . . . . . . . . . . 17 73 4.2. Data Correlation . . . . . . . . . . . . . . . . . . . . 18 74 4.3. Packet Reordering . . . . . . . . . . . . . . . . . . . . 19 75 5. Results of the Alternate Marking Experiment . . . . . . . . . 19 76 5.1. Controlled Domain requirement . . . . . . . . . . . . . . 21 77 6. Compliance with Guidelines from RFC 6390 . . . . . . . . . . 21 78 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 79 8. Security Considerations . . . . . . . . . . . . . . . . . . . 23 80 9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 24 81 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 25 82 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 25 83 11.1. Normative References . . . . . . . . . . . . . . . . . . 25 84 11.2. Informative References . . . . . . . . . . . . . . . . . 25 85 Appendix A. Changes Log . . . . . . . . . . . . . . . . . . . . 27 86 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27 88 1. Introduction 90 Nowadays, most Service Providers' networks carry traffic with 91 contents that are highly sensitive to packet loss [RFC7680], delay 92 [RFC7679], and jitter [RFC3393]. 94 In view of this scenario, Service Providers need methodologies and 95 tools to monitor and measure network performance with an adequate 96 accuracy, in order to constantly control the quality of experience 97 perceived by their customers. On the other hand, performance 98 monitoring provides useful information for improving network 99 management (e.g., isolation of network problems, troubleshooting, 100 etc.). 102 A lot of work related to Operations, Administration, and Maintenance 103 (OAM), which also includes performance monitoring techniques, has 104 been done by Standards Developing Organizations (SDOs): [RFC7276] 105 provides a good overview of existing OAM mechanisms defined in the 106 IETF, ITU-T, and IEEE. In the IETF, a lot of work has been done on 107 fault detection and connectivity verification, while a minor effort 108 has been thus far dedicated to performance monitoring. The IPPM WG 109 has defined standard metrics to measure network performance; however, 110 the methods developed in this WG mainly refer to focus on Active 111 measurement techniques. More recently, the MPLS WG has defined 112 mechanisms for measuring packet loss, one-way and two-way delay, and 113 delay variation in MPLS networks [RFC6374], but their applicability 114 to Passive measurements has some limitations, especially for pure 115 connection-less networks. 117 The lack of adequate tools to measure packet loss with the desired 118 accuracy drove an effort to design a new method for the performance 119 monitoring of live traffic, which is easy to implement and deploy. 120 The effort led to the method described in this document and in the 121 paper [IEEE-Network-PNPM]: basically, it is a Passive performance 122 monitoring technique, potentially applicable to any kind of packet- 123 based traffic, including Ethernet, IP, and MPLS, both unicast and 124 multicast. The method addresses primarily packet loss measurement, 125 but it can be easily extended to one-way or two-way delay and delay 126 variation measurements as well. 128 The method has been explicitly designed for Passive measurements, but 129 it can also be used with Active probes. Passive measurements are 130 usually more easily understood by customers and provide much better 131 accuracy, especially for packet loss measurements. 133 RFC 7799 [RFC7799] defines Passive and Hybrid Methods of Measurement. 134 In particular, Passive Methods of Measurement are based solely on 135 observations of an undisturbed and unmodified packet stream of 136 interest; Hybrid Methods are Methods of Measurement that use a 137 combination of Active Methods and Passive Methods. 139 Taking into consideration these definitions, the Alternate-Marking 140 Method could be considered Hybrid or Passive, depending on the case. 141 In the case where the marking method is obtained by changing existing 142 field values of the packets the technique is Hybrid. In the case 143 where the marking field is dedicated, reserved, and included in the 144 protocol specification, the Alternate-Marking technique can be 145 considered as Passive. 147 1.1. Requirements Language 149 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 150 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 151 "OPTIONAL" in this document are to be interpreted as described in BCP 152 14 [RFC2119] [RFC8174] when, and only when, they appear in all 153 capitals, as shown here. 155 2. Overview of the Method 157 In order to perform packet loss measurements on a production traffic 158 flow, different approaches exist. The most intuitive one consists in 159 numbering the packets so that each router that receives the flow can 160 immediately detect a packet that is missing. This approach, though 161 very simple in theory, is not simple to achieve: it requires the 162 insertion of a sequence number into each packet, and the devices must 163 be able to extract the number and check it in real time. Such a task 164 can be difficult to implement on live traffic: if UDP is used as the 165 transport protocol, the sequence number is not available; on the 166 other hand, if a higher-layer sequence number (e.g., in the RTP 167 header) is used, extracting that information from each packet and 168 processing it in real time could overload the device. 170 An alternate approach is to count the number of packets sent on one 171 end, count the number of packets received on the other end, and 172 compare the two values. This operation is much simpler to implement, 173 but it requires the devices performing the measurement to be in sync: 174 in order to compare two counters, it is required that they refer 175 exactly to the same set of packets. Since a flow is continuous and 176 cannot be stopped when a counter has to be read, it can be difficult 177 to determine exactly when to read the counter. A possible solution 178 to overcome this problem is to virtually split the flow in 179 consecutive blocks by periodically inserting a delimiter so that each 180 counter refers exactly to the same block of packets. The delimiter 181 could be, for example, a special packet inserted artificially into 182 the flow. However, delimiting the flow using specific packets has 183 some limitations. First, it requires generating additional packets 184 within the flow and requires the equipment to be able to process 185 those packets. In addition, the method is vulnerable to out-of-order 186 reception of delimiting packets and, to a lesser extent, to their 187 loss. 189 The method proposed in this document follows the second approach, but 190 it doesn't use additional packets to virtually split the flow in 191 blocks. Instead, it "marks" the packets so that the packets 192 belonging to the same block will have the same color, whilst 193 consecutive blocks will have different colors. Each change of color 194 represents a sort of auto-synchronization signal that guarantees the 195 consistency of measurements taken by different devices along the 196 path. 198 Figure 1 represents a very simple network and shows how the method 199 can be used to measure packet loss on different network segments: by 200 enabling the measurement on several interfaces along the path, it is 201 possible to perform link monitoring, node monitoring, or end-to-end 202 monitoring. The method is flexible enough to measure packet loss on 203 any segment of the network and can be used to isolate the faulty 204 element. 206 Traffic Flow 207 ========================================================> 208 +------+ +------+ +------+ +------+ 209 ---<> R1 <>-----<> R2 <>-----<> R3 <>-----<> R4 <>--- 210 +------+ +------+ +------+ +------+ 211 . . . . . . 212 . . . . . . 213 . <------> <-------> . 214 . Node Packet Loss Link Packet Loss . 215 . . 216 <---------------------------------------------------> 217 End-to-End Packet Loss 219 Figure 1: Available Measurements 221 3. Detailed Description of the Method 223 This section describes, in detail, how the method operates. A 224 special emphasis is given to the measurement of packet loss, which 225 represents the core application of the method, but applicability to 226 delay and jitter measurements is also considered. 228 3.1. Packet Loss Measurement 230 The basic idea is to virtually split traffic flows into consecutive 231 blocks: each block represents a measurable entity unambiguously 232 recognizable by all network devices along the path. By counting the 233 number of packets in each block and comparing the values measured by 234 different network devices along the path, it is possible to measure 235 packet loss occurred in any single block between any two points. 237 As discussed in the previous section, a simple way to create the 238 blocks is to "color" the traffic (two colors are sufficient), so that 239 packets belonging to different consecutive blocks will have different 240 colors. Whenever the color changes, the previous block terminates 241 and the new one begins. Hence, all the packets belonging to the same 242 block will have the same color and packets of different consecutive 243 blocks will have different colors. The number of packets in each 244 block depends on the criterion used to create the blocks: 246 o if the color is switched after a fixed number of packets, then 247 each block will contain the same number of packets (except for any 248 losses); and 250 o if the color is switched according to a fixed timer, then the 251 number of packets may be different in each block depending on the 252 packet rate. 254 The following figure shows how a flow looks like when it is split in 255 traffic blocks with colored packets. 257 A: packet with A coloring 258 B: packet with B coloring 260 | | | | | 261 | | Traffic Flow | | 262 -------------------------------------------------------------------> 263 BBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA 264 -------------------------------------------------------------------> 265 ... | Block 5 | Block 4 | Block 3 | Block 2 | Block 1 266 | | | | | 268 Figure 2: Traffic Coloring 270 Figure 3 shows how the method can be used to measure link packet loss 271 between two adjacent nodes. 273 Referring to the figure, let's assume we want to monitor the packet 274 loss on the link between two routers: router R1 and router R2. 275 According to the method, the traffic is colored alternatively with 276 two different colors: A and B. Whenever the color changes, the 277 transition generates a sort of square-wave signal, as depicted in the 278 following figure. 280 Color A ----------+ +-----------+ +---------- 281 | | | | 282 Color B +-----------+ +-----------+ 283 Block n ... Block 3 Block 2 Block 1 284 <---------> <---------> <---------> <---------> <---------> 286 Traffic Flow 287 ===========================================================> 288 Color ...AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA... 289 ===========================================================> 291 Figure 3: Computation of Link Packet Loss 293 Traffic coloring can be done by R1 itself if the traffic is not 294 already colored. R1 needs two counters, C(A)R1 and C(B)R1, on its 295 egress interface: C(A)R1 counts the packets with color A and C(B)R1 296 counts those with color B. As long as traffic is colored as A, only 297 counter C(A)R1 will be incremented, while C(B)R1 is not incremented; 298 conversely, when the traffic is colored as B, only C(B)R1 is 299 incremented. C(A)R1 and C(B)R1 can be used as reference values to 300 determine the packet loss from R1 to any other measurement point down 301 the path. Router R2, similarly, will need two counters on its 302 ingress interface, C(A)R2 and C(B)R2, to count the packets received 303 on that interface and colored with A and B, respectively. When an A 304 block ends, it is possible to compare C(A)R1 and C(A)R2 and calculate 305 the packet loss within the block; similarly, when the successive B 306 block terminates, it is possible to compare C(B)R1 with C(B)R2, and 307 so on, for every successive block. 309 Likewise, by using two counters on the R2 egress interface, it is 310 possible to count the packets sent out of the R2 interface and use 311 them as reference values to calculate the packet loss from R2 to any 312 measurement point down R2. 314 Using a fixed timer for color switching offers better control over 315 the method: the (time) length of the blocks can be chosen large 316 enough to simplify the collection and the comparison of measures 317 taken by different network devices. It's preferable to read the 318 value of the counters not immediately after the color switch: some 319 packets could arrive out of order and increment the counter 320 associated with the previous block (color), so it is worth waiting 321 for some time. A safe choice is to wait L/2 time units (where L is 322 the duration for each block) after the color switch, to read the 323 still counter of the previous color, so the possibility of reading a 324 running counter instead of a still one is minimized. The drawback is 325 that the longer the duration of the block, the less frequent the 326 measurement can be taken. 328 The following table shows how the counters can be used to calculate 329 the packet loss between R1 and R2. The first column lists the 330 sequence of traffic blocks, while the other columns contain the 331 counters of A-colored packets and B-colored packets for R1 and R2. 332 In this example, we assume that the values of the counters are reset 333 to zero whenever a block ends and its associated counter has been 334 read: with this assumption, the table shows only relative values, 335 which is the exact number of packets of each color within each block. 336 If the values of the counters were not reset, the table would contain 337 cumulative values, but the relative values could be determined simply 338 by the difference from the value of the previous block of the same 339 color. 341 The color is switched on the basis of a fixed timer (not shown in the 342 table), so the number of packets in each block is different. 344 +-------+--------+--------+--------+--------+------+ 345 | Block | C(A)R1 | C(B)R1 | C(A)R2 | C(B)R2 | Loss | 346 +-------+--------+--------+--------+--------+------+ 347 | 1 | 375 | 0 | 375 | 0 | 0 | 348 | 2 | 0 | 388 | 0 | 388 | 0 | 349 | 3 | 382 | 0 | 381 | 0 | 1 | 350 | 4 | 0 | 377 | 0 | 374 | 3 | 351 | ... | ... | ... | ... | ... | ... | 352 | 2n | 0 | 387 | 0 | 387 | 0 | 353 | 2n+1 | 379 | 0 | 377 | 0 | 2 | 354 +-------+--------+--------+--------+--------+------+ 356 Figure 4: Evaluation of Counters for Packet Loss Measurements 358 During an A block (blocks 1, 3, and 2n+1), all the packets are 359 A-colored; therefore, the C(A) counters are incremented to the number 360 seen on the interface, while C(B) counters are zero. Conversely, 361 during a B block (blocks 2, 4, and 2n), all the packets are 362 B-colored: C(A) counters are zero, while C(B) counters are 363 incremented. 365 When a block ends (because of color switching), the relative counters 366 stop incrementing; it is possible to read them, compare the values 367 measured on routers R1 and R2, and calculate the packet loss within 368 that block. 370 For example, looking at the table above, during the first block 371 (A-colored), C(A)R1 and C(A)R2 have the same value (375), which 372 corresponds to the exact number of packets of the first block (no 373 loss). Also, during the second block (B-colored), R1 and R2 counters 374 have the same value (388), which corresponds to the number of packets 375 of the second block (no loss). During the third and fourth blocks, 376 R1 and R2 counters are different, meaning that some packets have been 377 lost: in the example, one single packet (382-381) was lost during 378 block three, and three packets (377-374) were lost during block four. 380 The method applied to R1 and R2 can be extended to any other router 381 and applied to more complex networks, as far as the measurement is 382 enabled on the path followed by the traffic flow(s) being observed. 384 It's worth mentioning two different strategies that can be used when 385 implementing the method: 387 o flow-based: the flow-based strategy is used when only a limited 388 number of traffic flows need to be monitored. According to this 389 strategy, only a subset of the flows is colored. Counters for 390 packet loss measurements can be instantiated for each single flow, 391 or for the set as a whole, depending on the desired granularity. 392 A relevant problem with this approach is the necessity to know in 393 advance the path followed by flows that are subject to 394 measurement. Path rerouting and traffic load-balancing increase 395 the issue complexity, especially for unicast traffic. The problem 396 is easier to solve for multicast traffic, where load-balancing is 397 seldom used and static joins are frequently used to force traffic 398 forwarding and replication. 400 o link-based: measurements are performed on all the traffic on a 401 link-by-link basis. The link could be a physical link or a 402 logical link. Counters could be instantiated for the traffic as a 403 whole or for each traffic class (in case it is desired to monitor 404 each class separately), but in the second case, a couple of 405 counters are needed for each class. 407 As mentioned, the flow-based measurement requires the identification 408 of the flow to be monitored and the discovery of the path followed by 409 the selected flow. It is possible to monitor a single flow or 410 multiple flows grouped together, but in this case, measurement is 411 consistent only if all the flows in the group follow the same path. 412 Moreover, if a measurement is performed by grouping many flows, it is 413 not possible to determine exactly which flow was affected by packet 414 loss. In order to have measures per single flow, it is necessary to 415 configure counters for each specific flow. Once the flow(s) to be 416 monitored has been identified, it is necessary to configure the 417 monitoring on the proper nodes. Configuring the monitoring means 418 configuring the rule to intercept the traffic and configuring the 419 counters to count the packets. To have just an end-to-end 420 monitoring, it is sufficient to enable the monitoring on the first- 421 and last-hop routers of the path: the mechanism is completely 422 transparent to intermediate nodes and independent from the path 423 followed by traffic flows. On the contrary, to monitor the flow on a 424 hop-by-hop basis along its whole path, it is necessary to enable the 425 monitoring on every node from the source to the destination. In case 426 the exact path followed by the flow is not known a priori (i.e., the 427 flow has multiple paths to reach the destination), it is necessary to 428 enable the monitoring system on every path: counters on interfaces 429 traversed by the flow will report packet count, whereas counters on 430 other interfaces will be null. 432 3.1.1. Coloring the Packets 434 The coloring operation is fundamental in order to create packet 435 blocks. This implies choosing where to activate the coloring and how 436 to color the packets. 438 In case of flow-based measurements, the flow to monitor can be 439 defined by a set of selection rules (e.g., header fields) used to 440 match a subset of the packets; in this way, it is possible to control 441 the number of involved nodes, the path followed by the packets, and 442 the size of the flows. It is possible, in general, to have multiple 443 coloring nodes or a single coloring node that is easier to manage and 444 doesn't raise any risk of conflict. Coloring in multiple nodes can 445 be done, and the requirement is that the coloring must change 446 periodically between the nodes according to the timing considerations 447 in Section 3.2; so every node that is designated as a measurement 448 point along the path should be able to identify unambiguously the 449 colored packets. Furthermore, [RFC8889] generalizes the coloring for 450 multipoint-to-multipoint flow. In addition, it can be advantageous 451 to color the flow as close as possible to the source because it 452 allows an end-to-end measure if a measurement point is enabled on the 453 last-hop router as well. 455 For link-based measurements, all traffic needs to be colored when 456 transmitted on the link. If the traffic had already been colored, 457 then it has to be re-colored because the color must be consistent on 458 the link. This means that each hop along the path must (re-)color 459 the traffic; the color is not required to be consistent along 460 different links. 462 Traffic coloring can be implemented by setting a specific bit in the 463 packet header and changing the value of that bit periodically. How 464 to choose the marking field depends on the application and is out of 465 scope here. However, some applications are reported in Section 5. 467 3.1.2. Counting the Packets 469 For flow-based measurements, assuming that the coloring of the 470 packets is performed only by the source nodes, the nodes between 471 source and destination (included) have to count the colored packets 472 that they receive and forward: this operation can be enabled on every 473 router along the path or only on a subset, depending on which network 474 segment is being monitored (a single link, a particular metro area, 475 the backbone, or the whole path). Since the color switches 476 periodically between two values, two counters (one for each value) 477 are needed: one counter for packets with color A and one counter for 478 packets with color B. For each flow (or group of flows) being 479 monitored and for every interface where the monitoring is Active, a 480 couple of counters are needed. For example, in order to separately 481 monitor three flows on a router with four interfaces involved, 24 482 counters are needed (two counters for each of the three flows on each 483 of the four interfaces). Furthermore, [RFC8889] generalizes the 484 counting for multipoint-to-multipoint flow. 486 In case of link-based measurements, the behavior is similar except 487 that coloring and counting operations are performed on a link-by-link 488 basis at each endpoint of the link. 490 Another important aspect to take into consideration is when to read 491 the counters: in order to count the exact number of packets of a 492 block, the routers must perform this operation when that block has 493 ended; in other words, the counter for color A must be read when the 494 current block has color B, in order to be sure that the value of the 495 counter is stable. This task can be accomplished in two ways. The 496 general approach suggests reading the counters periodically, many 497 times during a block duration, and comparing these successive 498 readings: when the counter stops incrementing, it means that the 499 current block has ended, and its value can be elaborated safely. 500 Alternatively, if the coloring operation is performed on the basis of 501 a fixed timer, it is possible to configure the reading of the 502 counters according to that timer: for example, reading the counter 503 for color A every period in the middle of the subsequent block with 504 color B is a safe choice. A sufficient margin should be considered 505 between the end of a block and the reading of the counter, in order 506 to take into account any out-of-order packets. 508 3.1.3. Collecting Data and Calculating Packet Loss 510 The nodes enabled to perform performance monitoring collect the value 511 of the counters, but they are not able to directly use this 512 information to measure packet loss, because they only have their own 513 samples. For this reason, an external Network Management System 514 (NMS) can be used to collect and elaborate data and to perform packet 515 loss calculation. The NMS compares the values of counters from 516 different nodes and can calculate if some packets were lost (even a 517 single packet) and where those packets were lost. 519 The value of the counters needs to be transmitted to the NMS as soon 520 as it has been read. This can be accomplished by using SNMP or FTP 521 and can be done in Push Mode or Polling Mode. In the first case, 522 each router periodically sends the information to the NMS; in the 523 latter case, it is the NMS that periodically polls routers to collect 524 information. In any case, the NMS has to collect all the relevant 525 values from all the routers within one cycle of the timer. 527 It would also be possible to use a protocol to exchange values of 528 counters between the two endpoints in order to let them perform the 529 packet loss calculation for each traffic direction. 531 3.2. Timing Aspects 533 This document introduces two color-switching methods: one is based on 534 a fixed number of packets, and the other is based on a fixed timer. 535 But the method based on a fixed timer is preferable because it is 536 more deterministic, and it will be considered in the rest of the 537 document. 539 In general, clocks in network devices are not accurate and for this 540 reason, there is a clock error between the measurement points R1 and 541 R2. But, to implement the methodology, they must be synchronized to 542 the same clock reference with an accuracy of +/- L/2 time units, 543 where L is the fixed time duration of the block. So each colored 544 packet can be assigned to the right batch by each router. This is 545 because the minimum time distance between two packets of the same 546 color but that belong to different batches is L time units. 548 In practice, in addition to clock errors, the delay between 549 measurement points also affects the implementation of the methodology 550 because each packet can be delayed differently, and this can produce 551 out of order at batch boundaries. This means that, without 552 considering clock error, we wait L/2 after color switching to be sure 553 to take a still counter. 555 In summary, we need to take into account two contributions: clock 556 error between network devices and the interval we need to wait to 557 avoid packets being out of order because of network delay. 559 The following figure explains both issues. 561 ...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB... 562 |<======================================>| 563 | L | 564 ...=========>|<==================><==================>|<==========... 565 | L/2 L/2 | 566 |<===>| |<===>| 567 d | | d 568 |<==========================>| 569 available counting interval 571 Figure 5: Timing Aspects 573 It is assumed that all network devices are synchronized to a common 574 reference time with an accuracy of +/- A/2. Thus, the difference 575 between the clock values of any two network devices is bounded by A. 577 The guard band d is given by: 579 d = A + D_max - D_min, 581 where A is the clock accuracy, D_max is an upper bound on the network 582 delay between the network devices, and D_min is a lower bound on the 583 delay. 585 The available counting interval is L - 2d that must be > 0. 587 The condition that must be satisfied and is a requirement on the 588 synchronization accuracy is: 590 d < L/2. 592 3.3. One-Way Delay Measurement 594 The same principle used to measure packet loss can be applied also to 595 one-way delay measurement. There are three alternatives, as 596 described hereinafter. 598 Note that, for all the one-way delay alternatives described in the 599 next sections, by summing the one-way delays of the two directions of 600 a path, it is always possible to measure the two-way delay (round- 601 trip "virtual" delay). 603 3.3.1. Single-Marking Methodology 605 The alternation of colors can be used as a time reference to 606 calculate the delay. Whenever the color changes (which means that a 607 new block has started), a network device can store the timestamp of 608 the first packet of the new block; that timestamp can be compared 609 with the timestamp of the same packet on a second router to compute 610 packet delay. When looking at Figure 2, R1 stores the timestamp 611 TS(A1)R1 when it sends the first packet of block 1 (A-colored), the 612 timestamp TS(B2)R1 when it sends the first packet of block 2 613 (B-colored), and so on for every other block. R2 performs the same 614 operation on the receiving side, recording TS(A1)R2, TS(B2)R2, and so 615 on. Since the timestamps refer to specific packets (the first packet 616 of each block), we are sure that timestamps compared to compute delay 617 refer to the same packets. By comparing TS(A1)R1 with TS(A1)R2 (and 618 similarly TS(B2)R1 with TS(B2)R2, and so on), it is possible to 619 measure the delay between R1 and R2. In order to have more 620 measurements, it is possible to take and store more timestamps, 621 referring to other packets within each block. 623 In order to coherently compare timestamps collected on different 624 routers, the clocks on the network nodes must be in sync. 625 Furthermore, a measurement is valid only if no packet loss occurs and 626 if packet misordering can be avoided; otherwise, the first packet of 627 a block on R1 could be different from the first packet of the same 628 block on R2 (for instance, if that packet is lost between R1 and R2 629 or it arrives after the next one). 631 The following table shows how timestamps can be used to calculate the 632 delay between R1 and R2. The first column lists the sequence of 633 blocks, while other columns contain the timestamp referring to the 634 first packet of each block on R1 and R2. The delay is computed as a 635 difference between timestamps. For the sake of simplicity, all the 636 values are expressed in milliseconds. 638 +-------+---------+---------+---------+---------+-------------+ 639 | Block | TS(A)R1 | TS(B)R1 | TS(A)R2 | TS(B)R2 | Delay R1-R2 | 640 +-------+---------+---------+---------+---------+-------------+ 641 | 1 | 12.483 | - | 15.591 | - | 3.108 | 642 | 2 | - | 6.263 | - | 9.288 | 3.025 | 643 | 3 | 27.556 | - | 30.512 | - | 2.956 | 644 | 4 | - | 18.113 | - | 21.269 | 3.156 | 645 | ... | ... | ... | ... | ... | ... | 646 | 2n | 77.463 | - | 80.501 | - | 3.038 | 647 | 2n+1 | - | 24.333 | - | 27.433 | 3.100 | 648 +-------+---------+---------+---------+---------+-------------+ 650 Figure 6: Evaluation of Timestamps for Delay Measurements 652 The first row shows timestamps taken on R1 and R2, respectively, and 653 refers to the first packet of block 1 (which is A-colored). Delay 654 can be computed as a difference between the timestamp on R2 and the 655 timestamp on R1. Similarly, the second row shows timestamps (in 656 milliseconds) taken on R1 and R2 and refers to the first packet of 657 block 2 (which is B-colored). By comparing timestamps taken on 658 different nodes in the network and referring to the same packets 659 (identified using the alternation of colors), it is possible to 660 measure delay on different network segments. 662 For the sake of simplicity, in the above example, a single 663 measurement is provided within a block, taking into account only the 664 first packet of each block. The number of measurements can be easily 665 increased by considering multiple packets in the block: for instance, 666 a timestamp could be taken every N packets, thus generating multiple 667 delay measurements. Taking this to the limit, in principle, the 668 delay could be measured for each packet by taking and comparing the 669 corresponding timestamps (possible but impractical from an 670 implementation point of view). 672 3.3.1.1. Mean Delay 674 As mentioned before, the method previously exposed for measuring the 675 delay is sensitive to out-of-order reception of packets. In order to 676 overcome this problem, a different approach has been considered: it 677 is based on the concept of mean delay. The mean delay is calculated 678 by considering the average arrival time of the packets within a 679 single block. The network device locally stores a timestamp for each 680 packet received within a single block: summing all the timestamps and 681 dividing by the total number of packets received, the average arrival 682 time for that block of packets can be calculated. By subtracting the 683 average arrival times of two adjacent devices, it is possible to 684 calculate the mean delay between those nodes. When computing the 685 mean delay, the measurement error could be augmented by accumulating 686 the measurement error of a lot of packets. This method is robust to 687 out-of-order packets and also to packet loss (only a small error is 688 introduced). Moreover, it greatly reduces the number of timestamps 689 (only one per block for each network device) that have to be 690 collected by the management system. On the other hand, it only gives 691 one measure for the duration of the block, and it doesn't give the 692 minimum, maximum, and median delay values [RFC6703]. This limitation 693 could be overcome by reducing the duration of the block (for 694 instance, from 5 minutes to a few seconds), which implicates a highly 695 optimized implementation of the method. 697 3.3.2. Double-Marking Methodology 699 The Single-Marking methodology for one-way delay measurement is 700 sensitive to out-of-order reception of packets. The first approach 701 to overcome this problem has been described before and is based on 702 the concept of mean delay. But the limitation of mean delay is that 703 it doesn't give information about the delay value's distribution for 704 the duration of the block. Additionally, it may be useful to have 705 not only the mean delay but also the minimum, maximum, and median 706 delay values and, in wider terms, to know more about the statistic 707 distribution of delay values. So, in order to have more information 708 about the delay and to overcome out-of-order issues, a different 709 approach can be introduced; it is based on a Double-Marking 710 methodology. 712 Basically, the idea is to use the first marking to create the 713 alternate flow and, within this colored flow, a second marking to 714 select the packets for measuring delay/jitter. The first marking is 715 needed for packet loss and mean delay measurement. The second 716 marking creates a new set of marked packets that are fully identified 717 over the network, so that a network device can store the timestamps 718 of these packets; these timestamps can be compared with the 719 timestamps of the same packets on a second router to compute packet 720 delay values for each packet. The number of measurements can be 721 easily increased by changing the frequency of the second marking. 722 But the frequency of the second marking must not be too high in order 723 to avoid out-of-order issues. Between packets with the second 724 marking, there should be a security time gap (e.g., this gap could 725 be, at the minimum, the mean network delay calculated with the 726 previous methodology) to avoid out-of-order issues and also to have a 727 number of measurement packets that are rate independent. If a 728 second-marking packet is lost, the delay measurement for the 729 considered block is corrupted and should be discarded. 731 Mean delay is calculated on all the packets of a sample and is a 732 simple computation to be performed for a Single-Marking Method. In 733 some cases, the mean delay measure is not sufficient to characterize 734 the sample, and more statistics of delay extent data are needed, 735 e.g., percentiles, variance, and median delay values. The 736 conventional range (maximum-minimum) should be avoided for several 737 reasons, including stability of the maximum delay due to the 738 influence by outliers. RFC 5481 [RFC5481], Section 6.5 highlights 739 how the 99.9th percentile of delay and delay variation is more 740 helpful to performance planners. To overcome this drawback, the idea 741 is to couple the mean delay measure for the entire batch with a 742 Double-Marking Method, where a subset of batch packets is selected 743 for extensive delay calculation by using a second marking. In this 744 way, it is possible to perform a detailed analysis on these double- 745 marked packets. Please note that there are classic algorithms for 746 median and variance calculation, but they are out of the scope of 747 this document. The comparison between the mean delay for the entire 748 batch and the mean delay on these double-marked packets gives useful 749 information since it is possible to understand if the Double-Marking 750 measurements are actually representative of the delay trends. 752 3.4. Delay Variation Measurement 754 Similar to one-way delay measurement (both for Single Marking and 755 Double Marking), the method can also be used to measure the inter- 756 arrival jitter. We refer to the definition in RFC 3393 [RFC3393]. 757 The alternation of colors, for a Single-Marking Method, can be used 758 as a time reference to measure delay variations. In case of Double 759 Marking, the time reference is given by the second-marked packets. 760 Considering the example depicted in Figure 2, R1 stores the timestamp 761 TS(A)R1 whenever it sends the first packet of a block, and R2 stores 762 the timestamp TS(B)R2 whenever it receives the first packet of a 763 block. The inter-arrival jitter can be easily derived from one-way 764 delay measurement, by evaluating the delay variation of consecutive 765 samples. 767 The concept of mean delay can also be applied to delay variation, by 768 evaluating the average variation of the interval between consecutive 769 packets of the flow from R1 to R2. 771 4. Considerations 773 This section highlights some considerations about the methodology. 775 4.1. Synchronization 777 The Alternate-Marking technique does not require a strong 778 synchronization, especially for packet loss and two-way delay 779 measurement. Only one-way delay measurement requires network devices 780 to have synchronized clocks. 782 Color switching is the reference for all the network devices, and the 783 only requirement to be achieved is that all network devices have to 784 recognize the right batch along the path. 786 If the length of the measurement period is L time units, then all 787 network devices must be synchronized to the same clock reference with 788 an accuracy of +/- L/2 time units (without considering network 789 delay). This level of accuracy guarantees that all network devices 790 consistently match the color bit to the correct block. For example, 791 if the color is toggled every second (L = 1 second), then clocks must 792 be synchronized with an accuracy of +/- 0.5 second to a common time 793 reference. 795 This synchronization requirement can be satisfied even with a 796 relatively inaccurate synchronization method. This is true for 797 packet loss and two-way delay measurement, but not for one-way delay 798 measurement, where clock synchronization must be accurate. 800 Therefore, a system that uses only packet loss and two-way delay 801 measurement does not require synchronization. This is because the 802 value of the clocks of network devices does not affect the 803 computation of the two-way delay measurement. 805 4.2. Data Correlation 807 Data correlation is the mechanism to compare counters and timestamps 808 for packet loss, delay, and delay variation calculation. It could be 809 performed in several ways depending on the Alternate-Marking 810 application and use case. Some possibilities are to: 812 o use a centralized solution using NMS to correlate data; and 814 o define a protocol-based distributed solution by introducing a new 815 protocol or by extending the existing protocols (e.g., see RFC 816 6374 [RFC6374] or the Two-Way Active Measurement Protocol (TWAMP) 817 as defined in RFC 5357 [RFC5357] or the One-Way Active Measurement 818 Protocol (OWAMP) as defined in RFC 4656 [RFC4656]) in order to 819 communicate the counters and timestamps between nodes. 821 In the following paragraphs, an example data correlation mechanism is 822 explained and could be used independently of the adopted solutions. 824 When data is collected on the upstream and downstream nodes, e.g., 825 packet counts for packet loss measurement or timestamps for packet 826 delay measurement, and is periodically reported to or pulled by other 827 nodes or an NMS, a certain data correlation mechanism SHOULD be in 828 use to help the nodes or NMS tell whether any two or more packet 829 counts are related to the same block of markers or if any two 830 timestamps are related to the same marked packet. 832 The Alternate-Marking Method described in this document literally 833 splits the packets of the measured flow into different measurement 834 blocks; in addition, a Block Number (BN) could be assigned to each 835 such measurement block. The BN is generated each time a node reads 836 the data (packet counts or timestamps) and is associated with each 837 packet count and timestamp reported to or pulled by other nodes or 838 NMSs. The value of a BN could be calculated as the modulo of the 839 local time (when the data are read) and the interval of the marking 840 time period. 842 When the nodes or NMS see, for example, the same BNs associated with 843 two packet counts from an upstream and a downstream node, 844 respectively, it considers that these two packet counts correspond to 845 the same block, i.e., these two packet counts belong to the same 846 block of markers from the upstream and downstream nodes. The 847 assumption of this BN mechanism is that the measurement nodes are 848 time synchronized. This requires the measurement nodes to have a 849 certain time synchronization capability (e.g., the Network Time 850 Protocol (NTP) [RFC5905] or the IEEE 1588 Precision Time Protocol 851 (PTP) [IEEE-1588]). Synchronization aspects are further discussed in 852 Section 4.1. 854 4.3. Packet Reordering 856 Due to ECMP, packet reordering is very common in an IP network. The 857 accuracy of a marking-based PM, especially packet loss measurement, 858 may be affected by packet reordering. Take a look at the following 859 example: 861 Block : 1 | 2 | 3 | 4 | 5 |... 862 --------|---------|---------|---------|---------|---------|--- 863 Node R1 : AAAAAAA | BBBBBBB | AAAAAAA | BBBBBBB | AAAAAAA |... 864 Node R2 : AAAAABB | AABBBBA | AAABAAA | BBBBBBA | ABAAABA |... 866 Figure 7: Packet Reordering 868 In Figure 7, the packet stream for Node R1 isn't being reordered and 869 can be safely assigned to interval blocks, but the packet stream for 870 Node R2 is being reordered; so, looking at the packet with the marker 871 of "B" in block 3, there is no safe way to tell whether the packet 872 belongs to block 2 or block 4. 874 In general, there is the need to assign packets with the marker of 875 "B" or "A" to the right interval blocks. Most of the packet 876 reordering occurs at the edge of adjacent blocks, and they are easy 877 to handle if the interval of each block is sufficiently large. Then, 878 it can be assumed that the packets with different markers belong to 879 the block that they are closer to. If the interval is small, it is 880 difficult and sometimes impossible to determine to which block a 881 packet belongs. 883 To choose a proper interval is important, and how to choose a proper 884 interval is out of the scope of this document. But an implementation 885 SHOULD provide a way to configure the interval and allow a certain 886 degree of packet reordering. 888 5. Results of the Alternate Marking Experiment 890 The methodology described in the previous sections can be applied to 891 various performance measurement problems, as explained in [RFC8321]. 892 The only requirement is to select and mark the flow to be monitored; 893 in this way, packets are batched by the sender, and each batch is 894 alternately marked such that it can be easily recognized by the 895 receiver. 897 Either one or two flag bits might be available for marking in 898 different deployments: 900 One flag: packet loss measurement SHOULD be done as described in 901 Section 3.1, while delay measurement MAY be done according to the 902 single-marking method described in Section 3.3.1. Mean delay 903 (Section 3.3.1.1) is NOT RECOMMENDED since it implies more 904 computational load. 906 Two flags: packet loss measurement SHOULD be done as described in 907 Section 3.1, while delay measurement SHOULD be done according to 908 double-marking method Section 3.3.2. In this case single-marking 909 MAY also be used in combination with double-marking and the two 910 approaches provide slightly different pieces of information that 911 can be combined to have a more robust data set. 913 The experiment with Alternate Marking methodologies confirmed the 914 following benefits: 916 o easy implementation: it can be implemented by using features 917 already available on major routing platforms, or by applying an 918 optimized implementation of the method for both legacy and newest 919 technologies; 921 o low computational effort: the additional load on processing is 922 negligible; 924 o accurate loss and delay measurements: single packet loss 925 granularity is achieved with a Passive measurement; 927 o potential applicability to any kind of packet-based or frame-based 928 traffic: Ethernet, IP, MPLS, etc., and both unicast and multicast; 930 o robustness: the method can easily tolerate out-of-order packets, 931 and it's not based on "special" packets whose loss could have a 932 negative impact; 934 o flexibility: all the timestamp formats are allowed, because they 935 are managed out of band. The format (the Network Time Protocol 936 (NTP) [RFC5905] or the IEEE 1588 Precision Time Protocol (PTP) 937 [IEEE-1588]) depends on the precision you want; and 939 o no interoperability issues: the features required are available on 940 all current routing platforms. Both a centralized or distributed 941 solution can be used to harvest data from the routers. 943 Note that [I-D.zhou-ippm-enhanced-alternate-marking] extends the 944 Alternate Marking Data Fields, to provide enhanced capabilities and 945 allow advanced functionalities. 947 5.1. Controlled Domain requirement 949 The Alternate Marking Method is an example of a solution limited to a 950 controlled domain [RFC8799]. 952 A controlled domain is a managed network that selects, monitors, and 953 controls access by enforcing policies at the domain boundaries, in 954 order to discard undesired external packets entering the domain and 955 check internal packets leaving the domain. It does not necessarily 956 mean that a controlled domain is a single administrative domain or a 957 single organization. A controlled domain can correspond to a single 958 administrative domain or multiple administrative domains under a 959 defined network management. It must be possible to control the 960 domain boundaries, and use specific precautions if traffic traverses 961 the Internet. 963 For security reasons, the Alternate Marking Method is RECOMMENDED 964 only for controlled domains. 966 6. Compliance with Guidelines from RFC 6390 968 RFC 6390 [RFC6390] defines a framework and a process for developing 969 Performance Metrics for protocols above and below the IP layer (such 970 as IP-based applications that operate over reliable or datagram 971 transport protocols). 973 This document doesn't aim to propose a new Performance Metric but 974 rather a new Method of Measurement for a few Performance Metrics that 975 have already been standardized. Nevertheless, it's worth applying 976 guidelines from [RFC6390] to the present document, in order to 977 provide a more complete and coherent description of the proposed 978 method. We used a combination of the Performance Metric Definition 979 template defined in Section 5.4 of [RFC6390] and the Dependencies 980 laid out in Section 5.5 of that document. 982 o Metric Name / Metric Description: as already stated, this document 983 doesn't propose any new Performance Metrics. On the contrary, it 984 describes a novel method for measuring packet loss [RFC7680]. The 985 same concept, with small differences, can also be used to measure 986 delay [RFC7679] and jitter [RFC3393]. The document mainly 987 describes the applicability to packet loss measurement. 989 o Method of Measurement or Calculation: according to the method 990 described in the previous sections, the number of packets lost is 991 calculated by subtracting the value of the counter on the source 992 node from the value of the counter on the destination node. Both 993 counters must refer to the same color. The calculation is 994 performed when the value of the counters is in a steady state. 995 The steady state is an intrinsic characteristic of the marking 996 method counters because the alternation of color makes the 997 counters associated with each color still one at a time for the 998 duration of a marking period. 1000 o Units of Measurement: the method calculates and reports the exact 1001 number of packets sent by the source node and not received by the 1002 destination node. 1004 o Measurement Point(s) with Potential Measurement Domain: the 1005 measurement can be performed between adjacent nodes, on a per-link 1006 basis, or along a multi-hop path, provided that the traffic under 1007 measurement follows that path. In case of a multi-hop path, the 1008 measurements can be performed both end-to-end and hop-by-hop. 1010 o Measurement Timing: the method has a constraint on the frequency 1011 of measurements. This is detailed in Section 3.2, where it is 1012 specified that the marking period and the guard band interval are 1013 strictly related each other to avoid out-of-order issues. That is 1014 because, in order to perform a measurement, the counter must be in 1015 a steady state, and this happens when the traffic is being colored 1016 with the alternate color. 1018 o Implementation: the method uses one or two marking bits to color 1019 the packets; this enables the use of policy configurations on the 1020 router to color the packets and accordingly configure the counter 1021 for each color. The path followed by traffic being measured 1022 should be known in advance in order to configure the counters 1023 along the path and be able to compare the correct values. 1025 o Verification: both in the lab and in the operational network, the 1026 methodology has been tested and experimented for packet loss and 1027 delay measurements by using traffic generators together with 1028 precision test instruments and network emulators. 1030 o Use and Applications: the method can be used to measure packet 1031 loss with high precision on live traffic; moreover, by combining 1032 end-to-end and per-link measurements, the method is useful to 1033 pinpoint the single link that is experiencing loss events. 1035 o Reporting Model: the value of the counters has to be sent to a 1036 centralized management system that performs the calculations; such 1037 samples must contain a reference to the time interval they refer 1038 to, so that the management system can perform the correct 1039 correlation; the samples have to be sent while the corresponding 1040 counter is in a steady state (within a time interval); otherwise, 1041 the value of the sample should be stored locally. 1043 o Dependencies: the values of the counters have to be correlated to 1044 the time interval they refer to. 1046 o Organization of Results: the Method of Measurement produces 1047 singletons. 1049 o Parameters: currently, the main parameter of the method is the 1050 time interval used to alternate the colors and read the counters. 1052 7. IANA Considerations 1054 This document has no IANA actions. 1056 8. Security Considerations 1058 This document specifies a method to perform measurements in the 1059 context of a Service Provider's network and has not been developed to 1060 conduct Internet measurements, so it does not directly affect 1061 Internet security nor applications that run on the Internet. 1062 However, implementation of this method must be mindful of security 1063 and privacy concerns. 1065 There are two types of security concerns: potential harm caused by 1066 the measurements and potential harm to the measurements. 1068 o Harm caused by the measurement: the measurements described in this 1069 document are Passive, so there are no new packets injected into 1070 the network causing potential harm to the network itself and to 1071 data traffic. Nevertheless, the method implies modifications on 1072 the fly to a header or encapsulation of the data packets: this 1073 must be performed in a way that doesn't alter the quality of 1074 service experienced by packets subject to measurements and that 1075 preserves stability and performance of routers doing the 1076 measurements. One of the main security threats in OAM protocols 1077 is network reconnaissance; an attacker can gather information 1078 about the network performance by passively eavesdropping on OAM 1079 messages. The advantage of the methods described in this document 1080 is that the marking bits are the only information that is 1081 exchanged between the network devices. Therefore, Passive 1082 eavesdropping on data-plane traffic does not allow attackers to 1083 gain information about the network performance. 1085 o Harm to the Measurement: the measurements could be harmed by 1086 routers altering the marking of the packets or by an attacker 1087 injecting artificial traffic. Authentication techniques, such as 1088 digital signatures, may be used where appropriate to guard against 1089 injected traffic attacks. Since the measurement itself may be 1090 affected by routers (or other network devices) along the path of 1091 IP packets intentionally altering the value of marking bits of 1092 packets, as mentioned above, the mechanism specified in this 1093 document can be applied just in the context of a controlled 1094 domain; thus, the routers (or other network devices) are locally 1095 administered and this type of attack can be avoided. In addition, 1096 an attacker can't gain information about network performance from 1097 a single monitoring point; it must use synchronized monitoring 1098 points at multiple points on the path, because they have to do the 1099 same kind of measurement and aggregation that Service Providers 1100 using Alternate Marking must do. 1102 The privacy concerns of network measurement are limited because the 1103 method only relies on information contained in the header or 1104 encapsulation without any release of user data. Although information 1105 in the header or encapsulation is metadata that can be used to 1106 compromise the privacy of users, the limited marking technique in 1107 this document seems unlikely to substantially increase the existing 1108 privacy risks from header or encapsulation metadata. It might be 1109 theoretically possible to modulate the marking to serve as a covert 1110 channel, but it would have a very low data rate if it is to avoid 1111 adversely affecting the measurement systems that monitor the marking. 1113 Delay attacks are another potential threat in the context of this 1114 document. Delay measurement is performed using a specific packet in 1115 each block, marked by a dedicated color bit. Therefore, a 1116 man-in-the-middle attacker can selectively induce synthetic delay 1117 only to delay-colored packets, causing systematic error in the delay 1118 measurements. As discussed in previous sections, the methods 1119 described in this document rely on an underlying time synchronization 1120 protocol. Thus, by attacking the time protocol, an attacker can 1121 potentially compromise the integrity of the measurement. A detailed 1122 discussion about the threats against time protocols and how to 1123 mitigate them is presented in RFC 7384 [RFC7384]. 1125 9. Contributors 1127 Tianran Zhou 1128 Huawei Technologies 1129 Email: zhoutianran@huawei.com 1131 Xiao Min 1132 ZTE Corp. 1133 Email: xiao.min2@zte.com.cn 1135 10. Acknowledgements 1137 The authors would like to thank Alessandro Capello, Luca Castaldelli, 1138 Mach Chen and Lianshu Zheng for their contribution to the definition 1139 and the implementation of the method. 1141 The authors would also thank Martin Duke and Tommy Pauly for their 1142 assistance and their detailed and precious reviews. 1144 11. References 1146 11.1. Normative References 1148 [IEEE-1588] 1149 IEEE, "IEEE Standard for a Precision Clock Synchronization 1150 Protocol for Networked Measurement and Control Systems", 1151 IEEE Std 1588-2008. 1153 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1154 Requirement Levels", BCP 14, RFC 2119, 1155 DOI 10.17487/RFC2119, March 1997, 1156 . 1158 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 1159 Metric for IP Performance Metrics (IPPM)", RFC 3393, 1160 DOI 10.17487/RFC3393, November 2002, 1161 . 1163 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 1164 "Network Time Protocol Version 4: Protocol and Algorithms 1165 Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, 1166 . 1168 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1169 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1170 May 2017, . 1172 11.2. Informative References 1174 [I-D.zhou-ippm-enhanced-alternate-marking] 1175 Zhou, T., Fioccola, G., Liu, Y., Lee, S., Cociglio, M., 1176 and W. Li, "Enhanced Alternate Marking Method", draft- 1177 zhou-ippm-enhanced-alternate-marking-07 (work in 1178 progress), July 2021. 1180 [IEEE-Network-PNPM] 1181 IEEE Network, "AM-PM: Efficient Network Telemetry using 1182 Alternate Marking", DOI 10.1109/MNET.2019.1800152, 2019. 1184 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 1185 Zekauskas, "A One-way Active Measurement Protocol 1186 (OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006, 1187 . 1189 [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. 1190 Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", 1191 RFC 5357, DOI 10.17487/RFC5357, October 2008, 1192 . 1194 [RFC5481] Morton, A. and B. Claise, "Packet Delay Variation 1195 Applicability Statement", RFC 5481, DOI 10.17487/RFC5481, 1196 March 2009, . 1198 [RFC6374] Frost, D. and S. Bryant, "Packet Loss and Delay 1199 Measurement for MPLS Networks", RFC 6374, 1200 DOI 10.17487/RFC6374, September 2011, 1201 . 1203 [RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New 1204 Performance Metric Development", BCP 170, RFC 6390, 1205 DOI 10.17487/RFC6390, October 2011, 1206 . 1208 [RFC6703] Morton, A., Ramachandran, G., and G. Maguluri, "Reporting 1209 IP Network Performance Metrics: Different Points of View", 1210 RFC 6703, DOI 10.17487/RFC6703, August 2012, 1211 . 1213 [RFC7276] Mizrahi, T., Sprecher, N., Bellagamba, E., and Y. 1214 Weingarten, "An Overview of Operations, Administration, 1215 and Maintenance (OAM) Tools", RFC 7276, 1216 DOI 10.17487/RFC7276, June 2014, 1217 . 1219 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in 1220 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384, 1221 October 2014, . 1223 [RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, 1224 Ed., "A One-Way Delay Metric for IP Performance Metrics 1225 (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January 1226 2016, . 1228 [RFC7680] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, 1229 Ed., "A One-Way Loss Metric for IP Performance Metrics 1230 (IPPM)", STD 82, RFC 7680, DOI 10.17487/RFC7680, January 1231 2016, . 1233 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 1234 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 1235 May 2016, . 1237 [RFC8321] Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli, 1238 L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi, 1239 "Alternate-Marking Method for Passive and Hybrid 1240 Performance Monitoring", RFC 8321, DOI 10.17487/RFC8321, 1241 January 2018, . 1243 [RFC8799] Carpenter, B. and B. Liu, "Limited Domains and Internet 1244 Protocols", RFC 8799, DOI 10.17487/RFC8799, July 2020, 1245 . 1247 [RFC8889] Fioccola, G., Ed., Cociglio, M., Sapio, A., and R. Sisto, 1248 "Multipoint Alternate-Marking Method for Passive and 1249 Hybrid Performance Monitoring", RFC 8889, 1250 DOI 10.17487/RFC8889, August 2020, 1251 . 1253 Appendix A. Changes Log 1255 Changes from RFC 8321 include: 1257 o Minor editorial changes 1259 o Replacement of the section on "Applications, Implementation, and 1260 Deployment" with "Finding of the Alternate Marking Implementations 1261 and Deployments" 1263 o Moved advantages and benefits of the method from "Introduction" to 1264 the new section on "Finding of the Alternate Marking 1265 Implementations and Deployments" 1267 o Removed section on "Hybrid Measurement" 1269 Authors' Addresses 1271 Giuseppe Fioccola (editor) 1272 Huawei Technologies 1273 Riesstrasse, 25 1274 Munich 80992 1275 Germany 1277 Email: giuseppe.fioccola@huawei.com 1278 Mauro Cociglio 1279 Telecom Italia 1280 Via Reiss Romoli, 274 1281 Torino 10148 1282 Italy 1284 Email: mauro.cociglio@telecomitalia.it 1286 Greg Mirsky 1287 Ericsson 1289 Email: gregimirsky@gmail.com 1291 Tal Mizrahi 1292 Huawei Technologies 1294 Email: tal.mizrahi.phd@gmail.com