idnits 2.17.00 (12 Aug 2021) /tmp/idnits5308/draft-ietf-ippm-metric-registry-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The document has examples using IPv4 documentation addresses according to RFC6890, but does not use any IPv6 documentation addresses. Maybe there should be IPv6 examples, too? Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 8, 2016) is 2142 days in the past. Is this intentional? Checking references for intended status: Best Current Practice ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC2679' is defined on line 1121, but no explicit reference was found in the text == Unused Reference: 'RFC3393' is defined on line 1134, but no explicit reference was found in the text == Unused Reference: 'RFC4566' is defined on line 1154, but no explicit reference was found in the text == Unused Reference: 'RFC5481' is defined on line 1173, but no explicit reference was found in the text == Unused Reference: 'RFC5905' is defined on line 1177, but no explicit reference was found in the text == Unused Reference: 'RFC6776' is defined on line 1187, but no explicit reference was found in the text == Unused Reference: 'RFC6792' is defined on line 1193, but no explicit reference was found in the text == Unused Reference: 'RFC7003' is defined on line 1198, but no explicit reference was found in the text == Unused Reference: 'I-D.ietf-ippm-active-passive' is defined on line 1218, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2141 (Obsoleted by RFC 8141) ** Downref: Normative reference to an Informational RFC: RFC 2330 ** Obsolete normative reference: RFC 4148 (Obsoleted by RFC 6248) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) ** Downref: Normative reference to an Informational RFC: RFC 6248 -- Obsolete informational reference (is this intentional?): RFC 2679 (Obsoleted by RFC 7679) -- Obsolete informational reference (is this intentional?): RFC 4566 (Obsoleted by RFC 8866) == Outdated reference: draft-ietf-ippm-active-passive has been published as RFC 7799 == Outdated reference: draft-ietf-ippm-initial-registry has been published as RFC 8912 Summary: 5 errors (**), 0 flaws (~~), 12 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group M. Bagnulo 3 Internet-Draft UC3M 4 Intended status: Best Current Practice B. Claise 5 Expires: January 9, 2017 Cisco Systems, Inc. 6 P. Eardley 7 BT 8 A. Morton 9 AT&T Labs 10 A. Akhter 11 Consultant 12 July 8, 2016 14 Registry for Performance Metrics 15 draft-ietf-ippm-metric-registry-07 17 Abstract 19 This document defines the format for the Performance Metrics registry 20 and defines the IANA Registry for Performance Metrics. This document 21 also gives a set of guidelines for Registered Performance Metric 22 requesters and reviewers. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at http://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on January 9, 2017. 41 Copyright Notice 43 Copyright (c) 2016 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 59 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 60 3. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 61 4. Motivation for a Performance Metrics Registry . . . . . . . . 7 62 4.1. Interoperability . . . . . . . . . . . . . . . . . . . . 7 63 4.2. Single point of reference for Performance Metrics . . . . 8 64 4.3. Side benefits . . . . . . . . . . . . . . . . . . . . . . 8 65 5. Criteria for Performance Metrics Registration . . . . . . . . 9 66 6. Performance Metric Registry: Prior attempt . . . . . . . . . 9 67 6.1. Why this Attempt Will Succeed . . . . . . . . . . . . . . 10 68 7. Definition of the Performance Metric Registry . . . . . . . . 10 69 7.1. Summary Category . . . . . . . . . . . . . . . . . . . . 12 70 7.1.1. Identifier . . . . . . . . . . . . . . . . . . . . . 12 71 7.1.2. Name . . . . . . . . . . . . . . . . . . . . . . . . 13 72 7.1.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . 14 73 7.1.4. Description . . . . . . . . . . . . . . . . . . . . . 14 74 7.2. Metric Definition Category . . . . . . . . . . . . . . . 14 75 7.2.1. Reference Definition . . . . . . . . . . . . . . . . 14 76 7.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 15 77 7.3. Method of Measurement Category . . . . . . . . . . . . . 15 78 7.3.1. Reference Method . . . . . . . . . . . . . . . . . . 15 79 7.3.2. Packet Stream Generation . . . . . . . . . . . . . . 16 80 7.3.3. Traffic Filter . . . . . . . . . . . . . . . . . . . 16 81 7.3.4. Sampling Distribution . . . . . . . . . . . . . . . . 17 82 7.3.5. Run-time Parameters . . . . . . . . . . . . . . . . . 17 83 7.3.6. Role . . . . . . . . . . . . . . . . . . . . . . . . 18 84 7.4. Output Category . . . . . . . . . . . . . . . . . . . . . 18 85 7.4.1. Type . . . . . . . . . . . . . . . . . . . . . . . . 19 86 7.4.2. Reference Definition . . . . . . . . . . . . . . . . 19 87 7.4.3. Metric Units . . . . . . . . . . . . . . . . . . . . 19 88 7.5. Administrative information . . . . . . . . . . . . . . . 19 89 7.5.1. Status . . . . . . . . . . . . . . . . . . . . . . . 19 90 7.5.2. Requester . . . . . . . . . . . . . . . . . . . . . . 19 91 7.5.3. Revision . . . . . . . . . . . . . . . . . . . . . . 19 92 7.5.4. Revision Date . . . . . . . . . . . . . . . . . . . . 20 93 7.6. Comments and Remarks . . . . . . . . . . . . . . . . . . 20 94 8. The Life-Cycle of Registered Performance Metrics . . . . . . 20 95 8.1. Adding new Performance Metrics to the Performance Metrics 96 Registry . . . . . . . . . . . . . . . . . . . . . . . . 20 98 8.2. Revising Registered Performance Metrics . . . . . . . . . 21 99 8.3. Deprecating Registered Performance Metrics . . . . . . . 22 100 9. Security considerations . . . . . . . . . . . . . . . . . . . 23 101 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 102 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 24 103 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 24 104 12.1. Normative References . . . . . . . . . . . . . . . . . . 24 105 12.2. Informative References . . . . . . . . . . . . . . . . . 25 106 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 27 108 1. Introduction 110 The IETF specifies and uses Performance Metrics of protocols and 111 applications transported over its protocols. Performance metrics are 112 such an important part of the operations of IETF protocols that 113 [RFC6390] specifies guidelines for their development. 115 The definition and use of Performance Metrics in the IETF happens in 116 various working groups (WG), most notably: 118 The "IP Performance Metrics" (IPPM) WG is the WG primarily 119 focusing on Performance Metrics definition at the IETF. 121 The "Metric Blocks for use with RTCP's Extended Report Framework" 122 (XRBLOCK) WG recently specified many Performance Metrics related 123 to "RTP Control Protocol Extended Reports (RTCP XR)" [RFC3611], 124 which establishes a framework to allow new information to be 125 conveyed in RTCP, supplementing the original report blocks defined 126 in "RTP: A Transport Protocol for Real-Time Applications", 127 [RFC3550]. 129 The "Benchmarking Methodology" WG (BMWG) defined many Performance 130 Metrics for use in laboratory benchmarking of inter-networking 131 technologies. 133 The "IP Flow Information eXport" (IPFIX) concluded WG specified an 134 IANA process for new Information Elements. Some Performance 135 Metrics related Information Elements are proposed on regular 136 basis. 138 The "Performance Metrics for Other Layers" (PMOL) concluded WG, 139 defined some Performance Metrics related to Session Initiation 140 Protocol (SIP) voice quality [RFC6035]. 142 It is expected that more Performance Metrics will be defined in the 143 future, not only IP-based metrics, but also metrics which are 144 protocol-specific and application-specific. 146 However, despite the importance of Performance Metrics, there are two 147 related problems for the industry. First, how to ensure that when 148 one party requests another party to measure (or report or in some way 149 act on) a particular Performance Metric, then both parties have 150 exactly the same understanding of what Performance Metric is being 151 referred to. Second, how to discover which Performance Metrics have 152 been specified, so as to avoid developing new Performance Metric that 153 is very similar, but not quite inter-operable. The problems can be 154 addressed by creating a registry of performance metrics. The usual 155 way in which IETF organizes namespaces is with Internet Assigned 156 Numbers Authority (IANA) registries, and there is currently no 157 Performance Metrics Registry maintained by the IANA. 159 This document therefore requests that IANA create and maintain a 160 Performance Metrics Registry, according to the maintenance procedures 161 and the Performance Metrics Registry format defined in this memo. 162 Although the Registry format is primarily for use by IANA, any other 163 organization that wishes to create a Performance Metrics Registry MAY 164 use the same format for their purposes. The authors make no 165 guarantee of the format's applicability to any possible set of 166 Performance Metrics envisaged by other organizations, but encourage 167 others to apply it. In the remainder of this document, unless we 168 explicitly say so, we will refer to the IANA-maintained Performance 169 Metrics Registry as simply the Performance Metrics Registry. 171 2. Terminology 173 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 174 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 175 "OPTIONAL" in this document are to be interpreted as described in 176 [RFC2119]. 178 Performance Metric: A Performance Metric is a quantitative measure 179 of performance, targeted to an IETF-specified protocol or targeted 180 to an application transported over an IETF-specified protocol. 181 Examples of Performance Metrics are the FTP response time for a 182 complete file download, the DNS response time to resolve the IP 183 address, a database logging time, etc. This definition is 184 consistent with the definition of metric in [RFC2330] and broader 185 than the definition of performance metric in [RFC6390]. 187 Registered Performance Metric: A Registered Performance Metric is a 188 Performance Metric expressed as an entry in the Performance Metric 189 Registry, administered by IANA. Such a performance metric has met 190 all the registry review criteria defined in this document in order 191 to included in the registry. 193 Performance Metrics Registry: The IANA registry containing 194 Registered Performance Metrics. 196 Proprietary Registry: A set of metrics that are registered in a 197 proprietary registry, as opposed to Performance Metrics Registry. 199 Performance Metrics Experts: The Performance Metrics Experts is a 200 group of designated experts [RFC5226] selected by the IESG to 201 validate the Performance Metrics before updating the Performance 202 Metrics Registry. The Performance Metrics Experts work closely 203 with IANA. 205 Parameter: An input factor defined as a variable in the definition 206 of a Performance Metric. A numerical or other specified factor 207 forming one of a set that defines a metric or sets the conditions 208 of its operation. All Parameters must be known to measure using a 209 metric and interpret the results. There are two types of 210 Parameters, Fixed and Run-time parameters. For the Fixed 211 Parameters, the value of the variable is specified in the 212 Performance Metrics Registry entry and different Fixed Parameter 213 values results in different Registered Performance Metrics. For 214 the Run-time Parameters, the value of the variable is defined when 215 the metric measurement method is executed and a given Registered 216 Performance Metric supports multiple values for the parameter. 217 Although Run-time Parameters do not change the fundamental nature 218 of the Performance Metric's definition, some have substantial 219 influence on the network property being assessed and 220 interpretation of the results. 222 Note: Consider the case of packet loss in the following two 223 Active Measurement Method cases. The first case is packet loss 224 as background loss where the Run-time Parameter set includes a 225 very sparse Poisson stream, and only characterizes the times 226 when packets were lost. Actual user streams likely see much 227 higher loss at these times, due to tail drop or radio errors. 228 The second case is packet loss as inverse of throughput where 229 the Run-time Parameter set includes a very dense, bursty 230 stream, and characterizes the loss experienced by a stream that 231 approximates a user stream. These are both "loss metrics", but 232 the difference in interpretation of the results is highly 233 dependent on the Run-time Parameters (at least), to the extreme 234 where we are actually using loss to infer its compliment: 235 delivered throughput. 237 Active Measurement Method: Methods of Measurement conducted on 238 traffic which serves only the purpose of measurement and is 239 generated for that reason alone, and whose traffic characteristics 240 are known a priori. The complete definition of Active Methods is 241 specified in section 3.4 of[RFC7799]. Examples of Active 242 Measurement Methods are the measurement methods for the One way 243 delay metric defined in [RFC7679] and the one for round trip delay 244 defined in [RFC2681]. 246 Passive Measurement Method: Methods of Measurement conducted on 247 network traffic, generated either from the end users or from 248 network elements that would exist regardless whether the 249 measurement was being conducted or not. The complete definition 250 of Passive Methods is specified in section 3.6 of [RFC7799]. One 251 characteristic of Passive Measurement Methods is that sensitive 252 information may be observed, and as a consequence, stored in the 253 measurement system. 255 Hybrid Measurement Method: Hybrid Methods are Methods of Measurement 256 that use a combination of Active Methods and Passive Methods, to 257 assess Active Metrics, Passive Metrics, or new metrics derived 258 from the a priori knowledge and observations of the stream of 259 interest. The complete definition of Hybrid Methods is specified 260 in section 3.8 of [RFC7799]. 262 3. Scope 264 This document is meant mainly for two different audiences. For those 265 defining new Registered Performance Metrics, it provides 266 specifications and best practices to be used in deciding which 267 Registered Performance Metrics are useful for a measurement study, 268 instructions for writing the text for each column of the Registered 269 Performance Metrics, and information on the supporting documentation 270 required for the new Performance Metrics Registry entry (up to and 271 including the publication of one or more RFCs or I-Ds describing it). 272 For the appointed Performance Metrics Experts and for IANA personnel 273 administering the new IANA Performance Metric Registry, it defines a 274 set of acceptance criteria against which these proposed Registered 275 Performance Metrics should be evaluated. In addition, this document 276 may be useful for other organization who are defining a Performance 277 Metric registry of its own, who can rely on the Performance Metric 278 registry defined in this document. 280 This Performance Metric Registry is applicable to Performance Metrics 281 issued from Active Measurement, Passive Measurement, and any other 282 form of Performance Metric. This registry is designed to encompass 283 Performance Metrics developed throughout the IETF and especially for 284 the technologies specified in the following working groups: IPPM, 285 XRBLOCK, IPFIX, and BMWG. This document analyzes an prior attempt to 286 set up a Performance Metric Registry, and the reasons why this design 287 was inadequate [RFC6248]. Finally, this document gives a set of 288 guidelines for requesters and expert reviewers of candidate 289 Registered Performance Metrics. 291 This document makes no attempt to populate the Performance Metrics 292 Registry with initial entries. It does provides a few examples that 293 are merely illustrations and should not be included in the registry 294 at this point in time. 296 Based on [RFC5226] Section 4.3, this document is processed as Best 297 Current Practice (BCP) [RFC2026]. 299 4. Motivation for a Performance Metrics Registry 301 In this section, we detail several motivations for the Performance 302 Metric Registry. 304 4.1. Interoperability 306 As any IETF registry, the primary use for a registry is to manage a 307 namespace for its use within one or more protocols. In the 308 particular case of the Performance Metric Registry, there are two 309 types of protocols that will use the Performance Metrics in the 310 Performance Metrics Registry during their operation (by referring to 311 the Index values): 313 o Control protocol: this type of protocols is used to allow one 314 entity to request another entity to perform a measurement using a 315 specific metric defined by the Performance Metrics Registry. One 316 particular example is the LMAP framework [RFC7594]. Using the 317 LMAP terminology, the Performance Metrics Registry is used in the 318 LMAP Control protocol to allow a Controller to request a 319 measurement task to one or more Measurement Agents. In order to 320 enable this use case, the entries of the Performance Metric 321 Registry must be well enough defined to allow a Measurement Agent 322 implementation to trigger a specific measurement task upon the 323 reception of a control protocol message. This requirement heavily 324 constrains the type of entries that are acceptable for the 325 Performance Metric Registry. 327 o Report protocol: This type of protocols is used to allow an entity 328 to report measurement results to another entity. By referencing 329 to a specific Performance Metric Registry, it is possible to 330 properly characterize the measurement result data being reported. 331 Using the LMAP terminology, the Performance Metrics Registry is 332 used in the Report protocol to allow a Measurement Agent to report 333 measurement results to a Collector. 335 It should be noted that the LMAP framework explicitly allows for 336 using not only the IANA-maintained Performance Metrics Registry but 337 also other registries containing Performance Metrics, either defined 338 by other organizations or private ones. However, others who are 339 creating Registries to be used in the context of an LMAP framework 340 are encouraged to use the Registry format defined in this document, 341 because this makes it easier for developers of LMAP Measurement 342 Agents (MAs) to programmatically use information found in those other 343 Registries' entries. 345 4.2. Single point of reference for Performance Metrics 347 A Performance Metrics Registry serves as a single point of reference 348 for Performance Metrics defined in different working groups in the 349 IETF. As we mentioned earlier, there are several WGs that define 350 Performance Metrics in the IETF and it is hard to keep track of all 351 them. This results in multiple definitions of similar Performance 352 Metrics that attempt to measure the same phenomena but in slightly 353 different (and incompatible) ways. Having a registry would allow 354 both the IETF community and external people to have a single list of 355 relevant Performance Metrics defined by the IETF (and others, where 356 appropriate). The single list is also an essential aspect of 357 communication about Performance Metrics, where different entities 358 that request measurements, execute measurements, and report the 359 results can benefit from a common understanding of the referenced 360 Performance Metric. 362 4.3. Side benefits 364 There are a couple of side benefits of having such a registry. 365 First, the Performance Metrics Registry could serve as an inventory 366 of useful and used Performance Metrics, that are normally supported 367 by different implementations of measurement agents. Second, the 368 results of measurements using the Performance Metrics would be 369 comparable even if they are performed by different implementations 370 and in different networks, as the Performance Metric is properly 371 defined. BCP 176 [RFC6576] examines whether the results produced by 372 independent implementations are equivalent in the context of 373 evaluating the completeness and clarity of metric specifications. 374 This BCP defines the standards track advancement testing for (active) 375 IPPM metrics, and the same process will likely suffice to determine 376 whether Registered Performance Metrics are sufficiently well 377 specified to result in comparable (or equivalent) results. 378 Registered Performance Metrics which have undergone such testing 379 SHOULD be noted, with a reference to the test results. 381 5. Criteria for Performance Metrics Registration 383 It is neither possible nor desirable to populate the Performance 384 Metrics Registry with all combinations of Parameters of all 385 Performance Metrics. The Registered Performance Metrics should be: 387 1. interpretable by the user. 389 2. implementable by the software designer, 391 3. deployable by network operators, 393 4. accurate, for interoperability and deployment across vendors, 395 5. Operationally useful, so that it has significant industry 396 interest and/or has seen deployment, 398 6. Sufficiently tightly defined, so that different values for the 399 Run-time Parameters does not change the fundamental nature of the 400 measurement, nor change the practicality of its implementation. 402 In essence, there needs to be evidence that a candidate Registered 403 Performance Metric has significant industry interest, or has seen 404 deployment, and there is agreement that the candidate Registered 405 Performance Metric serves its intended purpose. 407 6. Performance Metric Registry: Prior attempt 409 There was a previous attempt to define a metric registry RFC 4148 410 [RFC4148]. However, it was obsoleted by RFC 6248 [RFC6248] because 411 it was "found to be insufficiently detailed to uniquely identify IPPM 412 metrics... [there was too much] variability possible when 413 characterizing a metric exactly" which led to the RFC4148 registry 414 having "very few users, if any". 416 A couple of interesting additional quotes from RFC 6248 might help 417 understand the issues related to that registry. 419 1. "It is not believed to be feasible or even useful to register 420 every possible combination of Type P, metric parameters, and 421 Stream parameters using the current structure of the IPPM Metrics 422 Registry." 424 2. "The registry structure has been found to be insufficiently 425 detailed to uniquely identify IPPM metrics." 427 3. "Despite apparent efforts to find current or even future users, 428 no one responded to the call for interest in the RFC 4148 429 registry during the second half of 2010." 431 The current approach learns from this by tightly defining each 432 Registered Performance Metric with only a few variable (Run-time) 433 Parameters to be specified by the measurement designer, if any. The 434 idea is that entries in the Performance Metrics Registry stem from 435 different measurement methods which require input (Run-time) 436 parameters to set factors like source and destination addresses 437 (which do not change the fundamental nature of the measurement). The 438 downside of this approach is that it could result in a large number 439 of entries in the Performance Metrics Registry. There is agreement 440 that less is more in this context - it is better to have a reduced 441 set of useful metrics rather than a large set of metrics, some with 442 with questionable usefulness. 444 6.1. Why this Attempt Will Succeed 446 As mentioned in the previous section, one of the main issues with the 447 previous registry was that the metrics contained in the registry were 448 too generic to be useful. This document specifies stricter criteria 449 for performance metric registration (see section 6), and imposes a 450 group of Performance Metrics Experts that will provide guidelines to 451 assess if a Performance Metric is properly specified. 453 Another key difference between this attempt and the previous one is 454 that in this case there is at least one clear user for the 455 Performance Metrics Registry: the LMAP framework and protocol. 456 Because the LMAP protocol will use the Performance Metrics Registry 457 values in its operation, this actually helps to determine if a metric 458 is properly defined. In particular, since we expect that the LMAP 459 control protocol will enable a controller to request a measurement 460 agent to perform a measurement using a given metric by embedding the 461 Performance Metric Registry value in the protocol, a metric is 462 properly specified if it is defined well-enough so that it is 463 possible (and practical) to implement the metric in the measurement 464 agent. This was the failure of the previous attempt: a registry 465 entry with an undefined Type-P (section 13 of RFC 2330 [RFC2330]) 466 allows implementation to be ambiguous. 468 7. Definition of the Performance Metric Registry 470 In this section we define the columns of the Performance Metric 471 Registry. This Performance Metric Registry is applicable to 472 Performance Metrics issued from Active Measurement, Passive 473 Measurement, and any other form of Performance Metric. Because of 474 that, it may be the case that some of the columns defined are not 475 applicable for a given type of metric. If this is the case, the 476 column(s) SHOULD be populated with the "NA" value (Non Applicable). 477 However, the "NA" value MUST NOT be used by any metric in the 478 following columns: Identifier, Name, URI, Status, Requester, 479 Revision, Revision Date, Description. In addition, it may be 480 possible that, in the future, a new type of metric requires 481 additional columns. Should that be the case, it is possible to add 482 new columns to the registry. The specification defining the new 483 column(s) must define how to populate the new column(s) for existing 484 entries. 486 The columns of the Performance Metric Registry are defined next. The 487 columns are grouped into "Categories" to facilitate the use of the 488 registry. Categories are described at the 7.x heading level, and 489 columns are at the 7.x.y heading level. The Figure below illustrates 490 this organization. An entry (row) therefore gives a complete 491 description of a Registered Performance Metric. 493 Each column serves as a check-list item and helps to avoid omissions 494 during registration and expert review. 496 Registry Categories and Columns, shown as 497 Category 498 ------------------ 499 Column | Column | 501 Summary 502 ------------------------------- 503 Identifier | Name | URIs | Description | 505 Metric Definition 506 ----------------------------------------- 507 Reference Definition | Fixed Parameters | 509 Method of Measurement 510 --------------------------------------------------------------------- 511 Reference | Packet | Traffic | Sampling | Run-time | Role | 512 Method | Stream | Filter | Distribution | Parameters | | 513 | Generation | 514 Output 515 ----------------------------- 516 | Type | Reference | Units | 517 | | Definition | | 519 Administrative Information 520 ---------------------------------- 521 Status |Request | Rev | Rev.Date | 523 Comments and Remarks 524 -------------------- 526 7.1. Summary Category 528 7.1.1. Identifier 530 A numeric identifier for the Registered Performance Metric. This 531 identifier MUST be unique within the Performance Metric Registry. 533 The Registered Performance Metric unique identifier is a 16-bit 534 integer (range 0 to 65535). When adding newly Registered Performance 535 Metrics to the Performance Metric Registry, IANA should assign the 536 lowest available identifier to the next Registered Performance 537 Metric. 539 7.1.2. Name 541 As the name of a Registered Performance Metric is the first thing a 542 potential human implementor will use when determining whether it is 543 suitable for their measurement study, it is important to be as 544 precise and descriptive as possible. In future, users will review 545 the names to determine if the metric they want to measure has already 546 been registered, or if a similar entry is available as a basis for 547 creating a new entry. 549 Names are composed of the following elements, seperated by an 550 underscore character "_": 552 MetricType_MethodLayer_SubTypeMethod_... Spec_Units_Output 554 o MetricType: a combination of the directional properties and the 555 metric measured, such as RTDelay, OWDelay, 557 o Method: One of the methods defined in [RFC7799], such as Active, 558 Passive, HybridType1, 560 o SubTypeMethod: One or more sub-types to further describe the 561 features of the entry, such as ICMP, IP, UDP, Poisson, Periodic, 563 o 565 o Spec: RFC that specifies this entry in the form RFCXXXXsecY, such 566 as RFC7799sec3. Note: this is not the Primary Reference 567 specification for the metric; it will be blank until the RFC 568 number is assigned. 570 o Units: The units of measurement for the output, such as Seconds, 571 RatioPercent, 573 o Output: The type of resulting ouptut of measurement, such as 574 Singleton, Minimum, Maximum, Median, 95%tile, 576 An example is: 578 RTDelay_Active_UDP_Poisson_RFCXXXXsecY_Seconds_95th%tile 580 as described in section 4 of [I-D.ietf-ippm-initial-registry]. 582 Note that private registries following the format described here 583 SHOULD use the prefix "Priv_" on any name to avoid unintended 584 conflicts (further considerations are described in section 10). 585 Private registry entries usually have no specifying RFC, thus the 586 Spec: element has no clear interpretation. 588 7.1.3. URIs 590 The URIs column MUST contain a URI [RFC3986] that uniquely identifies 591 the metric. This URI is a URN [RFC2141]. The URI is automatically 592 generated by prepending the prefix urn:ietf:params:ippm:metric: to 593 the metric name. The resulting URI is globally unique. 595 The URIs column MUST contain a second URI which is a URL [RFC3986] 596 and uniquely identifies and locates the metric entry so it is 597 accessible through the Internet. The URL points to a file containing 598 the human-readable information of exactly one registry entry. 599 Ideally, the file will be HTML-formated and contain URLs to 600 referenced sections of HTML-ized RFCs. The separate files for 601 different entries can be more easily edited and re-used when 602 preparing new entries. The exact composition of each metric URL will 603 be determined by IANA, but there will be some overlap with the URN 604 described above. The major sections of 605 [I-D.ietf-ippm-initial-registry] provide an example in HTML form 606 (sections 4 and above). 608 7.1.4. Description 610 A Registered Performance Metric description is a written 611 representation of a particular Performance Metrics Registry entry. 612 It supplements the Registered Performance Metric name to help 613 Performance Metrics Registry users select relevant Registered 614 Performance Metrics. 616 7.2. Metric Definition Category 618 This category includes columns to prompt all necessary details 619 related to the metric definition, including the RFC reference and 620 values of input factors, called fixed parameters, which are left open 621 in the RFC but have a particular value defined by the performance 622 metric. 624 7.2.1. Reference Definition 626 This entry provides a reference (or references) to the relevant 627 section(s) of the document(s) that define the metric, as well as any 628 supplemental information needed to ensure an unambiguous definition 629 for implementations. The reference needs to be an immutable 630 document, such as an RFC; for other standards bodies, it is likely to 631 be necessary to reference a specific, dated version of a 632 specification. 634 7.2.2. Fixed Parameters 636 Fixed Parameters are Parameters whose value must be specified in the 637 Performance Metrics Registry. The measurement system uses these 638 values. 640 Where referenced metrics supply a list of Parameters as part of their 641 descriptive template, a sub-set of the Parameters will be designated 642 as Fixed Parameters. For example, for active metrics, Fixed 643 Parameters determine most or all of the IPPM Framework convention 644 "packets of Type-P" as described in [RFC2330], such as transport 645 protocol, payload length, TTL, etc. An example for passive metrics 646 is for RTP packet loss calculation that relies on the validation of a 647 packet as RTP which is a multi-packet validation controlled by 648 MIN_SEQUENTIAL as defined by [RFC3550]. Varying MIN_SEQUENTIAL 649 values can alter the loss report and this value could be set as a 650 Fixed Parameter. 652 Parameters MUST have well defined names. For human readers, the 653 hanging indent style is preferred, and the names and definitions that 654 do not appear in the Reference Method Specification MUST appear in 655 this column. 657 Parameters MUST have a well-specified data format. 659 A Parameter which is a Fixed Parameter for one Performance Metrics 660 Registry entry may be designated as a Run-time Parameter for another 661 Performance Metrics Registry entry. 663 7.3. Method of Measurement Category 665 This category includes columns for references to relevant sections of 666 the RFC(s) and any supplemental information needed to ensure an 667 unambiguous method for implementations. 669 7.3.1. Reference Method 671 This entry provides references to relevant sections of the RFC(s) 672 describing the method of measurement, as well as any supplemental 673 information needed to ensure unambiguous interpretation for 674 implementations referring to the RFC text. 676 Specifically, this section should include pointers to pseudocode or 677 actual code that could be used for an unambigious implementation. 679 7.3.2. Packet Stream Generation 681 This column applies to Performance Metrics that generate traffic for 682 a part of their Measurement Method purposes including but not 683 necessarily limited to Active metrics. The generated traffic is 684 referred as stream and this columns describe its characteristics. 686 Each entry for this column contains the following information: 688 o Value: The name of the packet stream scheduling discipline 690 o Reference: the specification where the stream is defined 692 The packet generation stream may require parameters such as the the 693 average packet rate and distribution truncation value for streams 694 with Poisson-distributed inter-packet sending times. In case such 695 parameters are needed, they should be included either in the Fixed 696 parameter column or in the run time parameter column, depending on 697 wether they will be fixed or will be an input for the metric. 699 The simplest example of stream specification is Singleton scheduling 700 (see [RFC2330]), where a single atomic measurement is conducted. 701 Each atomic measurement could consist of sending a single packet 702 (such as a DNS request) or sending several packets (for example, to 703 request a webpage). Other streams support a series of atomic 704 measurements in a "sample", with a schedule defining the timing 705 between each transmitted packet and subsequent measurement. 706 Principally, two different streams are used in IPPM metrics, Poisson 707 distributed as described in [RFC2330] and Periodic as described in 708 [RFC3432]. Both Poisson and Periodic have their own unique 709 parameters, and the relevant set of parameters names and values 710 should be included either in the Fixed Parameters column or in the 711 Run-time parameter column. 713 7.3.3. Traffic Filter 715 This column applies to Performance Metrics that observe packets 716 flowing through (the device with) the measurement agent i.e. that is 717 not necessarily addressed to the measurement agent. This includes 718 but is not limited to Passive Metrics. The filter specifies the 719 traffic that is measured. This includes protocol field values/ 720 ranges, such as address ranges, and flow or session identifiers. 722 The traffic filter itself depends on needs of the metric itself and a 723 balance of operators measurement needs and user's need for privacy. 724 Mechanics for conveying the filter criteria might be the BPF (Berkley 725 Packet Filter) or PSAMP [RFC5475] Property Match Filtering which 726 reuses IPFIX [RFC7012]. An example BPF string for matching TCP/80 727 traffic to remote destination net 192.0.2.0/24 would be "dst net 728 192.0.2.0/24 and tcp dst port 80". More complex filter engines might 729 be supported by the implementation that might allow for matching 730 using Deep Packet Inspection (DPI) technology. 732 The traffic filter includes the following information: 734 Type: the type of traffic filter used, e.g. BPF, PSAMP, OpenFlow 735 rule, etc. as defined by a normative reference 737 Value: the actual set of rules expressed 739 7.3.4. Sampling Distribution 741 The sampling distribution defines out of all the packets that match 742 the traffic filter, which one of those are actually used for the 743 measurement. One possibility is "all" which implies that all packets 744 matching the Traffic filter are considered, but there may be other 745 sampling strategies. It includes the following information: 747 Value: the name of the sampling distribution 749 Reference definition: pointer to the specification where the 750 sampling distribution is properly defined. 752 The sampling distribution may require parameters. In case such 753 parameters are needed, they should be included either in the Fixed 754 parameter column or in the run time parameter column, depending on 755 wether they will be fixed or will be an input for the metric. 757 Sampling and Filtering Techniques for IP Packet Selection are 758 documented in the PSAMP (Packet Sampling) [RFC5475], while the 759 Framework for Packet Selection and Reporting, [RFC5474] provides more 760 background information. The sampling distribution parameters might 761 be expressed in terms of the Information Model for Packet Sampling 762 Exports, [RFC5477], and the Flow Selection Techniques, [RFC7014]. 764 7.3.5. Run-time Parameters 766 Run-Time Parameters are Parameters that must be determined, 767 configured into the measurement system, and reported with the results 768 for the context to be complete. However, the values of these 769 parameters is not specified in the Performance Metrics Registry (like 770 the Fixed Parameters), rather these parameters are listed as an aid 771 to the measurement system implementer or user (they must be left as 772 variables, and supplied on execution). 774 Where metrics supply a list of Parameters as part of their 775 descriptive template, a sub-set of the Parameters will be designated 776 as Run-Time Parameters. 778 Parameters MUST have well defined names. For human readers, the 779 hanging indent style is preferred, and the names and definitions that 780 do not appear in the Reference Method Specification MUST appear in 781 this column. 783 A Data Format for each Run-time Parameter MUST be specified in this 784 column, to simplify the control and implementation of measurement 785 devices. For example, parameters that include an IPv4 address can be 786 encoded as a 32 bit integer (i.e. binary base64 encoded value) or ip- 787 address as defined in [RFC6991]. The actual encoding(s) used must be 788 explicitly defined for each Run-time parameter. IPv6 addresses and 789 options MUST be accomodated, allowing Registered Metrics to be used 790 in either address family. 792 Examples of Run-time Parameters include IP addresses, measurement 793 point designations, start times and end times for measurement, and 794 other information essential to the method of measurement. 796 7.3.6. Role 798 In some method of measurements, there may be several roles defined 799 e.g. on a one-way packet delay active measurement, there is one 800 measurement agent that generates the packets and the other one that 801 receives the packets. This column contains the name of the role for 802 this particular entry. In the previous example, there should be two 803 entries in the registry, one for each role, so that when a 804 measurement agent is instructed to perform the one way delay source 805 metric know that it is supposed to generate packets. The values for 806 this field are defined in the reference method of measurement. 808 7.4. Output Category 810 For entries which involve a stream and many singleton measurements, a 811 statistic may be specified in this column to summarize the results to 812 a single value. If the complete set of measured singletons is 813 output, this will be specified here. 815 Some metrics embed one specific statistic in the reference metric 816 definition, while others allow several output types or statistics. 818 7.4.1. Type 820 This column contains the name of the output type. The output type 821 defines a single type of result that the metric produces. It can be 822 the raw results (packet send times and singleton metrics), or it can 823 be a summary statistic. The specification of the output type MUST 824 define the format of the output. In some systems, format 825 specifications will simplify both measurement implementation and 826 collection/storage tasks. Note that if two different statistics are 827 required from a single measurement (for example, both "Xth percentile 828 mean" and "Raw"), then a new output type must be defined ("Xth 829 percentile mean AND Raw"). 831 7.4.2. Reference Definition 833 This column contains a pointer to the specification where the output 834 type is defined 836 7.4.3. Metric Units 838 The measured results must be expressed using some standard dimension 839 or units of measure. This column provides the units. 841 When a sample of singletons (see [RFC2330] for definitions of these 842 terms) is collected, this entry will specify the units for each 843 measured value. 845 7.5. Administrative information 847 7.5.1. Status 849 The status of the specification of this Registered Performance 850 Metric. Allowed values are 'current' and 'deprecated'. All newly 851 defined Information Elements have 'current' status. 853 7.5.2. Requester 855 The requester for the Registered Performance Metric. The requester 856 MAY be a document, such as RFC, or person. 858 7.5.3. Revision 860 The revision number of a Registered Performance Metric, starting at 0 861 for Registered Performance Metrics at time of definition and 862 incremented by one for each revision. 864 7.5.4. Revision Date 866 The date of acceptance or the most recent revision for the Registered 867 Performance Metric. 869 7.6. Comments and Remarks 871 Besides providing additional details which do not appear in other 872 categories, this open Category (single column) allows for unforeseen 873 issues to be addressed by simply updating this informational entry. 875 8. The Life-Cycle of Registered Performance Metrics 877 Once a Performance Metric or set of Performance Metrics has been 878 identified for a given application, candidate Performance Metrics 879 Registry entry specifications in accordance with Section 7 are 880 submitted to IANA to follow the process for review by the Performance 881 Metric Experts, as defined below. This process is also used for 882 other changes to the Performance Metric Registry, such as deprecation 883 or revision, as described later in this section. 885 It is also desirable that the author(s) of a candidate Performance 886 Metrics Registry entry seek review in the relevant IETF working 887 group, or offer the opportunity for review on the WG mailing list. 889 8.1. Adding new Performance Metrics to the Performance Metrics Registry 891 Requests to change Registered Performance Metrics in the Performance 892 Metric Registry are submitted to IANA, which forwards the request to 893 a designated group of experts (Performance Metric Experts) appointed 894 by the IESG; these are the reviewers called for by the Expert Review 895 RFC5226 policy defined for the Performance Metric Registry. The 896 Performance Metric Experts review the request for such things as 897 compliance with this document, compliance with other applicable 898 Performance Metric-related RFCs, and consistency with the currently 899 defined set of Registered Performance Metrics. 901 Authors are expected to review compliance with the specifications in 902 this document to check their submissions before sending them to IANA. 904 The Performance Metric Experts should endeavor to complete referred 905 reviews in a timely manner. If the request is acceptable, the 906 Performance Metric Experts signify their approval to IANA, which 907 updates the Performance Metric Registry. If the request is not 908 acceptable, the Performance Metric Experts can coordinate with the 909 requester to change the request to be compliant. The Performance 910 Metric Experts may also choose in exceptional circumstances to reject 911 clearly frivolous or inappropriate change requests outright. 913 This process should not in any way be construed as allowing the 914 Performance Metric Experts to overrule IETF consensus. Specifically, 915 any Registered Performance Metrics that were added with IETF 916 consensus require IETF consensus for revision or deprecation. 918 Decisions by the Performance Metric Experts may be appealed as in 919 Section 7 of RFC5226. 921 8.2. Revising Registered Performance Metrics 923 A request for Revision is only permissible when the changes maintain 924 backward-compatibility with implementations of the prior Performance 925 Metrics Registry entry describing a Registered Performance Metric 926 (entries with lower revision numbers, but the same Identifier and 927 Name). 929 The purpose of the Status field in the Performance Metric Registry is 930 to indicate whether the entry for a Registered Performance Metric is 931 'current' or 'deprecated'. 933 In addition, no policy is defined for revising IANA Performance 934 Metric entries or addressing errors therein. To be certain, changes 935 and deprecations within the Performance Metric Registry are not 936 encouraged, and should be avoided to the extent possible. However, 937 in recognition that change is inevitable, the provisions of this 938 section address the need for revisions. 940 Revisions are initiated by sending a candidate Registered Performance 941 Metric definition to IANA, as in Section 8, identifying the existing 942 Performance Metrics Registry entry. 944 The primary requirement in the definition of a policy for managing 945 changes to existing Registered Performance Metrics is avoidance of 946 interoperability problems; Performance Metric Experts must work to 947 maintain interoperability above all else. Changes to Registered 948 Performance Metrics may only be done in an inter-operable way; 949 necessary changes that cannot be done in a way to allow 950 interoperability with unchanged implementations must result in the 951 creation of a new Registered Performance Metric and possibly the 952 deprecation of the earlier metric. 954 A change to a Registered Performance Metric is held to be backward- 955 compatible only when: 957 1. "it involves the correction of an error that is obviously only 958 editorial; or" 960 2. "it corrects an ambiguity in the Registered Performance Metric's 961 definition, which itself leads to issues severe enough to prevent 962 the Registered Performance Metric's usage as originally defined; 963 or" 965 3. "it corrects missing information in the metric definition without 966 changing its meaning (e.g., the explicit definition of 'quantity' 967 semantics for numeric fields without a Data Type Semantics 968 value); or" 970 4. "it harmonizes with an external reference that was itself 971 corrected." 973 If an Performance Metric revision is deemed permissible by the 974 Performance Metric Experts, according to the rules in this document, 975 IANA makes the change in the Performance Metric Registry. The 976 requester of the change is appended to the requester in the 977 Performance Metrics Registry. 979 Each Registered Performance Metric in the Performance Metrics 980 Registry has a revision number, starting at zero. Each change to a 981 Registered Performance Metric following this process increments the 982 revision number by one. 984 When a revised Registered Performance Metric is accepted into the 985 Performance Metric Registry, the date of acceptance of the most 986 recent revision is placed into the revision Date column of the 987 registry for that Registered Performance Metric. 989 Where applicable, additions to Registered Performance Metrics in the 990 form of text Comments or Remarks should include the date, but such 991 additions may not constitute a revision according to this process. 993 Older version(s) of the updated metric entries are kept in the 994 registry for archival purposes. The older entries are kept with all 995 fields unmodified (version, revision date) except for the status 996 field that is changed to "Deprecated". 998 8.3. Deprecating Registered Performance Metrics 1000 Changes that are not permissible by the above criteria for Registered 1001 Performance Metric's revision may only be handled by deprecation. A 1002 Registered Performance Metric MAY be deprecated and replaced when: 1004 1. "the Registered Performance Metric definition has an error or 1005 shortcoming that cannot be permissibly changed as in 1006 Section Revising Registered Performance Metrics; or" 1008 2. "the deprecation harmonizes with an external reference that was 1009 itself deprecated through that reference's accepted deprecation 1010 method; or" 1012 A request for deprecation is sent to IANA, which passes it to the 1013 Performance Metric Expert for review. When deprecating an 1014 Performance Metric, the Performance Metric description in the 1015 Performance Metric Registry must be updated to explain the 1016 deprecation, as well as to refer to any new Performance Metrics 1017 created to replace the deprecated Performance Metric. 1019 The revision number of a Registered Performance Metric is incremented 1020 upon deprecation, and the revision Date updated, as with any 1021 revision. 1023 The use of deprecated Registered Performance Metrics should result in 1024 a log entry or human-readable warning by the respective application. 1026 Names and Metric ID of deprecated Registered Performance Metrics must 1027 not be reused. 1029 The deprecated entries are kept with all fields unmodified, except 1030 the version, revision date, and the status field (changed to 1031 "Deprecated"). 1033 9. Security considerations 1035 This draft doesn't introduce any new security considerations for the 1036 Internet. However, the definition of Performance Metrics may 1037 introduce some security concerns, and should be reviewed with 1038 security in mind. 1040 10. IANA Considerations 1042 This document specifies the procedure for Performance Metrics 1043 Registry setup. IANA is requested to create a new registry for 1044 Performance Metrics called "Registered Performance Metrics" with the 1045 columns defined in Section 7. 1047 New assignments for Performance Metric Registry will be administered 1048 by IANA through Expert Review [RFC5226], i.e., review by one of a 1049 group of experts, the Performance Metric Experts, appointed by the 1050 IESG upon recommendation of the Transport Area Directors. The 1051 experts can be initially drawn from the Working Group Chairs and 1052 document editors of the Performance Metrics Directorate among other 1053 sources of experts. 1055 The Identifier values from 64512 to 65536 are reserved for private 1056 use. Names starting with the prefix Priv_ are reserved for private 1057 use, and are not considered for registration. 1059 This document requests the allocation of the URI prefix 1060 urn:ietf:params:ippm:metric for the purpose of generating URIs for 1061 Registered Performance Metrics. 1063 11. Acknowledgments 1065 Thanks to Brian Trammell and Bill Cerveny, IPPM chairs, for leading 1066 some brainstorming sessions on this topic. Thanks to Barbara Stark 1067 and Juergen Schoenwaelder for the detailed feedback and suggestions. 1069 12. References 1071 12.1. Normative References 1073 [RFC2026] Bradner, S., "The Internet Standards Process -- Revision 1074 3", BCP 9, RFC 2026, DOI 10.17487/RFC2026, October 1996, 1075 . 1077 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1078 Requirement Levels", BCP 14, RFC 2119, 1079 DOI 10.17487/RFC2119, March 1997, 1080 . 1082 [RFC2141] Moats, R., "URN Syntax", RFC 2141, DOI 10.17487/RFC2141, 1083 May 1997, . 1085 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 1086 "Framework for IP Performance Metrics", RFC 2330, 1087 DOI 10.17487/RFC2330, May 1998, 1088 . 1090 [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform 1091 Resource Identifier (URI): Generic Syntax", STD 66, 1092 RFC 3986, DOI 10.17487/RFC3986, January 2005, 1093 . 1095 [RFC4148] Stephan, E., "IP Performance Metrics (IPPM) Metrics 1096 Registry", BCP 108, RFC 4148, DOI 10.17487/RFC4148, August 1097 2005, . 1099 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 1100 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 1101 DOI 10.17487/RFC5226, May 2008, 1102 . 1104 [RFC6248] Morton, A., "RFC 4148 and the IP Performance Metrics 1105 (IPPM) Registry of Metrics Are Obsolete", RFC 6248, 1106 DOI 10.17487/RFC6248, April 2011, 1107 . 1109 [RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New 1110 Performance Metric Development", BCP 170, RFC 6390, 1111 DOI 10.17487/RFC6390, October 2011, 1112 . 1114 [RFC6576] Geib, R., Ed., Morton, A., Fardid, R., and A. Steinmitz, 1115 "IP Performance Metrics (IPPM) Standard Advancement 1116 Testing", BCP 176, RFC 6576, DOI 10.17487/RFC6576, March 1117 2012, . 1119 12.2. Informative References 1121 [RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 1122 Delay Metric for IPPM", RFC 2679, DOI 10.17487/RFC2679, 1123 September 1999, . 1125 [RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, 1126 Ed., "A One-Way Delay Metric for IP Performance Metrics 1127 (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January 1128 2016, . 1130 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 1131 Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, 1132 September 1999, . 1134 [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation 1135 Metric for IP Performance Metrics (IPPM)", RFC 3393, 1136 DOI 10.17487/RFC3393, November 2002, 1137 . 1139 [RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network 1140 performance measurement with periodic streams", RFC 3432, 1141 DOI 10.17487/RFC3432, November 2002, 1142 . 1144 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 1145 Jacobson, "RTP: A Transport Protocol for Real-Time 1146 Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, 1147 July 2003, . 1149 [RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed., 1150 "RTP Control Protocol Extended Reports (RTCP XR)", 1151 RFC 3611, DOI 10.17487/RFC3611, November 2003, 1152 . 1154 [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session 1155 Description Protocol", RFC 4566, DOI 10.17487/RFC4566, 1156 July 2006, . 1158 [RFC5474] Duffield, N., Ed., Chiou, D., Claise, B., Greenberg, A., 1159 Grossglauser, M., and J. Rexford, "A Framework for Packet 1160 Selection and Reporting", RFC 5474, DOI 10.17487/RFC5474, 1161 March 2009, . 1163 [RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F. 1164 Raspall, "Sampling and Filtering Techniques for IP Packet 1165 Selection", RFC 5475, DOI 10.17487/RFC5475, March 2009, 1166 . 1168 [RFC5477] Dietz, T., Claise, B., Aitken, P., Dressler, F., and G. 1169 Carle, "Information Model for Packet Sampling Exports", 1170 RFC 5477, DOI 10.17487/RFC5477, March 2009, 1171 . 1173 [RFC5481] Morton, A. and B. Claise, "Packet Delay Variation 1174 Applicability Statement", RFC 5481, DOI 10.17487/RFC5481, 1175 March 2009, . 1177 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 1178 "Network Time Protocol Version 4: Protocol and Algorithms 1179 Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, 1180 . 1182 [RFC6035] Pendleton, A., Clark, A., Johnston, A., and H. Sinnreich, 1183 "Session Initiation Protocol Event Package for Voice 1184 Quality Reporting", RFC 6035, DOI 10.17487/RFC6035, 1185 November 2010, . 1187 [RFC6776] Clark, A. and Q. Wu, "Measurement Identity and Information 1188 Reporting Using a Source Description (SDES) Item and an 1189 RTCP Extended Report (XR) Block", RFC 6776, 1190 DOI 10.17487/RFC6776, October 2012, 1191 . 1193 [RFC6792] Wu, Q., Ed., Hunt, G., and P. Arden, "Guidelines for Use 1194 of the RTP Monitoring Framework", RFC 6792, 1195 DOI 10.17487/RFC6792, November 2012, 1196 . 1198 [RFC7003] Clark, A., Huang, R., and Q. Wu, Ed., "RTP Control 1199 Protocol (RTCP) Extended Report (XR) Block for Burst/Gap 1200 Discard Metric Reporting", RFC 7003, DOI 10.17487/RFC7003, 1201 September 2013, . 1203 [RFC7012] Claise, B., Ed. and B. Trammell, Ed., "Information Model 1204 for IP Flow Information Export (IPFIX)", RFC 7012, 1205 DOI 10.17487/RFC7012, September 2013, 1206 . 1208 [RFC7014] D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow 1209 Selection Techniques", RFC 7014, DOI 10.17487/RFC7014, 1210 September 2013, . 1212 [RFC7594] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., 1213 Aitken, P., and A. Akhter, "A Framework for Large-Scale 1214 Measurement of Broadband Performance (LMAP)", RFC 7594, 1215 DOI 10.17487/RFC7594, September 2015, 1216 . 1218 [I-D.ietf-ippm-active-passive] 1219 Morton, A., "Active and Passive Metrics and Methods (and 1220 everything in-between, or Hybrid)", draft-ietf-ippm- 1221 active-passive-06 (work in progress), January 2016. 1223 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 1224 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 1225 May 2016, . 1227 [I-D.ietf-ippm-initial-registry] 1228 Morton, A., Bagnulo, M., Eardley, P., and K. D'Souza, 1229 "Initial Performance Metric Registry Entries", draft-ietf- 1230 ippm-initial-registry-00 (work in progress), March 2016. 1232 [RFC6991] Schoenwaelder, J., Ed., "Common YANG Data Types", 1233 RFC 6991, DOI 10.17487/RFC6991, July 2013, 1234 . 1236 Authors' Addresses 1237 Marcelo Bagnulo 1238 Universidad Carlos III de Madrid 1239 Av. Universidad 30 1240 Leganes, Madrid 28911 1241 SPAIN 1243 Phone: 34 91 6249500 1244 Email: marcelo@it.uc3m.es 1245 URI: http://www.it.uc3m.es 1247 Benoit Claise 1248 Cisco Systems, Inc. 1249 De Kleetlaan 6a b1 1250 1831 Diegem 1251 Belgium 1253 Email: bclaise@cisco.com 1255 Philip Eardley 1256 BT 1257 Adastral Park, Martlesham Heath 1258 Ipswich 1259 ENGLAND 1261 Email: philip.eardley@bt.com 1263 Al Morton 1264 AT&T Labs 1265 200 Laurel Avenue South 1266 Middletown, NJ 1267 USA 1269 Email: acmorton@att.com 1271 Aamer Akhter 1272 Consultant 1273 118 Timber Hitch 1274 Cary, NC 1275 USA 1277 Email: aakhter@gmail.com