idnits 2.17.00 (12 Aug 2021) /tmp/idnits50259/draft-spiegel-ippm-ioam-rawexport-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (March 19, 2018) is 1523 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-17) exists of draft-ietf-ippm-ioam-data-02 == Outdated reference: A later version (-05) exists of draft-brockners-ippm-ioam-geneve-00 == Outdated reference: A later version (-03) exists of draft-brockners-ippm-ioam-vxlan-gpe-00 -- Obsolete informational reference (is this intentional?): RFC 5226 (Obsoleted by RFC 8126) Summary: 0 errors (**), 0 flaws (~~), 5 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 ippm M. Spiegel 3 Internet-Draft Barefoot Networks 4 Intended status: Informational F. Brockners 5 Expires: September 20, 2018 S. Bhandari 6 R. Sivakolundu 7 Cisco 8 March 19, 2018 10 In-situ OAM raw data export with IPFIX 11 draft-spiegel-ippm-ioam-rawexport-00 13 Abstract 15 In-situ Operations, Administration, and Maintenance (IOAM) records 16 operational and telemetry information in the packet while the packet 17 traverses a path between two points in the network. This document 18 discusses how In-situ Operations, Administration, and Maintenance 19 (IOAM) information can be exported in raw, i.e. uninterpreted, format 20 from network devices to systems, such as monitoring or analytics 21 systems using IPFIX. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at http://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on September 20, 2018. 40 Copyright Notice 42 Copyright (c) 2018 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents 47 (http://trustee.ietf.org/license-info) in effect on the date of 48 publication of this document. Please review these documents 49 carefully, as they describe your rights and restrictions with respect 50 to this document. Code Components extracted from this document must 51 include Simplified BSD License text as described in Section 4.e of 52 the Trust Legal Provisions and are provided without warranty as 53 described in the Simplified BSD License. 55 Table of Contents 57 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 58 1.1. Requirements . . . . . . . . . . . . . . . . . . . . . . 4 59 1.2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 5 60 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 5 61 3. IPFIX for IOAM raw data export . . . . . . . . . . . . . . . 6 62 3.1. Key IPFIX information elements leveraged for IOAM raw 63 data export . . . . . . . . . . . . . . . . . . . . . . . 6 64 3.2. New IPFIX information elements leveraged for IOAM raw 65 data export . . . . . . . . . . . . . . . . . . . . . . . 7 66 3.2.1. ioamReportFlags . . . . . . . . . . . . . . . . . . . 7 67 3.2.2. ioamEncapsulationType . . . . . . . . . . . . . . . . 7 68 3.2.3. ioamPreallocatedTraceData . . . . . . . . . . . . . . 8 69 3.2.4. ioamIncrementalTraceData . . . . . . . . . . . . . . 9 70 3.2.5. ioamE2EData . . . . . . . . . . . . . . . . . . . . . 9 71 3.2.6. ioamPOTData . . . . . . . . . . . . . . . . . . . . . 10 72 4. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 10 73 4.1. Fixed Length IP Packet . . . . . . . . . . . . . . . . . 10 74 4.2. Variable Length IP Packet (length < 255) . . . . . . . . 11 75 4.3. Variable Length IP Packet (length > 255) . . . . . . . . 12 76 4.4. Variable Length ETHERNET Packet (length < 255) . . . . . 13 77 4.5. Variable Length IP Packet with Fixed Length IOAM 78 Incremental Trace Data . . . . . . . . . . . . . . . . . 14 79 4.6. Variable Length IP Packet with Variable Length IOAM 80 Incremental Trace Data . . . . . . . . . . . . . . . . . 15 81 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 82 6. Manageability Considerations . . . . . . . . . . . . . . . . 17 83 7. Security Considerations . . . . . . . . . . . . . . . . . . . 17 84 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 17 85 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 17 86 9.1. Normative References . . . . . . . . . . . . . . . . . . 17 87 9.2. Informative References . . . . . . . . . . . . . . . . . 18 88 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 18 90 1. Introduction 92 In-situ Operations, Administration, and Maintenance (IOAM) records 93 operational and telemetry information in the packet while the packet 94 traverses a path between two points in the network. IOAM data fields 95 are defined in [I-D.ietf-ippm-ioam-data]. This document discusses 96 how In-situ Operations, Administration, and Maintenance (IOAM) 97 information can be exported in raw format, i.e. uninterpreted format, 98 from network devices to systems, such as monitoring or analytics 99 systems using IPFIX [RFC7011]. 101 "Raw export of IOAM data" refers to a mode of operation where a node 102 exports the IOAM data as it is received in the packet. The exporting 103 node neither interprets, aggregates nor reformats the IOAM data 104 before it is exported. Raw export of IOAM data is to support an 105 operational model where the processing and interpretation of IOAM 106 data is decoupled from the operation of encapsulating/updating/ 107 decapsulating IOAM data, which is also referred to as IOAM data-plane 108 operation. The figure below shows the separation of concerns for 109 IOAM export: Exporting IOAM data is performed by the "IOAM node" 110 which performs IOAM data-plane operation, whereas the interpretation 111 of IOAM data is performed by the IOAM data processing system. The 112 separation of concerns is to off-load interpretation, aggregation and 113 formatting of IOAM data from the node which performs data-plane 114 operations. In other words, a node which is focused on data-plane 115 operations, i.e. forwarding of packets and handling IOAM data will 116 not be tasked to also interpret the IOAM data, but can leave this 117 task to another system. Note that for scalability reasons, a single 118 IOAM node could choose to export IOAM data to several IOAM data 119 processing systems. 121 +-------------+ 122 | Monitoring/ | 123 | Analytics | 124 | system | 125 +-------------+ 126 ^ 127 | Processed/interpreted/aggregated 128 | IOAM data 129 | 130 +-------------+ 131 | IOAM data | 132 | processing | 133 | system | 134 +-------------+ 135 ^ 136 | Raw export of 137 | IOAM data 138 | 139 +------------+ 140 | | 141 ..--| IOAM node |--.. 142 | | 143 +------------+ 145 IOAM node: IOAM encapsulating, IOAM decapsulating or 146 IOAM transit node. 148 IOAM data processing system: System that receives raw IOAM data 149 and provides for formatting, aggregation and 150 interpretation of the IOAM data. 152 Monitoring/Analytics system: System that receives telemetry and 153 other operational information from a variety of 154 sources and provides for correlation and 155 interpretation of the data received. 157 Raw export of IOAM data is typically generated by network devices at 158 the edges of the network. Deployment and use-case dependent, such as 159 in cases where the operator is interested in dropped packets, raw 160 export of IOAM data may be generated by IOAM transit nodes. 162 1.1. Requirements 164 Requirements for raw export of IOAM data: 166 o Export all IOAM information contained in a packet. 168 o Export a specific IOAM data type - Incremental Trace type, 169 Preallocated Trace type, Proof of Transit type, Edge to Edge type. 171 o Support coalescing of the IOAM data from multiple packets into a 172 single raw export packet. 174 o Support export of additional parts of the packet, other than the 175 IOAM data as part of the raw export. This could be parts of the 176 packet header and/or parts of the packet payload. This additional 177 information provides context to the IOAM data (e.g. to be used for 178 flow identification) and is to enable the IOAM data processing 179 system to perform further analysis on the received data. 181 o Report the reason why IOAM data was exported. The "reason for 182 export" is to complement the IOAM data retrieved from the packet. 183 For example, if a packet was dropped by a node due to congestion, 184 it could be helpful to export the IOAM data of this dropped packet 185 along with an indication that the packet that the IOAM data 186 belongs to was dropped due to congestion. 188 1.2. Scope 190 This document discusses raw export of IOAM data using IPFIX. 192 The following is considered out of scope for this document: 194 o Protocols other than IPFIX for raw export of IOAM data. 196 o Interpretation or aggregation of IOAM data prior to exporting. 198 o Configuration of network devices so that they can determine when 199 to generate IOAM reports, and what information to include in those 200 reports. 202 o Events that trigger generation of IOAM reports. 204 o Selection of particular destinations within distributed telemetry 205 monitoring systems, to which IOAM reports will be sent. 207 o Export format for flow statistics or processed/interpreted/ 208 aggregated IOAM data. 210 2. Conventions 212 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 213 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 214 document are to be interpreted as described in [RFC2119]. 216 Abbreviations used in this document: 218 E2E: Edge to Edge 220 IOAM: In-situ Operations, Administration, and Maintenance 222 MTU: Maximum Transmit Unit 224 OAM: Operations, Administration, and Maintenance 226 POT: Proof of Transit 228 3. IPFIX for IOAM raw data export 230 IPFIX, being a generic export protocol, can export any Information 231 Elements as long as they are described in the information model. 232 IPFIX protocol is well suited for and is defined as the protocol for 233 exporting packet samples in [RFC5476]. 235 IPFIX/PSAMP [RFC7011], [RFC5476] already define most of the 236 information elements needed for exporting sections of packets needed 237 for deriving context and raw IOAM data export. This document 238 specifies required extensions of the IPFIX information model for 239 meeting the requirements in Section 1.1. 241 3.1. Key IPFIX information elements leveraged for IOAM raw data export 243 The existing IPFIX Information Elements that are required for IOAM 244 raw data export are listed here. Their details are available in 245 IANA's IPFIX registry [IANA-IPFIX]. 247 The elements used to carry the sections of the packets including IOAM 248 data within it are as follows: 250 313 - ipHeaderPacketSection 252 315 - dataLinkFrameSection 254 The following Information Elements will be used to provide context to 255 the ipHeaderPacketSection and dataLinkFrameSection as described in 256 [IANA-IPFIX]: 258 408 - dataLinkFrameType 260 409 - sectionOffset 262 410 - sectionExportedOctets 263 The following Information Element will be used to provide forwarding 264 status of the flow and any attached reasons. 266 89 - forwardingStatus 268 3.2. New IPFIX information elements leveraged for IOAM raw data export 270 IOAM data raw export using IPFIX requires a set of new information 271 elements which are described in this section. 273 3.2.1. ioamReportFlags 275 Description: 277 This Information Element describes properties associated with an IOAM 278 report. 280 The ioamReportFlags data type is an 8-bit field. The following bits 281 are defined here: 283 Bit 0 Dropped Association - Dropped packet of interest. 285 Bit 1 Congested Queue Association - Indicates the presence of 286 congestion on a monitored queue. 288 Bit 2 Tracked Flow Association - Matched a flow of interest. 290 Bit 3-7 Reserved 292 IANA is requested to create a new subregistry for IOAM Report Flags 293 and fill it with the initial list from the description. New 294 assignments for IOAM Encapsulation Types are administered by IANA 295 through Expert Review [RFC5226] i.e., review by one of a group of 296 experts designated by an IETF Area Director. 298 Abstract Data Type: unsigned8 300 Data Type Semantics: flags 302 ElementId: TBD1 304 Status: current 306 3.2.2. ioamEncapsulationType 308 Description: 310 This Information Element specifies the type of encapsulation to 311 interpret ioamPreallocatedTraceHeader, ioamIncrementalTraceHeader, 312 ioamE2EHeader, ioamPOTHeader. 314 The following ioamEncapsulationType values are defined here: 316 0 None : IOAM data follows format defined in 317 [I-D.ietf-ippm-ioam-data] 319 1 GRE : IOAM data follows format defined in [I-D.weis-ippm-ioam-gre] 321 2 VXLAN-GPE : IOAM data follows format defined in 322 [I-D.brockners-ippm-ioam-vxlan-gpe] 324 3 GENEVE : IOAM data follows format defined in 325 [I-D.brockners-ippm-ioam-geneve] 327 4 NSH : IOAM data follows format defined in 328 [I-D.brockners-sfc-ioam-nsh] 330 IANA is requested to create a new subregistry for IOAM Encapsulation 331 Types and fill it with the initial list from the description. New 332 assignments for IOAM Encapsulation Types are administered by IANA 333 through Expert Review [RFC5226] i.e., review by one of a group of 334 experts designated by an IETF Area Director. 336 Abstract Data Type: unsigned8 338 Data Type Semantics: identifier 340 ElementId: TBD2 342 Status: current 344 3.2.3. ioamPreallocatedTraceData 346 Description: 348 This Information Element carries n octets of IOAM Preallocated Trace 349 data defined in [I-D.ietf-ippm-ioam-data]. 351 The format of the data is determined by the ioamEncapsulationType 352 information element, if present. When the ioamEncapsulationType 353 information element is present and has a value other than "None", and 354 with sufficient length, this element may also report octets from 355 subsequent headers and payload. If no ioamEncapsulationType 356 information element is present, then the encapsulation type shall be 357 assumed to be "None" and this information element only contains 358 octets from the IOAM Preallocated Trace Option. 360 Abstract Data Type: octetArray 362 ElementId: TBD3 364 Status: current 366 3.2.4. ioamIncrementalTraceData 368 Description: 370 This Information Element carries n octets of IOAM Incremental Trace 371 data defined in [I-D.ietf-ippm-ioam-data]. 373 The format of the data is determined by the ioamEncapsulationType 374 information element, if present. When the ioamEncapsulationType 375 information element is present and has a value other than "None", and 376 with sufficient length, this element may also report octets from 377 subsequent headers and payload. If no ioamEncapsulationType 378 information element is present, then the encapsulation type shall be 379 assumed to be "None" and this information element only contains 380 octets from the IOAM Incremental Trace Option. 382 Abstract Data Type: octetArray 384 ElementId: TBD4 386 Status: current 388 3.2.5. ioamE2EData 390 Description: 392 This Information Element carries n octets of IOAM E2E data defined in 393 [I-D.ietf-ippm-ioam-data]. 395 The format of the data is determined by the ioamEncapsulationType 396 information element, if present. When the ioamEncapsulationType 397 information element is present and has a value other than "None", and 398 with sufficient length, this element may also report octets from 399 subsequent headers and payload. If no ioamEncapsulationType 400 information element is present, then the encapsulation type shall be 401 assumed to be "None" and this information element only contains 402 octets from the IOAM Edge-to-Edge Option. 404 Abstract Data Type: octetArray 405 ElementId: TBD5 407 Status: current 409 3.2.6. ioamPOTData 411 Description: 413 This Information Element carries n octets of IOAM POT data defined in 414 [I-D.ietf-ippm-ioam-data]. 416 The format of the data is determined by the ioamEncapsulationType 417 information element, if present. When the ioamEncapsulationType 418 information element is present and has a value other than "None", and 419 with sufficient length, this element may also report octets from 420 subsequent headers and payload. If no ioamEncapsulationType 421 information element is present, then the encapsulation type shall be 422 assumed to be "None" and this information element only contains 423 octets from the IOAM Proof of Transit Option. 425 Abstract Data Type: octetArray 427 ElementId: TBD6 429 Status: current 431 4. Examples 433 This section shows a set of examples of how IOAM information along 434 with other parts of the packet can be carried using IPFIX. 436 4.1. Fixed Length IP Packet 438 This examples shows a fixed length IP packet. IOAM data is part of 439 the ipHeaderPacketSection. 441 0 1 2 3 442 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 443 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+--- 444 | Version Number | Length | | 445 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 446 | Export Time (seconds) |IPFIX 447 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Message 448 | Sequence Number |Header 449 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 450 | Observation Domain ID | | 451 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 452 | Set ID ( = Template ID) | Length |SetHdr 453 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 454 |ioamReportFlags| fwdingStatus | sectionExportedOctets | | 455 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 456 | ipHeaderPacketSection (start) | | 457 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Record1 458 | ... | | 459 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 460 | ipHeaderPacketSection (end) | | 461 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 462 |ioamReportFlags| fwdingStatus | sectionExportedOctets | | 463 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 464 | ipHeaderPacketSection (start) | | 465 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Record2 466 | ... | | 467 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 468 | ipHeaderPacketSection (end) | | 469 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 471 4.2. Variable Length IP Packet (length < 255) 473 This examples shows a variable length IP packet, with length < 255 474 bytes. IOAM data is part of the ipHeaderPacketSection. 476 0 1 2 3 477 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 478 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 479 | Version Number | Length | | 480 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 481 | Export Time (seconds) |IPFIX 482 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Message 483 | Sequence Number |Header 484 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 485 | Observation Domain ID | | 486 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 487 | Set ID ( = Template ID) | Length |SetHdr 488 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 489 |ioamReportFlags| fwdingStatus | paddingOctets | Length (< 255)| | 490 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 491 | ipHeaderPacketSection (start) | | 492 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Record1 493 | ... | | 494 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 495 | ipHeaderPacketSection (end) | | 496 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 497 |ioamReportFlags| fwdingStatus | paddingOctets | Length (< 255)| | 498 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 499 | ipHeaderPacketSection (start) | | 500 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Record2 501 | ... | | 502 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 503 | ipHeaderPacketSection (end) | | 504 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 506 4.3. Variable Length IP Packet (length > 255) 508 This examples shows a variable length IP packet, with length > 255 509 bytes. IOAM data is part of the ipHeaderPacketSection. 511 0 1 2 3 512 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 513 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 514 | Version Number | Length | | 515 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 516 | Export Time (seconds) |IPFIX 517 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Message 518 | Sequence Number |Header 519 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 520 | Observation Domain ID | | 521 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 522 | Set ID ( = Template ID) | Length |SetHdr 523 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 524 |ioamReportFlags| fwdingStatus | paddingOctets | | 525 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 526 | paddingOctets | 255 | Length (0 to 65535) | | 527 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 528 | ipHeaderPacketSection (start) |Record1 529 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 530 | ... | | 531 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 532 | ipHeaderPacketSection (end) | | 533 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 535 4.4. Variable Length ETHERNET Packet (length < 255) 537 This examples shows a variable length Ethernet packet, with length < 538 255 bytes. IOAM data is part of the dataLinkFrameSection. 540 0 1 2 3 541 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 542 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 543 | Version Number | Length | | 544 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 545 | Export Time (seconds) |IPFIX 546 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Message 547 | Sequence Number |Header 548 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 549 | Observation Domain ID | | 550 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 551 | Set ID ( = Template ID) | Length |SetHdr 552 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 553 |ioamReportFlags| fwdingStatus | dataLinkFrameType = 0x01 | | 554 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 555 | dataLinkFrameSize | paddingOctets | Length (< 255)| | 556 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 557 | dataLinkFrameSection (start) |Record1 558 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 559 | ... | | 560 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 561 | dataLinkFrameSection (end) | | 562 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-- 564 4.5. Variable Length IP Packet with Fixed Length IOAM Incremental Trace 565 Data 567 This examples shows a variable length IP packet with length < 255 568 bytes and fixed length ioamIncrementalTraceData carried separately. 570 0 1 2 3 571 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 572 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+--- 573 | Version Number | Length | | 574 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 575 | Export Time (seconds) |IPFIX 576 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Message 577 | Sequence Number |Header 578 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 579 | Observation Domain ID | | 580 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+--- 581 | Set ID ( = Template ID) | Length |SetHdr 582 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+--- 583 | ioamIncrementalTraceData (start) | | 584 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 585 | ... | | 586 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 587 | ioamIncrementalTraceData (end) | | 588 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Record1 589 |ioamReportFlags| fwdingStatus |ioamEncapType | Length (< 255)| | 590 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 591 | ipHeaderPacketSection (start) | | 592 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 593 | ... | | 594 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 595 | ipHeaderPacketSection (end) | | 596 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+--- 598 4.6. Variable Length IP Packet with Variable Length IOAM Incremental 599 Trace Data 601 This examples shows a variable length IP packet with length < 255 602 bytes and variable length ioamIncrementalTraceData with length < 255 603 bytes carried separately. 605 0 1 2 3 606 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 607 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+--- 608 | Version Number | Length | | 609 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 610 | Export Time (seconds) |IPFIX 611 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Message 612 | Sequence Number |Header 613 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 614 | Observation Domain ID | | 615 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+--- 616 | Set ID ( = Template ID) | Length |SetHdr 617 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+--- 618 |ioamReportFlags| fwdingStatus |ioamEncapType | Length (< 255)| | 619 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 620 | ioamIncrementalTraceData (start) | | 621 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 622 | ... | | 623 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 624 | ioamIncrementalTraceData (end) | | 625 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Record1 626 | paddingOctets | Length (< 255)| | 627 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 628 | ipHeaderPacketSection (start) | | 629 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 630 | ... | | 631 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 632 | ipHeaderPacketSection (end) | | 633 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+--- 635 5. IANA Considerations 637 IANA is requested to allocate code points for the following 638 Information Elements in [IANA-IPFIX]: 640 TBD1 ioamReportFlags 642 TBD2 ioamEncapsulationType 644 TBD3 ioamPreallocatedTraceData 646 TBD4 ioamIncrementalTraceData 648 TBD5 ioamE2EData 650 TBD6 ioamPOTData 651 See Section 3.2 for further details. 653 IANA is requested to create subregistries for ioamReportFlags defined 654 in Section 3.2.1 and ioamEncapsulationType defined in Section 3.2.2. 656 6. Manageability Considerations 658 Manageability considerations will be addressed in a later version of 659 this document.. 661 7. Security Considerations 663 Security considerations will be addressed in a later version of this 664 document. 666 8. Acknowledgements 668 The authors would like to thank Barak Gafni, Tal Mizrahi, John Lemon, 669 and Aviv Kfir for their thoughts and comments on raw IOAM data 670 export. 672 9. References 674 9.1. Normative References 676 [I-D.ietf-ippm-ioam-data] 677 Brockners, F., Bhandari, S., Pignataro, C., Gredler, H., 678 Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov, 679 P., Chang, R., daniel.bernier@bell.ca, d., and J. Lemon, 680 "Data Fields for In-situ OAM", draft-ietf-ippm-ioam- 681 data-02 (work in progress), March 2018. 683 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 684 Requirement Levels", BCP 14, RFC 2119, 685 DOI 10.17487/RFC2119, March 1997, . 688 [RFC5476] Claise, B., Ed., Johnson, A., and J. Quittek, "Packet 689 Sampling (PSAMP) Protocol Specifications", RFC 5476, 690 DOI 10.17487/RFC5476, March 2009, . 693 [RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, 694 "Specification of the IP Flow Information Export (IPFIX) 695 Protocol for the Exchange of Flow Information", STD 77, 696 RFC 7011, DOI 10.17487/RFC7011, September 2013, 697 . 699 9.2. Informative References 701 [I-D.brockners-ippm-ioam-geneve] 702 Brockners, F., Bhandari, S., Govindan, V., Pignataro, C., 703 Gredler, H., Leddy, J., Youell, S., Mizrahi, T., Lapukhov, 704 P., Gafni, B., Kfir, A., and M. Spiegel, "Geneve 705 encapsulation for In-situ OAM Data", draft-brockners-ippm- 706 ioam-geneve-00 (work in progress), March 2018. 708 [I-D.brockners-ippm-ioam-vxlan-gpe] 709 Brockners, F., Bhandari, S., Govindan, V., Pignataro, C., 710 Gredler, H., Leddy, J., Youell, S., Mizrahi, T., Kfir, A., 711 Gafni, B., Lapukhov, P., and M. Spiegel, "VXLAN-GPE 712 Encapsulation for In-situ OAM Data", draft-brockners-ippm- 713 ioam-vxlan-gpe-00 (work in progress), March 2018. 715 [I-D.brockners-sfc-ioam-nsh] 716 Brockners, F., Bhandari, S., Govindan, V., Pignataro, C., 717 Gredler, H., Leddy, J., Youell, S., Mizrahi, T., Mozes, 718 D., Lapukhov, P., and R. Chang, "NSH Encapsulation for In- 719 situ OAM Data", draft-brockners-sfc-ioam-nsh-01 (work in 720 progress), March 2018. 722 [I-D.weis-ippm-ioam-gre] 723 Weis, B., Brockners, F., crhill@cisco.com, c., Bhandari, 724 S., Govindan, V., Pignataro, C., Gredler, H., Leddy, J., 725 Youell, S., Mizrahi, T., Kfir, A., Gafni, B., Lapukhov, 726 P., and M. Spiegel, "GRE Encapsulation for In-situ OAM 727 Data", draft-weis-ippm-ioam-gre-00 (work in progress), 728 March 2018. 730 [IANA-IPFIX] 731 "IP Flow Information Export (IPFIX) Entities", 732 . 734 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 735 IANA Considerations Section in RFCs", RFC 5226, 736 DOI 10.17487/RFC5226, May 2008, . 739 Authors' Addresses 740 Mickey Spiegel 741 Barefoot Networks 742 4750 Patrick Henry Drive 743 Santa Clara, CA 95054 744 US 746 Email: mspiegel@barefootnetworks.com 748 Frank Brockners 749 Cisco Systems, Inc. 750 Hansaallee 249, 3rd Floor 751 DUESSELDORF, NORDRHEIN-WESTFALEN 40549 752 Germany 754 Email: fbrockne@cisco.com 756 Shwetha Bhandari 757 Cisco Systems, Inc. 758 Cessna Business Park, Sarjapura Marathalli Outer Ring Road 759 Bangalore, KARNATAKA 560 087 760 India 762 Email: shwethab@cisco.com 764 Ramesh Sivakolundu 765 Cisco Systems, Inc. 766 170 West Tasman Dr. 767 SAN JOSE, CA 95134 768 U.S.A. 770 Email: sramesh@cisco.com