idnits 2.17.00 (12 Aug 2021) /tmp/idnits19517/draft-boschi-ipfix-reducing-redundancy-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 20. -- Found old boilerplate from RFC 3978, Section 5.5 on line 1722. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 1698. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 1698. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 1704. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == There are 44 instances of lines with non-ascii characters in the document. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 28 longer pages, the longest (page 8) being 71 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 28 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 228 instances of too long lines in the document, the longest one being 7 characters in excess of 72. ** The abstract seems to contain references ([RFC2119]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. == There are 3 instances of lines with private range IPv4 addresses in the document. If these are generic example addresses, they should be changed to use any of the ranges defined in RFC 6890 (or successor): 192.0.2.x, 198.51.100.x or 203.0.113.x. == There are 1 instance of lines with non-RFC3849-compliant IPv6 addresses in the document. If these are example addresses, they should be changed. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The exact meaning of the all-uppercase expression 'NOT REQUIRED' is not defined in RFC 2119. If it is intended as a requirements expression, it should be rewritten using one of the combinations defined in RFC 2119; otherwise it should not be all-uppercase. == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: MUST not have information elements in common), to avoid potential collisions. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 25, 2006) is 5808 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'RFC1889' is mentioned on line 241, but not defined ** Obsolete undefined reference: RFC 1889 (Obsoleted by RFC 3550) -- Possible downref: Non-RFC (?) normative reference: ref. 'IPFIX-PROTO' -- Possible downref: Non-RFC (?) normative reference: ref. 'IPFIX-INFO' -- Possible downref: Non-RFC (?) normative reference: ref. 'PSAMP-PROTO' -- Unexpected draft version: The latest known version of draft-ietf-ipfix-arch is -02, but you're referring to -11. == Outdated reference: draft-ietf-ipfix-as has been published as RFC 5472 == Outdated reference: draft-ietf-psamp-sample-tech has been published as RFC 5475 == Outdated reference: draft-ietf-psamp-info has been published as RFC 5477 == Outdated reference: A later version (-06) exists of draft-ietf-psamp-mib-05 == Outdated reference: draft-ietf-psamp-framework has been published as RFC 5474 Summary: 6 errors (**), 0 flaws (~~), 14 warnings (==), 12 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet-Draft E. Boschi 3 draft-boschi-ipfix-reducing-redundancy-02.txt Hitachi Europe 4 Expires: December 27, 2006 L. Mark 5 Fraunhofer FOKUS 6 B. Claise 7 Cisco Systems 9 June 25, 2006 11 Reducing redundancy in IPFIX and PSAMP reports 12 draft-boschi-ipfix-reducing-redundancy-02.txt 14 Status of this Memo 16 By submitting this Internet-Draft, each author represents that 17 any applicable patent or other IPR claims of which he or she is 18 aware have been or will be disclosed, and any of which he or she 19 becomes aware will be disclosed, in accordance with Section 6 of 20 BCP 79. 22 Internet-Drafts are working documents of the Internet 23 Engineering Task Force (IETF), its areas, and its working 24 groups. Note that other groups may also distribute working 25 documents as Internet-Drafts. 27 Internet-Drafts are draft documents valid for a maximum of six 28 months and may be updated, replaced, or obsoleted by other 29 documents at any time. It is inappropriate to use 30 Internet-Drafts as reference material or to cite them other than 31 as "work in progress." 33 The list of current Internet-Drafts can be accessed at 34 http://www.ietf.org/ietf/1id-abstracts.txt. 36 The list of Internet-Draft Shadow Directories can be accessed at 37 http://www.ietf.org/shadow.html. 39 This Internet-Draft will expire on December 27, 2006. 41 Copyright Notice 43 Copyright (C) The Internet Society (2006). 45 Reducing redundancy in IPFIX and PSAMP reports 47 Abstract 49 This document describes a bandwidth saving method for exporting 50 flow or packet information using the IP Flow Information Export 51 (IPFIX) protocol. As the PSAMP protocol is based on IPFIX, these 52 considerations are valid for PSAMP exports as well. 54 This method works by separating information common to several 55 flow records from information specific to an individual flow 56 record. Common flow information is exported only once in a data 57 record defined by an option template, while the rest of the 58 specific flow information is associated with the common 59 information via a unique identifier. 61 Conventions used in this document 63 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 64 NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 65 "OPTIONAL" in this document are to be interpreted as described 66 in RFC 2119 [RFC2119]. 68 Table of Contents 70 Copyright Notice·············································1 71 Abstract·····················································2 72 1. Introduction············································3 73 1.1 IPFIX Documents Overview······························3 74 1.2 PSAMP Documents Overview······························3 75 2. Terminology·············································4 76 2.1 Terminology Summary Table.····························8 77 2.2 IPFIX Flows versus PSAMP Packets······················8 78 3. Problem Statement and High Level Solution···············8 79 3.1 Per Flow Data Reduction·······························8 80 3.1.1 Unique Data Reduction..................................8 81 3.1.2 Multiple Data Reduction................................9 82 3.2 Per Packet Data Reduction····························11 83 4. Specifications for bandwidth saving information export·12 84 4.1 Per Flow Data Reduction······························13 85 4.1.1 Unique Data Reduction.................................13 86 4.1.2 Multiple Data Reduction...............................14 87 4.2 Per-Packet Data Reduction····························14 88 5. Transport Protocol Choice······························15 89 5.1 SCTP·················································15 90 5.2 UDP··················································15 91 5.3 TCP··················································15 92 6. commonPropertiesID Management··························16 93 7. The Collecting Process Side····························16 94 7.1 SCTP·················································17 95 7.2 UDP··················································17 96 7.3 TCP··················································17 97 8. Export and Evaluation Considerations···················17 98 8.1 Transport Protocol Choice····························18 99 8.2 Reduced Size Encoding································18 100 8.3 CommonPropertiesID vs. TemplateID scope··············18 101 8.4 Efficiency Gain······································18 102 9. IANA Considerations····································18 103 10. Security Considerations································18 104 11. Appendix A: Examples···································19 105 11.1 Per Flow Data Reduction······························19 106 11.1.1 Unique Data Reduction................................19 107 11.1.2 Multiple Data Reduction..............................22 108 11.2 Per-Packet Information Export························24 109 12. References·············································26 110 12.1 Normative References·································26 111 12.2 Informative References·······························27 113 Reducing redundancy in IPFIX and PSAMP reports 115 13. Author's Addresses·····································27 116 14. Intellectual Property Statement························28 117 15. Copyright Statement····································28 118 16. Disclaimer·············································28 120 1. Introduction 122 The IPFIX working group has specified a protocol to export IP 123 Flow information [IPFIX-PROTO]. This protocol is designed to 124 export information about IP traffic flows and related 125 measurement data, where a flow is defined by a set of key 126 attributes (e.g. source and destination IP address, source and 127 destination port, etc.). However, thanks to its template 128 mechanism, the IPFIX protocol can export any type of 129 information, as long as the information element is specified in 130 [IPFIX-INFO] or registered with IANA. 132 Regardless of the flow attributes content, flow records with 133 common attributes export the same values in every single flow 134 record. These common attributes may represent values common to 135 a collection of flows or packets, or values that are invariant 136 over time. The reduction of redundant data from the export 137 stream can result in a significant reduction of the transferred 138 data. 140 This draft specifies a way to export these invariant or common 141 attributes only once, while the rest of the flow specific 142 attributes are exported in regular data records. Unique common 143 properties identifiers are used to link data records and the 144 common attributes. 146 The proposed method is applicable to IPFIX flow and to PSAMP per 147 packet information, without any changes to both the IPFIX and 148 PSAMP protocol specifications. 150 1.1 IPFIX Documents Overview 152 The IPFIX protocol [IPFIX-PROTO] provides network administrators 153 with access to IP flow information. The architecture for the 154 export of measured IP flow information out of an IPFIX exporting 155 process to a collecting process is defined in [IPFIX-ARCH], per 156 the requirements defined in [RFC3917]. This document specifies 157 how IPFIX data record and templates are carried via a 158 congestion-aware transport protocol from IPFIX exporting 159 processes to IPFIX collecting process. IPFIX has a formal 160 description of IPFIX information elements, their name, type and 161 additional semantic information, as specified in [IPFIX-INFO]. 162 Finally [IPFIX-AS] describes what type of applications can use 163 the IPFIX protocol and how they can use the information 164 provided. It furthermore shows how the IPFIX framework relates 165 to other architectures and frameworks. 167 1.2 PSAMP Documents Overview 169 The document "A Framework for Packet Selection and Reporting" 170 [PSAMP-FMWK], describes the PSAMP framework for network elements 171 to select subsets of packets by statistical and other methods, 172 and to export a stream of reports on the selected packets to a 173 collector. The set of packet selection techniques (sampling, 174 filtering, and hashing) supported by PSAMP are described in 175 "Sampling and Filtering Techniques for IP Packet Selection" 176 [PSAMP-TECH]. The PSAMP protocol [PSAMP-PROTO] specifies the 177 export of packet information from a PSAMP exporting process to a 179 Reducing redundancy in IPFIX and PSAMP reports 181 PSAMP collecting process. Like IPFIX, PSAMP has a formal 182 description of its information elements, their name, type and 183 additional semantic information. The PSAMP information model is 184 defined in [PSAMP-INFO]. Finally [PSAMP-MIB] describes the PSAMP 185 Management Information Base. 187 2. Terminology 189 The terms in this section are in line with the IPFIX terminology 190 section [IPFIX-PROTO], and [PSAMP-PROTO]. Note that this 191 document selected the IPFIX definition of the term Exporting 192 Process [IPFIX-PROTO], as this definition is more generic than 193 the PSAMP definition [PSAMP-PROTO]. 195 Observation Point 197 An Observation Point is a location in the network where IP 198 packets can be observed. Examples include: a line to which 199 a probe is attached, a shared medium, such as an Ethernet- 200 based LAN, a single port of a router, or a set of 201 interfaces (physical or logical) of a router. 203 Note that every Observation Point is associated with an 204 Observation Domain (defined below), and that one 205 Observation Point may be a superset of several other 206 Observation Points. For example one Observation Point can 207 be an entire line card. That would be the superset of the 208 individual Observation Points at the line card's 209 interfaces. 211 Observation Domain 213 An Observation Domain is the largest set of Observation 214 Points for which Flow information can be aggregated by a 215 Metering Process. For example, a router line card may be 216 an Observation Domain if it is composed of several 217 interfaces, each of which is an Observation Point. In the 218 IPFIX Message it generates, the Observation Domain includes 219 its Observation Domain ID, which is unique per Exporting 220 Process. That way, the Collecting Process can identify the 221 specific Observation Domain from the Exporter that sends 222 the IPFIX Messages. Every Observation Point is associated 223 with an Observation Domain. It is RECOMMENDED that 224 Observation Domain IDs are also unique per IPFIX Device. 226 IP Traffic Flow or Flow 228 There are several definitions of the term 'flow' being used 229 by the Internet community. Within the context of IPFIX we 230 use the following definition: 232 A Flow is defined as a set of IP packets passing an 233 Observation Point in the network during a certain time 234 interval. All packets belonging to a particular Flow have 235 a set of common properties. Each property is defined as 236 the result of applying a function to the values of: 238 1. one or more packet header field (e.g. destination IP 239 address), transport header field (e.g. destination port 240 number), or application header field (e.g. RTP header 241 fields [RFC1889]) 243 2. one or more characteristics of the packet itself 244 (e.g. number of MPLS labels, etc...) 246 Reducing redundancy in IPFIX and PSAMP reports 248 3. one or more of fields derived from packet treatment 249 (e.g. next hop IP address, the output interface, etc...) 251 A packet is defined to belong to a Flow if it completely 252 satisfies all the defined properties of the Flow. 254 This definition covers the range from a Flow containing all 255 packets observed at a network interface to a Flow 256 consisting of just a single packet between two 257 applications. It includes packets selected by a sampling 258 mechanism. 260 Flow Record 262 A Flow Record contains information about a specific Flow 263 that was observed at an Observation Point. A Flow Record 264 contains measured properties of the Flow (e.g. the total 265 number of bytes for all the Flow's packets) and usually 266 characteristic properties of the Flow (e.g. source IP 267 address). 269 Metering Process 271 The Metering Process generates Flow Records. Inputs to the 272 process are packet headers and characteristics observed at 273 an Observation Point, and packet treatment at the 274 Observation Point (for example the selected output 275 interface). 277 The Metering Process consists of a set of functions that 278 includes packet header capturing, timestamping, sampling, 279 classifying, and maintaining Flow Records. 281 The maintenance of Flow Records may include creating new 282 records, updating existing ones, computing Flow statistics, 283 deriving further Flow properties, detecting Flow 284 expiration, passing Flow Records to the Exporting Process, 285 and deleting Flow Records. 287 Exporting Process 289 The Exporting Process sends Flow Records to one or more 290 Collecting Processes. The Flow Records are generated by 291 one or more Metering Processes. 293 Exporter 295 A device which hosts one or more Exporting Processes is 296 termed an Exporter. 298 IPFIX Device 300 An IPFIX Device hosts at least one Exporting Process. It 301 may host further Exporting processes and arbitrary numbers 302 of Observation Points and Metering Process. 304 Collecting Process 306 A Collecting Process receives Flow Records from one or more 307 Exporting Processes. The Collecting Process might process 308 or store received Flow Records, but such actions are out of 309 scope for this document. 311 Template 313 Reducing redundancy in IPFIX and PSAMP reports 315 Template is an ordered sequence of pairs, 316 used to completely specify the structure and semantics of a 317 particular set of information that needs to be communicated 318 from an IPFIX Device to a Collector. Each Template is 319 uniquely identifiable by means of a Template ID. 321 Template Record 323 A Template Record defines the structure and interpretation 324 of fields in a Data Record. 326 Data Record 328 A Data Record is a record that contains values of the 329 parameters corresponding to a Template Record. 331 Options Template Record 333 An Options Template Record is a Template Record that 334 defines the structure and interpretation of fields in a 335 Data Record, including defining how to scope the 336 applicability of the Data Record. 338 Set 340 Set is a generic term for a collection of records that have 341 a similar structure. In an IPFIX Message, one or more Sets 342 follow the Message Header. 344 There are three different types of Sets: Template Set, 345 Options Template Set, and Data Set. 347 Template Set 349 A Template Set is a collection of one or more Template 350 Records that have been grouped together in an IPFIX 351 Message. 353 Options Template Set 355 An Options Template Set is a collection of one or more 356 Options Template Records that have been grouped together in 357 an IPFIX Message. 359 Data Set 361 A Data Set is one or more Data Records, of the same type, 362 that are grouped together in an IPFIX Message. Each Data 363 Record is previously defined by a Template Record or an 364 Options Template Record. 366 Information Element 368 An Information Element is a protocol and encoding 369 independent description of an attribute which may appear in 370 an IPFIX Record. The IPFIX information model [IPFIX-INFO] 371 defines the base set of Information Elements for IPFIX. 372 The type associated with an Information Element indicates 373 constraints on what it may contain and also determines the 374 valid encoding mechanisms for use in IPFIX. 376 Observed Packet Stream 378 The Observed Packet Stream is the set of all packets 379 observed at the Observation Point. 381 Reducing redundancy in IPFIX and PSAMP reports 383 Packet Content 385 The packet content denotes the union of the packet header 386 (which includes link layer, network layer and other 387 encapsulation headers) and the packet payload. 389 Selection Process 391 A Selection Process takes the Observed Packet Stream as its 392 input and selects a subset of that stream as its output. 394 Selector 396 A Selector defines the action of a Selection Process on a 397 single packet of its input. If selected, the packet 398 becomes an element of the output Packet Stream. 400 The Selector can make use of the following information in 401 determining whether a packet is selected: 403 (i) the Packet Content; 405 (ii) information derived from the packet's treatment 406 at the Observation Point; 408 (iii) any selection state that may be maintained by the 409 Selection Process. 411 PSAMP Device 413 A PSAMP Device is a device hosting at least an Observation 414 Point, a Selection Process and an Exporting Process. 415 Typically, corresponding Observation Point(s), Selection 416 Process(es) and Exporting Process(es) are co-located at 417 this device, for example at a router. 419 Filtering 421 A filter is a Selector that selects a packet 422 deterministically based on the Packet Content, or its 423 treatment, or functions of these occurring in the Selection 424 State. Examples include field match Filtering, and Hash- 425 based Selection. 427 CommonPropertiesID 429 An identifier of a set of common properties that is locally 430 unique to an Exporting Process and to Observation Domain. 431 This ID can be used to link to information reported in 432 separate records. See [IPFIX-INFO] for the Information 433 Element definition. 435 Common Properties 437 Common Properties are a collection of one or more 438 attributes shared by a set of different Flow Records. Each 439 set of Common Properties is uniquely identifiable by means 440 of a commonPropertiesID. 442 Specific Properties 444 Specific Properties are a collection of one or more 445 attributes reported in a Flow Record that are not included 446 in the Common Properties defined for that Flow Record. 448 Reducing redundancy in IPFIX and PSAMP reports 450 2.1 Terminology Summary Table. 452 +------------------+---------------------------------------------+ 453 | | Contents | 454 | +--------------------+------------------------+ 455 | Set | Template | Record | 456 +------------------+--------------------+------------------------+ 457 | Data Set | / | Data Record(s) | 458 +------------------+--------------------+------------------------+ 459 | Template Set | Template Record(s) | / | 460 +------------------+--------------------+------------------------+ 461 | Options Template | Options Template | / | 462 | Set | Record(s) | | 463 +------------------+--------------------+------------------------+ 465 Figure 1: Terminology Summary Table 467 A Data Set is composed of Data Record(s). No Template Record is 468 included. A Template Record or an Options Template Record 469 defines the Data Record. 471 A Template Set contains only Template Record(s). 473 An Options Template Set contains only Options Template 474 Record(s). 476 2.2 IPFIX Flows versus PSAMP Packets 478 As described in [PSAMP-PROTO], the major difference between 479 IPFIX and PSAMP is that the IPFIX protocol exports Flow Records 480 while the PSAMP protocol exports Packet Records. From a pure 481 export point of view, IPFIX will not distinguish a Flow Record 482 composed of several packets aggregated together from a Flow 483 Record composed of a single packet. So the PSAMP export can be 484 seen as special IPFIX Flow Record containing information about a 485 single packet. 487 For this document clarity, the term Flow Record represents a 488 generic term expressing an IPFIX Flow Record or a PSAMP packet 489 record, as foreseen by its definition. However, when 490 appropriate, a clear distinction between Flow Record or packet 491 Record will be made. 493 3. Problem Statement and High Level Solution 495 Several Flow Records often share a set of common properties. 496 Repeating the information about these common properties for 497 every Flow Record introduces a huge amount of redundancy. This 498 draft proposes a method to reduce this redundancy. The next 499 section describes the generic concept. Section 3.1.2 identifies 500 that the proposed solution can be applied multiple times. 501 Section 3.2 utilizes the concept to export per-packet 502 information. 504 3.1 Per Flow Data Reduction 506 3.1.1 Unique Data Reduction 508 Consider a set of properties "A", e.g. common sourceAddressA and 509 sourcePortA, equivalent for each Flow Records exported. Figure 2 511 Reducing redundancy in IPFIX and PSAMP reports 513 shows how this information is repeated with classical IPFIX Flow 514 Records, expressing the waste of bandwidth to export redundant 515 information. 517 +----------------+-------------+---------------------------+ 518 | sourceAddressA | sourcePortA | | 519 +----------------+-------------+---------------------------+ 520 | sourceAddressA | sourcePortA | | 521 +----------------+-------------+---------------------------+ 522 | sourceAddressA | sourcePortA | | 523 +----------------+-------------+---------------------------+ 524 | sourceAddressA | sourcePortA | | 525 +----------------+-------------+---------------------------+ 526 | ... | ... | ... | 527 +----------------+-------------+---------------------------+ 529 Figure 2: Common and Specific Properties exported in the same 530 record 532 Figure 3 shows how this information is exported when applying 533 the specifications of this document. The Common Properties are 534 separated from the Specific Properties for each Flow Record. 535 The Common Properties would be exported only once in a specific 536 Data Record (defined by an Option Template), while each Flow 537 Record contains a pointer to the Common Properties A, along with 538 its Flow specific information. In order to maintain the 539 relationship between these sets of properties, we introduce 540 indices (index A) for the Common Properties that are unique for 541 all Common Properties entries within an Observation Domain. The 542 purpose of the indices is to serve as a "key" identifying "rows" 543 of the Common Properties table. The rows are then referenced by 544 the Specific Properties by using the appropriate value for the 545 Common Properties identifier. 547 +------------------------+-----------------+-------------+ 548 | index for properties A | sourceAddressA | sourcePortA | 549 +------------------------+-----------------+-------------+ 550 | ... | ... | ... | 551 +------------------------+-----------------+-------------+ 553 +------------------------+---------------------------+ 554 | index for properties A | | 555 +------------------------+---------------------------+ 556 | index for properties A | | 557 +------------------------+---------------------------+ 558 | index for properties A | | 559 +------------------------+---------------------------+ 560 | index for properties A | | 561 +------------------------+---------------------------+ 563 Figure 3: Common and Specific Properties exported in different 564 records 566 This unique export of the Common Properties results in a 567 decrease of the bandwidth requirements from the Exporter to the 568 Collector. 570 3.1.2 Multiple Data Reduction 572 A Flow Record can refer to one or more Common Properties sets; 573 the use of multiple Common Properties can lead to more efficient 574 exports. Note that in the case of multiple Common Properties, 575 the different sets of Common Properties MUST be disjoint (i.e. 577 Reducing redundancy in IPFIX and PSAMP reports 579 MUST not have information elements in common), to avoid 580 potential collisions. 582 Consider a set of properties "A", e.g. common sourceAddressA and 583 sourcePortA and another set of properties "B", e.g. 584 destinationAddressB and destinationPortB. Figure 4 shows how 585 this information is repeated with classical IPFIX export in 586 several Flow Records. 588 +--------+--------+---------+---------+---------------------+ 589 |srcAddrA|srcPortA|destAddrB|destPortB| | 590 +--------+--------+---------+---------+---------------------+ 591 |srcAddrA|srcPortA|destAddrB|destPortB| | 592 +--------+--------+---------+---------+---------------------+ 593 |srcAddrA|srcPortA|destAddrB|destPortB| | 594 +--------+--------+---------+---------+---------------------+ 595 |srcAddrA|srcPortA|destAddrB|destPortB| | 596 +--------+--------+---------+---------+---------------------+ 597 | ... | ... | ... | ... | ... | 598 +--------+--------+---------+---------+---------------------+ 600 Figure 4: Common and Specific Properties exported in the same 601 record 603 We can separate the Common Properties into the properties A 604 composed of sourceAddressA and sourcePortA, and into the 605 properties B composed of destinationAddressB and 606 destinationPortB. The Flow Record that only contain the 607 property A will only contain the index for property A, the Flow 608 Record that only contain the property B will contain the index 609 for property B, while the Flow Record that contain both the 610 properties A and B contains both indexes (see Figure 5). 612 +-------------------+-----------------+-------------+ 613 | index for prop. A | sourceAddressA | sourcePortA | 614 +-------------------+-----------------+-------------+ 616 +-------------------+---------------------+------------------+ 617 | index for prop. B | destinationAddressB | destinationPortB | 618 +-------------------+---------------------+------------------+ 620 +-----------------+-----------------+-----------------------+ 621 |index for prop. A|index for prop. B| | 622 +-----------------+-----------------+-----------------------+ 623 |index for prop. A|index for prop. B| | 624 +-----------------+-----------------+-----------------------+ 625 |index for prop. A|index for prop. B| | 626 +-----------------+-----------------+-----------------------+ 627 |index for prop. A|index for prop. B| | 628 +-----------------+-----------------+-----------------------+ 629 | ... | ... | ... | 630 +-----------------+-----------------+-----------------------+ 632 Figure 5: Multiple Common (above) and Specific Properties 633 (below) exported in different records 635 The advantage of the multiple Common Properties is that the 636 objective of reducing the bandwidth is met while the number of 637 index is kept to a minimum. Indeed, an alternative solution 638 would have been to have an extra index for the property C, 639 composed of sourceAddressA, sourcePortA, destinationAddressB, 640 destinationPortB. 642 Reducing redundancy in IPFIX and PSAMP reports 644 3.2 Per Packet Data Reduction 646 The PSAMP protocol can be used for the export of per-packet 647 information. In this case the specific packet of observation 648 could be considered a special case of a Flow (a Flow Record 649 composed of a single packet) and consequently per-packet 650 information could be exported using Flow Records. However, if 651 filtering is applied to select a subset of all packets, using 652 IPFIX to export per-packet information is relatively inefficient 653 since all packets belonging to the same series share common 654 attributes (e.g. source address, destination address, etc). 656 A first example of the per packet data reduction is the 657 measurement of One-Way Delay (OWD), where the exact same 658 specific packet must be observed at the source and destination 659 of the path to be measured. By subtracting the time of 660 observation of the same packet at the two end points with 661 synchronized clocks, the OWD is computed. As the OWD is measured 662 for a specific application on which a Service Level Agreement 663 (SLA) is bound, this translates into the observation of packets 664 with specific properties, results of filtering. For example, 665 all the packets of a specific source and destination IP 666 addresses, of a specific DSCP value, and of a specific 667 destination transport port. In order to match the identical 668 packet at both Observation Points, a series of packets with 669 those properties must be observed on both ends of the 670 measurements. This implies the export of a series of Flow 671 Records composed of two types of information: some common 672 information for all packets, and some unique information about 673 packets in order to generate a unique identifier for each packet 674 passing this Observation Point (for example, a hash value on the 675 invariant fields of the packet). So, the two IPFIX Devices 676 composing the measurements end points can individually apply the 677 redundancy technique described in this draft in order to save 678 some bandwidth for the Flow Records export. 680 A second example of per packet data reduction is trajectory 681 sampling. 683 [*** TODO: make the distinction between 1. temporal export of 684 same information from one PSAMP device 2. export of similar 685 information from different devices. The method in this document 686 only applies to 1.] 688 A third example of per packet data reduction is One-packet flows 689 exported from a single router with a zero second export. 691 [*** TODO: This would be an example of the I.E. 313 – 692 ipHeaderPacketSection and I.E 314 – ipPayloadPacketSection in 693 PSAMP] 695 Figure 6, which displays the high level solution for the per 696 packet reduction, depicts three packets belonging to Flow A (and 697 therefore sharing the set of Common Properties A) and one packet 698 belonging to Flow B, respectively. It shows export records 699 containing packet specific information and the Common Properties 700 (source and destination address). The Common Properties 701 introduce a huge amount of redundancy, as they are repeated for 702 every packet in every Data Record. 704 Reducing redundancy in IPFIX and PSAMP reports 706 +----------+-----------+--------------------------+ 707 | srcAddrA | destAddrA | | 708 +----------+-----------+--------------------------+ 709 | srcAddrA | destAddrA | | 710 +----------+-----------+--------------------------+ 711 | srcAddrB | destAddrB | | 712 +----------+-----------+--------------------------+ 713 | srcAddrA | destAddrA | | 714 +----------+-----------+--------------------------+ 716 Figure 6: Common and Specific Properties represented in one 717 record 719 In Figure 7 we separate Common Properties from Specific 720 Properties, i.e. Common Properties from specific packet 721 information. In order to maintain the relation between Specific 722 (Packet) Properties and Common Properties we introduce indices 723 (index A and index B), as previously explained. 725 +----------+-----------+------------------------+ 726 | srcAddrA | destAddrA | index for properties A | 727 +----------+-----------+------------------------+ 728 | srcAddrB | destAddrB | index for properties B | 729 +----------+-----------+------------------------+ 731 +------------------------+------------------+ 732 | index for properties A | | 733 +------------------------+------------------+ 734 | index for properties A | | 735 +------------------------+------------------+ 736 | index for properties B | | 737 +------------------------+------------------+ 738 | index for properties A | | 739 +------------------------+------------------+ 741 Figure 7: Common and Specific (packet) Properties exported 742 separately 744 4. Specifications for bandwidth saving information export 746 The IPFIX protocol [IPFIX-PROTO] is Template based. Templates 747 define how data should be exported, describing data fields 748 together with their type and meaning. IPFIX specifies two types 749 of Templates: the Template Record and the Options Template 750 Record. The difference between the two is that the Options 751 Template Record includes the notion of scope, defining how to 752 scope the applicability of the Data Record. The scope, which is 753 only available in the Options Template Record, gives the context 754 of the reported Information Elements in the Data Records. The 755 Template Records and Options Template Records are necessary to 756 decode the Data Records. Indeed, by only looking at the Data 757 Records themselves, this is impossible to distinguish a Data 758 Record defined by Template Record from a Data Record defined by 759 an Option Template Record. To export information more 760 efficiently, this specification proposes to group Flow Records 761 by their common properties. We define Common Properties as a 762 collection of attributes shared by a set of different Flow 763 Records. 765 Reducing redundancy in IPFIX and PSAMP reports 767 4.1 Per Flow Data Reduction 769 4.1.1 Unique Data Reduction 771 As explained in Figure 8, the information is split into two 772 parts, using two different Data Records. Common Properties MUST 773 be exported via Data Records defined by an Option Template 774 Record and MUST be sent only once with SCTP and TCP. These 775 properties represent values common to several Flow Records (e.g. 776 IP source and destination address). The Common Properties Data 777 Records MUST be sent prior to the corresponding Specific 778 Properties Data Records. The Data Records reporting Specific 779 Properties MUST be associated with the Data Records reporting 780 the Common Properties using a unique identifier for the Common 781 Properties, the commonPropertiesID Information Element. The 782 commonPropertiesID MUST be exported as the scope in the Options 783 Template Record, and also exported in the associated Template 784 Record. 786 +---------------------------+ +---------------------+ 787 | Common Properties | | Specific Properties | Template 788 | Option Template Record | | Template Record | Definition 789 | | | | 790 | scope: commonPropertiesID | | commonPropertiesID | 791 | Common Properties | | Specific Properties | 792 +------------+--------------+ +---------+-----------+ 793 .............|...............................|....................... 794 | | 795 +------------v-------------+ +----------v----------+ 796 | Common Properties | | Specific Properties |+ Exported 797 | Data Record |------> Data Records || Data 798 +--------------------------+ +---------------------+| Records 799 +---------------------+ 801 Figure 8: Template Record and Data Record dependencies 803 The Common Properties are valid for all Flow Records containing 804 the associated commonPropertiesID. Since the commonPropertiesID 805 is a 64-bit data type, this method limits the number of active 806 data reduction to 2**64 per Exporting Process and Observation 807 Domain. 808 The assignment of Flow Records to common attributes could be 809 alternatively provided by the templateID Information Element 810 (instead of the commonPropertiesID Information Element). In this 811 case, the scope in the Common Properties Option Template Record 812 must contain the Template ID used in the Specific Properties 813 Template Record, as displayed in Figure 9. The Common 814 Properties are valid for all data records of the specified 815 Template. In this case the use of commonPropertiesID is not 816 required. 818 Reducing redundancy in IPFIX and PSAMP reports 820 +---------------------------+ +---------------------+ 821 | Common Properties | | Specific Properties | Template 822 | Option Template Record | | Template Record | Definition 823 | | | | 824 | scope: Template ID | | Specific Properties | 825 | Common Sroperties | | | 826 +------------+--------------+ +---------+-----------+ 827 .............|...............................|....................... 828 | | 829 +------------v-------------+ +----------v----------+ 830 | Common Properties | | Specific Properties |+ Exported 831 | Data Record |------> Data Records || Data 832 +--------------------------+ +---------------------+| Records 833 +---------------------+ 835 Figure 9: Template Records and Data Records linked with 836 TemplateID 838 4.1.2 Multiple Data Reduction 840 If a set of Flow Records share multiple sets of Common 841 Properties, multiple commonPropertiesID instances MAY be used to 842 increase export efficiency even further, as displayed in the 843 Figure 10. 845 +----------------------------+ +---------------------+ 846 | Common Properties | | Specific Properties | Template 847 | Option Template Record | | Template Record | Definition 848 | | | | 849 | Scope: commonPropertiesID1 | | commonPropertiesID1 | 850 | Scope: commonPropertiesID2 | | commonPropertiesID2 | 851 | Common Properties | | Specific Properties | 852 +------------+---------------+ +---------+-----------+ 853 .............|...............................|....................... 854 | | 855 +------------v-------------+ +----------v----------+ 856 | Common Properties | | Specific Properties |+ Exported 857 | Data Record |------> Data Records || Data 858 +--------------------------+ +---------------------+| Records 859 +---------------------+ 861 Figure 10: Multiple data reduction 863 4.2 Per-Packet Data Reduction 865 From the IPFIX protocol, there are no differences between the 866 Flow Record or per packet record data reduction, except maybe 867 the terminology where the Specific Properties could be called 868 packet specific properties in the following Figure 11. 870 Reducing redundancy in IPFIX and PSAMP reports 872 +---------------------------+ +---------------------+ 873 | Common Properties | | Specific Properties | Template 874 | Option Template Record | | Template Record | Definition 875 | | | | 876 | scope: commonPropertiesID | | commonPropertiesID | 877 | Common Properties | | Specific Properties | 878 +------------+--------------+ +---------+-----------+ 879 .............|...............................|....................... 880 | | 881 +------------v-------------+ +----------v----------+ 882 | Common Properties | | Specific Properties |+ Exported 883 | Data Record |------> Data Records || Data 884 +--------------------------+ +---------------------+| Records 885 +---------------------+ 887 Figure 11: Per-packet data reduction 889 5. Transport Protocol Choice 891 This document follows the IPFIX transport protocol 892 specifications defined in [IPFIX-PROTO]. However, depending on 893 the transport protocol choice, this document imposes some more 894 constraints. If SCTP is selected as the IPFIX protocol, the SCTP 895 sub-section specifications MUST be respected. If UDP is selected 896 as the IPFIX protocol, the UDP sub-section specifications MUST 897 be respected. If TCP is selected as the IPFIX protocol, the TCP 898 sub-section specifications MUST be respected. 900 5.1 SCTP 902 The active Common Properties MUST be sent after the SCTP 903 association establishment before the corresponding Specific 904 Properties Data Records. In case of SCTP association re- 905 establishment, all active Common Properties MUST be re-sent 906 before the corresponding Specific Properties Data Records. 908 The Common Properties Flow Records MUST be sent on a reliable 909 SCTP stream. 911 5.2 UDP 913 Common Properties Data Records MUST be re-sent at regular 914 intervals, whose frequency MUST be configurable. 915 CommonPropertiesIDs have a specified lifetime during which they 916 cannot be reused. After that time a commonPropertiesID can be 917 assigned to another set of Common Properties. CommonPropertiesID 918 whose lifetime has longer expired SHOULD be preferred. The 919 lifetime MUST be configurable. 921 5.3 TCP 923 Common Properties MUST be sent after the TCP connection 924 establishment before the corresponding Specific Properties Data 925 Records. In case of TCP connection re-establishment, all active 926 Common Properties MUST be re-sent before the corresponding 927 Specific Properties Data Records. 929 Reducing redundancy in IPFIX and PSAMP reports 931 6. commonPropertiesID Management 933 The commonPropertiesID is an identifier of a set of common 934 properties that is locally unique to an Exporting Process and to 935 Observation Domain. The Exporting Process MUST manage the 936 commonPropertiesIDs allocations for its Observation Domains. 937 Different Observation Domains from the same Exporter MAY use the 938 same commonPropertiesID value to refer to different sets of 939 Common Properties. 941 The commonPropertiesID values MAY be assigned sequentially, but 942 it’s NOT REQUIRED. Particular commonPropertiesID ranges or 943 values MAY have explicit meanings for the IPFIX Device. For 944 example, commonPropertiesID values may be assigned based on the 945 result of a hash function, etc... 947 Using a 64-bit commonPropertiesID Information Element allows the 948 export of 2**64 -1 active sets of Common Properties, per 949 Observation Domain, per Exporting Process. 951 CommonPropertiesIDs that are not used anymore SHOULD be 952 withdrawn. The Common Properties ID withdrawal message is an 953 Option Data Record consisting of only one scope field namely the 954 CommonPropertiesID and no non-scope fields. 956 0 1 2 3 957 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 958 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 959 | Set ID = 3 | Length = 14 octets | 960 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 961 | Template ID = 259 | Field Count = 1 | 962 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 963 | Scope Field count = 1 |0| commonPropertiesID = XX | 964 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 965 | Scope 1 Field Length = 8 | 966 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 968 Figure 12: CommonPropertiesID withdrawal template 970 0 1 2 3 971 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 972 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 973 | Set ID = 259 | Length = 12 octets | 974 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 975 | N | 976 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 977 | ... | 978 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 980 Figure 13: CommonPropertiesID withdrawal record, withdrawing 981 CommonPropertiesID N 983 7. The Collecting Process Side 985 The Collecting Process can either store the Flow Records as they 986 arrive, without reconstructing the initial Flow Record, or 987 reconstruct the initial Flow Record. In the former case there 988 might be less storage capacity required at the Collector side. 989 In the latter the collector job is more complex and time- 990 consuming due to the higher resource demand for record 991 processing in real time. 993 Reducing redundancy in IPFIX and PSAMP reports 995 Like TemplateIDs the CommonPropertiesIDs are generated 996 dynamically by the Exporting Process. The CommonPropertiesIDs 997 are only valid within the protocol stack. Hence a restart of the 998 exporting process may imply a renumbering of CommonProperiesIDs. 999 For this reason it is not recommended to use the 1000 CommonPropertiesIds outside the protocol stack e.g. to store 1001 them within a database. Outside the protocol stack there is 1002 additional information needed to keep a non-ambiguous 1003 association between the related Common Properties and Specific 1004 Properties. 1006 If the Collecting Process has received the Specific Properties 1007 Data Record before the associated Common Properties Data Record, 1008 the Collecting Process MAY store the Specific Properties Data 1009 Record and await the retransmission or out-of-order arrival of 1010 the Common Properties Data Record. 1012 If a Collection Process receives a CommonPropertiesID Withdraw 1013 Record, the Collection Process MUST expire the related Common 1014 Properties data. 1016 If SCTP is selected as the IPFIX protocol, the SCTP sub-section 1017 specifications MUST be respected. If UDP is selected as the 1018 IPFIX protocol, the UDP sub-section specifications MUST be 1019 respected. If TCP is selected as the IPFIX protocol, the TCP 1020 sub-section specifications MUST be respected. 1022 7.1 SCTP 1024 When the SCTP association is reset, either gracefully or 1025 abnormally, the Collecting Processes MUST delete all 1026 commonPropertiesID values associated with that association. 1028 7.2 UDP 1030 The Collecting Process associates a lifetime with each 1031 commonPropertiesID. The mapping of Data Records to Common 1032 Properties uses the most recent Common Properties definition 1033 associated to the specified commonPropertiesID. The lifetime of 1034 the CommonPropertiesID ends on the receipt of a 1035 CommonPropertiesID withdrawal record. If there is no flow 1036 definition associated with that commonPropertiesID or the 1037 lifetime of the flow definition has expired, no mapping is 1038 possible. In this case the Collecting Process MAY store the 1039 Specific Properties and await the retransmission or out-of-order 1040 arrival of the Common Properties. 1042 7.3 TCP 1044 When the TCP connection is reset, either gracefully or 1045 abnormally, the Collecting Processes MUST expire all 1046 commonPropertiesID values corresponding to that connection. 1048 8. Export and Evaluation Considerations 1050 The main advantage of the method specified in this document is 1051 the reduction in the amount of measurement data that has to be 1052 transferred from the Exporter to the Collector. In addition 1053 there might be less storage capacity required at the Collector 1054 side if the Collector decides to store the Flow Records as they 1055 arrive, without reconstructing the initial Flow Record. 1057 On the other hand, these methods require additional resources on 1058 both the Exporter and the Collector. The Exporter has to manage 1059 Common Properties information and to assign commonPropertiesId 1061 Reducing redundancy in IPFIX and PSAMP reports 1063 values to Flow Records. The Collector has to process records 1064 described by two templates instead of just one. Additional 1065 effort is also required when post processing the measurement 1066 data, in order to correlate Flow Records with Common Properties 1067 information. 1069 8.1 Transport Protocol Choice 1071 The proposed method is most effective using a reliable transport 1072 protocol for the transfer of the Common Properties. Therefore 1073 the use of SCTP or TCP is recommended. However, if the path from 1074 the Exporting Process to the Collecting Process is not fully 1075 reliable, the SCTP or TCP retransmission might reduce the 1076 benefits of this specification. If the path from the Exporting 1077 Process to the Collecting Process is full reliable, the use of 1078 UDP is less effective because the common properties have to be 1079 re-sent regularly. 1081 8.2 Reduced Size Encoding 1083 The transfer of the CommonPropertiesIDs originates some 1084 overhead. Note that IPFIX allows reduced-size encoding of 1085 Information Elements. In cases where the range of the 1086 commonPropertiesID can be restricted, reduced-size encoding can 1087 be applied also to the commonPropertiesID, and would result in a 1088 further bandwidth efficiency gain. 1090 8.3 CommonPropertiesID vs. TemplateID scope 1092 The assignment of Flow Records to common attributes could be 1093 done via the CommonPropertiesID and alternatively via the 1094 templateID Information Element. In the second case the 1095 commonPropertiesID is not required: this reduces the overhead 1096 but the Exporting Process must use one templateID per set of 1097 Common Properties. In the general case, this method is not 1098 scalable, but it can be suitable for certain applications. 1100 8.4 Efficiency Gain 1102 The example in section 11.2 below uses IPFIX to export 1103 measurement data for each received packet. In that case, for a 1104 flow of 1000 packets the amount of data can be decreased more 1105 than 33 percent. 1107 While the goal of this specification is to reduce the bandwidth, 1108 the efficiency might be limited. Indeed, the efficiency gain 1109 is based on the numerous redundant information in flows. While 1110 the Exporting Process can evaluate the direct gain for the Flow 1111 Records to be exported, it can’t predict whether future Flow 1112 Records would contain the information specified by active 1113 commonPropertiesID values. This implies that the efficiency 1114 factor of this specification is higher for specific applications 1115 where filtering is involved, such as one-way delay or trajectory 1116 sampling. 1118 9. IANA Considerations 1120 This document has no actions for IANA. 1122 10. Security Considerations 1124 For the proposed use of the IPFIX protocol for bandwidth-saving 1125 export the security considerations as for the IPFIX protocol 1126 apply. 1128 Reducing redundancy in IPFIX and PSAMP reports 1130 11. Appendix A: Examples 1132 11.1 Per Flow Data Reduction 1134 11.1.1 Unique Data Reduction 1136 In this section we show how flow information can be exported 1137 efficiently using the method described in this draft. Let's 1138 suppose we have to periodically export data about two IPv6 1139 Flows. 1141 In this example we report the following information: 1143 Flow| dstIPv6Address | dst- |nPkts|nBytes 1144 | | Port | | 1145 ---------------------------------------------------------------- 1146 A |5F05:2000:80AD:5800:0058:0800:2023:1D71| 80 | 30 | 6000 1147 | | | | 1148 A |5F05:2000:80AD:5800:0058:0800:2023:1D71| 80 | 50 | 9500 1149 | | | | 1150 B |5F05:2000:80AD:5800:0058:00AA:00B7:AF2B| 1932 | 60 | 8000 1151 | | | | 1152 A |5F05:2000:80AD:5800:0058:0800:2023:1D71| 80 | 40 | 6500 1153 | | | | 1154 A |5F05:2000:80AD:5800:0058:0800:2023:1D71| 80 | 60 | 9500 1155 | | | | 1156 B |5F05:2000:80AD:5800:0058:00AA:00B7:AF2B| 1932 | 54 | 7600 1158 The Common Properties in this case are the destination IPv6 1159 address and the destination port. We first define an Option 1160 Template that contains the following Information Elements: 1162 - Scope: the commonPropertiesID, with a type of 137 [IPFIX- 1163 INFO] and a length of 8 octets. 1165 - The destination IPv6 address, destinationIPv6Address 1166 [IPFIX-INFO], with a type of 28 and a length of 16 octets 1168 - The destination port, destinationTransportPort [IPFIX-INFO] 1169 with a type of 11, and a length of 2 octets 1171 Figure 14 shows the Option template defining the Common 1172 Properties with commonPropertiesID as scope: 1174 Reducing redundancy in IPFIX and PSAMP reports 1176 0 1 2 3 1177 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1178 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1179 | Set ID = 3 | Length = 24 octets | 1180 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1181 | Template ID = 257 | Field Count = 3 | 1182 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1183 | Scope Field count = 1 |0| commonPropertiesID = 137 | 1184 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1185 | Scope 1 Field Length = 8 |0| destinationIPv6Address = 28| 1186 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1187 | Field Length = 16 |0|destinationTransportPort = 11| 1188 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1189 | Field Length = 2 | (Padding) | 1190 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1192 Figure 14: Common Properties Option Template 1194 The Specific Properties Template consists of the information not 1195 contained in the Option Templates, i.e. flow specific 1196 information, in this case the number of packets and the number 1197 of bytes to be reported. Additionally, this Template contains 1198 the commonPropertiesID. In Data Records, the value of this field 1199 will contain one of the unique indices of the Option Records 1200 exported before. It contains the following Information Elements 1201 (see also Figure 15): 1203 - commonPropertiesID with a length of 8 octets 1205 - The number of packets of the Flow: inPacketDeltaCount in 1206 [IPFIX-INFO], with a length of 4 octets 1208 - The number of octets of the Flow: inOctetDeltaCount in 1209 [IPFIX-INFO], with a length of 4 octets 1211 0 1 2 3 1212 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1213 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1214 | Set ID = 2 | Length = 20 octets | 1215 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1216 | Template ID = 258 | Field Count = 4 | 1217 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1218 |0| commonPropertiesID = 137 | Field Length = 8 | 1219 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1220 |0| inPacketDeltaCount = 2 | Field Length = 4 | 1221 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1222 |0| inOctetDeltaCount = 1 | Field Length = 4 | 1223 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1225 Figure 15: Specific Properties Template 1227 Considering the data shown at the beginning of this example, the 1228 following two Data Records will be exported: 1230 Common- | dstAddress | dst- 1231 PropertiesID | | Port 1232 -------------+-----------------------------------------+------- 1233 101 | 5F05:2000:80AD:5800:0058:0800:2023:1D71 | 80 1234 | | 1235 102 | 5F05:2000:80AD:5800:0058:00AA:00B7:AF2B | 1932 1237 Reducing redundancy in IPFIX and PSAMP reports 1239 The Data Records reporting the Common Properties will look like: 1241 0 1 2 3 1242 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1243 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1244 | Set ID = 257 | Length = 60 octets | 1245 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1246 | 101 | 1247 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1248 | ... | 1249 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1250 | 5F05:2000: ... | 1251 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1252 | ... 80AD:5800: ... | 1253 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1254 | ... 0058:0800: ... | 1255 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1256 | ... 2023:1D71 | 1257 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1258 | 80 | (Padding) | 1259 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1260 | 102 | 1261 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1262 | ... | 1263 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1264 | 5F05:2000: ... | 1265 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1266 | ... 80AD:5800: ... | 1267 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1268 | ... 0058:00AA: ... | 1269 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1270 | ... 00B7:AF2B | 1271 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1272 | 1932 | (Padding) | 1273 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1275 Figure 16: Data Records reporting Common Properties 1277 The Data Records will in turn be: 1279 commonPropertiesID | inPacketDeltaCount | inOctetDeltaCount 1280 --------------------------------------------------------------- 1281 101 | 30 | 6000 1282 101 | 50 | 9500 1283 102 | 60 | 8000 1284 101 | 40 | 6500 1285 101 | 60 | 9500 1286 102 | 54 | 7600 1288 Figure 17 shows the first Data Record listed in the table: 1290 0 1 2 3 1291 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1292 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1293 | Set ID = 258 | Length = 16 | 1294 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1295 | 101 | 1296 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1297 | ... | 1298 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1299 | 30 | 6000 | 1300 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1302 Figure 17: Data Record reporting Specific Properties 1304 Reducing redundancy in IPFIX and PSAMP reports 1306 11.1.2 Multiple Data Reduction 1308 In this example we export the following flow information: 1310 Flow | srcAddr | srcPort | dstAddr | dstPort | nPackets | nBytes 1311 ---------------------------------------------------------------- 1312 A |10.0.0.1 | 1932 |10.0.1.2 | 80 | 30 | 6000 1313 B |10.0.0.3 | 2032 |10.0.1.2 | 80 | 50 | 9500 1315 Figure 18 shows the Option Templates, containing the Common 1316 Properties together with the commonPropertiesID as Scope. 1318 In the first Common Properties Option Template we export the 1319 following Information Elements: 1321 - Scope 1: the Common Properties ID, commonPropertiesId with 1322 a type of 137 [IPFIX-INFO]. Note that the commonProperties 1323 IE has a length of 8 octets, but if smaller size is 1324 sufficient to carry any value the Exporter may need to 1325 deliver, reduced size encoding can be used. In this example 1326 we use reduced sizing, of 4 octets. 1328 - the source IPv4 Address, sourceIPv4Address [IPFIX-INFO], 1329 with a type of 8 and a length of 4 octets 1331 - the source Port, sourceTransportPort [IPFIX-INFO], with a 1332 type of 7 and a length of 2 octets 1334 The second Option Template contains the following Information 1335 Elements: 1337 - Scope 2: the commonPropertiesID, with a type of 137 [IPFIX- 1338 INFO] and a length of 4 octets (reduced sizing). 1340 - the destination IPv4 Address, destinationIPv4Address 1341 [IPFIX-INFO], with a type of 12 and a length of 4 octets 1343 - the destination port, destinationTransportPort [IPFIX-INFO] 1344 with a type of 11, and a length of 2 octets 1346 The commonPropertiesId Information Element [NOTE: to be included 1347 in IPFIX-INFO], is used in both cases as the Scope Field. 1349 0 1 2 3 1350 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1351 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1352 | Set ID = 3 | Length = 24 octets | 1353 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1354 | Template ID = 256 | Field Count = 3 | 1355 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1356 | Scope Field count = 1 |0| commonPropertiesID = 137 | 1357 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1358 | Scope 1 Field Length = 4 |0| sourceIPv4Address = 8 | 1359 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1360 | Field Length = 4 |0| transportSourcePort = 7 | 1361 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1362 | Field Length = 2 | (Padding) | 1363 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1365 Reducing redundancy in IPFIX and PSAMP reports 1367 0 1 2 3 1368 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1369 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1370 | Set ID = 3 | Length = 24 octets | 1371 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1372 | Template ID = 257 | Field Count = 3 | 1373 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1374 | Scope Field count = 1 |0| commonPropertiesID = 137 | 1375 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1376 | Scope 1 Field Length = 4 |0| destinationIPv4Address = 12| 1377 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1378 | Field Length = 4 |0|transportDestinationPort = 11| 1379 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1380 | Field Length = 2 | (Padding) | 1381 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1383 Figure 18: Example Common Properties Template 1385 Considering the values given at the beginning of this section we 1386 will export the Common Properties using the following Data 1387 Records: 1389 commonPropertiesID | sourceAddress | sourcePort 1390 --------------------+-----------------+------------- 1391 101 | 10.0.0.1 | 1932 1392 102 | 10.0.0.3 | 2032 1394 and 1396 commonPropertiesID | dstAddress | dstPort 1397 --------------------+---------------+----------- 1398 103 | 10.0.1.2 | 80 1400 The Specific Properties Template consists of the information not 1401 contained in the Option Templates, i.e. flow specific 1402 information. Additionally, this Template contains the two 1403 commonPropertiesID. In Data Records, the values of each of these 1404 fields will contain one of the unique indices specified in the 1405 Option Records exported previously. 1407 Figure 19 displays the Template including the commonPropertiesID 1408 plus the Specific Properties. In this example we export the 1409 following Information Elements: 1411 - commonPropertiesID for the source fields with a length of 4 1412 octets (reduced size encoding) 1414 - commonPropertiesID for the destination fields with a length 1415 of 4 octets (reduced size encoding) 1417 - the number of packets of the Flow: inPacketDeltaCount in 1418 [IPFIX-INFO], with a length of 4 octets 1420 - the number of octets of the Flow: inOctetDeltaCount in 1421 [IPFIX-INFO], with a length of 4 octets 1423 Reducing redundancy in IPFIX and PSAMP reports 1425 0 1 2 3 1426 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1427 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1428 | Set ID = 2 | Length = 24 octets | 1429 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1430 | Template ID = 259 | Field Count = 4 | 1431 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1432 |0| commonPropertiesID = 137 | Field Length = 4 | 1433 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1434 |0| commonPropertiesID = 137 | Field Length = 4 | 1435 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1436 |0| inPacketDeltaCount = 2 | Field Length = 4 | 1437 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1438 |0| inOctetDeltaCount = 1 | Field Length = 4 | 1439 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1441 Figure 19: Example Specific Properties Template 1443 Considering the values given at the beginning of this section, 1444 the Data Records of the two flows will look like: 1446 0 1 2 3 1447 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1448 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1449 | Set ID = 256 | Length = 28 | 1450 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1451 | 101 | 1452 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1453 | 103 | 1454 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1455 | 30 | 6000 | 1456 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1457 | 102 | 1458 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1459 | 103 | 1460 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1461 | 50 | 9500 | 1462 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1464 Figure 20: Specific Properties 1466 11.2 Per-Packet Information Export 1468 This section demonstrates per-packet information export to 1469 support passive One-Way Delay (OWD) measurement. The Templates 1470 required for exporting measurement data of this kind are 1471 illustrated in the figures below. 1472 Figure 21 shows the Option Template containing the information 1473 concerning Flows using the commonPropertiesID as scope. In the 1474 Common Properties Template we export the following Information 1475 Elements: 1477 - the source IPv4 Address, sourceIPv4Address [IPFIX-INFO], 1478 with a type of 8 and a length of 4 octets 1480 - the destination IPv4 Address, destinationIPv4Address 1481 [IPFIX-INFO], with a type of 12 and a length of 4 octets 1483 - the Class of Service field, ClassOfServiceIPv4 [IPFIX- 1484 INFO], with a type of 5 and a length of 1 octet 1486 - the Protocol Identifier, protocolIdentifier [IPFIX-INFO], 1487 with a type of 4 and a length of 1 octet 1489 Reducing redundancy in IPFIX and PSAMP reports 1491 - source port, sourceTransportPort [IPFIX-INFO], with a type 1492 of 7 and and a length of 2 octets 1494 - destination port, destinationTransportPort [IPFIX-INFO], 1495 with a type of 11 and a length of 2 octets 1497 The commonPropertiesID Information Element, is used as the Scope 1498 Field. 1500 0 1 2 3 1501 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1502 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1503 | Set ID = 3 | Length = 40 octets | 1504 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1505 | Template ID = 256 | Field Count = 7 | 1506 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1507 | Scope Field count = 1 |0| commonPropertiesID = XX | 1508 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1509 | Scope 1 Field Length = 4 |0| sourceIPv4Address = 8 | 1510 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1511 | Field Length = 4 |0| destinationIPv4Address = 12 | 1512 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1513 | Field Length = 4 |0| classOfServiceIPv4 = 5 | 1514 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1515 | Field Length = 1 |0| protocolIdentifier = 4 | 1516 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1517 | Field Length = 1 |0| transportSourcePort = 7 | 1518 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1519 | Field Length = 2 |0|transportDestinationPort = 11| 1520 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1521 | Field Length = 2 | (Padding) | 1522 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1524 Figure 21: Example Flow Properties Template 1526 For passive One-Way-Delay measurement, the Packet Properties 1527 Template, or Specific Properties Template, consists of at least 1528 Timestamp and Packet ID. Additionally, this template contains a 1529 commonPropertiesId field to associate the packet with a Flow. 1531 Figure 22 displays the template with the packet properties. In 1532 this example we export the following Information Elements: 1534 - commonPropertiesID. In this case reduced size encoding is 1535 used, and the Information Element is declared with a length 1536 of 4 octets instead of 8. 1538 - packetTimestamp, packetID, and packetLength. Since 1539 packetTimestamp, packetID, and packetLength are not (yet) 1540 IETF-defined information elements, we export them as 1541 enterprise-specific IEs. The three IEs have respectively a 1542 type of 220, 221, and 222 and a length of 8, 4, and 4 1543 octets. 1545 Reducing redundancy in IPFIX and PSAMP reports 1547 0 1 2 3 1548 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 1549 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1550 | Set ID = 2 | Length = 36 octets | 1551 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1552 | Template ID = 257 | Field Count = 4 | 1553 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1554 |0| commonPropertiesID = 137 | Field Length = 4 | 1555 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1556 |1| packetTimestamp = 220 | Field Length = 8 | 1557 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1558 | Enterprise number | 1559 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1560 |1| packetID = 221 | Field Length = 4 | 1561 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1562 | Enterprise number | 1563 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1564 |1| packetLength = 222 | Field Length = 4 | 1565 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1566 | Enterprise number | 1567 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1569 Figure 22: Example Packet Properties Template 1571 At the collection point, packet records from the two measurement 1572 points are gathered and correlated by means of the packet ID. 1573 The resulting delay data records are exported in a similar 1574 manner as the packet data. One-way delay data is associated with 1575 flow information by the commonPropertiesId field. The OWD 1576 properties contain the Packet Pair ID (which is the packet ID of 1577 the two contributing packet records), the timestamp of the 1578 packet passing the reference monitor point in order to 1579 reconstruct a time series, the calculated delay value, and the 1580 commonPropertiesID. 1582 In this example using IPFIX to export the measurement data for 1583 each received packet 30 bytes have to be transferred 1584 (sourceAddressV4=4, destinationAddressV4=4, classOfServiceV4=1, 1585 protocolIdentifier=1, sourceTransportPort=2, 1586 destionationTransportPort=2, packetTimestamp=8, packetID=4, 1587 packetLength=4). Without considering the IPFIX protocol overhead 1588 a flow of 1000 packets produces 30000 bytes of measurement data. 1589 Using the proposed optimization each packet produces an export 1590 of only 20 bytes (packetTimestamp=8, packetID=4, packetLength=4, 1591 commonPropertiesID=4). The export of the flow information 1592 produces 18 bytes (sourceAddressV4=4, destinationAddressV4=4, 1593 classOfServiceV4=1, protocolIdentifier=1, sourceTransportPort=2, 1594 destionationTransportPort=2, commonPropertiesID =4). For a flow 1595 of 1000 packets this sums up to 20018 bytes. This is a decrease 1596 of more than 33 percent. 1598 12. References 1600 12.1 Normative References 1602 [RFC2119] Bradner, S., "Key words for use in RFCs to 1603 Indicate Requirement Levels", BCP 14, RFC 2119, 1604 March 1997 1606 [IPFIX-PROTO] Benoit Claise et Al.: IPFIX Protocol 1607 Specification, IETF draft work in progress 1608 , April 2006 1610 Reducing redundancy in IPFIX and PSAMP reports 1612 [IPFIX-INFO] J. Quittek, S.Bryant, B.Claise, J. Meyer: 1613 Information Model for IP Flow Information Export 1614 Internet-draft work in progress , September 2005 1617 [PSAMP-PROTO] Benoit Claise: PSAMP Protocol Specification, 1618 Internet Draft , 1619 March 2006 1621 12.2 Informative References 1623 [IPFIX-ARCH] Sadasivan, G., Brownlee, N., Claise, B., Quittek, 1624 J., "Architecture Model for IP Flow Information 1625 Export" draft-ietf-ipfix-arch-11.txt, May 2005 1627 [IPFIX-AS] Zseby, T., Boschi, E., Brownlee, N., Claise, B., 1628 "IPFIX Applicability", draft-ietf-ipfix-as-06.txt, 1629 May 2005 1631 [PSAMP-TECH] T. Zseby, M. Molina, N. Duffield, S. Niccolini, F. 1632 Raspall, "Sampling and Filtering Techniques for IP 1633 Packet Selection" draft-ietf-psamp-sample-tech- 1634 07.txt 1636 [PSAMP-INFO] T. Dietz, F. Dressler, G. Carle, B. Claise, 1637 "Information Model for Packet Sampling Exports", 1638 draft-ietf-psamp-info-03.txt 1640 [PSAMP-MIB] T. Dietz, B. Claise "Definitions of Managed Objects 1641 for Packet Sampling" draft-ietf-psamp-mib-05.txt 1643 [PSAMP-FMWK] D. Chiou, B. Claise, N. Duffield, A. Greenberg, M. 1644 Grossglauser, P. Marimuthu, J. Rexford, G. 1645 Sadasivan, "A Framework for Passive Packet 1646 Measurement" draft-ietf-psamp-framework-10.txt 1648 [RFC3917] Quittek, J., Zseby, T., Claise, B., Zander, S., 1649 "Requirements for IP Flow Information Export" RFC 1650 3917, October 2004 1652 13. Author's Addresses 1654 Elisa Boschi 1655 Hitachi Europe SAS 1656 Immeuble Le Theleme 1657 1503 Route des Dolines 1658 06560 Valbonne, France 1659 Phone: +33 4 89874180 1660 Email: elisa.boschi@hitachi-eu.com 1662 Lutz Mark 1663 Fraunhofer Institute for Open Communication Systems 1664 Kaiserin-Augusta-Allee 31 1665 10589 Berlin 1666 Germany 1667 Phone: +49-30-34 63 7306 1668 Fax: +49-30-34 53 8306 1669 Email: mark@fokus.fraunhofer.de 1671 Benoit Claise 1672 Cisco Systems 1673 De Kleetlaan 6a b1 1674 Diegem 1813 1675 Belgium 1677 Reducing redundancy in IPFIX and PSAMP reports 1679 Phone: +32 2 704 5622 1680 Email: bclaise@cisco.com 1682 14. Intellectual Property Statement 1684 The IETF takes no position regarding the validity or scope of 1685 any Intellectual Property Rights or other rights that might be 1686 claimed to pertain to the implementation or use of the 1687 technology described in this document or the extent to which any 1688 license under such rights might or might not be available; nor 1689 does it represent that it has made any independent effort to 1690 identify any such rights. Information on the procedures with 1691 respect to rights in RFC documents can be found in BCP 78 and 1692 BCP 79. 1693 Copies of IPR disclosures made to the IETF Secretariat and any 1694 assurances of licenses to be made available, or the result of an 1695 attempt made to obtain a general license or permission for the 1696 use of such proprietary rights by implementers or users of this 1697 specification can be obtained from the IETF on-line IPR 1698 repository at http://www.ietf.org/ipr. 1700 The IETF invites any interested party to bring to its attention 1701 any copyrights, patents or patent applications, or other 1702 proprietary rights that may cover technology that may be 1703 required to implement this standard. Please address the 1704 information to the IETF at ietf-ipr@ietf.org. 1706 15. Copyright Statement 1708 Copyright (C) The Internet Society (2006). This document is 1709 subject to the rights, licenses and restrictions contained in 1710 BCP 78, and except as set forth therein, the authors retain all 1711 their rights. 1713 16. Disclaimer 1715 This document and the information contained herein are provided 1716 on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE 1717 REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND 1718 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, 1719 EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY 1720 THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY 1721 RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS 1722 FOR A PARTICULAR PURPOSE.