idnits 2.17.00 (12 Aug 2021) /tmp/idnits19911/draft-ietf-httpbis-header-structure-18.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([2], [3], [4], [5], [1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (April 19, 2020) is 761 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1' on line 1570 -- Looks like a reference, but probably isn't: '2' on line 1572 -- Looks like a reference, but probably isn't: '3' on line 1574 -- Looks like a reference, but probably isn't: '4' on line 1576 -- Looks like a reference, but probably isn't: '5' on line 1578 == Missing Reference: 'RFCxxxx' is mentioned on line 333, but not defined == Missing Reference: 'RFC3986' is mentioned on line 339, but not defined -- Looks like a reference, but probably isn't: '6' on line 1631 Summary: 1 error (**), 0 flaws (~~), 3 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 HTTP M. Nottingham 3 Internet-Draft Fastly 4 Intended status: Standards Track P-H. Kamp 5 Expires: October 21, 2020 The Varnish Cache Project 6 April 19, 2020 8 Structured Field Values for HTTP 9 draft-ietf-httpbis-header-structure-18 11 Abstract 13 This document describes a set of data types and associated algorithms 14 that are intended to make it easier and safer to define and handle 15 HTTP header and trailer fields, known as "Structured Fields", 16 "Structured Headers", or "Structured Trailers". It is intended for 17 use by specifications of new HTTP fields that wish to use a common 18 syntax that is more restrictive than traditional HTTP field values. 20 Note to Readers 22 _RFC EDITOR: please remove this section before publication_ 24 Discussion of this draft takes place on the HTTP working group 25 mailing list (ietf-http-wg@w3.org), which is archived at 26 https://lists.w3.org/Archives/Public/ietf-http-wg/ [1]. 28 Working Group information can be found at https://httpwg.github.io/ 29 [2]; source code and issues list for this draft can be found at 30 https://github.com/httpwg/http-extensions/labels/header-structure 31 [3]. 33 Tests for implementations are collected at https://github.com/httpwg/ 34 structured-header-tests [4]. 36 Implementations are tracked at https://github.com/httpwg/wiki/wiki/ 37 Structured-Headers [5]. 39 Status of This Memo 41 This Internet-Draft is submitted in full conformance with the 42 provisions of BCP 78 and BCP 79. 44 Internet-Drafts are working documents of the Internet Engineering 45 Task Force (IETF). Note that other groups may also distribute 46 working documents as Internet-Drafts. The list of current Internet- 47 Drafts is at https://datatracker.ietf.org/drafts/current/. 49 Internet-Drafts are draft documents valid for a maximum of six months 50 and may be updated, replaced, or obsoleted by other documents at any 51 time. It is inappropriate to use Internet-Drafts as reference 52 material or to cite them other than as "work in progress." 54 This Internet-Draft will expire on October 21, 2020. 56 Copyright Notice 58 Copyright (c) 2020 IETF Trust and the persons identified as the 59 document authors. All rights reserved. 61 This document is subject to BCP 78 and the IETF Trust's Legal 62 Provisions Relating to IETF Documents 63 (https://trustee.ietf.org/license-info) in effect on the date of 64 publication of this document. Please review these documents 65 carefully, as they describe your rights and restrictions with respect 66 to this document. Code Components extracted from this document must 67 include Simplified BSD License text as described in Section 4.e of 68 the Trust Legal Provisions and are provided without warranty as 69 described in the Simplified BSD License. 71 Table of Contents 73 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 74 1.1. Intentionally Strict Processing . . . . . . . . . . . . . 4 75 1.2. Notational Conventions . . . . . . . . . . . . . . . . . 5 76 2. Defining New Structured Fields . . . . . . . . . . . . . . . 5 77 3. Structured Data Types . . . . . . . . . . . . . . . . . . . . 8 78 3.1. Lists . . . . . . . . . . . . . . . . . . . . . . . . . . 9 79 3.1.1. Inner Lists . . . . . . . . . . . . . . . . . . . . . 9 80 3.1.2. Parameters . . . . . . . . . . . . . . . . . . . . . 10 81 3.2. Dictionaries . . . . . . . . . . . . . . . . . . . . . . 11 82 3.3. Items . . . . . . . . . . . . . . . . . . . . . . . . . . 12 83 3.3.1. Integers . . . . . . . . . . . . . . . . . . . . . . 13 84 3.3.2. Decimals . . . . . . . . . . . . . . . . . . . . . . 13 85 3.3.3. Strings . . . . . . . . . . . . . . . . . . . . . . . 13 86 3.3.4. Tokens . . . . . . . . . . . . . . . . . . . . . . . 14 87 3.3.5. Byte Sequences . . . . . . . . . . . . . . . . . . . 14 88 3.3.6. Booleans . . . . . . . . . . . . . . . . . . . . . . 15 89 4. Working With Structured Fields in HTTP . . . . . . . . . . . 15 90 4.1. Serializing Structured Fields . . . . . . . . . . . . . . 15 91 4.1.1. Serializing a List . . . . . . . . . . . . . . . . . 16 92 4.1.2. Serializing a Dictionary . . . . . . . . . . . . . . 18 93 4.1.3. Serializing an Item . . . . . . . . . . . . . . . . . 18 94 4.1.4. Serializing an Integer . . . . . . . . . . . . . . . 19 95 4.1.5. Serializing a Decimal . . . . . . . . . . . . . . . . 20 96 4.1.6. Serializing a String . . . . . . . . . . . . . . . . 20 97 4.1.7. Serializing a Token . . . . . . . . . . . . . . . . . 21 98 4.1.8. Serializing a Byte Sequence . . . . . . . . . . . . . 21 99 4.1.9. Serializing a Boolean . . . . . . . . . . . . . . . . 22 100 4.2. Parsing Structured Fields . . . . . . . . . . . . . . . . 22 101 4.2.1. Parsing a List . . . . . . . . . . . . . . . . . . . 24 102 4.2.2. Parsing a Dictionary . . . . . . . . . . . . . . . . 25 103 4.2.3. Parsing an Item . . . . . . . . . . . . . . . . . . . 26 104 4.2.4. Parsing an Integer or Decimal . . . . . . . . . . . . 28 105 4.2.5. Parsing a String . . . . . . . . . . . . . . . . . . 30 106 4.2.6. Parsing a Token . . . . . . . . . . . . . . . . . . . 30 107 4.2.7. Parsing a Byte Sequence . . . . . . . . . . . . . . . 31 108 4.2.8. Parsing a Boolean . . . . . . . . . . . . . . . . . . 32 109 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32 110 6. Security Considerations . . . . . . . . . . . . . . . . . . . 32 111 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 32 112 7.1. Normative References . . . . . . . . . . . . . . . . . . 32 113 7.2. Informative References . . . . . . . . . . . . . . . . . 33 114 7.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 34 115 Appendix A. Frequently Asked Questions . . . . . . . . . . . . . 34 116 A.1. Why not JSON? . . . . . . . . . . . . . . . . . . . . . . 34 117 Appendix B. Implementation Notes . . . . . . . . . . . . . . . . 35 118 Appendix C. Changes . . . . . . . . . . . . . . . . . . . . . . 35 119 C.1. Since draft-ietf-httpbis-header-structure-17 . . . . . . 36 120 C.2. Since draft-ietf-httpbis-header-structure-16 . . . . . . 36 121 C.3. Since draft-ietf-httpbis-header-structure-15 . . . . . . 36 122 C.4. Since draft-ietf-httpbis-header-structure-14 . . . . . . 36 123 C.5. Since draft-ietf-httpbis-header-structure-13 . . . . . . 37 124 C.6. Since draft-ietf-httpbis-header-structure-12 . . . . . . 37 125 C.7. Since draft-ietf-httpbis-header-structure-11 . . . . . . 37 126 C.8. Since draft-ietf-httpbis-header-structure-10 . . . . . . 37 127 C.9. Since draft-ietf-httpbis-header-structure-09 . . . . . . 38 128 C.10. Since draft-ietf-httpbis-header-structure-08 . . . . . . 38 129 C.11. Since draft-ietf-httpbis-header-structure-07 . . . . . . 39 130 C.12. Since draft-ietf-httpbis-header-structure-06 . . . . . . 39 131 C.13. Since draft-ietf-httpbis-header-structure-05 . . . . . . 39 132 C.14. Since draft-ietf-httpbis-header-structure-04 . . . . . . 39 133 C.15. Since draft-ietf-httpbis-header-structure-03 . . . . . . 40 134 C.16. Since draft-ietf-httpbis-header-structure-02 . . . . . . 40 135 C.17. Since draft-ietf-httpbis-header-structure-01 . . . . . . 40 136 C.18. Since draft-ietf-httpbis-header-structure-00 . . . . . . 40 137 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 40 138 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 41 140 1. Introduction 142 Specifying the syntax of new HTTP header (and trailer) fields is an 143 onerous task; even with the guidance in Section 8.3.1 of [RFC7231], 144 there are many decisions - and pitfalls - for a prospective HTTP 145 field author. 147 Once a field is defined, bespoke parsers and serializers often need 148 to be written, because each field value has slightly different 149 handling of what looks like common syntax. 151 This document introduces a set of common data structures for use in 152 definitions of new HTTP field values to address these problems. In 153 particular, it defines a generic, abstract model for them, along with 154 a concrete serialization for expressing that model in HTTP [RFC7230] 155 header and trailer fields. 157 A HTTP field that is defined as a "Structured Header" or "Structured 158 Trailer" (if the field can be either, it is a "Structured Field") 159 uses the types defined in this specification to define its syntax and 160 basic handling rules, thereby simplifying both its definition by 161 specification writers and handling by implementations. 163 Additionally, future versions of HTTP can define alternative 164 serializations of the abstract model of these structures, allowing 165 fields that use that model to be transmitted more efficiently without 166 being redefined. 168 Note that it is not a goal of this document to redefine the syntax of 169 existing HTTP fields; the mechanisms described herein are only 170 intended to be used with those that explicitly opt into them. 172 Section 2 describes how to specify a Structured Field. 174 Section 3 defines a number of abstract data types that can be used in 175 Structured Fields. 177 Those abstract types can be serialized into and parsed from HTTP 178 field values using the algorithms described in Section 4. 180 1.1. Intentionally Strict Processing 182 This specification intentionally defines strict parsing and 183 serialization behaviors using step-by-step algorithms; the only error 184 handling defined is to fail the operation altogether. 186 It is designed to encourage faithful implementation and therefore 187 good interoperability. Therefore, an implementation that tried to be 188 "helpful" by being more tolerant of input would make interoperability 189 worse, since that would create pressure on other implementations to 190 implement similar (but likely subtly different) workarounds. 192 In other words, strict processing is an intentional feature of this 193 specification; it allows non-conformant input to be discovered and 194 corrected by the producer early, and avoids both interoperability and 195 security issues that might otherwise result. 197 Note that as a result of this strictness, if a field is appended to 198 by multiple parties (e.g., intermediaries, or different components in 199 the sender), an error in one party's value is likely to cause the 200 entire field value to fail parsing. 202 1.2. Notational Conventions 204 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 205 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 206 "OPTIONAL" in this document are to be interpreted as described in BCP 207 14 [RFC2119] [RFC8174] when, and only when, they appear in all 208 capitals, as shown here. 210 This document uses algorithms to specify parsing and serialization 211 behaviors, and the Augmented Backus-Naur Form (ABNF) notation of 212 [RFC5234] to illustrate expected syntax in HTTP header fields. In 213 doing so, it uses the VCHAR, SP, DIGIT, ALPHA and DQUOTE rules from 214 [RFC5234]. It also includes the tchar rule from [RFC7230]. 216 When parsing from HTTP fields, implementations MUST follow the 217 algorithms, but MAY vary in implementation so as the behaviors are 218 indistinguishable from specified behavior. If there is disagreement 219 between the parsing algorithms and ABNF, the specified algorithms 220 take precedence. In some places, the algorithms are "greedy" with 221 whitespace, but this should not affect conformance. 223 For serialization to HTTP fields, the ABNF illustrates the range of 224 acceptable wire representations with as much fidelity as possible, 225 and the algorithms define the recommended way to produce them. 226 Implementations MAY vary from the specified behavior so long as the 227 output still matches the ABNF. 229 2. Defining New Structured Fields 231 To specify a HTTP field as a Structured Field, its authors needs to: 233 o Reference this specification. Recipients and generators of the 234 field need to know that the requirements of this document are in 235 effect. 237 o Identify whether the field is a Structured Header (i.e., it can 238 only be used in the header section - the common case), a 239 Structured Trailer (only in the trailer section), or a Structured 240 Field (both). 242 o Specify the type of the field value; either List (Section 3.1), 243 Dictionary (Section 3.2), or Item (Section 3.3). 245 o Define the semantics of those structures. 247 o Specify any additional constraints upon the structures used, as 248 well as the consequences when those constraints are violated. 250 Typically, this means that a field definition will specify the top- 251 level type - List, Dictionary or Item - and then define its allowable 252 types, and constraints upon them. For example, a header defined as a 253 List might have all Integer members, or a mix of types; a header 254 defined as an Item might allow only Strings, and additionally only 255 strings beginning with the letter "Q". Likewise, Inner Lists are 256 only valid when a field definition explicitly allows them. 258 When parsing fails, the entire field is ignored (see Section 4.2); in 259 most situations, violating field-specific constraints should have the 260 same effect. Thus, if a header is defined as an Item and required to 261 be an Integer, but a String is received, the field will by default be 262 ignored. If the field requires different error handling, this should 263 be explicitly specified. 265 Both Items and Inner Lists allow parameters as an extensibility 266 mechanism; this means that values can later be extended to 267 accommodate more information, if need be. To preserve forward 268 compatibility, field specifications are discouraged from defining the 269 presence of an unrecognized Parameter as an error condition. 271 To further assure that this extensibility is available in the future, 272 and to encourage consumers to use a complete parser implementation, a 273 field definition can specify that "grease" Parameters be added by 274 senders. A specification could stipulate that all Parameters that 275 fit a defined pattern are reserved for this use and then encourage 276 them to be sent on some portion of requests. This helps to 277 discourage recipients from writing a parser that does not account for 278 Parameters. 280 Specifications that use Dictionaries can also allow for forward 281 compatibility by requiring that the presence of - as well as value 282 and type associated with - unknown members be ignored. Later 283 specifications can then add additional members, specifying 284 constraints on them as appropriate. 286 An extension to a structured field can then require that an entire 287 field value be ignored by a recipient that understands the extension 288 if constraints on the value it defines are not met. 290 A field definition cannot relax the requirements of this 291 specification because doing so would preclude handling by generic 292 software; they can only add additional constraints (for example, on 293 the numeric range of Integers and Decimals, the format of Strings and 294 Tokens, the types allowed in a Dictionary's values, or the number of 295 Items in a List). Likewise, field definitions can only use this 296 specification for the entire field value, not a portion thereof. 298 This specification defines minimums for the length or number of 299 various structures supported by implementations. It does not specify 300 maximum sizes in most cases, but authors should be aware that HTTP 301 implementations do impose various limits on the size of individual 302 fields, the total number of fields, and/or the size of the entire 303 header or trailer section. 305 Specifications can refer to a field name as a "structured header 306 name", "structured trailer name" or "structured field name" as 307 appropriate. Likewise, they can refer its field value as a 308 "structured header value", "structured trailer value" or "structured 309 field value" as necessary. Field definitions are encouraged to use 310 the ABNF rules beginning with "sh-" defined in this specification; 311 other rules in this specification are not intended for their use. 313 For example, a fictitious Foo-Example header field might be specified 314 as: 316 --8<-- 317 42. Foo-Example Header 319 The Foo-Example HTTP header field conveys information about how 320 much Foo the message has. 322 Foo-Example is a Item Structured Header [RFCxxxx]. Its value MUST be 323 an Integer (Section Y.Y of [RFCxxxx]). Its ABNF is: 325 Foo-Example = sh-integer 327 Its value indicates the amount of Foo in the message, and MUST 328 be between 0 and 10, inclusive; other values MUST cause 329 the entire header field to be ignored. 331 The following parameters are defined: 332 * A Parameter whose name is "foourl", and whose value is a String 333 (Section Y.Y of [RFCxxxx]), conveying the Foo URL 334 for the message. See below for processing requirements. 336 "foourl" contains a URI-reference (Section 4.1 of [RFC3986]). If 337 its value is not a valid URI-reference, the entire header field 338 MUST be ignored. If its value is a relative reference (Section 4.2 339 of [RFC3986]), it MUST be resolved (Section 5 of [RFC3986]) before 340 being used. 342 For example: 344 Foo-Example: 2; foourl="https://foo.example.com/" 345 -->8-- 347 3. Structured Data Types 349 This section defines the abstract value types that can be composed 350 into Structured Fields. The ABNF provided represents the on-wire 351 format in HTTP field values. 353 In summary: 355 o There are three top-level types that a HTTP field can be defined 356 as: Lists, Dictionaries, and Items. 358 o Lists and Dictionaries are containers; their members can be Items 359 or Inner Lists (which are themselves arrays of Items). 361 o Both Items and Inner Lists can be parameterized with key/value 362 pairs. 364 3.1. Lists 366 Lists are arrays of zero or more members, each of which can be an 367 Item (Section 3.3) or an Inner List (Section 3.1.1), both of which 368 can be Parameterized (Section 3.1.2). 370 The ABNF for Lists in HTTP fields is: 372 sh-list = list-member *( *SP "," *SP list-member ) 373 list-member = sh-item / inner-list 375 Each member is separated by a comma and optional whitespace. For 376 example, a field whose value is defined as a List of Strings could 377 look like: 379 Example-StrListHeader: "foo", "bar", "It was the best of times." 381 An empty List is denoted by not serializing the field at all. 383 Note that Lists can have their members split across multiple lines 384 inside a header or trailer section, as per Section 3.2.2 of 385 [RFC7230]; for example, the following are equivalent: 387 Example-Hdr: foo, bar 389 and 391 Example-Hdr: foo 392 Example-Hdr: bar 394 However, individual members of a List cannot be safely split between 395 across lines; see Section 4.2 for details. 397 Parsers MUST support Lists containing at least 1024 members. Field 398 specifications can constrain the types and cardinality of individual 399 List values as they require. 401 3.1.1. Inner Lists 403 An Inner List is an array of zero or more Items (Section 3.3). Both 404 the individual Items and the Inner List itself can be Parameterized 405 (Section 3.1.2). 407 The ABNF for Inner Lists is: 409 inner-list = "(" *SP [ sh-item *( 1*SP sh-item ) *SP ] ")" 410 parameters 412 Inner Lists are denoted by surrounding parenthesis, and have their 413 values delimited by a single space. A field whose value is defined 414 as a List of Inner Lists of Strings could look like: 416 Example-StrListListHeader: ("foo" "bar"), ("baz"), ("bat" "one"), () 418 Note that the last member in this example is an empty Inner List. 420 A header field whose value is defined as a List of Inner Lists with 421 Parameters at both levels could look like: 423 Example-ListListParam: ("foo"; a=1;b=2);lvl=5, ("bar" "baz");lvl=1 425 Parsers MUST support Inner Lists containing at least 256 members. 426 Field specifications can constrain the types and cardinality of 427 individual Inner List members as they require. 429 3.1.2. Parameters 431 Parameters are an ordered map of key-values pairs that are associated 432 with an Item (Section 3.3) or Inner List (Section 3.1.1). The keys 433 are unique within the scope the Parameters they occur within, and the 434 values are bare items (i.e., they themselves cannot be parameterized; 435 see Section 3.3). 437 The ABNF for Parameters is: 439 parameters = *( ";" *SP parameter ) 440 parameter = param-name [ "=" param-value ] 441 param-name = key 442 key = ( lcalpha / "*" ) 443 *( lcalpha / DIGIT / "_" / "-" / "." / "*" ) 444 lcalpha = %x61-7A ; a-z 445 param-value = bare-item 447 A parameter is separated from its Item or Inner List and other 448 parameters by a semicolon. For example: 450 Example-ParamListHeader: abc;a=1;b=2; cde_456, (ghi;jk=4 l);q="9";r=w 452 Parameters whose value is Boolean true MUST omit that value when 453 serialized. For example: 455 Example-IntHeader: 1; a; b=?0 457 Note that this requirement is only on serialization; parsers are 458 still required to correctly handle the true value when it appears in 459 a parameter. 461 Parsers MUST support at least 256 parameters on an Item or Inner 462 List, and support parameter keys with at least 64 characters. Field 463 specifications can constrain the types and cardinality of individual 464 parameter names and values as they require. 466 3.2. Dictionaries 468 Dictionaries are ordered maps of name-value pairs, where the names 469 are short, textual strings and the values are items (Section 3.3) or 470 arrays of items, both of which can be Parameterized (Section 3.1.2). 471 There can be zero or more members, and their names are unique in the 472 scope of the Dictionary they occur within. 474 Implementations MUST provide access to Dictionaries both by index and 475 by name. Specifications MAY use either means of accessing the 476 members. 478 The ABNF for Dictionaries is: 480 sh-dictionary = dict-member *( *SP "," *SP dict-member ) 481 dict-member = member-name [ "=" member-value ] 482 member-name = key 483 member-value = sh-item / inner-list 485 Members are separated by a comma with optional whitespace, while 486 names and values are separated by "=" (without whitespace). For 487 example: 489 Example-DictHeader: en="Applepie", da=:w4ZibGV0w6ZydGU=: 491 Members whose value is Boolean true MUST omit that value when 492 serialized. For example, here both "b" and "c" are true: 494 Example-DictHeader: a=?0, b, c; foo=bar 496 Note that this requirement is only on serialization; parsers are 497 still required to correctly handle the true Boolean value when it 498 appears in Dictionary values. 500 A Dictionary with a member whose value is an Inner List of tokens: 502 Example-DictListHeader: rating=1.5, feelings=(joy sadness) 504 A Dictionary with a mix of singular and list values, some with 505 Parameters: 507 Example-MixDict: a=(1 2), b=3, c=4;aa=bb, d=(5 6);valid 508 As with lists, an empty Dictionary is represented by omitting the 509 entire field. 511 Typically, a field specification will define the semantics of 512 Dictionaries by specifying the allowed type(s) for individual member 513 names, as well as whether their presence is required or optional. 514 Recipients MUST ignore names that are undefined or unknown, unless 515 the field's specification specifically disallows them. 517 Note that dictionaries can have their members split across multiple 518 lines inside a header or trailer section; for example, the following 519 are equivalent: 521 Example-Hdr: foo=1, bar=2 523 and 525 Example-Hdr: foo=1 526 Example-Hdr: bar=2 528 However, individual members of a Dictionary cannot be safely split 529 between lines; see Section 4.2 for details. 531 Parsers MUST support Dictionaries containing at least 1024 name/value 532 pairs, and names with at least 64 characters. 534 3.3. Items 536 An Item can be a Integer (Section 3.3.1), Decimal (Section 3.3.2), 537 String (Section 3.3.3), Token (Section 3.3.4), Byte Sequence 538 (Section 3.3.5), or Boolean (Section 3.3.6). It can have associated 539 Parameters (Section 3.1.2). 541 The ABNF for Items is: 543 sh-item = bare-item parameters 544 bare-item = sh-integer / sh-decimal / sh-string / sh-token 545 / sh-binary / sh-boolean 547 For example, a header field that is defined to be an Item that is an 548 Integer might look like: 550 Example-IntItemHeader: 5 552 or with Parameters: 554 Example-IntItemHeader: 5; foo=bar 556 3.3.1. Integers 558 Integers have a range of -999,999,999,999,999 to 999,999,999,999,999 559 inclusive (i.e., up to fifteen digits, signed), for IEEE 754 560 compatibility ([IEEE754]). 562 The ABNF for Integers is: 564 sh-integer = ["-"] 1*15DIGIT 566 For example: 568 Example-IntegerHeader: 42 570 Note that commas in Integers are used in this section's prose only 571 for readability; they are not valid in the wire format. 573 3.3.2. Decimals 575 Decimals are numbers with an integer and a fractional component. The 576 integer component has at most 12 digits; the fractional component has 577 at most three digits. 579 The ABNF for decimals is: 581 sh-decimal = ["-"] 1*12DIGIT "." 1*3DIGIT 583 For example, a header whose value is defined as a Decimal could look 584 like: 586 Example-DecimalHeader: 4.5 588 Note that the serialisation algorithm (Section 4.1.5) rounds input 589 with more than three digits of precision in the fractional component. 590 If an alternative rounding strategy is desired, this should be 591 specified by the header definition to occur before serialisation. 593 3.3.3. Strings 595 Strings are zero or more printable ASCII [RFC0020] characters (i.e., 596 the range %x20 to %x7E). Note that this excludes tabs, newlines, 597 carriage returns, etc. 599 The ABNF for Strings is: 601 sh-string = DQUOTE *(chr) DQUOTE 602 chr = unescaped / escaped 603 unescaped = %x20-21 / %x23-5B / %x5D-7E 604 escaped = "\" ( DQUOTE / "\" ) 606 Strings are delimited with double quotes, using a backslash ("\") to 607 escape double quotes and backslashes. For example: 609 Example-StringHeader: "hello world" 611 Note that Strings only use DQUOTE as a delimiter; single quotes do 612 not delimit Strings. Furthermore, only DQUOTE and "\" can be 613 escaped; other characters after "\" MUST cause parsing to fail. 615 Unicode is not directly supported in Strings, because it causes a 616 number of interoperability issues, and - with few exceptions - field 617 values do not require it. 619 When it is necessary for a field value to convey non-ASCII content, a 620 Byte Sequence (Section 3.3.5) can be specified, along with a 621 character encoding (preferably [UTF-8]). 623 Parsers MUST support Strings (after any decoding) with at least 1024 624 characters. 626 3.3.4. Tokens 628 Tokens are short textual words; their abstract model is identical to 629 their expression in the HTTP field value serialization. 631 The ABNF for Tokens is: 633 sh-token = ( ALPHA / "*" ) *( tchar / ":" / "/" ) 635 Parsers MUST support Tokens with at least 512 characters. 637 Note that Token allows the same characters as the "token" ABNF rule 638 defined in [RFC7230], with the exceptions that the first character is 639 required to be either ALPHA or "*", and ":" and "/" are also allowed 640 in subsequent characters. 642 3.3.5. Byte Sequences 644 Byte Sequences can be conveyed in Structured Fields. 646 The ABNF for a Byte Sequence is: 648 sh-binary = ":" *(base64) ":" 649 base64 = ALPHA / DIGIT / "+" / "/" / "=" 651 A Byte Sequence is delimited with colons and encoded using base64 652 ([RFC4648], Section 4). For example: 654 Example-BinaryHdr: :cHJldGVuZCB0aGlzIGlzIGJpbmFyeSBjb250ZW50Lg==: 656 Parsers MUST support Byte Sequences with at least 16384 octets after 657 decoding. 659 3.3.6. Booleans 661 Boolean values can be conveyed in Structured Fields. 663 The ABNF for a Boolean is: 665 sh-boolean = "?" boolean 666 boolean = "0" / "1" 668 A Boolean is indicated with a leading "?" character followed by a "1" 669 for a true value or "0" for false. For example: 671 Example-BoolHdr: ?1 673 4. Working With Structured Fields in HTTP 675 This section defines how to serialize and parse Structured Fields in 676 field values, and protocols compatible with them (e.g., in HTTP/2 677 [RFC7540] before HPACK [RFC7541] is applied). 679 4.1. Serializing Structured Fields 681 Given a structure defined in this specification, return an ASCII 682 string suitable for use in a HTTP field value. 684 1. If the structure is a Dictionary or List and its value is empty 685 (i.e., it has no members), do not serialize the field at all 686 (i.e., omit both the field-name and field-value). 688 2. If the structure is a List, let output_string be the result of 689 running Serializing a List (Section 4.1.1) with the structure. 691 3. Else if the structure is a Dictionary, let output_string be the 692 result of running Serializing a Dictionary (Section 4.1.2) with 693 the structure. 695 4. Else if the structure is an Item, let output_string be the result 696 of running Serializing an Item (Section 4.1.3) with the 697 structure. 699 5. Else, fail serialization. 701 6. Return output_string converted into an array of bytes, using 702 ASCII encoding [RFC0020]. 704 4.1.1. Serializing a List 706 Given an array of (member_value, parameters) tuples as input_list, 707 return an ASCII string suitable for use in a HTTP field value. 709 1. Let output be an empty string. 711 2. For each (member_value, parameters) of input_list: 713 1. If member_value is an array, append the result of running 714 Serializing an Inner List (Section 4.1.1.1) with 715 (member_value, parameters) to output. 717 2. Otherwise, append the result of running Serializing an Item 718 (Section 4.1.3) with (member_value, parameters) to output. 720 3. If more member_values remain in input_list: 722 1. Append "," to output. 724 2. Append a single SP to output. 726 3. Return output. 728 4.1.1.1. Serializing an Inner List 730 Given an array of (member_value, parameters) tuples as inner_list, 731 and parameters as list_parameters, return an ASCII string suitable 732 for use in a HTTP field value. 734 1. Let output be the string "(". 736 2. For each (member_value, parameters) of inner_list: 738 1. Append the result of running Serializing an Item 739 (Section 4.1.3) with (member_value, parameters) to output. 741 2. If more values remain in inner_list, append a single SP to 742 output. 744 3. Append ")" to output. 746 4. Append the result of running Serializing Parameters 747 (Section 4.1.1.2) with list_parameters to output. 749 5. Return output. 751 4.1.1.2. Serializing Parameters 753 Given an ordered Dictionary as input_parameters (each member having a 754 param_name and a param_value), return an ASCII string suitable for 755 use in a HTTP field value. 757 1. Let output be an empty string. 759 2. For each param_name with a value of param_value in 760 input_parameters: 762 1. Append ";" to output. 764 2. Append the result of running Serializing a Key 765 (Section 4.1.1.3) with param_name to output. 767 3. If param_value is not Boolean true: 769 1. Append "=" to output. 771 2. Append the result of running Serializing a bare Item 772 (Section 4.1.3.1) with param_value to output. 774 3. Return output. 776 4.1.1.3. Serializing a Key 778 Given a key as input_key, return an ASCII string suitable for use in 779 a HTTP field value. 781 1. Convert input_key into a sequence of ASCII characters; if 782 conversion fails, fail serialization. 784 2. If input_key contains characters not in lcalpha, DIGIT, "_", "-", 785 ".", or "*" fail serialization. 787 3. If the first character of input_key is not lcalpha or "*", fail 788 serialization. 790 4. Let output be an empty string. 792 5. Append input_key to output. 794 6. Return output. 796 4.1.2. Serializing a Dictionary 798 Given an ordered Dictionary as input_dictionary (each member having a 799 member_name and a tuple value of (member_value, parameters)), return 800 an ASCII string suitable for use in a HTTP field value. 802 1. Let output be an empty string. 804 2. For each member_name with a value of (member_value, parameters) 805 in input_dictionary: 807 1. Append the result of running Serializing a Key 808 (Section 4.1.1.3) with member's member_name to output. 810 3. If member_value is Boolean true: 812 1. Append the result of running Serializing Parameters 813 (Section 4.1.1.2) with parameters to output. 815 4. Otherwise: 817 1. Append "=" to output. 819 2. If member_value is an array, append the result of running 820 Serializing an Inner List (Section 4.1.1.1) with 821 (member_value, parameters) to output. 823 3. Otherwise, append the result of running Serializing an Item 824 (Section 4.1.3) with (member_value, parameters) to output. 826 5. If more members remain in input_dictionary: 828 1. Append "," to output. 830 2. Append a single SP to output. 832 6. Return output. 834 4.1.3. Serializing an Item 836 Given an Item as bare_item and Parameters as item_parameters, return 837 an ASCII string suitable for use in a HTTP field value. 839 1. Let output be an empty string. 841 2. Append the result of running Serializing a Bare Item 842 Section 4.1.3.1 with bare_item to output. 844 3. Append the result of running Serializing Parameters 845 Section 4.1.1.2 with item_parameters to output. 847 4. Return output. 849 4.1.3.1. Serializing a Bare Item 851 Given an Item as input_item, return an ASCII string suitable for use 852 in a HTTP field value. 854 1. If input_item is an Integer, return the result of running 855 Serializing an Integer (Section 4.1.4) with input_item. 857 2. If input_item is a Decimal, return the result of running 858 Serializing a Decimal (Section 4.1.5) with input_item. 860 3. If input_item is a String, return the result of running 861 Serializing a String (Section 4.1.6) with input_item. 863 4. If input_item is a Token, return the result of running 864 Serializing a Token (Section 4.1.7) with input_item. 866 5. If input_item is a Boolean, return the result of running 867 Serializing a Boolean (Section 4.1.9) with input_item. 869 6. If input_item is a Byte Sequence, return the result of running 870 Serializing a Byte Sequence (Section 4.1.8) with input_item. 872 7. Otherwise, fail serialization. 874 4.1.4. Serializing an Integer 876 Given an Integer as input_integer, return an ASCII string suitable 877 for use in a HTTP field value. 879 1. If input_integer is not an integer in the range of 880 -999,999,999,999,999 to 999,999,999,999,999 inclusive, fail 881 serialization. 883 2. Let output be an empty string. 885 3. If input_integer is less than (but not equal to) 0, append "-" to 886 output. 888 4. Append input_integer's numeric value represented in base 10 using 889 only decimal digits to output. 891 5. Return output. 893 4.1.5. Serializing a Decimal 895 Given a decimal number as input_decimal, return an ASCII string 896 suitable for use in a HTTP field value. 898 1. If input_decimal is not a decimal number, fail serialization. 900 2. If input_decimal has more than three significant digits to the 901 right of the decimal point, round it to three decimal places, 902 rounding the final digit to the nearest value, or to the even 903 value if it is equidistant. 905 3. If input_decimal has more than 12 significant digits to the left 906 of the decimal point after rounding, fail serialization. 908 4. Let output be an empty string. 910 5. If input_decimal is less than (but not equal to) 0, append "-" 911 to output. 913 6. Append input_decimal's integer component represented in base 10 914 (using only decimal digits) to output; if it is zero, append 915 "0". 917 7. Append "." to output. 919 8. If input_decimal's fractional component is zero, append "0" to 920 output. 922 9. Otherwise, append the significant digits of input_decimal's 923 fractional component represented in base 10 (using only decimal 924 digits) to output. 926 10. Return output. 928 4.1.6. Serializing a String 930 Given a String as input_string, return an ASCII string suitable for 931 use in a HTTP field value. 933 1. Convert input_string into a sequence of ASCII characters; if 934 conversion fails, fail serialization. 936 2. If input_string contains characters in the range %x00-1f or %x7f 937 (i.e., not in VCHAR or SP), fail serialization. 939 3. Let output be an empty string. 941 4. Append DQUOTE to output. 943 5. For each character char in input_string: 945 1. If char is "\" or DQUOTE: 947 1. Append "\" to output. 949 2. Append char to output. 951 6. Append DQUOTE to output. 953 7. Return output. 955 4.1.7. Serializing a Token 957 Given a Token as input_token, return an ASCII string suitable for use 958 in a HTTP field value. 960 1. Convert input_token into a sequence of ASCII characters; if 961 conversion fails, fail serialization. 963 2. If the first character of input_token is not ALPHA or "*", or the 964 remaining portion contains a character not in tchar, ":" or "/", 965 fail serialization. 967 3. Let output be an empty string. 969 4. Append input_token to output. 971 5. Return output. 973 4.1.8. Serializing a Byte Sequence 975 Given a Byte Sequence as input_bytes, return an ASCII string suitable 976 for use in a HTTP field value. 978 1. If input_bytes is not a sequence of bytes, fail serialization. 980 2. Let output be an empty string. 982 3. Append ":" to output. 984 4. Append the result of base64-encoding input_bytes as per 985 [RFC4648], Section 4, taking account of the requirements below. 987 5. Append ":" to output. 989 6. Return output. 991 The encoded data is required to be padded with "=", as per [RFC4648], 992 Section 3.2. 994 Likewise, encoded data SHOULD have pad bits set to zero, as per 995 [RFC4648], Section 3.5, unless it is not possible to do so due to 996 implementation constraints. 998 4.1.9. Serializing a Boolean 1000 Given a Boolean as input_boolean, return an ASCII string suitable for 1001 use in a HTTP field value. 1003 1. If input_boolean is not a boolean, fail serialization. 1005 2. Let output be an empty string. 1007 3. Append "?" to output. 1009 4. If input_boolean is true, append "1" to output. 1011 5. If input_boolean is false, append "0" to output. 1013 6. Return output. 1015 4.2. Parsing Structured Fields 1017 When a receiving implementation parses HTTP fields that are known to 1018 be Structured Fields, it is important that care be taken, as there 1019 are a number of edge cases that can cause interoperability or even 1020 security problems. This section specifies the algorithm for doing 1021 so. 1023 Given an array of bytes input_bytes that represents the chosen 1024 field's field-value (which is empty if that field is not present), 1025 and field_type (one of "dictionary", "list", or "item"), return the 1026 parsed header value. 1028 1. Convert input_bytes into an ASCII string input_string; if 1029 conversion fails, fail parsing. 1031 2. Discard any leading SP characters from input_string. 1033 3. If field_type is "list", let output be the result of running 1034 Parsing a List (Section 4.2.1) with input_string. 1036 4. If field_type is "dictionary", let output be the result of 1037 running Parsing a Dictionary (Section 4.2.2) with input_string. 1039 5. If field_type is "item", let output be the result of running 1040 Parsing an Item (Section 4.2.3) with input_string. 1042 6. Discard any leading SP characters from input_string. 1044 7. If input_string is not empty, fail parsing. 1046 8. Otherwise, return output. 1048 When generating input_bytes, parsers MUST combine all lines in the 1049 same section (header or trailer) that case-insensitively match the 1050 field name into one comma-separated field-value, as per [RFC7230], 1051 Section 3.2.2; this assures that the entire field value is processed 1052 correctly. 1054 For Lists and Dictionaries, this has the effect of correctly 1055 concatenating all of the field's lines, as long as individual members 1056 of the top-level data structure are not split across multiple header 1057 instances. 1059 Strings split across multiple field lines will have unpredictable 1060 results, because comma(s) and whitespace inserted upon combination 1061 will become part of the string output by the parser. Since 1062 concatenation might be done by an upstream intermediary, the results 1063 are not under the control of the serializer or the parser. 1065 Tokens, Integers, Decimals and Byte Sequences cannot be split across 1066 multiple field lines because the inserted commas will cause parsing 1067 to fail. 1069 If parsing fails - including when calling another algorithm - the 1070 entire field value MUST be ignored (i.e., treated as if the field 1071 were not present in the section). This is intentionally strict, to 1072 improve interoperability and safety, and specifications referencing 1073 this document are not allowed to loosen this requirement. 1075 Note that this requirement does not apply to an implementation that 1076 is not parsing the field; for example, an intermediary is not 1077 required to strip a failing header field from a message before 1078 forwarding it. 1080 4.2.1. Parsing a List 1082 Given an ASCII string as input_string, return an array of 1083 (item_or_inner_list, parameters) tuples. input_string is modified to 1084 remove the parsed value. 1086 1. Let members be an empty array. 1088 2. While input_string is not empty: 1090 1. Append the result of running Parsing an Item or Inner List 1091 (Section 4.2.1.1) with input_string to members. 1093 2. Discard any leading SP characters from input_string. 1095 3. If input_string is empty, return members. 1097 4. Consume the first character of input_string; if it is not 1098 ",", fail parsing. 1100 5. Discard any leading SP characters from input_string. 1102 6. If input_string is empty, there is a trailing comma; fail 1103 parsing. 1105 3. No structured data has been found; return members (which is 1106 empty). 1108 4.2.1.1. Parsing an Item or Inner List 1110 Given an ASCII string as input_string, return the tuple 1111 (item_or_inner_list, parameters), where item_or_inner_list can be 1112 either a single bare item, or an array of (bare_item, parameters) 1113 tuples. input_string is modified to remove the parsed value. 1115 1. If the first character of input_string is "(", return the result 1116 of running Parsing an Inner List (Section 4.2.1.2) with 1117 input_string. 1119 2. Return the result of running Parsing an Item (Section 4.2.3) with 1120 input_string. 1122 4.2.1.2. Parsing an Inner List 1124 Given an ASCII string as input_string, return the tuple (inner_list, 1125 parameters), where inner_list is an array of (bare_item, parameters) 1126 tuples. input_string is modified to remove the parsed value. 1128 1. Consume the first character of input_string; if it is not "(", 1129 fail parsing. 1131 2. Let inner_list be an empty array. 1133 3. While input_string is not empty: 1135 1. Discard any leading SP characters from input_string. 1137 2. If the first character of input_string is ")": 1139 1. Consume the first character of input_string. 1141 2. Let parameters be the result of running Parsing 1142 Parameters (Section 4.2.3.2) with input_string. 1144 3. Return the tuple (inner_list, parameters). 1146 3. Let item be the result of running Parsing an Item 1147 (Section 4.2.3) with input_string. 1149 4. Append item to inner_list. 1151 5. If the first character of input_string is not SP or ")", fail 1152 parsing. 1154 4. The end of the inner list was not found; fail parsing. 1156 4.2.2. Parsing a Dictionary 1158 Given an ASCII string as input_string, return an ordered map whose 1159 values are (item_or_inner_list, parameters) tuples. input_string is 1160 modified to remove the parsed value. 1162 1. Let dictionary be an empty, ordered map. 1164 2. While input_string is not empty: 1166 1. Let this_key be the result of running Parsing a Key 1167 (Section 4.2.3.3) with input_string. 1169 2. If the first character of input_string is "=": 1171 1. Consume the first character of input_string. 1173 2. Let member be the result of running Parsing an Item or 1174 Inner List (Section 4.2.1.1) with input_string. 1176 3. Otherwise: 1178 1. Let value be Boolean true. 1180 2. Let parameters be the result of running Parsing 1181 Parameters Section 4.2.3.2 with input_string. 1183 3. Let member be the tuple (value, parameters). 1185 4. Add name this_key with value member to dictionary. If 1186 dictionary already contains a name this_key (comparing 1187 character-for-character), overwrite its value. 1189 5. Discard any leading SP characters from input_string. 1191 6. If input_string is empty, return dictionary. 1193 7. Consume the first character of input_string; if it is not 1194 ",", fail parsing. 1196 8. Discard any leading SP characters from input_string. 1198 9. If input_string is empty, there is a trailing comma; fail 1199 parsing. 1201 3. No structured data has been found; return dictionary (which is 1202 empty). 1204 4.2.3. Parsing an Item 1206 Given an ASCII string as input_string, return a (bare_item, 1207 parameters) tuple. input_string is modified to remove the parsed 1208 value. 1210 1. Let bare_item be the result of running Parsing a Bare Item 1211 (Section 4.2.3.1) with input_string. 1213 2. Let parameters be the result of running Parsing Parameters 1214 (Section 4.2.3.2) with input_string. 1216 3. Return the tuple (bare_item, parameters). 1218 4.2.3.1. Parsing a Bare Item 1220 Given an ASCII string as input_string, return a bare Item. 1221 input_string is modified to remove the parsed value. 1223 1. If the first character of input_string is a "-" or a DIGIT, 1224 return the result of running Parsing an Integer or Decimal 1225 (Section 4.2.4) with input_string. 1227 2. If the first character of input_string is a DQUOTE, return the 1228 result of running Parsing a String (Section 4.2.5) with 1229 input_string. 1231 3. If the first character of input_string is ":", return the result 1232 of running Parsing a Byte Sequence (Section 4.2.7) with 1233 input_string. 1235 4. If the first character of input_string is "?", return the result 1236 of running Parsing a Boolean (Section 4.2.8) with input_string. 1238 5. If the first character of input_string is an ALPHA or "*", return 1239 the result of running Parsing a Token (Section 4.2.6) with 1240 input_string. 1242 6. Otherwise, the item type is unrecognized; fail parsing. 1244 4.2.3.2. Parsing Parameters 1246 Given an ASCII string as input_string, return an ordered map whose 1247 values are bare Items. input_string is modified to remove the parsed 1248 value. 1250 1. Let parameters be an empty, ordered map. 1252 2. While input_string is not empty: 1254 1. If the first character of input_string is not ";", exit the 1255 loop. 1257 2. Consume a ";" character from the beginning of input_string. 1259 3. Discard any leading SP characters from input_string. 1261 4. let param_name be the result of running Parsing a Key 1262 (Section 4.2.3.3) with input_string. 1264 5. Let param_value be Boolean true. 1266 6. If the first character of input_string is "=": 1268 1. Consume the "=" character at the beginning of 1269 input_string. 1271 2. Let param_value be the result of running Parsing a Bare 1272 Item (Section 4.2.3.1) with input_string. 1274 7. Append key param_name with value param_value to parameters. 1275 If parameters already contains a name param_name (comparing 1276 character-for-character), overwrite its value. 1278 3. Return parameters. 1280 4.2.3.3. Parsing a Key 1282 Given an ASCII string as input_string, return a key. input_string is 1283 modified to remove the parsed value. 1285 1. If the first character of input_string is not lcalpha or "*", 1286 fail parsing. 1288 2. Let output_string be an empty string. 1290 3. While input_string is not empty: 1292 1. If the first character of input_string is not one of lcalpha, 1293 DIGIT, "_", "-", ".", or "*", return output_string. 1295 2. Let char be the result of consuming the first character of 1296 input_string. 1298 3. Append char to output_string. 1300 4. Return output_string. 1302 4.2.4. Parsing an Integer or Decimal 1304 Given an ASCII string as input_string, return an Integer or Decimal. 1305 input_string is modified to remove the parsed value. 1307 NOTE: This algorithm parses both Integers (Section 3.3.1) and 1308 Decimals (Section 3.3.2), and returns the corresponding structure. 1310 1. Let type be "integer". 1312 2. Let sign be 1. 1314 3. Let input_number be an empty string. 1316 4. If the first character of input_string is "-", consume it and 1317 set sign to -1. 1319 5. If input_string is empty, there is an empty integer; fail 1320 parsing. 1322 6. If the first character of input_string is not a DIGIT, fail 1323 parsing. 1325 7. While input_string is not empty: 1327 1. Let char be the result of consuming the first character of 1328 input_string. 1330 2. If char is a DIGIT, append it to input_number. 1332 3. Else, if type is "integer" and char is ".": 1334 1. If input_number contains more than 12 characters, fail 1335 parsing. 1337 2. Otherwise, append char to input_number and set type to 1338 "decimal". 1340 4. Otherwise, prepend char to input_string, and exit the loop. 1342 5. If type is "integer" and input_number contains more than 15 1343 characters, fail parsing. 1345 6. If type is "decimal" and input_number contains more than 16 1346 characters, fail parsing. 1348 8. If type is "integer": 1350 1. Parse input_number as an integer and let output_number be 1351 the product of the result and sign. 1353 2. If output_number is outside the range -999,999,999,999,999 1354 to 999,999,999,999,999 inclusive, fail parsing. 1356 9. Otherwise: 1358 1. If the final character of input_number is ".", fail parsing. 1360 2. If the number of characters after "." in input_number is 1361 greater than three, fail parsing. 1363 3. Parse input_number as a decimal number and let output_number 1364 be the product of the result and sign. 1366 10. Return output_number. 1368 4.2.5. Parsing a String 1370 Given an ASCII string as input_string, return an unquoted String. 1371 input_string is modified to remove the parsed value. 1373 1. Let output_string be an empty string. 1375 2. If the first character of input_string is not DQUOTE, fail 1376 parsing. 1378 3. Discard the first character of input_string. 1380 4. While input_string is not empty: 1382 1. Let char be the result of consuming the first character of 1383 input_string. 1385 2. If char is a backslash ("\"): 1387 1. If input_string is now empty, fail parsing. 1389 2. Let next_char be the result of consuming the first 1390 character of input_string. 1392 3. If next_char is not DQUOTE or "\", fail parsing. 1394 4. Append next_char to output_string. 1396 3. Else, if char is DQUOTE, return output_string. 1398 4. Else, if char is in the range %x00-1f or %x7f (i.e., is not 1399 in VCHAR or SP), fail parsing. 1401 5. Else, append char to output_string. 1403 5. Reached the end of input_string without finding a closing DQUOTE; 1404 fail parsing. 1406 4.2.6. Parsing a Token 1408 Given an ASCII string as input_string, return a Token. input_string 1409 is modified to remove the parsed value. 1411 1. If the first character of input_string is not ALPHA or "*", fail 1412 parsing. 1414 2. Let output_string be an empty string. 1416 3. While input_string is not empty: 1418 1. If the first character of input_string is not in tchar, ":" 1419 or "/", return output_string. 1421 2. Let char be the result of consuming the first character of 1422 input_string. 1424 3. Append char to output_string. 1426 4. Return output_string. 1428 4.2.7. Parsing a Byte Sequence 1430 Given an ASCII string as input_string, return a Byte Sequence. 1431 input_string is modified to remove the parsed value. 1433 1. If the first character of input_string is not ":", fail parsing. 1435 2. Discard the first character of input_string. 1437 3. If there is not a ":" character before the end of input_string, 1438 fail parsing. 1440 4. Let b64_content be the result of consuming content of 1441 input_string up to but not including the first instance of the 1442 character ":". 1444 5. Consume the ":" character at the beginning of input_string. 1446 6. If b64_content contains a character not included in ALPHA, DIGIT, 1447 "+", "/" and "=", fail parsing. 1449 7. Let binary_content be the result of Base 64 Decoding [RFC4648] 1450 b64_content, synthesizing padding if necessary (note the 1451 requirements about recipient behavior below). 1453 8. Return binary_content. 1455 Because some implementations of base64 do not allow reject of encoded 1456 data that is not properly "=" padded (see [RFC4648], Section 3.2), 1457 parsers SHOULD NOT fail when it is not present, unless they cannot be 1458 configured to do so. 1460 Because some implementations of base64 do not allow rejection of 1461 encoded data that has non-zero pad bits (see [RFC4648], Section 3.5), 1462 parsers SHOULD NOT fail when it is present, unless they cannot be 1463 configured to do so. 1465 This specification does not relax the requirements in [RFC4648], 1466 Section 3.1 and 3.3; therefore, parsers MUST fail on characters 1467 outside the base64 alphabet, and on line feeds in encoded data. 1469 4.2.8. Parsing a Boolean 1471 Given an ASCII string as input_string, return a Boolean. input_string 1472 is modified to remove the parsed value. 1474 1. If the first character of input_string is not "?", fail parsing. 1476 2. Discard the first character of input_string. 1478 3. If the first character of input_string matches "1", discard the 1479 first character, and return true. 1481 4. If the first character of input_string matches "0", discard the 1482 first character, and return false. 1484 5. No value has matched; fail parsing. 1486 5. IANA Considerations 1488 This document has no actions for IANA. 1490 6. Security Considerations 1492 The size of most types defined by Structured Fields is not limited; 1493 as a result, extremely large fields could be an attack vector (e.g., 1494 for resource consumption). Most HTTP implementations limit the sizes 1495 of individual fields as well as the overall header or trailer section 1496 size to mitigate such attacks. 1498 It is possible for parties with the ability to inject new HTTP fields 1499 to change the meaning of a Structured Field. In some circumstances, 1500 this will cause parsing to fail, but it is not possible to reliably 1501 fail in all such circumstances. 1503 7. References 1505 7.1. Normative References 1507 [RFC0020] Cerf, V., "ASCII format for network interchange", STD 80, 1508 RFC 20, DOI 10.17487/RFC0020, October 1969, 1509 . 1511 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1512 Requirement Levels", BCP 14, RFC 2119, 1513 DOI 10.17487/RFC2119, March 1997, 1514 . 1516 [RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data 1517 Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006, 1518 . 1520 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 1521 Specifications: ABNF", STD 68, RFC 5234, 1522 DOI 10.17487/RFC5234, January 2008, 1523 . 1525 [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 1526 Protocol (HTTP/1.1): Message Syntax and Routing", 1527 RFC 7230, DOI 10.17487/RFC7230, June 2014, 1528 . 1530 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1531 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1532 May 2017, . 1534 7.2. Informative References 1536 [IEEE754] IEEE, "IEEE Standard for Floating-Point Arithmetic", 1537 IEEE 754-2019, DOI 10.1109/IEEESTD.2019.8766229, 1538 ISBN 978-1-5044-5924-2, July 2019, 1539 . 1541 [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer 1542 Protocol (HTTP/1.1): Semantics and Content", RFC 7231, 1543 DOI 10.17487/RFC7231, June 2014, 1544 . 1546 [RFC7493] Bray, T., Ed., "The I-JSON Message Format", RFC 7493, 1547 DOI 10.17487/RFC7493, March 2015, 1548 . 1550 [RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext 1551 Transfer Protocol Version 2 (HTTP/2)", RFC 7540, 1552 DOI 10.17487/RFC7540, May 2015, 1553 . 1555 [RFC7541] Peon, R. and H. Ruellan, "HPACK: Header Compression for 1556 HTTP/2", RFC 7541, DOI 10.17487/RFC7541, May 2015, 1557 . 1559 [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data 1560 Interchange Format", STD 90, RFC 8259, 1561 DOI 10.17487/RFC8259, December 2017, 1562 . 1564 [UTF-8] Yergeau, F., "UTF-8, a transformation format of ISO 1565 10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, November 1566 2003, . 1568 7.3. URIs 1570 [1] https://lists.w3.org/Archives/Public/ietf-http-wg/ 1572 [2] https://httpwg.github.io/ 1574 [3] https://github.com/httpwg/http-extensions/labels/header-structure 1576 [4] https://github.com/httpwg/structured-header-tests 1578 [5] https://github.com/httpwg/wiki/wiki/Structured-Headers 1580 [6] https://github.com/httpwg/structured-header-tests 1582 Appendix A. Frequently Asked Questions 1584 A.1. Why not JSON? 1586 Earlier proposals for Structured Fields were based upon JSON 1587 [RFC8259]. However, constraining its use to make it suitable for 1588 HTTP header fields required senders and recipients to implement 1589 specific additional handling. 1591 For example, JSON has specification issues around large numbers and 1592 objects with duplicate members. Although advice for avoiding these 1593 issues is available (e.g., [RFC7493]), it cannot be relied upon. 1595 Likewise, JSON strings are by default Unicode strings, which have a 1596 number of potential interoperability issues (e.g., in comparison). 1597 Although implementers can be advised to avoid non-ASCII content where 1598 unnecessary, this is difficult to enforce. 1600 Another example is JSON's ability to nest content to arbitrary 1601 depths. Since the resulting memory commitment might be unsuitable 1602 (e.g., in embedded and other limited server deployments), it's 1603 necessary to limit it in some fashion; however, existing JSON 1604 implementations have no such limits, and even if a limit is 1605 specified, it's likely that some field definition will find a need to 1606 violate it. 1608 Because of JSON's broad adoption and implementation, it is difficult 1609 to impose such additional constraints across all implementations; 1610 some deployments would fail to enforce them, thereby harming 1611 interoperability. In short, if it looks like JSON, people will be 1612 tempted to use a JSON parser / serializer on field values. 1614 Since a major goal for Structured Fields is to improve 1615 interoperability and simplify implementation, these concerns led to a 1616 format that requires a dedicated parser and serializer. 1618 Additionally, there were widely shared feelings that JSON doesn't 1619 "look right" in HTTP fields. 1621 Appendix B. Implementation Notes 1623 A generic implementation of this specification should expose the top- 1624 level serialize (Section 4.1) and parse (Section 4.2) functions. 1625 They need not be functions; for example, it could be implemented as 1626 an object, with methods for each of the different top-level types. 1628 For interoperability, it's important that generic implementations be 1629 complete and follow the algorithms closely; see Section 1.1. To aid 1630 this, a common test suite is being maintained by the community at 1631 https://github.com/httpwg/structured-header-tests [6]. 1633 Implementers should note that Dictionaries and Parameters are order- 1634 preserving maps. Some fields may not convey meaning in the ordering 1635 of these data types, but it should still be exposed so that 1636 applications which need to use it will have it available. 1638 Likewise, implementations should note that it's important to preserve 1639 the distinction between Tokens and Strings. While most programming 1640 languages have native types that map to the other types well, it may 1641 be necessary to create a wrapper "token" object or use a parameter on 1642 functions to assure that these types remain separate. 1644 The serialization algorithm is defined in a way that it is not 1645 strictly limited to the data types defined in Section 3 in every 1646 case. For example, Decimals are designed to take broader input and 1647 round to allowed values. 1649 Appendix C. Changes 1651 _RFC Editor: Please remove this section before publication._ 1653 C.1. Since draft-ietf-httpbis-header-structure-17 1655 o Editorial improvements. 1657 C.2. Since draft-ietf-httpbis-header-structure-16 1659 o Editorial improvements. 1661 o Discussion on forwards compatibility. 1663 C.3. Since draft-ietf-httpbis-header-structure-15 1665 o Editorial improvements. 1667 o Use HTTP field terminology more consistently, in line with recent 1668 changes to HTTP-core. 1670 o String length requirements apply to decoded strings (#1051). 1672 o Correctly round decimals in serialisation (#1043). 1674 o Clarify input to serialisation algorithms (#1055). 1676 o Omitted True dictionary value can have parameters (#1083). 1678 o Keys can now start with '*' (#1068). 1680 C.4. Since draft-ietf-httpbis-header-structure-14 1682 o Editorial improvements. 1684 o Allow empty dictionary values (#992). 1686 o Change value of omitted parameter value to True (#995). 1688 o Explain more about splitting dictionaries and lists across header 1689 instances (#997). 1691 o Disallow HTAB, replace OWS with spaces (#998). 1693 o Change byte sequence delimiters from "*" to ":" (#991). 1695 o Allow tokens to start with "*" (#991). 1697 o Change Floats to fixed-precision Decimals (#982). 1699 o Round the fractional component of decimal, rather than truncating 1700 it (#982). 1702 o Handle duplicate dictionary and parameter keys by overwriting 1703 their values, rather than failing (#997). 1705 o Allow "." in key (#1027). 1707 o Check first character of key in serialisation (#1037). 1709 o Talk about greasing headers (#1015). 1711 C.5. Since draft-ietf-httpbis-header-structure-13 1713 o Editorial improvements. 1715 o Define "structured header name" and "structured header value" 1716 terms (#908). 1718 o Corrected text about valid characters in strings (#931). 1720 o Removed most instances of the word "textual", as it was redundant 1721 (#915). 1723 o Allowed parameters on Items and Inner Lists (#907). 1725 o Expand the range of characters in token (#961). 1727 o Disallow OWS before ";" delimiter in parameters (#961). 1729 C.6. Since draft-ietf-httpbis-header-structure-12 1731 o Editorial improvements. 1733 o Reworked float serialisation (#896). 1735 o Don't add a trailing space in inner-list (#904). 1737 C.7. Since draft-ietf-httpbis-header-structure-11 1739 o Allow * in key (#844). 1741 o Constrain floats to six digits of precision (#848). 1743 o Allow dictionary members to have parameters (#842). 1745 C.8. Since draft-ietf-httpbis-header-structure-10 1747 o Update abstract (#799). 1749 o Input and output are now arrays of bytes (#662). 1751 o Implementations need to preserve difference between token and 1752 string (#790). 1754 o Allow empty dictionaries and lists (#781). 1756 o Change parameterized lists to have primary items (#797). 1758 o Allow inner lists in both dictionaries and lists; removes lists of 1759 lists (#816). 1761 o Subsume Parameterised Lists into Lists (#839). 1763 C.9. Since draft-ietf-httpbis-header-structure-09 1765 o Changed Boolean from T/F to 1/0 (#784). 1767 o Parameters are now ordered maps (#765). 1769 o Clamp integers to 15 digits (#737). 1771 C.10. Since draft-ietf-httpbis-header-structure-08 1773 o Disallow whitespace before items properly (#703). 1775 o Created "key" for use in dictionaries and parameters, rather than 1776 relying on identifier (#702). Identifiers have a separate minimum 1777 supported size. 1779 o Expanded the range of special characters allowed in identifier to 1780 include all of ALPHA, ".", ":", and "%" (#702). 1782 o Use "?" instead of "!" to indicate a Boolean (#719). 1784 o Added "Intentionally Strict Processing" (#684). 1786 o Gave better names for referring specs to use in Parameterised 1787 Lists (#720). 1789 o Added Lists of Lists (#721). 1791 o Rename Identifier to Token (#725). 1793 o Add implementation guidance (#727). 1795 C.11. Since draft-ietf-httpbis-header-structure-07 1797 o Make Dictionaries ordered mappings (#659). 1799 o Changed "binary content" to "byte sequence" to align with Infra 1800 specification (#671). 1802 o Changed "mapping" to "map" for #671. 1804 o Don't fail if byte sequences aren't "=" padded (#658). 1806 o Add Booleans (#683). 1808 o Allow identifiers in items again (#629). 1810 o Disallowed whitespace before items (#703). 1812 o Explain the consequences of splitting a string across multiple 1813 headers (#686). 1815 C.12. Since draft-ietf-httpbis-header-structure-06 1817 o Add a FAQ. 1819 o Allow non-zero pad bits. 1821 o Explicitly check for integers that violate constraints. 1823 C.13. Since draft-ietf-httpbis-header-structure-05 1825 o Reorganise specification to separate parsing out. 1827 o Allow referencing specs to use ABNF. 1829 o Define serialisation algorithms. 1831 o Refine relationship between ABNF, parsing and serialisation 1832 algorithms. 1834 C.14. Since draft-ietf-httpbis-header-structure-04 1836 o Remove identifiers from item. 1838 o Remove most limits on sizes. 1840 o Refine number parsing. 1842 C.15. Since draft-ietf-httpbis-header-structure-03 1844 o Strengthen language around failure handling. 1846 C.16. Since draft-ietf-httpbis-header-structure-02 1848 o Split Numbers into Integers and Floats. 1850 o Define number parsing. 1852 o Tighten up binary parsing and give it an explicit end delimiter. 1854 o Clarify that mappings are unordered. 1856 o Allow zero-length strings. 1858 o Improve string parsing algorithm. 1860 o Improve limits in algorithms. 1862 o Require parsers to combine header fields before processing. 1864 o Throw an error on trailing garbage. 1866 C.17. Since draft-ietf-httpbis-header-structure-01 1868 o Replaced with draft-nottingham-structured-headers. 1870 C.18. Since draft-ietf-httpbis-header-structure-00 1872 o Added signed 64bit integer type. 1874 o Drop UTF8, and settle on BCP137 ::EmbeddedUnicodeChar for h1- 1875 unicode-string. 1877 o Change h1_blob delimiter to ":" since "'" is valid t_char 1879 Acknowledgements 1881 Many thanks to Matthew Kerwin for his detailed feedback and careful 1882 consideration during the development of this specification. 1884 Thanks also to Ian Clelland, Roy Fielding, Anne van Kesteren, Kazuho 1885 Oku, Evert Pot, Julian Reschke, Martin Thomson, Mike West, and 1886 Jeffrey Yasskin for their contributions. 1888 Authors' Addresses 1890 Mark Nottingham 1891 Fastly 1893 Email: mnot@mnot.net 1894 URI: https://www.mnot.net/ 1896 Poul-Henning Kamp 1897 The Varnish Cache Project 1899 Email: phk@varnish-cache.org