idnits 2.17.00 (12 Aug 2021) /tmp/idnits15174/draft-goessner-dispatch-jsonpath-00.txt: -(2): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding -(5): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == There are 7 instances of lines with non-ascii characters in the document. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack a Security Considerations section. == There are 1 instance of lines with non-RFC2606-compliant FQDNs in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (12 July 2020) is 671 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '0' on line 563 -- Looks like a reference, but probably isn't: '3' on line 566 -- Looks like a reference, but probably isn't: '2' on line 565 -- Looks like a reference, but probably isn't: '1' on line 564 Summary: 1 error (**), 0 flaws (~~), 4 warnings (==), 6 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group S. Gössner 3 Internet-Draft Fachhochschule Dortmund 4 Intended status: Standards Track C. Bormann, Ed. 5 Expires: 13 January 2021 Universität Bremen TZI 6 12 July 2020 8 JSONPath -- XPath for JSON 9 draft-goessner-dispatch-jsonpath-00 11 Abstract 13 insert abstract here 15 Contributing 17 This document picks up the popular JSONPath specification dated 18 2007-02-21 and provides a more normative definition for it. It is 19 intended as a submission to the IETF DISPATCH WG, in order to find 20 the right way to complete standardization of this specification. In 21 its current state, it is a strawman document showing what needs to be 22 covered. 24 Comments and issues can be directed at the github repository _insert 25 repo here_ as well as (for the time when the more permanent home is 26 being decided) at the dispatch@ietf.org mailing list. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at https://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on 13 January 2021. 45 Copyright Notice 47 Copyright (c) 2020 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 52 license-info) in effect on the date of publication of this document. 53 Please review these documents carefully, as they describe your rights 54 and restrictions with respect to this document. Code Components 55 extracted from this document must include Simplified BSD License text 56 as described in Section 4.e of the Trust Legal Provisions and are 57 provided without warranty as described in the Simplified BSD License. 59 Table of Contents 61 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 62 1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 63 1.2. Inspired by XPath . . . . . . . . . . . . . . . . . . . . 3 64 1.3. Overview of JSONPath Expressions . . . . . . . . . . . . 4 65 2. JSONPath Examples . . . . . . . . . . . . . . . . . . . . . . 7 66 3. Detailed definition . . . . . . . . . . . . . . . . . . . . . 9 67 4. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 10 68 5. IANA considerations . . . . . . . . . . . . . . . . . . . . . 10 69 6. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 70 6.1. Normative References . . . . . . . . . . . . . . . . . . 10 71 6.2. Informative References . . . . . . . . . . . . . . . . . 11 72 Appendix A. Early JSONPath implementations . . . . . . . . . . . 11 73 A.1. Implementation . . . . . . . . . . . . . . . . . . . . . 11 74 A.2. Usage . . . . . . . . . . . . . . . . . . . . . . . . . . 12 75 A.3. Parameters . . . . . . . . . . . . . . . . . . . . . . . 12 76 A.4. Return value . . . . . . . . . . . . . . . . . . . . . . 12 77 A.5. JavaScript Example . . . . . . . . . . . . . . . . . . . 12 78 A.6. PHP example . . . . . . . . . . . . . . . . . . . . . . . 12 79 A.7. Results . . . . . . . . . . . . . . . . . . . . . . . . . 13 80 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 13 81 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13 83 1. Introduction 85 This document picks up the popular JSONPath specification dated 86 2007-02-21 [JSONPath-orig] and provides a more normative definition 87 for it. It is intended as a submission to the IETF DISPATCH WG, in 88 order to find the right way to complete standardization of this 89 specification. In its current state, it is a strawman document 90 showing what needs to be covered. 92 1.1. Terminology 94 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 95 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 96 "OPTIONAL" in this document are to be interpreted as described in 97 BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all 98 capitals, as shown here. 100 The grammatical rules in this document are to be interpreted as 101 described in [RFC5234]. 103 The terminology of [RFC8259] applies. 105 Data Item: A structure complying to the generic data model of JSON, 106 i.e., composed of containers such as arrays and maps (JSON 107 objects), and of atomic data such as null, true, false, numbers, 108 and text strings. 110 Object: Used in its generic sense, e.g., for programming language 111 objects. When a JSON Object as defined in [RFC8259] is meant, we 112 specifically say JSON Object. 114 Query: Short name for JSONPath expression. 116 Argument: Short name for the JSON data item a JSONPath expression is 117 applied to. 119 Output Path: A simple form of JSONPath expression that identifies a 120 Position by providing a query that results in exactly that 121 position. Similar to, but syntactically different from, a JSON 122 Pointer [RFC6901]. 124 Position: A JSON data item identical to or nested within the JSON 125 data item to which the query is applied to, expressed either by 126 the value of that data item or by providing a JSONPath Output 127 Path. 129 1.2. Inspired by XPath 131 A frequently emphasized advantage of XML is the availability of 132 plenty tools to analyse, transform and selectively extract data out 133 of XML documents. [XPath] is one of these powerful tools. 135 In 2007, the need for something solving the same class of problems 136 for the emerging JSON community became apparent, specifically for: 138 * Finding data interactively and extracting them out of [RFC8259] 139 data items without special scripting. 141 * Specifying the relevant parts of the JSON data in a request by a 142 client, so the server can reduce the data in the server response, 143 minimizing bandwidth usage. 145 So how does such a tool look like when done for JSON? When defining 146 a JSONPath, how should expressions look like? 148 The XPath expression 150 /store/book[1]/title 152 looks like 154 x.store.book[0].title 156 or 158 x['store']['book'][0]['title'] 160 in popular programming languages such as JavaScript, Python and PHP, 161 with a variable x holding the JSON data item. Here we observe that 162 such languages already have a fundamentally XPath-like feature built 163 in. 165 The JSONPath tool in question should: 167 * be naturally based on those language characteristics. 169 * cover only essential parts of XPath 1.0. 171 * be lightweight in code size and memory consumption. 173 * be runtime efficient. 175 1.3. Overview of JSONPath Expressions 177 JSONPath expressions always apply to a JSON data item in the same way 178 as XPath expressions are used in combination with an XML document. 179 Since a JSON data item is usually anonymous and doesn't necessarily 180 have a "root member object", JSONPath used the abstract name "$" to 181 refer to the top level object of the data item. 183 JSONPath expressions can use the _dot-notation_ 185 $.store.book[0].title 187 or the _bracket-notation_ 188 $['store']['book'][0]['title'] 190 for paths input to a JSONPath processor. Where a JSONPath processor 191 uses JSONPath expressions for internal purposes or as output paths, 192 these will always be converted to the more general _bracket- 193 notation_. 195 JSONPath allows the wildcard symbol "*" for member names and array 196 indices. It borrows the descendant operator ".." from [E4X] and the 197 array slice syntax proposal "[start:end:step]" [SLICE] from 198 ECMASCRIPT 4. 200 JSONPath can employ an _underlying scripting language_, expressions 201 of which, written in parentheses: "()", can be used as an 202 alternative to explicit names or indices as in: 204 $.store.book[(@.length-1)].title 206 The symbol "@" is used for the current object. Filter expressions 207 are supported via the syntax "?()" as in 209 $.store.book[?(@.price < 10)].title 211 Here is a complete overview and a side by side comparison of the 212 JSONPath syntax elements with its XPath counterparts. 214 +=======+==================+=======================================+ 215 | XPath | JSONPath | Description | 216 +=======+==================+=======================================+ 217 | / | $ | the root object/element | 218 +-------+------------------+---------------------------------------+ 219 | . | @ | the current object/element | 220 +-------+------------------+---------------------------------------+ 221 | / | "." or "[]" | child operator | 222 +-------+------------------+---------------------------------------+ 223 | .. | n/a | parent operator | 224 +-------+------------------+---------------------------------------+ 225 | // | .. | nested descendants. JSONPath borrows | 226 | | | this syntax from E4X. | 227 +-------+------------------+---------------------------------------+ 228 | * | * | wildcard. All objects/elements | 229 | | | regardless of their names. | 230 +-------+------------------+---------------------------------------+ 231 | @ | n/a | attribute access. JSON data items | 232 | | | don't have attributes. | 233 +-------+------------------+---------------------------------------+ 234 | [] | [] | subscript operator. XPath uses it to | 235 | | | iterate over element collections and | 236 | | | for predicates. In JavaScript and | 237 | | | JSON it is the native array operator. | 238 +-------+------------------+---------------------------------------+ 239 | | | [,] | Union operator in XPath results in a | 240 | | | combination of node sets. JSONPath | 241 | | | allows alternate names or array | 242 | | | indices as a set. | 243 +-------+------------------+---------------------------------------+ 244 | n/a | [start:end:step] | array slice operator borrowed from | 245 | | | ES4. | 246 +-------+------------------+---------------------------------------+ 247 | [] | ?() | applies a filter (script) expression. | 248 +-------+------------------+---------------------------------------+ 249 | n/a | () | script expression, using the | 250 | | | underlying script engine. | 251 +-------+------------------+---------------------------------------+ 252 | () | n/a | grouping in Xpath | 253 +-------+------------------+---------------------------------------+ 255 Table 1: Overview over JSONPath, comparing to XPath 257 XPath has a lot more to offer (location paths in unabbreviated 258 syntax, operators and functions) than listed here. Moreover there is 259 a significant difference how the subscript operator works in Xpath 260 and JSONPath: 262 * Square brackets in XPath expressions always operate on the _node 263 set_ resulting from the previous path fragment. Indices always 264 start at 1. 266 * With JSONPath, square brackets operate on the _object_ or _array_ 267 addressed by the previous path fragment. Indices always start at 268 0. 270 2. JSONPath Examples 272 This section provides some more examples for JSONPath expressions. 273 The examples are based on a simple JSON data item patterned after a 274 typical XML example representing a bookstore (that also has 275 bicycles): 277 { "store": { 278 "book": [ 279 { "category": "reference", 280 "author": "Nigel Rees", 281 "title": "Sayings of the Century", 282 "price": 8.95 283 }, 284 { "category": "fiction", 285 "author": "Evelyn Waugh", 286 "title": "Sword of Honour", 287 "price": 12.99 288 }, 289 { "category": "fiction", 290 "author": "Herman Melville", 291 "title": "Moby Dick", 292 "isbn": "0-553-21311-3", 293 "price": 8.99 294 }, 295 { "category": "fiction", 296 "author": "J. R. R. Tolkien", 297 "title": "The Lord of the Rings", 298 "isbn": "0-395-19395-8", 299 "price": 22.99 300 } 301 ], 302 "bicycle": { 303 "color": "red", 304 "price": 19.95 305 } 306 } 307 } 309 Figure 1: Example JSON data item 311 The examples in Table 2 presume an underlying script language that 312 allows obtaining the number of items in an array, testing for the 313 presence of a map member, and performing numeric comparisons of map 314 member values with a constant. 316 +======================+=========================+==================+ 317 | XPath | JSONPath | Result | 318 +======================+=========================+==================+ 319 | /store/book/author | $.store.book[*].author | the authors of | 320 | | | all books in | 321 | | | the store | 322 +----------------------+-------------------------+------------------+ 323 | //author | $..author | all authors | 324 +----------------------+-------------------------+------------------+ 325 | /store/* | $.store.* | all things in | 326 | | | store, which | 327 | | | are some books | 328 | | | and a red | 329 | | | bicycle. | 330 +----------------------+-------------------------+------------------+ 331 | /store//price | $.store..price | the price of | 332 | | | everything in | 333 | | | the store. | 334 +----------------------+-------------------------+------------------+ 335 | //book[3] | $..book[2] | the third book | 336 +----------------------+-------------------------+------------------+ 337 | //book[last()] | "$..book[(@.length-1)]" | the last book | 338 | | "$..book[-1:]" | in order. | 339 +----------------------+-------------------------+------------------+ 340 | //book[position()<3] | "$..book[0,1]" | the first two | 341 | | "$..book[:2]" | books | 342 +----------------------+-------------------------+------------------+ 343 | //book[isbn] | $..book[?(@.isbn)] | filter all | 344 | | | books with isbn | 345 | | | number | 346 +----------------------+-------------------------+------------------+ 347 | //book[price<10] | $..book[?(@.price<10)] | filter all | 348 | | | books cheapier | 349 | | | than 10 | 350 +----------------------+-------------------------+------------------+ 351 | //* | $..* | all Elements in | 352 | | | XML document. | 353 | | | All members of | 354 | | | JSON data item. | 355 +----------------------+-------------------------+------------------+ 357 Table 2: Example JSONPath expressions applied to the example JSON 358 data item 360 3. Detailed definition 362 [TBD: This section needs to be fleshed out in detail. The text given 363 here is intended to give the flavor of that detail, not to be the 364 actual definition that is to be defined.] 366 JSONPath expressions, "queries" for short in this specification, are 367 character strings, represented in UTF-8 unless otherwise required by 368 the context in which they are used. 370 When applied to a JSON data item, a query returns a (possibly empty) 371 list of "positions" in the data item that match the JSONPath 372 expression. 374 JSONPath = root *(step) 375 root = "$" 377 step = ".." name ; nested descendants 378 / "." name ; child (dot notation) 379 / "[" value-expression *("," value-expression) "]" 380 ; child[ren] (bracket notation) 381 / "[" value-expression *2(":" value-expression) "]" ; (slice) 382 value-expression = *DIGIT / name 383 / script-expression / filter-expression 384 name = "'" text "'" 385 / "*" ; wildcard 386 script-expression = "(" script ")" 387 filter-expression = "?(" script ")" 388 script = 389 text = 390 DIGIT = %x30-39 392 Figure 2: ABNF definition for JSONPath 394 Within a script, @ stands for the position under consideration. 395 [TBD: define underlying scripting language, if there is to be a 396 standard one] 398 [TBD: define handling of spaces] 399 A JSONPath starts at the root of the argument; the "current list" of 400 positions is initialized to that root. Each step applies the 401 semantics of that step to each of the positions in the "current 402 list", returning another list; the "current list" is replaced by the 403 concatenation of all these returned lists, and the next step begins. 404 When all steps have been performed, the "current list" is returned, 405 depending on the choices of the context either as a list of data 406 items or as a list of output paths. [TBD: define the order of that 407 list] 409 [TBD: Define all the steps] 411 [TBD: Define details of Output Path] 413 4. Discussion 415 * Currently only single quotes allowed inside of JSONPath 416 expressions. 418 * Script expressions inside of JSONPath locations are currently not 419 recursively evaluated by jsonPath. Only the global "$" and local 420 "@" symbols are expanded by a simple regular expression. 422 * An alternative for jsonPath to return false in case of no match 423 may be to return an empty array in future. [This is already done 424 in the above.] 426 5. IANA considerations 428 TBD: Define a media type for JSON Path expressions. 430 6. References 432 6.1. Normative References 434 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 435 Requirement Levels", BCP 14, RFC 2119, 436 DOI 10.17487/RFC2119, March 1997, 437 . 439 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 440 Specifications: ABNF", STD 68, RFC 5234, 441 DOI 10.17487/RFC5234, January 2008, 442 . 444 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 445 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 446 May 2017, . 448 [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data 449 Interchange Format", STD 90, RFC 8259, 450 DOI 10.17487/RFC8259, December 2017, 451 . 453 6.2. Informative References 455 [E4X] ISO, "Information technology -- ECMAScript for XML (E4X) 456 specification", ISO/IEC 22537:2006 , 2006. 458 [JSON-PHP] "JSON-PHP", January 2005, 459 . 461 [JSONPath-impl] 462 "jsonpath (Downloads)", n.d., 463 . 465 [JSONPath-orig] 466 Gössner, S., "JSONPath - XPath for JSON", 21 February 467 2007, . 469 [RFC6901] Bryan, P., Ed., Zyp, K., and M. Nottingham, Ed., 470 "JavaScript Object Notation (JSON) Pointer", RFC 6901, 471 DOI 10.17487/RFC6901, April 2013, 472 . 474 [SLICE] "Slice notation", n.d., 475 . 477 [XPath] Berglund, A., Boag, S., Chamberlin, D., Fernandez, M., 478 Kay, M., Robie, J., and J. Simeon, "XML Path Language 479 (XPath) 2.0 (Second Edition)", World Wide Web Consortium 480 Recommendation REC-xpath20-20101214, 14 December 2010, 481 . 483 Appendix A. Early JSONPath implementations 485 This appendix has been copied from the similar section in 486 [JSONPath-orig], with few changes. It is informative, intended to 487 supply more examples and give an impression for what could be a 488 typical JSONPath API. 490 A.1. Implementation 492 JSONPath is implemented in JavaScript for client-side usage and 493 ported over to PHP for use on the server. 495 A.2. Usage 497 All you need to do is downloading either of the files 499 * "jsonpath.js" [JSONPath-impl] 501 * "jsonpath.php" [JSONPath-impl] 503 include it in your program and use the simple API consisting of one 504 single function. 506 jsonPath(obj, expr [, args]) 508 A.3. Parameters 510 obj (object|array): Object representing the JSON data item. 512 expr (string): JSONPath expression string. 514 args (object|undefined): Object controlling path evaluation and 515 output. Currently only one member is supported. 517 args.resultType ("VALUE"|"PATH"): causes the result to be either 518 matching values (default) or normalized path expressions. 520 A.4. Return value 522 (array|false): Array holding either values or normalized path 523 expressions matching the input path expression, which can be used 524 for lazy evaluation. "false" in case of no match. 526 A.5. JavaScript Example 528 var o = { /*...*/ }, // the 'store' JSON object from above 529 res1 = jsonPath(o, "$..author").toJSONString(), 530 res2 = jsonPath(o, "$..author", 531 {resultType:"PATH"}).toJSONString(); 533 A.6. PHP example 535 We need here to convert the JSON string to a PHP array first. I am 536 using Michal Migurski's JSON parser [JSON-PHP] for that. 538 require_once('json.php'); // JSON parser 539 require_once('jsonpath.php'); // JSONPath evaluator 541 $json = '{ ... }'; // JSON data item from above 543 $parser = new Services_JSON(SERVICES_JSON_LOOSE_TYPE); 544 $o = $parser->decode($json); 545 $match1 = jsonPath($o, "$..author"); 546 $match2 = jsonPath($o, "$..author", 547 array("resultType" => "PATH")); 548 $res1 = $parser->encode($match1); 549 $res2 = $parser->encode($match2); 551 A.7. Results 553 Both JavaScript and PHP example result in the following JSON arrays 554 (as strings): 556 res1: 557 [ "Nigel Rees", 558 "Evelyn Waugh", 559 "Herman Melville", 560 "J. R. R. Tolkien" 561 ] 562 res2: 563 [ "$['store']['book'][0]['author']", 564 "$['store']['book'][1]['author']", 565 "$['store']['book'][2]['author']", 566 "$['store']['book'][3]['author']" 567 ] 569 Please note that the return value of jsonPath is an array, which is 570 also a valid JSON data item. So you might want to apply jsonPath to 571 the resulting data item again or use one of your favorite array 572 methods as sort with it. 574 Acknowledgements 576 This specification is based on Stefan Gössner's original online 577 article defining JSONPath [JSONPath-orig]. 579 The books example was taken from http://coli.lili.uni- 580 bielefeld.de/~andreas/Seminare/sommer02/books.xml -- a dead link now. 582 Authors' Addresses 583 Stefan Gössner 584 Fachhochschule Dortmund 585 Sonnenstraße 96 586 D-44139 Dortmund 587 Germany 589 Email: stefan.goessner@fh-dortmund.de 591 Carsten Bormann (editor) 592 Universität Bremen TZI 593 Postfach 330440 594 D-28359 Bremen 595 Germany 597 Phone: +49-421-218-63921 598 Email: cabo@tzi.org