idnits 2.17.00 (12 Aug 2021) /tmp/idnits17044/draft-melnikov-imap-partial-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 3 instances of too long lines in the document, the longest one being 7 characters in excess of 72. -- The draft header indicates that this document updates RFC4731, but the abstract doesn't seem to directly say this. It does mention RFC4731 though, so this could be OK. -- The draft header indicates that this document updates RFC5267, but the abstract doesn't seem to directly say this. It does mention RFC5267 though, so this could be OK. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year (Using the creation date from RFC4731, updated by this document, for RFC5378 checks: 2005-02-11) -- The document seems to contain a disclaimer for pre-RFC5378 work, and may have content which was first submitted before 10 November 2008. The disclaimer is necessary when there are original authors that you have been unable to contact, or if some do not wish to grant the BCP78 rights to the IETF Trust. If you are able to get all authors (current and original) to grant those rights, you can and should remove the disclaimer; otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (16 November 2021) is 185 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC7162' is defined on line 469, but no explicit reference was found in the text ** Obsolete normative reference: RFC 3501 (Obsoleted by RFC 9051) Summary: 2 errors (**), 0 flaws (~~), 2 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Melnikov 3 Internet-Draft Isode 4 Updates: 5267, 4731 (if approved) A. P. Achuthan 5 Intended status: Standards Track V. Nagulakonda 6 Expires: 20 May 2022 Yahoo! 7 L. Alves 8 Verizon 9 16 November 2021 11 IMAP Paged SEARCH & FETCH Extension 12 draft-melnikov-imap-partial-01 14 Abstract 16 The PARTIAL extension of the Internet Message Access Protocol (RFC 17 3501/RFC 9051) allows clients to limit the number of search results 18 returned, as well as to perform incremental (paged) searches. This 19 also helps servers to optimize resource usage when performing 20 searches. 22 This document extends PARTIAL SEARCH return option originally 23 specified in RFC 5267. It also clarifies some interactions between 24 RFC 5267 and RFC 4731/RFC 9051. 26 This document also describes the MESSAGELIMIT extension for 27 announcing a limit on the number of messages that can be processed in 28 a single FETCH/SEARCH/STORE/COPY/MOVE command. 30 Status of This Memo 32 This Internet-Draft is submitted in full conformance with the 33 provisions of BCP 78 and BCP 79. 35 Internet-Drafts are working documents of the Internet Engineering 36 Task Force (IETF). Note that other groups may also distribute 37 working documents as Internet-Drafts. The list of current Internet- 38 Drafts is at https://datatracker.ietf.org/drafts/current/. 40 Internet-Drafts are draft documents valid for a maximum of six months 41 and may be updated, replaced, or obsoleted by other documents at any 42 time. It is inappropriate to use Internet-Drafts as reference 43 material or to cite them other than as "work in progress." 45 This Internet-Draft will expire on 20 May 2022. 47 Copyright Notice 49 Copyright (c) 2021 IETF Trust and the persons identified as the 50 document authors. All rights reserved. 52 This document is subject to BCP 78 and the IETF Trust's Legal 53 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 54 license-info) in effect on the date of publication of this document. 55 Please review these documents carefully, as they describe your rights 56 and restrictions with respect to this document. Code Components 57 extracted from this document must include Simplified BSD License text 58 as described in Section 4.e of the Trust Legal Provisions and are 59 provided without warranty as described in the Simplified BSD License. 61 This document may contain material from IETF Documents or IETF 62 Contributions published or made publicly available before November 63 10, 2008. The person(s) controlling the copyright in some of this 64 material may not have granted the IETF Trust the right to allow 65 modifications of such material outside the IETF Standards Process. 66 Without obtaining an adequate license from the person(s) controlling 67 the copyright in such materials, this document may not be modified 68 outside the IETF Standards Process, and derivative works of it may 69 not be created outside the IETF Standards Process, except to format 70 it for publication as an RFC or to translate it into languages other 71 than English. 73 Table of Contents 75 1. Introduction and Overview . . . . . . . . . . . . . . . . . . 3 76 2. Document Conventions . . . . . . . . . . . . . . . . . . . . 3 77 3. The PARTIAL extension . . . . . . . . . . . . . . . . . . . . 3 78 3.1. Incremental SEARCH and partial results . . . . . . . . . 4 79 3.2. Interaction between PARTIAL, MIN, MAX and SAVE SEARCH 80 return options . . . . . . . . . . . . . . . . . . . . . 5 81 3.3. Extension to UID FETCH . . . . . . . . . . . . . . . . . 6 82 3.4. Use of PARTIAL and CONDSTORE IMAP extensions together . . 7 83 4. The MESSAGELIMIT extension . . . . . . . . . . . . . . . . . 7 84 4.1. Returning limits on the number of messages processed in a 85 single SEARCH/FETCH/STORE/COPY/MOVE command . . . . . . . 7 86 5. Formal syntax . . . . . . . . . . . . . . . . . . . . . . . . 8 87 6. Security Considerations . . . . . . . . . . . . . . . . . . . 9 88 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 89 7.1. Changes/additions to the IMAP4 capabilities registry . . 9 90 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 10 91 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 92 9.1. Normative References . . . . . . . . . . . . . . . . . . 10 93 9.2. Informative References . . . . . . . . . . . . . . . . . 10 94 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 95 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11 97 1. Introduction and Overview 99 This document defines an extension to the Internet Message Access 100 Protocol [RFC3501] for performing incremental searches and fetches. 101 This extension is compatible with both IMAP4rev1 [RFC3501] and 102 IMAP4rev2 [RFC9051]. 104 The PARTIAL extension of the Internet Message Access Protocol (RFC 105 3501/RFC 9051) allows clients to limit the number of search results 106 returned, as well as to perform incremental (paged) searches. This 107 also helps servers to optimize resource usage when performing 108 searches. 110 This document extends PARTIAL SEARCH return option originally 111 specified in RFC 5267. It also clarifies some interactions between 112 RFC 5267 and RFC 4731/RFC 9051. 114 This document also describes the MESSAGELIMIT extension for 115 announcing a limit on the number of messages that can be processed in 116 a single FETCH/SEARCH/STORE/COPY/MOVE command. 118 2. Document Conventions 120 In protocol examples, this document uses a prefix of "C: " to denote 121 lines sent by the client to the server, and "S: " for lines sent by 122 the server to the client. Lines prefixed with "// " are comments 123 explaining the previous protocol line. These prefixes and comments 124 are not part of the protocol. Lines without any of these prefixes 125 are continuations of the previous line, and no line break is present 126 in the protocol unless specifically mentioned. 128 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 129 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 130 "OPTIONAL" in this document are to be interpreted as described in BCP 131 14 [RFC2119] [RFC8174] when, and only when, they appear in all 132 capitals, as shown here. 134 Other capitalised words are IMAP keywords [RFC3501][RFC9051] or 135 keywords from this document. 137 3. The PARTIAL extension 139 An IMAP server advertises support for the PARTIAL extension by 140 including "PARTIAL" capability in the CAPABILITY response/response 141 code. 143 3.1. Incremental SEARCH and partial results 145 The PARTIAL search return option causes the server to provide in an 146 ESEARCH response a subset of the results denoted by the sequence 147 range given as the mandatory argument. The first result (message 148 with the lowest matching UID) is 1; thus, the first 500 results would 149 be obtained by a return option of "PARTIAL 1:500", and the second 500 150 by "PARTIAL 501:1000". This intentionally mirrors message sequence 151 numbers. 153 It is also possible to direct the server to start SEARCH from the 154 latest matching (with the highest UID) message. This can be done by 155 prepeding "-" to the index. For example -1 is the last message, -2 156 is next to the last and so on. Using this syntax helps server 157 implementations to optimize their SEARCHes. 159 A single command MUST NOT contain more than one PARTIAL or ALL search 160 return option -- that is, either one PARTIAL, one ALL, or neither 161 PARTIAL nor ALL is allowed. 163 For SEARCH results, the entire result list MUST be ordered in mailbox 164 order, that is, in UID or message sequence number order. 166 Where a PARTIAL search return option references results that do not 167 exist, by using a range which starts or ends higher (or lower) than 168 the current number of results, then the server returns the results 169 that are in the set. This yields a PARTIAL return data item that 170 has, as payload, the original range and a potentially missing set of 171 results that may be shorter than the extent of the range. If the 172 whole range references results that do not exist, a special value 173 "NIL" is returned by the server instead of the sequence set. 175 Clients need not request PARTIAL results in any particular order. 176 Because mailboxes may change, clients might wish to use PARTIAL in 177 combination with UPDATE (see [RFC5267] if the server also advertises 178 CONTEXT=SEARCH capability, especially if the intent is to walk a 179 large set of results; however, these return options do not interact 180 -- the UPDATE will provide notifications for all matching results. 182 // Let's assume that the A01 SEARCH without PARTIAL would return 183 // 23764 results. 184 C: A01 UID SEARCH RETURN (PARTIAL -1:-100) UNDELETED 185 UNKEYWORD $Junk 186 S: * ESEARCH (TAG "A01") UID PARTIAL (-1:-100 ...) 187 // 100 most recent results in set syntax elided. 188 S: A01 OK Completed. 190 // Let's assume that the A02 SEARCH without PARTIAL would return 191 // 23764 results. 192 C: A02 UID SEARCH RETURN (PARTIAL 23500:24000) UNDELETED 193 UNKEYWORD $Junk 194 C: A03 UID SEARCH RETURN (PARTIAL 1:500) UNDELETED 195 UNKEYWORD $Junk 196 C: A04 UID SEARCH RETURN (PARTIAL 24000:24500) UNDELETED 197 UNKEYWORD $Junk 198 S: * ESEARCH (TAG "A02") UID PARTIAL (23500:24000 ...) 199 // 264 results in set syntax elided, 200 // this spans the end of the results. 201 S: A02 OK Completed. 202 S: * ESEARCH (TAG "A03") UID PARTIAL (1:500 ...) 203 // 500 results in set syntax elided. 204 S: A03 OK Completed. 205 S: * ESEARCH (TAG "A04") UID PARTIAL (24000:24500 NIL) 206 // No results are present, this is beyond the end of the results. 207 S: A04 OK Completed. 209 3.2. Interaction between PARTIAL, MIN, MAX and SAVE SEARCH return 210 options 212 This section only applies if the server advertises PARTIAL IMAP 213 capability or CONTEXT=SEARCH [RFC5267], together with ESEARCH 214 [RFC4731] and/or IMAP4rev2"[RFC9051]. 216 The SAVE result option doesn't change whether the server would return 217 items corresponding to PARTIAL SEARCH result options. 219 As specified in Section 3.1, it is an error to specify both PARTIAL 220 and ALL result options in the same SEARCH command. 222 When the SAVE result option is combined with the PARTIAL result 223 option, and none of MIN/MAX/COUNT result options is present, the 224 corresponding PARTIAL is returned, and the "$" marker would contain 225 all messages returned by the PARTIAL result option. 227 When the SAVE + PARTIAL result options are combined with the MIN or 228 the MAX result option, and the COUNT result option is absent, the 229 corresponding PARTIAL result and MIN/MAX is returned (if the search 230 result is not empty), and the "$" marker would contain all messages 231 returned by the PARTIAL result option + the corresponding MIN/MAX 232 message. 234 If the SAVE + PARTIAL result options are combined with both MIN and 235 MAX result options, and the COUNT result options is absent, the 236 PARTIAL, MIN and MAX are returned (if the search result is not 237 empty), and the "$" marker would contain all messages returned by the 238 PARTIAL result option plus MIN and MAX messages. 240 If the SAVE + PARTIAL result options are combined with the COUNT 241 result option, the PARTIAL and COUNT are returned, and the "$" marker 242 would always contain all messages found by the SEARCH or UID SEARCH 243 command. 245 The following table summarizes the additional requirement on ESEARCH 246 server implementations described in this section. 248 +==============================+=====================+ 249 | Combination of Result option | "$" marker value | 250 +==============================+=====================+ 251 | SAVE PARTIAL | PARTIAL | 252 +------------------------------+---------------------+ 253 | SAVE PARTIAL MIN | PARTIAL & MIN | 254 +------------------------------+---------------------+ 255 | SAVE PARTIAL MAX | PARTIAL & MAX | 256 +------------------------------+---------------------+ 257 | SAVE PARTIAL MIN MAX | PARTIAL & MIN & MAX | 258 +------------------------------+---------------------+ 259 | SAVE PARTIAL COUNT [m] | all found messages | 260 +------------------------------+---------------------+ 262 Table 1 264 where '[m]' means optional "MIN" and/or "MAX" 266 3.3. Extension to UID FETCH 268 The PARTIAL extension also extends the UID FETCH command with a 269 PARTIAL FETCH modifier. The PARTIAL FETCH modifier has the same 270 syntax as the PARTIAL SEARCH result option. Presence of the PARTIAL 271 FETCH modifier instructs the server to only return FETCH results for 272 messages in the specified range. It is useful when the sequence-set 273 (first) parameter to the UID FETCH command includes unknown number of 274 messages. 276 // Returning information for the last 3 messages in the UID range 277 C: 10 UID FETCH 25900:26600 (UID FLAGS) (PARTIAL -1:-3) 278 S: * 12888 FETCH (FLAGS (\Seen) UID 25996) 279 S: * 12889 FETCH (FLAGS (\Flagged \Answered) UID 25997) 280 S: * 12890 FETCH (FLAGS () UID 26600) 281 S: 10 OK FETCH completed 282 // Returning information for the first 5 messages in the UID range 283 C: 11 UID FETCH 25900:26600 (UID FLAGS) (PARTIAL 1:5) 284 S: * 12591 FETCH (FLAGS (\Seen) UID 25900) 285 S: * 12592 FETCH (FLAGS (\Flagged) UID 25902) 286 S: * 12593 FETCH (FLAGS (\Answered) UID 26310) 287 S: * 12594 FETCH (FLAGS () UID 26311) 288 S: * 12595 FETCH (FLAGS (\Answered) UID 26498) 289 S: 11 OK FETCH completed 291 3.4. Use of PARTIAL and CONDSTORE IMAP extensions together 293 This section is informative. 295 The PARTIAL FETCH modifier can be combined with the CHANGEDSINCE 296 FETCH modifier. 298 // Returning information for the last 30 messages in the UID range 299 // that have any flag/keyword modified since modseq 98305 300 C: 101 UID FETCH 25900:26600 (UID FLAGS) (PARTIAL -1:-30 CHANGEDSINCE 98305) 301 S: * 12888 FETCH (FLAGS (\Flagged \Answered) MODSEQ (98306) UID 25997) 302 S: * 12890 FETCH (FLAGS () MODSEQ (98312) UID 26600) 303 S: 101 OK FETCH completed 305 Note that the order of PARTIAL and CHANGEDSINCE FETCH modifiers is 306 not important. 308 4. The MESSAGELIMIT extension 310 An IMAP server advertises support for the MESSAGELIMIT extension by 311 including "MESSAGELIMIT=" capability in the CAPABILITY 312 response/response code, where "" is a positive integer that 313 conveys the maximum number of messages that can be processed in a 314 single SEARCH/FETCH/STORE/COPY/MOVE command. 316 4.1. Returning limits on the number of messages processed in a single 317 SEARCH/FETCH/STORE/COPY/MOVE command 319 // Should this be a separate IMAP capability? 320 // Also, should this also affect SEARCH? If yes, do we need a way to 321 // specify SEARCH criterion for "all UIDs after" or "all UIDs before" 322 // a specific UID? 324 If a server implementation doesn't allow more than messages to be 325 operated on by a single SEARCH/FETCH/STORE/COPY/MOVE command, it MUST 326 return the MESSAGELIMIT response code defined below: 328 MESSAGELIMIT The server doesn't allow more than messages to be operated 329 on by a single SEARCH/FETCH/STORE/COPY/MOVE command. The 330 lowest processed UID is . The client needs to repeat 331 the operation for remaining messages. 333 C: 03 FETCH 10000:14589 (UID FLAGS) 334 S: * 14589 FETCH (FLAGS (\Seen) UID 25000) 335 S: * 14588 FETCH (FLAGS (\Answered) UID 24998) 336 S: ... further 997 fetch responses 337 S: * 13590 FETCH (FLAGS () UID 23221) 338 S: 03 OK [MESSAGELIMIT 1000 23221] FETCH completed with 1000 partial 339 results 341 In the above example the value is 1000 and the lowest 342 processed UID is 23221. 344 5. Formal syntax 346 The following syntax specification uses the Augmented Backus-Naur 347 Form (ABNF) notation as specified in [ABNF]. 349 Non-terminals referenced but not defined below are as defined by 350 IMAP4 [RFC3501]. 352 Except as noted otherwise, all alphabetic characters are case- 353 insensitive. The use of upper or lower case characters to define 354 token strings is for editorial clarity only. Implementations MUST 355 accept these strings in a case-insensitive fashion. 357 SP = 358 MINUS = "-" 360 capability =/ "PARTIAL" 361 ;; from [RFC3501] 363 modifier-partial = "PARTIAL" SP partial-range 365 partial-range-first = nz-number ":" nz-number 366 ;; Request to search from oldest (lowest UIDs) to 367 ;; more recent messages. 368 ;; A range 500:400 is the same as 400:500. 369 ;; This is similar to from [RFC3501], 370 ;; but cannot contain "*". 372 partial-range-last = MINUS nz-number ":" MINUS nz-number 373 ;; Request to search from newest (highest UIDs) to 374 ;; oldest messages. 376 ;; A range -500:-400 is the same as -400:-500. 378 partial-range = partial-range-first / partial-range-last 380 search-return-opt =/ modifier-partial 381 ;; All conform to , from [IMAP-ABNF]/[RFC9051] 383 search-return-data =/ ret-data-partial 385 ret-data-partial = "PARTIAL" 386 SP "(" partial-range SP partial-results ")" 387 ;; is the requested range. 389 partial-results = sequence-set / "NIL" 390 ;; from [RFC3501]. 391 ;; NIL indicates no results correspond to the requested range. 393 tagged-ext-simple =/ partial-range-last 395 fetch-modifier =/ modifier-partial 397 capability =/ "MESSAGELIMIT=" message-limit 398 ;; from [RFC3501] 400 message-limit = nz-number 402 resp-text-code =/ "MESSAGELIMIT" SP message-limit SP uniqueid 403 ;; No more than nz-number messages can be processed 404 ;; by any command at a time. The last (lowest) processed 405 ;; UID is uniqueid. 407 6. Security Considerations 409 TBD. 411 7. IANA Considerations 413 7.1. Changes/additions to the IMAP4 capabilities registry 415 IMAP4 capabilities are registered by publishing a standards track or 416 IESG approved Informational or Experimental RFC. The registry is 417 currently located at: 419 https://www.iana.org/assignments/imap4-capabilities 421 IANA is requested to add definition of the PARTIAL extension to point 422 to this document. 424 8. Acknowledgments 426 This document was motivated by Yahoo! team and their questions about 427 best client practices for dealing with large mailboxes. 429 This document uses lots of text from RFC 5267. Thus work of the RFC 430 5267 authors Dave Cridland and Curtis King is appreciated. 432 9. References 434 9.1. Normative References 436 [ABNF] Crocker, D., Ed. and P. Overell, Ed., "Augmented BNF for 437 Syntax Specifications: ABNF", RFC 5234, January 2008, 438 . 440 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 441 Requirement Levels", BCP 14, RFC 2119, 442 DOI 10.17487/RFC2119, March 1997, 443 . 445 [RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 446 4rev1", RFC 3501, DOI 10.17487/RFC3501, March 2003, 447 . 449 [RFC4731] Melnikov, A. and D. Cridland, "IMAP4 Extension to SEARCH 450 Command for Controlling What Kind of Information Is 451 Returned", RFC 4731, DOI 10.17487/RFC4731, November 2006, 452 . 454 [RFC5267] Cridland, D. and C. King, "Contexts for IMAP4", RFC 5267, 455 DOI 10.17487/RFC5267, July 2008, 456 . 458 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 459 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 460 May 2017, . 462 [RFC9051] Melnikov, A., Ed. and B. Leiba, Ed., "Internet Message 463 Access Protocol (IMAP) - Version 4rev2", RFC 9051, 464 DOI 10.17487/RFC9051, August 2021, 465 . 467 9.2. Informative References 469 [RFC7162] Melnikov, A. and D. Cridland, "IMAP Extensions: Quick Flag 470 Changes Resynchronization (CONDSTORE) and Quick Mailbox 471 Resynchronization (QRESYNC)", RFC 7162, 472 DOI 10.17487/RFC7162, May 2014, 473 . 475 Index 477 M 479 M 481 MESSAGELIMIT (response code) 482 Section 4.1, Paragraph 3.2.1 484 Authors' Addresses 486 Alexey Melnikov 487 Isode Limited 489 Email: alexey.melnikov@isode.com 490 URI: https://www.isode.com 492 Arun Prakash Achuthan 493 Yahoo! 495 Email: arunprakash@myyahoo.com 497 Vikram Nagulakonda 498 Yahoo! 500 Email: nvikram_imap@yahoo.com 502 Luis Alves 503 Yahoo! 505 Email: luis.alves@lafaspot.com