idnits 2.17.00 (12 Aug 2021) /tmp/idnits40562/draft-ietf-appsawg-multimailbox-search-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year (Using the creation date from RFC4466, updated by this document, for RFC5378 checks: 2005-03-09) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 3, 2014) is 2908 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 3501 (Obsoleted by RFC 9051) Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Applications Area Working Group B. Leiba 3 Internet-Draft Huawei Technologies 4 Obsoletes: 6237 (if approved) A. Melnikov 5 Updates: 4466 (if approved) Isode Limited 6 Intended status: Standards Track June 3, 2014 7 Expires: December 5, 2014 9 IMAP4 Multimailbox SEARCH Extension 10 draft-ietf-appsawg-multimailbox-search-01 12 Abstract 14 The IMAP4 specification allows the searching of only the selected 15 mailbox. A user often wants to search multiple mailboxes, and a 16 client that wishes to support this must issue a series of SELECT and 17 SEARCH commands, waiting for each to complete before moving on to the 18 next. This extension allows a client to search multiple mailboxes 19 with one command, limiting round trips delay, and not requiring 20 disruption of the currently selected mailbox. This extension also 21 uses MAILBOX and TAG fields in ESEARCH responses, allowing a client 22 to pipeline the searches if it chooses. This document updates RFC 23 4466 and obsoletes RFC 6237. 25 Status of This Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on December 5, 2014. 42 Copyright Notice 44 Copyright (c) 2014 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 60 1.1. Conventions Used in This Document . . . . . . . . . . . . . 3 61 2. New ESEARCH Command . . . . . . . . . . . . . . . . . . . . . 3 62 2.1. The ESEARCH Response . . . . . . . . . . . . . . . . . . . 4 63 2.2. Source Options: Specifying Mailboxes to Search . . . . . . 5 64 2.3. Strictness in Search Matches . . . . . . . . . . . . . . . 7 65 3. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 7 66 4. Formal Syntax . . . . . . . . . . . . . . . . . . . . . . . . 7 67 5. Security Considerations . . . . . . . . . . . . . . . . . . . 9 68 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 69 7. Implementation Status . . . . . . . . . . . . . . . . . . . . 9 70 8. Changes Since RFC 6237 . . . . . . . . . . . . . . . . . . . 10 71 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 10 72 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 73 10.1. Normative References . . . . . . . . . . . . . . . . . . . 11 74 10.2. Informative References . . . . . . . . . . . . . . . . . . 11 75 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11 77 1. Introduction 79 The IMAP4 specification allows the searching of only the selected 80 mailbox. A user often wants to search multiple mailboxes, and a 81 client that wishes to support this must issue a series of SELECT and 82 SEARCH commands, waiting for each to complete before moving on to the 83 next. The commands can't be pipelined, because the server might run 84 them in parallel, and the untagged SEARCH responses could not then be 85 distinguished from each other. 87 This extension allows a client to search multiple mailboxes with one 88 command, and includes MAILBOX and TAG fields in the ESEARCH response, 89 yielding the following advantages: 91 o A single command limits the number of round trips needed to search 92 a set of mailboxes. 94 o A single command eliminates the need to wait for one search to 95 complete before starting the next. 97 o A single command allows the server to optimize the search, if it 98 can. 100 o A command that is not dependent upon the selected mailbox 101 eliminates the need to disrupt the selection state or to open 102 another IMAP connection. 104 o The MAILBOX, UIDVALIDITY, and TAG fields in the responses allow a 105 client to distinguish which responses go with which search (and 106 which mailbox). A client can safely pipeline these search 107 commands without danger of confusion. The addition of the MAILBOX 108 and UIDVALIDITY fields updates the search-correlator item defined 109 in [RFC4466]. 111 This extension was previously published as experimental. There is 112 now implementation experience, giving confidence in the protocol, so 113 this document puts the extension on the Standards Track, with some 114 minor updates that were informed by the implementation experience. 115 [[CREF1: RFC Editor: Please remove this paragraph at publication.]] 117 1.1. Conventions Used in This Document 119 In examples, "C:" indicates lines sent by a client that is connected 120 to a server. "S:" indicates lines sent by the server to the client. 122 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 123 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 124 document are to be interpreted as described in RFC 2119 [RFC2119]. 126 2. New ESEARCH Command 128 Arguments: OPTIONAL source options 129 OPTIONAL result options 130 OPTIONAL charset specification (see [RFC2978]) 131 searching criteria (one or more) 133 Responses: REQUIRED untagged response: ESEARCH 135 Result: OK -- search completed 136 NO -- error: cannot search that charset or criteria 137 BAD -- command unknown or arguments invalid 139 This section defines a new ESEARCH command, which works similarly to 140 the UID SEARCH command described in Section 2.6.1 of [RFC4466] 141 (initially described in Section 6.4.4 of [RFC3501] and extended by 142 [RFC4731]). 144 The ESEARCH command further extends searching by allowing for 145 optional source and result options. This document does not define 146 any new result options (see Section 3.1 of [RFC4731]). A server that 147 supports this extension includes "MULTISEARCH" in its IMAP capability 148 string. 150 Because there has been confusion about this, it is worth pointing out 151 that with ESEARCH, as with any SEARCH or UID SEARCH command, it MUST 152 NOT be considered an error if the search terms include a range of 153 message numbers that extends (or, in fact, starts) beyond the end of 154 the mailbox. For example, a client might want to establish a rolling 155 window through the search results this way: 157 C: tag1 UID ESEARCH FROM "frobozz" 1:100 159 ...followed later by this: 161 C: tag1 UID ESEARCH FROM "frobozz" 101:200 163 ...and so on. This tells the server to match only the first hundred 164 messages in the mailbox the first time, the second hundred the second 165 time, etc. In fact, it might likely allow the server to optimize the 166 search significantly. In the above example, whether the mailbox 167 contains 50 or 150 or 250 messages, neither of the search commands 168 shown will result in an error. It is up to the client to know when 169 to stop moving its search window. 171 2.1. The ESEARCH Response 173 In response to an ESEARCH command, the server MUST return ESEARCH 174 responses [RFC4731] (that is, not SEARCH responses). Because message 175 numbers are not useful for mailboxes that are not selected, the 176 responses MUST contain information about UIDs, not message numbers. 177 This is true even if the source options specify that only the 178 selected mailbox be searched. 180 Presence of a source option in the absence of a result option implies 181 the "ALL" result option (see Section 3.1 of [RFC4731]). Note that 182 this is not the same as the result from the SEARCH command described 183 in the IMAP base protocol [RFC3501]. 185 Source options describe which mailboxes must be searched for 186 messages. An ESEARCH command with source options does not affect 187 which mailbox, if any, is currently selected, regardless of which 188 mailboxes are searched. 190 For each mailbox satisfying the source options, a single ESEARCH 191 response MUST be returned if any messages in that mailbox match the 192 search criteria. An ESEARCH response MUST NOT be returned for 193 mailboxes that contain no matching messages. This is true even when 194 result options such as MIN, MAX, and COUNT are specified (see 195 Section 3.1 of [RFC4731]), and the values returned (lowest UID 196 matched, highest UID matched, and number of messages matched, 197 respectively) apply to the mailbox reported in that ESEARCH response. 199 Note that it is possible for an ESEARCH command to return no untagged 200 responses (no ESEARCH responses at all), in the case that there are 201 no matches to the search in any of the mailboxes that satisfy the 202 source options. Clients can detect this situation by finding the 203 tagged OK response without having received any matching untagged 204 ESEARCH responses. 206 Each ESEARCH response MUST contain the MAILBOX, TAG, and UIDVALIDITY 207 correlators. Correlators allow clients to issue several ESEARCH 208 commands at once (pipelined). If the SEARCHRES [RFC5182] extension 209 is used in an ESEARCH command, that ESEARCH command MUST be executed 210 by the server after all previous SEARCH/ESEARCH commands have 211 completed and before any subsequent SEARCH/ESEARCH commands are 212 executed. The server MAY perform consecutive ESEARCH commands in 213 parallel as long as none of them use the SEARCHRES extension. 215 2.2. Source Options: Specifying Mailboxes to Search 217 The source options, if present, MUST contain a mailbox specifier as 218 defined in the IMAP NOTIFY extension [RFC5465], Section 6 (using the 219 "filter-mailboxes" ABNF item), with the following differences: 221 1. The "selected-delayed" specifier is not valid here. 223 2. A "subtree-one" specifier is added. The "subtree" specifier 224 results in a search of the specified mailbox and all selectable 225 mailboxes that are subordinate to it, through an indefinitely 226 deep hierarchy. The "subtree-one" specifier results in a search 227 of the specified mailbox and all selectable child mailboxes, one 228 hierarchy level down. 230 If "subtree" is specified, the server MUST defend against loops in 231 the hierarchy (for example, those caused by recursive file-system 232 links within the message store). The server SHOULD do this by 233 keeping track of the mailboxes that have been searched, and 234 terminating the hierarchy traversal when a repeat is found. If it 235 cannot do that, it MAY do it by limiting the hierarchy depth. 237 If the source options are not present, the value "selected" is 238 assumed -- that is, only the currently selected mailbox is searched. 240 The "personal" source option is a particularly convenient way to 241 search all of the current user's mailboxes. Note that there is no 242 way to use wildcard characters to search all mailboxes; the 243 "mailboxes" source option does not do wildcard expansion. 245 If the source options include (or default to) "selected", the IMAP 246 session MUST be in "selected" state. If the source options specify 247 other mailboxes and NOT "selected", then the IMAP session MUST be in 248 either "selected" or "authenticated" state. If the session is not in 249 a correct state, the ESEARCH command MUST return a "BAD" result. 251 The client SHOULD NOT provide source options that resolve to 252 including the same mailbox more than once. A server can, of course, 253 remove the duplicates before processing, but the server MAY return 254 "BAD" to an ESEARCH command with duplicate source mailboxes. 256 If the server supports the SEARCHRES [RFC5182] extension, then the 257 "SAVE" result option is valid only if "selected" is specified or 258 defaulted as the sole mailbox to be searched. If any source option 259 other than "selected" is specified, the ESEARCH command MUST return a 260 "BAD" result. 262 If the server supports the CONTEXT=SEARCH and/or CONTEXT=SORT 263 extension [RFC5267], then the following additional rules apply: 265 o The CONTEXT return option (Section 4.2 of [RFC5267]) can be used 266 with an ESEARCH command. 268 o If the UPDATE return option is used (Section 4.3 of [RFC5267]), it 269 MUST apply only to the currently selected mailbox. If UPDATE is 270 used and there is no mailbox currently selected, the ESEARCH 271 command MUST return a "BAD" result. 273 o The PARTIAL search return option (Section 4.4 of [RFC5267]) can be 274 used and applies to each mailbox searched by the ESEARCH command. 276 If the server supports the Access Control List (ACL) [RFC4314] 277 extension, then the logged-in user is required to have the "r" right 278 for each mailbox she wants to search. In addition, any mailboxes 279 that are not explicitly named (accessed through "personal" or 280 "subtree", for example) are required to have the "l" right. 281 Mailboxes matching the source options for which the logged-in user 282 lacks sufficient rights MUST be ignored by the ESEARCH command 283 processing. In particular, ESEARCH responses MUST NOT be returned 284 for those mailboxes. 286 2.3. Strictness in Search Matches 288 The base IMAP SEARCH command (Section 6.4.4. of [RFC3501]) requires 289 strict substring matching in text searches. Many servers, however, 290 use search engines that match strings in different ways, matching, 291 for example, "swim" to "swam" and "swum" as well, or only doing full 292 word matching (where "swim" will not match "swimming"). This is 293 covered by the "Fuzzy Search" extension to IMAP [RFC6203], and that 294 extension is compatible with this one and can be combined with it. 296 Whether or not Fuzzy Search is implemented or used, this extension 297 explicitly allows flexible searching with respect to TEXT and BODY 298 searches. Servers MAY use fuzzy text matching in multimailbox 299 searches. 301 3. Examples 303 In the following example, note that two ESEARCH commands are 304 pipelined, and that the server is running them in parallel, 305 interleaving a response to the second search amid the responses to 306 the first (watch the tags). 308 C: tag1 ESEARCH IN (mailboxes "folder1" subtree "folder2") unseen 309 C: tag2 ESEARCH IN (mailboxes "folder1" subtree-one "folder2") 310 subject "chad" 311 S: * ESEARCH (TAG "tag1" MAILBOX "folder1" UIDVALIDITY 1) UID ALL 312 4001,4003,4005,4007,4009 313 S: * ESEARCH (TAG "tag2" MAILBOX "folder1" UIDVALIDITY 1) UID ALL 314 3001:3004,3788 315 S: * ESEARCH (TAG "tag1" MAILBOX "folder2/banana" UIDVALIDITY 503) 316 UID ALL 3002,4004 317 S: * ESEARCH (TAG "tag1" MAILBOX "folder2/peach" UIDVALIDITY 3) UID 318 ALL 921691 319 S: tag1 OK done 320 S: * ESEARCH (TAG "tag2" MAILBOX "folder2/salmon" UIDVALIDITY 321 1111111) UID ALL 50003,50006,50009,50012 322 S: tag2 OK done 324 4. Formal Syntax 326 The following syntax specification uses the Augmented Backus-Naur 327 Form (ABNF) as described in [RFC5234]. Terms not defined here are 328 taken from [RFC3501], [RFC5465], or [RFC4466]. 330 command-auth =/ esearch 331 ; Update definition from IMAP base [RFC3501]. 332 ; Add new "esearch" command. 334 command-select =/ esearch 335 ; Update definition from IMAP base [RFC3501]. 336 ; Add new "esearch" command. 338 filter-mailboxes-other =/ ("subtree-one" SP one-or-more-mailbox) 339 ; Update definition from IMAP Notify [RFC5465]. 340 ; Add new "subtree-one" selector. 342 filter-mailboxes-selected = "selected" 343 ; Update definition from IMAP Notify [RFC5465]. 344 ; We forbid the use of "selected-delayed". 346 one-correlator = ("TAG" SP tag-string) / ("MAILBOX" SP astring) / 347 ("UIDVALIDITY" SP nz-number) 348 ; Each correlator MUST appear exactly once. 350 scope-option = scope-option-name [SP scope-option-value] 351 ; No options defined here. Syntax for future extensions. 353 scope-option-name = tagged-ext-label 354 ; No options defined here. Syntax for future extensions. 356 scope-option-value = tagged-ext-val 357 ; No options defined here. Syntax for future extensions. 359 scope-options = scope-option *(SP scope-option) 360 ; A given option may only appear once. 361 ; No options defined here. Syntax for future extensions. 363 esearch = "ESEARCH" [SP esearch-source-opts] 364 [SP search-return-opts] SP search-program 366 search-correlator = SP "(" one-correlator *(SP one-correlator) ")" 367 ; Updates definition in IMAP4 ABNF [RFC4466]. 369 esearch-source-opts = "IN" SP "(" source-mbox [SP 370 "(" scope-options ")"] ")" 372 source-mbox = filter-mailboxes *(SP filter-mailboxes) 373 ; "filter-mailboxes" is defined in IMAP Notify [RFC5465]. 374 ; See updated definition of filter-mailboxes-other, above. 375 ; See updated definition of filter-mailboxes-selected, above. 377 5. Security Considerations 379 This new IMAP ESEARCH command allows a single command to search many 380 mailboxes at once. On the one hand, a client could do that by 381 sending many IMAP SEARCH commands. On the other hand, this makes it 382 easier for a client to overwork a server, by sending a single command 383 that results in an expensive search of tens of thousands of 384 mailboxes. Server implementations need to be aware of that, and 385 provide mechanisms that prevent a client from adversely affecting 386 other users. Limitations on the number of mailboxes that may be 387 searched in one command, and/or on the server resources that will be 388 devoted to responding to a single client, are reasonable limitations 389 for an implementation to impose. 391 Implementations MUST, of course, apply access controls appropriately, 392 limiting a user's access to ESEARCH in the same way its access is 393 limited for any other IMAP commands. This extension has no data- 394 access risks beyond what may be there in the unextended IMAP 395 implementation. 397 Mailboxes matching the source options for which the logged-in user 398 lacks sufficient rights MUST be ignored by the ESEARCH command 399 processing (see the paragraph about this in Section 2.2). In 400 particular, any attempt to distinguish insufficient access from 401 non-existent mailboxes may expose information about the mailbox 402 hierarchy that isn't otherwise available to the client. 404 If "subtree" is specified, the server MUST defend against loops in 405 the hierarchy (see the paragraph about this in Section 2.2). 407 6. IANA Considerations 409 The "IMAP Capabilities" registry is currently located at 410 . 413 IANA is asked to change the reference for the IMAP capability 414 "MULTISEARCH" to point to this document. 416 7. Implementation Status 418 [[CREF2: RFC Editor: Please remove this section at publication.]] 420 This section records the status of known implementations of the 421 protocol defined by this specification at the time of posting of 422 this Internet-Draft, and is based on a proposal described in RFC 423 6982. The description of implementations in this section is 424 intended to assist the IETF in its decision processes in 425 progressing drafts to RFCs. Please note that the listing of any 426 individual implementation here does not imply endorsement by the 427 IETF. Furthermore, no effort has been spent to verify the 428 information presented here that was supplied by IETF contributors. 429 This is not intended as, and must not be construed to be, a 430 catalog of available implementations or their features. Readers 431 are advised to note that other implementations may exist. 433 According to RFC 6982, "this will allow reviewers and working 434 groups to assign due consideration to documents that have the 435 benefit of running code, which may serve as evidence of valuable 436 experimentation and feedback that have made the implemented 437 protocols more mature. It is up to the individual working groups 438 to use this information as they see fit". 440 The following implementations are known to exist: 442 o Oracle has a server implementation that is not currently in a 443 product. 445 o There is a client implementation that has been tested with the 446 Oracle server. No further information is available. 448 Interest has been expressed in creating the following 449 implementations: 451 o Isode Limited 453 8. Changes Since RFC 6237 455 o Change to Standards Track. 457 o Added paragraph about duplicate mailboxes. 459 o Added Section 2.3 about fuzzy search. 461 o Added Section 7, "Implementation Status". 462 [[CREF3: RFC Editor: Please remove this bullet at publication.]] 464 9. Acknowledgements 466 The authors gratefully acknowledge feedback provided by Timo 467 Sirainen, Peter Coates, Arnt Gulbrandsen, and Chris Newman. 469 10. References 471 10.1. Normative References 473 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 474 Requirement Levels", BCP 14, RFC 2119, March 1997. 476 [RFC2978] Freed, N. and J. Postel, "IANA Charset Registration 477 Procedures", BCP 19, RFC 2978, October 2000. 479 [RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 480 4rev1", RFC 3501, March 2003. 482 [RFC4314] Melnikov, A., "IMAP4 Access Control List (ACL) Extension", 483 RFC 4314, December 2005. 485 [RFC4466] Melnikov, A. and C. Daboo, "Collected Extensions to IMAP4 486 ABNF", RFC 4466, April 2006. 488 [RFC4731] Melnikov, A. and D. Cridland, "IMAP4 Extension to SEARCH 489 Command for Controlling What Kind of Information Is 490 Returned", RFC 4731, November 2006. 492 [RFC5182] Melnikov, A., "IMAP Extension for Referencing the Last 493 SEARCH Result", RFC 5182, March 2008. 495 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 496 Specifications: ABNF", STD 68, RFC 5234, January 2008. 498 [RFC5267] Cridland, D. and C. King, "Contexts for IMAP4", RFC 5267, 499 July 2008. 501 [RFC5465] Gulbrandsen, A., King, C., and A. Melnikov, "The IMAP 502 NOTIFY Extension", RFC 5465, February 2009. 504 10.2. Informative References 506 [RFC6203] Sirainen, T., "IMAP4 Extension for Fuzzy Search", RFC 507 6203, March 2011. 509 Authors' Addresses 511 Barry Leiba 512 Huawei Technologies 514 Phone: +1 646 827 0648 515 Email: barryleiba@computer.org 516 URI: http://internetmessagingtechnology.org/ 517 Alexey Melnikov 518 Isode Limited 519 5 Castle Business Village 520 36 Station Road 521 Hampton, Middlesex TW12 2BX 522 UK 524 Email: Alexey.Melnikov@isode.com 525 URI: http://www.melnikov.ca/