idnits 2.17.00 (12 Aug 2021) /tmp/idnits39834/draft-ietf-appsawg-multimailbox-search-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. == There are 1 instance of lines with non-RFC2606-compliant FQDNs in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year (Using the creation date from RFC4466, updated by this document, for RFC5378 checks: 2005-03-09) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (March 03, 2014) is 3000 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 3501 (Obsoleted by RFC 9051) Summary: 2 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Applications Area Working Group B. Leiba 3 Internet-Draft Huawei Technologies 4 Updates: 4466 (if approved) A. Melnikov 5 Obsoletes: 6237 (if approved) Isode Limited 6 Intended status: Standards Track March 03, 2014 7 Expires: September 02, 2014 9 IMAP4 Multimailbox SEARCH Extension 10 draft-ietf-appsawg-multimailbox-search-00 12 Abstract 14 The IMAP4 specification allows the searching of only the selected 15 mailbox. A user often wants to search multiple mailboxes, and a 16 client that wishes to support this must issue a series of SELECT and 17 SEARCH commands, waiting for each to complete before moving on to the 18 next. This extension allows a client to search multiple mailboxes 19 with one command, limiting round trips delay, and not requiring 20 disruption of the currently selected mailbox. This extension also 21 uses MAILBOX and TAG fields in ESEARCH responses, allowing a client 22 to pipeline the searches if it chooses. This document updates RFC 23 4466 and obsoletes RFC 6237. 25 Status of this Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on September 02, 2014. 42 Copyright Notice 44 Copyright (c) 2014 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents (http://trustee.ietf.org/ 49 license-info) in effect on the date of publication of this document. 50 Please review these documents carefully, as they describe your rights 51 and restrictions with respect to this document. Code Components 52 extracted from this document must include Simplified BSD License text 53 as described in Section 4.e of the Trust Legal Provisions and are 54 provided without warranty as described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2 59 1.1. Conventions Used in This Document . . . . . . . . . . . . . 3 60 2. New ESEARCH Command . . . . . . . . . . . . . . . . . . . . . 3 61 2.1. The ESEARCH Response . . . . . . . . . . . . . . . . . . . . 4 62 2.2. Source Options: Specifying Mailboxes to Search . . . . . . . 5 63 2.3. Strictness in Search Matches . . . . . . . . . . . . . . . . 6 64 3. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 65 4. Formal Syntax . . . . . . . . . . . . . . . . . . . . . . . . 7 66 5. Security Considerations . . . . . . . . . . . . . . . . . . . 8 67 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 68 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 9 69 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 9 70 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 9 72 1. Introduction 74 The IMAP4 specification allows the searching of only the selected 75 mailbox. A user often wants to search multiple mailboxes, and a 76 client that wishes to support this must issue a series of SELECT and 77 SEARCH commands, waiting for each to complete before moving on to the 78 next. The commands can't be pipelined, because the server might run 79 them in parallel, and the untagged SEARCH responses could not then be 80 distinguished from each other. 82 This extension allows a client to search multiple mailboxes with one 83 command, and includes MAILBOX and TAG fields in the ESEARCH response, 84 yielding the following advantages: 86 o A single command limits the number of round trips needed to search 87 a set of mailboxes. 89 o A single command eliminates the need to wait for one search to 90 complete before starting the next. 92 o A single command allows the server to optimize the search, if it 93 can. 95 o A command that is not dependent upon the selected mailbox 96 eliminates the need to disrupt the selection state or to open 97 another IMAP connection. 99 o The MAILBOX, UIDVALIDITY, and TAG fields in the responses allow a 100 client to distinguish which responses go with which search (and 101 which mailbox). A client can safely pipeline these search 102 commands without danger of confusion. The addition of the MAILBOX 103 and UIDVALIDITY fields updates the search-correlator item defined 104 in [RFC4466]. 106 This extension was previously published as experimental. There is 107 now implementation experience, giving confidence in the protocol, so 108 this document puts the extension on the Standards Track, with some 109 minor updates that were informed by the implementation experience. 111 1.1. Conventions Used in This Document 113 In examples, "C:" indicates lines sent by a client that is connected 114 to a server. "S:" indicates lines sent by the server to the client. 116 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 117 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 118 document are to be interpreted as described in RFC 2119 [RFC2119]. 120 2. New ESEARCH Command 122 Arguments: OPTIONAL source options 123 OPTIONAL result options 124 OPTIONAL charset specification (see [RFC2978]) 125 searching criteria (one or more) 127 Responses: REQUIRED untagged response: ESEARCH 129 Result: OK -- search completed 130 NO -- error: cannot search that charset or criteria 131 BAD -- command unknown or arguments invalid 133 This section defines a new ESEARCH command, which works similarly to 134 the UID SEARCH command described in Section 2.6.1 of [RFC4466] 135 (initially described in Section 6.4.4 of [RFC3501] and extended by 136 [RFC4731]). 138 The ESEARCH command further extends searching by allowing for 139 optional source and result options. This document does not define 140 any new result options (see Section 3.1 of [RFC4731]). A server that 141 supports this extension includes "MULTISEARCH" in its IMAP capability 142 string. 144 Because there has been confusion about this, it is worth pointing out 145 that with ESEARCH, as with *any* SEARCH or UID SEARCH command, it 146 MUST NOT be considered an error if the search terms include a range 147 of message numbers that extends (or, in fact, starts) beyond the end 148 of the mailbox. For example, a client might want to establish a 149 rolling window through the search results this way: 151 C: tag1 UID ESEARCH FROM "frobozz" 1:100 152 ...followed later by this: 154 C: tag1 UID ESEARCH FROM "frobozz" 101:200 156 ...and so on. This tells the server to match only the first hundred 157 messages in the mailbox the first time, the second hundred the second 158 time, etc. In fact, it might likely allow the server to optimize the 159 search significantly. In the above example, whether the mailbox 160 contains 50 or 150 or 250 messages, neither of the search commands 161 shown will result in an error. It is up to the client to know when 162 to stop moving its search window. 164 2.1. The ESEARCH Response 166 In response to an ESEARCH command, the server MUST return ESEARCH 167 responses [RFC4731] (that is, not SEARCH responses). Because message 168 numbers are not useful for mailboxes that are not selected, the 169 responses MUST contain information about UIDs, not message numbers. 170 This is true even if the source options specify that only the 171 selected mailbox be searched. 173 Presence of a source option in the absence of a result option implies 174 the "ALL" result option (see Section 3.1 of [RFC4731]). Note that 175 this is not the same as the result from the SEARCH command described 176 in the IMAP base protocol [RFC3501]. 178 Source options describe which mailboxes must be searched for 179 messages. An ESEARCH command with source options does not affect 180 which mailbox, if any, is currently selected, regardless of which 181 mailboxes are searched. 183 For each mailbox satisfying the source options, a single ESEARCH 184 response MUST be returned if any messages in that mailbox match the 185 search criteria. An ESEARCH response MUST NOT be returned for 186 mailboxes that contain no matching messages. This is true even when 187 result options such as MIN, MAX, and COUNT are specified (see Section 188 3.1 of [RFC4731]), and the values returned (lowest UID matched, 189 highest UID matched, and number of messages matched, respectively) 190 apply to the mailbox reported in that ESEARCH response. 192 Note that it is possible for an ESEARCH command to return *no* 193 untagged responses (no ESEARCH responses at all), in the case that 194 there are no matches to the search in any of the mailboxes that 195 satisfy the source options. Clients can detect this situation by 196 finding the tagged OK response without having received any matching 197 untagged ESEARCH responses. 199 Each ESEARCH response MUST contain the MAILBOX, TAG, and UIDVALIDITY 200 correlators. Correlators allow clients to issue several ESEARCH 201 commands at once (pipelined). If the SEARCHRES [RFC5182] extension 202 is used in an ESEARCH command, that ESEARCH command MUST be executed 203 by the server after all previous SEARCH/ESEARCH commands have 204 completed and before any subsequent SEARCH/ESEARCH commands are 205 executed. The server MAY perform consecutive ESEARCH commands in 206 parallel as long as none of them use the SEARCHRES extension. 208 2.2. Source Options: Specifying Mailboxes to Search 210 The source options, if present, MUST contain a mailbox specifier as 211 defined in the IMAP NOTIFY extension [RFC5465], Section 6 (using the 212 "filter-mailboxes" ABNF item), with the following differences: 214 1. The "selected-delayed" specifier is not valid here. 216 2. A "subtree-one" specifier is added. The "subtree" specifier 217 results in a search of the specified mailbox and all selectable 218 mailboxes that are subordinate to it, through an indefinitely 219 deep hierarchy. The "subtree-one" specifier results in a search 220 of the specified mailbox and all selectable child mailboxes, one 221 hierarchy level down. 223 If "subtree" is specified, the server MUST defend against loops in 224 the hierarchy (for example, those caused by recursive file-system 225 links within the message store). The server SHOULD do this by 226 keeping track of the mailboxes that have been searched, and 227 terminating the hierarchy traversal when a repeat is found. If it 228 cannot do that, it MAY do it by limiting the hierarchy depth. 230 If the source options are not present, the value "selected" is 231 assumed -- that is, only the currently selected mailbox is searched. 233 The "personal" source option is a particularly convenient way to 234 search all of the current user's mailboxes. Note that there is no 235 way to use wildcard characters to search all mailboxes; the 236 "mailboxes" source option does not do wildcard expansion. 238 If the source options include (or default to) "selected", the IMAP 239 session MUST be in "selected" state. If the source options specify 240 other mailboxes and NOT "selected", then the IMAP session MUST be in 241 either "selected" or "authenticated" state. If the session is not in 242 a correct state, the ESEARCH command MUST return a "BAD" result. 244 The client SHOULD NOT provide source options that resolve to 245 including the same mailbox more than once. A server can, of course, 246 remove the duplicates before processing, but the server MAY return 247 "BAD" to an ESEARCH command with duplicate source mailboxes. 249 If the server supports the SEARCHRES [RFC5182] extension, then the 250 "SAVE" result option is valid *only* if "selected" is specified or 251 defaulted as the sole mailbox to be searched. If any source option 252 other than "selected" is specified, the ESEARCH command MUST return a 253 "BAD" result. 255 If the server supports the CONTEXT=SEARCH and/or CONTEXT=SORT 256 extension [RFC5267], then the following additional rules apply: 258 o The CONTEXT return option (Section 4.2 of [RFC5267]) can be used 259 with an ESEARCH command. 261 o If the UPDATE return option is used (Section 4.3 of [RFC5267]), it 262 MUST apply ONLY to the currently selected mailbox. If UPDATE is 263 used and there is no mailbox currently selected, the ESEARCH 264 command MUST return a "BAD" result. 266 o The PARTIAL search return option (Section 4.4 of [RFC5267]) can be 267 used and applies to each mailbox searched by the ESEARCH command. 269 If the server supports the Access Control List (ACL) [RFC4314] 270 extension, then the logged-in user is required to have the "r" right 271 for each mailbox she wants to search. In addition, any mailboxes 272 that are not explicitly named (accessed through "personal" or 273 "subtree", for example) are required to have the "l" right. 274 Mailboxes matching the source options for which the logged-in user 275 lacks sufficient rights MUST be ignored by the ESEARCH command 276 processing. In particular, ESEARCH responses MUST NOT be returned 277 for those mailboxes. 279 2.3. Strictness in Search Matches 281 The base IMAP SEARCH command (Section 6.4.4. of [RFC3501]) requires 282 strict substring matching in text searches. Many servers, however, 283 use search engines that match strings in different ways, matching, 284 for example, "swim" to "swam" and "swum" as well, or only doing full 285 word matching (where "swim" will not match "swimming"). This is 286 covered by the "Fuzzy Search" extension to IMAP [RFC6203], and that 287 extension is compatible with this one and can be combined with it. 289 Whether or not Fuzzy Search is implemented or used, this extension 290 explicitly allows flexible searching with respect to TEXT and BODY 291 searches. Servers MAY use fuzzy text matching in multimailbox 292 searches. 294 3. Examples 296 In the following example, note that two ESEARCH commands are 297 pipelined, and that the server is running them in parallel, 298 interleaving a response to the second search amid the responses to 299 the first (watch the tags). 301 C: tag1 ESEARCH IN (mailboxes "folder1" subtree "folder2") unseen 302 C: tag2 ESEARCH IN (mailboxes "folder1" subtree-one "folder2") 303 subject "chad" 304 S: * ESEARCH (TAG "tag1" MAILBOX "folder1" UIDVALIDITY 1) UID ALL 305 4001,4003,4005,4007,4009 306 S: * ESEARCH (TAG "tag2" MAILBOX "folder1" UIDVALIDITY 1) UID ALL 307 3001:3004,3788 308 S: * ESEARCH (TAG "tag1" MAILBOX "folder2/banana" UIDVALIDITY 503) 309 UID ALL 3002,4004 310 S: * ESEARCH (TAG "tag1" MAILBOX "folder2/peach" UIDVALIDITY 3) UID 311 ALL 921691 312 S: tag1 OK done 313 S: * ESEARCH (TAG "tag2" MAILBOX "folder2/salmon" UIDVALIDITY 314 1111111) UID ALL 50003,50006,50009,50012 315 S: tag2 OK done 317 4. Formal Syntax 319 The following syntax specification uses the Augmented Backus-Naur 320 Form (ABNF) as described in [RFC5234]. Terms not defined here are 321 taken from [RFC3501], [RFC5465], or [RFC4466]. 323 command-auth =/ esearch 324 ; Update definition from IMAP base [RFC3501]. 325 ; Add new "esearch" command. 327 command-select =/ esearch 328 ; Update definition from IMAP base [RFC3501]. 329 ; Add new "esearch" command. 331 filter-mailboxes-other =/ ("subtree-one" SP one-or-more-mailbox) 332 ; Update definition from IMAP Notify [RFC5465]. 333 ; Add new "subtree-one" selector. 335 filter-mailboxes-selected = "selected" 336 ; Update definition from IMAP Notify [RFC5465]. 337 ; We forbid the use of "selected-delayed". 339 one-correlator = ("TAG" SP tag-string) / ("MAILBOX" SP astring) / 340 ("UIDVALIDITY" SP nz-number) 341 ; Each correlator MUST appear exactly once. 343 scope-option = scope-option-name [SP scope-option-value] 344 ; No options defined here. Syntax for future extensions. 346 scope-option-name = tagged-ext-label 347 ; No options defined here. Syntax for future extensions. 349 scope-option-value = tagged-ext-val 350 ; No options defined here. Syntax for future extensions. 352 scope-options = scope-option *(SP scope-option) 353 ; A given option may only appear once. 354 ; No options defined here. Syntax for future extensions. 356 esearch = "ESEARCH" [SP esearch-source-opts] 357 [SP search-return-opts] SP search-program 359 search-correlator = SP "(" one-correlator *(SP one-correlator) ")" 360 ; Updates definition in IMAP4 ABNF [RFC4466]. 362 esearch-source-opts = "IN" SP "(" source-mbox [SP 363 "(" scope-options ")"] ")" 365 source-mbox = filter-mailboxes *(SP filter-mailboxes) 366 ; "filter-mailboxes" is defined in IMAP Notify [RFC5465]. 367 ; See updated definition of filter-mailboxes-other, above. 368 ; See updated definition of filter-mailboxes-selected, above. 370 5. Security Considerations 372 This new IMAP ESEARCH command allows a single command to search many 373 mailboxes at once. On the one hand, a client could do that by 374 sending many IMAP SEARCH commands. On the other hand, this makes it 375 easier for a client to overwork a server, by sending a single command 376 that results in an expensive search of tens of thousands of 377 mailboxes. Server implementations need to be aware of that, and 378 provide mechanisms that prevent a client from adversely affecting 379 other users. Limitations on the number of mailboxes that may be 380 searched in one command, and/or on the server resources that will be 381 devoted to responding to a single client, are reasonable limitations 382 for an implementation to impose. 384 Implementations MUST, of course, apply access controls appropriately, 385 limiting a user's access to ESEARCH in the same way its access is 386 limited for any other IMAP commands. This extension has no data- 387 access risks beyond what may be there in the unextended IMAP 388 implementation. 390 Mailboxes matching the source options for which the logged-in user 391 lacks sufficient rights MUST be ignored by the ESEARCH command 392 processing (see the paragraph about this in Section 2.2). In 393 particular, any attempt to distinguish insufficient access from non- 394 existent mailboxes may expose information about the mailbox hierarchy 395 that isn't otherwise available to the client. 397 If "subtree" is specified, the server MUST defend against loops in 398 the hierarchy (see the paragraph about this in Section 2.2). 400 6. IANA Considerations 402 The "IMAP Capabilities" registry is currently located at . 405 IANA is asked to change the reference for the IMAP capability 406 "MULTISEARCH", to point to this document. 408 7. Acknowledgements 410 The authors gratefully acknowledge feedback provided by Timo 411 Sirainen, Peter Coates, Arnt Gulbrandsen, and Chris Newman. 413 8. References 415 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 416 Requirement Levels", BCP 14, RFC 2119, March 1997. 418 [RFC2978] Freed, N. and J. Postel, "IANA Charset Registration 419 Procedures", BCP 19, RFC 2978, October 2000. 421 [RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 422 4rev1", RFC 3501, March 2003. 424 [RFC4314] Melnikov, A., "IMAP4 Access Control List (ACL) Extension", 425 RFC 4314, December 2005. 427 [RFC4466] Melnikov, A. and C. Daboo, "Collected Extensions to IMAP4 428 ABNF", RFC 4466, April 2006. 430 [RFC4731] Melnikov, A. and D. Cridland, "IMAP4 Extension to SEARCH 431 Command for Controlling What Kind of Information Is 432 Returned", RFC 4731, November 2006. 434 [RFC5182] Melnikov, A., "IMAP Extension for Referencing the Last 435 SEARCH Result", RFC 5182, March 2008. 437 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 438 Specifications: ABNF", STD 68, RFC 5234, January 2008. 440 [RFC5267] Cridland, D. and C. King, "Contexts for IMAP4", RFC 5267, 441 July 2008. 443 [RFC5465] Gulbrandsen, A., King, C. and A. Melnikov, "The IMAP 444 NOTIFY Extension", RFC 5465, February 2009. 446 [RFC6203] Sirainen, T., "IMAP4 Extension for Fuzzy Search", RFC 447 6203, March 2011. 449 Authors' Addresses 451 Barry Leiba 452 Huawei Technologies 454 Phone: +1 646 827 0648 455 Email: barryleiba@computer.org 456 URI: http://internetmessagingtechnology.org/ 457 Alexey Melnikov 458 Isode Limited 459 5 Castle Business Village 460 36 Station Road 461 Hampton, Middlesex TW12 2BX 462 UK 464 Email: Alexey.Melnikov@isode.com 465 URI: http://www.melnikov.ca/