idnits 2.17.00 (12 Aug 2021) /tmp/idnits12422/draft-ietf-lemonade-compress-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 12. -- Found old boilerplate from RFC 3978, Section 5.5 on line 312. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 283. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 289. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 294. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing document type: Expected "INTERNET-DRAFT" in the upper left hand corner of the first page == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 17 instances of too long lines in the document, the longest one being 6 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 2006) is 5818 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'BINARY' is mentioned on line 177, but not defined == Unused Reference: 'STARTTLS' is defined on line 254, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2234 (ref. 'ABNF') (Obsoleted by RFC 4234) ** Obsolete normative reference: RFC 3501 (ref. 'IMAP') (Obsoleted by RFC 9051) ** Downref: Normative reference to an Informational RFC: RFC 1951 (ref. 'DEFLATE') Summary: 8 errors (**), 0 flaws (~~), 5 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group Arnt Gulbrandsen 2 Request for Comments: DRAFT Oryx Mail Systems GmbH 3 draft-ietf-lemonade-compress-01.txt June 2006 5 The IMAP COMPRESS=DEFLATE Extension 7 Status of this Memo 9 By submitting this Internet-Draft, each author represents that any 10 applicable patent or other IPR claims of which he or she is aware 11 have been or will be disclosed, and any of which he or she becomes 12 aware will be disclosed, in accordance with Section 6 of BCP 79. 14 Internet-Drafts are working documents of the Internet Engineering 15 Task Force (IETF), its areas, and its working groups. Note that 16 other groups may also distribute working documents as Internet- 17 Drafts. 19 Internet-Drafts are draft documents valid for a maximum of six 20 months and may be updated, replaced, or obsoleted by other documents 21 at any time. It is inappropriate to use Internet-Drafts as 22 reference material or to cite them other than as "work in progress". 24 The list of current Internet-Drafts can be accessed at 25 http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet- 26 Draft Shadow Directories can be accessed at 27 http://www.ietf.org/shadow.html. 29 Copyright Notice 31 Copyright (C) The Internet Society 2006. 33 Abstract 35 The COMPRESS=DEFLATE extension allows an IMAP connection to be 36 compressed using the DEFLATE algorithm, such that effective 37 compression is available even when TLS is used. 39 Conventions Used in This Document 41 The key words "REQUIRED", "MUST", "MUST NOT", "SHOULD", "SHOULD 42 NOT", and "MAY" in this document are to be interpreted as described 43 in "Key words for use in RFCs to Indicate Requirement Levels" 45 [KEYWORDS]. Formal syntax is defined by [ABNF] as modified by 46 [IMAP]. 48 In the example, "C:" and "S:" indicate lines sent by the client and 49 server respectively. 51 Introduction and Overview 53 An IMAP server that supports this extension announces 54 "COMPRESS=DEFLATE" as one of its capabilities. 56 The goal of COMPRESS=DEFLATE is to reduce the bandwidth usage of 57 IMAP. On regular IMAP connections, the PPP or MNP compression used 58 with many low-bandwidth links compresses IMAP well. However, when 59 TLS is used, PPP/MNP compression is ineffective. TLS too may provide 60 compression, but few or no implementations do so in practice. 62 In order to increase interoperation, it is desirable to have as few 63 different compression algorithms as possible, so this document 64 specifies only one. The DEFLATE algorithm is standard, widely 65 available, unencumbered by patents and fairly efficient. Hopefully 66 it will not be necessary to define additional algorithms. 68 The extension adds one new command (COMPRESS) and no new responses. 70 The COMPRESS Command 72 Arguments: Name of compression mechanism: "DEFLATE". 73 Direction: "UP", "DOWN" or "BOTH". 75 Responses: None 77 Result: OK The server will compress its responses (if the direction 78 is DOWN or BOTH) and expects the client to compress its 79 commands (if the direction is UP or BOTH). 80 NO The connection already is compressed, or the server 81 doesn't support the requested mechanism, or the direction 82 specified is unknown. 83 BAD Command unknown or invalid argument. 85 The COMPRESS command instructs the server to use the named 86 compression mechanism ("DEFLATE" is the only one defined) for future 87 commands and/or responses. If the direction specified is "UP", only 88 commands are compressed. If the direction specified is "DOWN", only 89 For DEFLATE (as for many other compression mechanisms), the 90 compressor can trade speed against quality. When decompressing 91 there isn't much of a tradeoff. Consequently, the client and server 92 are both free to pick the best reasonable rate of compression for 93 the data they send. 95 The client MUST NOT send additional commands until it has seen the 96 result of COMPRESS. 98 If both SASL/TLS and COMPRESS are in use, the data should be 99 compressed before it is encrypted (and decrypted before it is 100 decompressed), independent of the order in which the client issues 101 COMPRESS, AUTHENTICATE and STARTTLS. 103 Example 105 This example shows a simple login sequence. The client uses TLS for 106 privacy and [DEFLATE] for compression. 108 S: * OK [CAPABILITY IMAP4REV1 STARTTLS COMPRESS=DEFLATE] 109 C: a starttls 110 S: a OK 111 C: b compress deflate 112 S: b OK 113 C: c login arnt tnra 114 S: c OK 116 Compression Efficiency 118 IMAP poses some unusual problems for a compression layer. 120 Upstream is fairly simple. Most IMAP clients send the same few 121 commands again and again, so any compression algorith which can 122 exploit quotes works efficiently. The APPEND command is an 123 exception; clients which send many APPEND commands may want to take 124 special care. 126 Downstream has the unusual property that 3-4 kinds of data are sent, 127 confusing all dictionary-based compression algorithms. 129 The first type is IMAP responses. These are highly compressible; 130 zlib using its least CPU-intensive setting compresses typical 131 responses to 25-40% of their original size. 133 The second is email headers. These are equally compressible, and 134 benefit from using the same dictionary as the IMAP responses. 136 The third is email body text. Text is usually fairly short and 137 includes much ASCII, so the same compression dictionary will do a 138 good job here, too. When multiple messages in the same thread are 139 read at the same time, quoted lines etc. can often be compressed 140 almost to zero. 142 Finally, attachments (non-text email bodies) are transmitted, either 143 in [BINARY] form or encoded with base-64. 145 When attachments are retrieved in [BINARY] form, DEFLATE may be able 146 to compress them, but the format of the attachment is usually not 147 IMAP-like, so the dictionary built while compressing IMAP does not 148 help. The compressor has to adapt from IMAP to the attachment's 149 format, and then back. 151 When attachments are retrieved in base-64 form, the same problems 152 apply, but the base-64 encoding adds another problem. 8-bit 153 compression algorithms such as deflate work well on 8-bit file 154 formats, however base-64 turns a file into something resembling a 155 6-bit bytes in an 8-bit format. 157 A few file formats aren't compressible using deflate, e.g. .gz, .zip 158 and .jpg files. 160 According to the author's measurements, the compression level used 161 makes little difference. zlib's level 1 compresses IMAP almost as 162 well as level 9, and for the receiver, level 1 seems to require 163 (just a tiny bit) pmore CPU than level 9. Independent verification 164 is strongly desired. 166 Implementation Notes 168 When using the zlib library (see [DEFLATE]), the functions 169 deflateInit(), deflate(), inflateInit() and inflate() suffice to 170 implement this extension. 172 Note that when using TLS, compression may actually decrease the CPU 173 usage, depending on which algorithms are used in TLS. This is 174 because fewer bytes need to be encrypted, and encryption is 175 generally more expensive than compression. 177 A client can improve downstream compression by implementing [BINARY] 178 and using FETCH BINARY instead of FETCH BODY. 180 A server can improve downstream compression if it hints to the 181 compressor that the data type is about to change strongly, e.g. by 182 sending a Z_FULL_FLUSH at the start and end of large non-text 183 literals (before and after '*CHAR8' in the definition of literal in 184 RFC 3501, page 86). 186 A server can improve the CPU efficiency both of the server and the 187 client if it adjusts the compression level (e.g. using the 188 deflateParams() function in zlib) at these points. A very simple 189 strategy is to change the level 0 to at the start of a literal 190 provided the first two bytes are either 0x1F 0x8B (as in deflate- 191 compressed files) or 0xFF 0xD8 (JPEG), and to keep it at 1-5 the 192 rest of the time. 194 Formal Syntax 196 The following syntax specification uses the Augmented Backus-Naur 197 Form (ABNF) notation as specified in [ABNF]. Non-terminals 198 referenced but not defined below are as defined by [ABNF] (SP, CRLF) 199 or [IMAP] (all others). 201 Except as noted otherwise, all alphabetic characters are case- 202 insensitive. The use of upper or lower case characters to define 203 token strings is for editorial clarity only. Implementations MUST 204 accept these strings in a case-insensitive fashion. 206 command-any =/ compress 208 compress = "COMPRESS" SP algorithm SP ( "UP" / "DOWN" / 209 "BOTH" ) 211 algorithm = "DEFLATE" 213 Security considerations 215 (As for [TLSCOMP] RFC 3749.) 217 IANA Considerations 219 The IANA is requested to add COMPRESS=DEFLATE to the list of IMAP 220 extensions. 222 Credits 224 Quite a few people on the LEMONADE mailing list have offered 225 comments, including Dave Cridland, Ned Freed and Tony Hansen. And 226 various people in the rooms at meetings. Send me mail, I'll add you. 228 Open Issues 230 Both ends can already disable compression at any point by calling 231 deflateParams(). The only missing feature is for the client to 232 request that the server stop compressing - are there use-cases for 233 that? It requires adding more server-side state, so I'm wary. 235 What text and numbers are needed wrt. compression levels? A bit of 236 solid information is not amiss. 238 Normative References 240 [ABNF] Crocker, Overell, "Augmented BNF for Syntax 241 Specifications: ABNF", RFC 2234, Internet Mail 242 Consortium, Demon Internet Ltd, November 1997. 244 [IMAP] Crispin, "Internet Message Access Protocol - Version 245 4rev1", RFC 3501, University of Washington, June 2003. 247 [KEYWORDS] Bradner, "Key words for use in RFCs to Indicate 248 Requirement Levels", RFC 2119, Harvard University, March 249 1997. 251 [DEFLATE] Deutsch, "DEFLATE Compressed Data Format Specification 252 version 1.3", RFC 1951, Aladdin Enterprises, May 1996. 254 [STARTTLS] Newman, C. "Using TLS with IMAP, POP3 and ACAP", RFC 255 2595, June 1999. 257 Informative References 259 [TLSCOMP] Hollenbeck, "Transport Layer Security Protocol 260 Compression Methods", RFC 3749, VeriSign, May 2004. 262 Author's Address 264 Arnt Gulbrandsen 265 Oryx Mail Systems GmbH 266 Schweppermannstr. 8 267 D-81671 Muenchen 268 Germany 270 Fax: +49 89 4502 9758 272 Email: arnt@oryx.com 274 Intellectual Property Statement 276 The IETF takes no position regarding the validity or scope of any 277 Intellectual Property Rights or other rights that might be claimed to 278 pertain to the implementation or use of the technology described in this 279 document or the extent to which any license under such rights might or 280 might not be available; nor does it represent that it has made any 281 independent effort to identify any such rights. Information on the 282 procedures with respect to rights in RFC documents can be found in BCP 78 283 and BCP 79. 285 Copies of IPR disclosures made to the IETF Secretariat and any assurances 286 of licenses to be made available, or the result of an attempt made to 287 obtain a general license or permission for the use of such proprietary 288 rights by implementers or users of this specification can be obtained from 289 the IETF on-line IPR repository at http://www.ietf.org/ipr. 291 The IETF invites any interested party to bring to its attention any 292 copyrights, patents or patent applications, or other proprietary rights 293 that may cover technology that may be required to implement this standard. 294 Please address the information to the IETF at ietf-ipr@ietf.org. 296 Copyright Statement 298 Copyright (C) The Internet Society (2006). 300 This document is subject to the rights, licenses and restrictions 301 contained in BCP 78, and except as set forth therein, the authors retain 302 all their rights. 304 Disclaimer of Validity 306 This document and the information contained herein are provided on an "AS 307 IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS 308 SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING 309 TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT 310 LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT 311 INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR 312 FITNESS FOR A PARTICULAR PURPOSE. 314 Acknowledgment 316 Funding for the RFC Editor function is currently provided by the Internet 317 Society.