--- rfc3507.txt Mon Oct 25 15:23:56 2004 +++ rfc3507-fixed.txt Wed Jun 1 15:51:34 2005 @@ -587,7 +587,14 @@ message format of RFC 2822 [3] -- that is, a start-line (either a request line or a status line), a number of header fields (also known as "headers"), an empty line (i.e., a line with nothing preceding the - CRLF) indicating the end of the header fields, and a message-body. + CRLF) indicating the end of the header fields, and possibly a + message-body. + + The presence of a message-body is determined exclusively by the + presence and value of the Encapsulated header documented in Section + 4.4. Thus, the sender MUST include the Encapsulated header in every + ICAP message with message-body. The message-body syntax and semantics + are determined by the value of the Encapsulated header. The header lines of an ICAP message specify the ICAP resource being requested as well as other meta-data such as cache control @@ -601,6 +608,29 @@ allowing only one outstanding request on a transport connection at a time. Multiple parallel connections MAY be used as in HTTP. + An ICAP client sending a message-body MUST monitor the transport + connection for an early ICAP response (i.e., the response that comes + while the client is still transmitting the request). Such a response + may be a successful (e.g., 200 "OK") response or not (e.g., 400 "Bad + Request"). Just like HTTP rules in Section 8.2.2 of [4], this + requirement eliminates a deadlock when neither client nor server can + send more data. However, correct early response handling is more + important (and not limited to errors) for ICAP because ICAP servers + often have to respond early to avoid buffering the entire + encapsulated message. Early responses may also decrease end-user + perceived latency if the client pipelines received content to the + end-user. + + Regardless of the early response meaning and timing, the ICAP client + SHOULD finish sending the request. If the client chooses not to + finish the request, it MUST terminate the transport connection after + receiving the early response because the ICAP server would not be + able to detect the end of the ICAP request otherwise. ICAP + extensions (not defined in this document) MAY supersede these + requirements by documenting ways to abort the request without + terminating the transport connection abnormally. + + 4.2 ICAP URIs All ICAP requests specify the ICAP resource being requested from the @@ -777,6 +807,9 @@ 408 - Request timeout. ICAP server gave up waiting for a request from an ICAP client. + 418 - Bad composition. ICAP server needs encapsulated sections + different from those in the request. + 500 - Server error. Error on the ICAP server, such as "out of disk space". @@ -864,12 +897,28 @@ (See Examples in Sections 4.8.3 and 4.9.3.). The motivation behind this decision is described in Section 8.2. + HTTP chunked transfer-coding may include a trailer area containing + HTTP entity-header fields. Since ICAP requires support for chunked + transfer-coding, an ICAP agent MUST accept an encapsulated trailer, + if any (i.e., the presence of a trailer must not prevent ICAP + recipient from correctly parsing and handling an ICAP message). + Similar to other HTTP message parts, an ICAP server MUST send the + received trailer back to the ICAP client unless the ICAP server + modifies or strips trailers as a part of server content adaptation + actions. + + An ICAP agent MUST NOT send an ICAP header in a trailer area of the + ICAP message-body encoding unless it knows the recipient expects such + a header. This document does not define how such an expectation is + negotiated. In general, sending ICAP headers in the trailer makes it + impossible for the trailer recipient to distinguish HTTP headers from + ICAP headers. + 4.4.1 The "Encapsulated" Header The offset of each encapsulated section's start relative to the start of the encapsulating message's body is noted using the "Encapsulated" - header. This header MUST be included in every ICAP message. For - example, the header + header. For example, the header Encapsulated: req-hdr=0, res-hdr=45, res-body=100 @@ -879,7 +928,10 @@ decimal notation for consistency with HTTP's Content-Length header. The special entity "null-body" indicates there is no encapsulated - body in the ICAP message. + HTTP body in the ICAP message. An Encapsulated header value of + "null-body=0" describes a message-body of zero length, which is + syntactically equivalent to having no message-body. A value of + "null-body=0" is common for OPTIONS responses, for example. The syntax of an Encapsulated header is: @@ -905,23 +957,42 @@ appear in the encapsulating message-body MUST be the same as the order in which the parts are named in the Encapsulated header. In other words, the offsets listed in the Encapsulated line MUST be - monotonically increasing. In addition, the legal forms of the - Encapsulated header depend on the method being used (REQMOD, RESPMOD, - or OPTIONS). Specifically: + monotonically increasing. + + In addition, the legal forms of the Encapsulated header value depend + on the request method. The value MUST use the following grammar for + matching requests and 200 "OK" responses to those requests. REQMOD request encapsulated_list: [reqhdr] reqbody REQMOD response encapsulated_list: {[reqhdr] reqbody} | {[reshdr] resbody} RESPMOD request encapsulated_list: [reqhdr] [reshdr] resbody RESPMOD response encapsulated_list: [reshdr] resbody + OPTIONS request encapsulated_list: [optbody] OPTIONS response encapsulated_list: optbody In the above grammar, note that encapsulated headers are always - optional. At most one body per encapsulated message is allowed. If - no encapsulated body is presented, the "null-body" header is used + OPTIONAL. At most one encapsulated body per ICAP message is allowed. + If no encapsulated body is presented, the "null-body" header is used instead; this is useful because it indicates the length of the header section. + An ICAP server receiving encapsulated_list that does not match server + needs MAY respond with a 418 "Bad Composition" error. This situation + may happen, for example, when the server does not receive + encapsulated HTTP requests headers in a RESPMOD request but needs + them to process the encapsulated HTTP response. + + Interpretation of a message-body depends on the Encapsulated header + value. This specification defines Encapsulated value semantics for + three request methods and 200 "OK" responses to those requests. The + sender MUST NOT include a message-body in any other message unless it + knows the recipient can handle it; the mechanism to obtain such + knowledge is beyond the scope of this document. For example, requests + using extension methods and responses other than 200 "OK" must not + include a message-body unless the recipient knows how to interpret + it. + Examples of legal Encapsulated headers: /* REQMOD request: This encapsulated HTTP request's headers start @@ -1112,6 +1183,9 @@ symbol explained below), then the ICAP server MUST NOT respond with 100 Continue. + As prescribed in Section 4.1.1, 100 "Continue" and 204 "No Content" + responses must not have message-bodies by default. + When an ICAP client is performing a preview, it may not yet know how many bytes will ultimately be available in the arriving HTTP message that it is relaying to the HTTP server. Therefore, ICAP defines a @@ -1277,15 +1351,19 @@ 4.8.1 Request - In REQMOD mode, the ICAP request MUST contain an encapsulated HTTP - request. The headers and body (if any) MUST both be encapsulated, - except that hop-by-hop headers are not encapsulated. - - - - - + In REQMOD mode, the ICAP request contains an encapsulated HTTP + request. An HTTP request has at most two parts: HTTP request headers + (including HTTP Request-Line) and possibly an HTTP request body. An + ICAP client MUST encapsulate at least one part. If the request body + is not encapsulated, the client MUST use the "null-body" entity. + + To improve interoperability, an ICAP client SHOULD encapsulate all + available HTTP request parts unless it knows the ICAP server expects + just one part. Note that an HTTP trailer, if any, is a part of the + chunked HTTP request body and, hence, may be present in an ICAP + REQMOD request even if HTTP request headers are not encapsulated. + An ICAP client MUST NOT encapsulate HTTP hop-by-hop request headers. Elson & Cerpa Informational [Page 23] @@ -1494,13 +1572,21 @@ 4.9.1 Request - Using encapsulation described in Section 4.4, the header and body of - the HTTP response to be modified MUST be included in the ICAP body. - If available, the header of the original client request SHOULD also - be included. As with the other method, the hop-by-hop headers of the - encapsulated messages MUST NOT be forwarded. The Encapsulated header - MUST indicate the byte-offsets of the beginning of each of these four - parts. + In RESPMOD mode, the ICAP request contains optional encapsulated HTTP + request headers and an encapsulated HTTP response. An HTTP response + has at most two parts: HTTP response headers (including HTTP + Status-Line) and possibly an HTTP response body. An ICAP client MUST + encapsulate at least one of those two parts. If the HTTP response + body is not encapsulated, the client MUST use the "null-body" entity. + + To improve interoperability, an ICAP client SHOULD encapsulate HTTP + request headers and all available HTTP response parts unless it knows + the ICAP server expects something else. Note that an HTTP trailer, + if any, is a part of the chunked HTTP response body and, hence, may + be present in an ICAP RESPMOD request even if HTTP response headers + are not encapsulated. + + An ICAP client MUST NOT encapsulate HTTP hop-by-hop response headers. 4.9.2 Response @@ -1616,9 +1702,8 @@ The OPTIONS method consists of a request-line, as described in Section 4.3.2, such as the following example: - OPTIONS icap://icap.server.net/sample-service ICAP/1.0 User-Agent: - ICAP-client-XYZ/1.001 - + OPTIONS icap://icap.server.net/sample-service ICAP/1.0 + User-Agent: ICAP-client-XYZ/1.001 @@ -1631,6 +1716,12 @@ Other headers are also allowed as described in Section 4.3.1 and Section 4.3.2 (for example, Host). + Some ICAP servers may not be able to handle OPTIONS requests with + message-body because earlier protocol specifications did not + explicitly allow or prohibit such requests. An ICAP client MUST NOT + send an OPTIONS request with a message-body, unless the client knows + that the server can handle such a request. + 4.10.2 OPTIONS Response The OPTIONS response consists of a status line as described in @@ -2026,19 +2117,41 @@ encoding within the encapsulated body section as defined in HTTP/1.1 [4]. This requires that ICAP client implementations convert incoming objects "on the fly" to chunked from whatever transfer-encoding on - which they arrive. However, the transformation is simple: - - - For objects arriving using "Content-Length" headers, one big chunk - can be created of the same size as indicated in the Content-Length - header. - - - For objects arriving using a TCP close to signal the end of the - object, each incoming group of bytes read from the OS can be - converted into a chunk (by writing the length of the bytes read, - followed by the bytes themselves) + which they arrive. A straightforward conversion approach is + highlighted below. - - For objects arriving using chunked encoding, they can be - retransmitted as is (without re-chunking). + As object content comes in, the ICAP client converts all available + content bytes into a single chunk to be sent to the ICAP server. If + incoming content is chunked-encoded, the client decodes the encoding + first, to get access to object content. The client follows HTTP rules + to detect the end of the incoming HTTP message. For example, if the + client gets an HTTP message with Content-Length of 100KB and gets the + first 100 bytes of that message content, the client can send the + first 100 bytes as a single complete chunk. The client should neither + (a) wait a long time for all 100KB to arrive or (b) announce a 100KB + chunk but send the first 100 bytes only. + + The above straightforward process can be optimized to minimize + copying of content bytes, even if the incoming content is chunked. + For example, an implementation can wait a little for more HTTP + content (or the entire HTTP chunk) to become available before forming + and sending a chunk to the ICAP server. + + When object content length is known a priori, it is tempting to + declare a single chunk of matching size and then forward incoming + object data as it comes in, without any additional encoding efforts. + Similarly, it is tempting to forward already chunked content "as is", + without re-chunking it first. + + However, unrecoverable errors may occur when an ICAP client promises + to send chunk content that it does not yet have because the promised + data may never arrive due to origin server or network errors. + Chunked coding does not have a mechanism to terminate a chunk + prematurely; the ICAP server would expect all promised bytes. Thus, + if ICAP client receives fewer than expected HTTP bytes, it has no + other choice but to close the ICAP connection. A straightforward + approach described above does not make false promises and avoids the + problem. 6.4 Distinct URIs for Distinct Services