Chunked transfer encoding is a method in HTTP 1.1 for streaming data. Data is divided into non-overlapping chunks, each prefixed by its size. The transmission concludes with a zero-length chunk. This allows persistent connections for dynamically generated content and facilitates sending additional header fields after the message body.
Chunked transfer is considered acceptable in HTTP 1.1, even without explicit specification. The format involves expressing chunk size in hexadecimal ASCII, optional parameters, and a termination sequence for each chunk. Trailers, specified using TE and Trailers header fields, can follow the terminating chunk. Overall, chunked transfer encoding is useful for efficiently transmitting data in a streaming fashion, especially when content size is unknown beforehand or additional headers are required after the message body.
The "Transfer-Encoding: chunked" mechanism is used to send data in variable-sized chunks. These chunks are represented in hexadecimal format within the HTTP message. This is done for efficiency and ease of parsing. Hexadecimal notation allows for a compact representation of the chunk size, making it more efficient for both transmission and parsing compared to decimal.
In contrast, if the entire content is known before sending. A content-length header can be set. It describes the whole length of the trsamitted content. The "Content-Length" header in HTTP represents the length of the message body in decimal format. This is because it's a straightforward way to specify the number of octets (bytes) in the body of the HTTP message. Decimal notation is commonly used for human readability and ease of parsing. After octet-stream according to the content length is read, the connection can be closed.
In this example a http response sends random bytes as chunks. Those chunks are decoded and displayed here.